Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.......................XXX..................................] duration: 5.957757058s FAIL (2025-07-19 20:12:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.957787867s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [........................XXX.................................] duration: 5.857496483s FAIL (2025-07-19 20:12:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.857524997s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....................XXXXXX.................................] duration: 5.875817846s FAIL (2025-07-19 20:11:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.875839959s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [......................XXXXXX................................] duration: 5.977648369s FAIL (2025-07-19 20:11:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.977675842s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.........................XXX................................] duration: 5.600243627s FAIL (2025-07-19 20:11:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.600279255s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.......................XXXXXX...............................] duration: 5.895259755s FAIL (2025-07-19 20:10:47 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.895287657s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [..........................XXX...............................] duration: 5.66620404s FAIL (2025-07-19 20:10:46 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.666229459s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...........................XXX..............................] duration: 5.878011103s FAIL (2025-07-19 20:10:19 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.87804097s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [........................XXXXXX..............................] duration: 5.933648989s FAIL (2025-07-19 20:09:46 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.933715275s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [............................XXX.............................] duration: 5.680454084s FAIL (2025-07-19 20:09:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.680480224s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.........................XXXXXX.............................] duration: 5.982078301s FAIL (2025-07-19 20:09:19 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.982099552s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.............................XXX............................] duration: 5.671930794s FAIL (2025-07-19 20:08:48 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.671958648s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..........................XXXXXX............................] duration: 6.023828077s FAIL (2025-07-19 20:08:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.02385105s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [..............................XXX...........................] duration: 5.716201569s FAIL (2025-07-19 20:07:56 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.716227268s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...........................XXXXXX...........................] duration: 5.988030192s FAIL (2025-07-19 20:07:19 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.988058938s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [............................XXXXXX..........................] duration: 5.874587301s FAIL (2025-07-19 20:06:43 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.874616346s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...............................XXX..........................] duration: 5.708888577s FAIL (2025-07-19 20:06:43 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.708933042s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.............................XXXXXX.........................] duration: 5.897733316s FAIL (2025-07-19 20:05:58 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.89776194s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [................................XXX.........................] duration: 5.816001078s FAIL (2025-07-19 20:05:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.816032577s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..............................XXXXXX........................] duration: 5.965090594s FAIL (2025-07-19 20:05:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.965121194s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.................................XXX........................] duration: 5.624927165s FAIL (2025-07-19 20:04:26 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.624958757s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...............................XXXXXX.......................] duration: 5.839055366s FAIL (2025-07-19 20:04:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.839083761s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [..................................XXX.......................] duration: 5.676425133s FAIL (2025-07-19 20:03:18 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.676452274s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [................................XXXXXX......................] duration: 5.862253931s FAIL (2025-07-19 20:03:16 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.862277867s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.................................XXXXXX.....................] duration: 5.817605657s FAIL (2025-07-19 20:03:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.817657546s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..................................XXXXXX....................] duration: 5.8813144s FAIL (2025-07-19 20:02:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.881341112s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...................................XXX......................] duration: 5.748487288s FAIL (2025-07-19 20:02:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.74851411s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...................................XXXXXX...................] duration: 5.89971015s FAIL (2025-07-19 20:02:03 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.899740348s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [....................................XXX.....................] duration: 6.001214844s FAIL (2025-07-19 20:01:50 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.001235203s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.....................................XXX....................] duration: 5.681814735s FAIL (2025-07-19 20:01:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.681844892s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [....................................XXXXXX..................] duration: 6.002104521s FAIL (2025-07-19 20:01:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.00213072s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [......................................XXX...................] duration: 6.162433863s FAIL (2025-07-19 20:00:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.162460394s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.....................................XXXXXX.................] duration: 5.987172474s FAIL (2025-07-19 20:00:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.987198774s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.......................................XXX..................] duration: 5.917131681s FAIL (2025-07-19 20:00:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.917165797s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [......................................XXXXXX................] duration: 5.953918597s FAIL (2025-07-19 19:59:35 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.953944877s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [........................................XXX.................] duration: 5.618125222s FAIL (2025-07-19 19:59:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.618152744s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.........................................XXX................] duration: 5.626864131s FAIL (2025-07-19 19:58:31 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.626898798s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.......................................XXXXXX...............] duration: 5.782912808s FAIL (2025-07-19 19:58:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.78293994s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [..........................................XXX...............] duration: 5.834180221s FAIL (2025-07-19 19:57:32 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.834203114s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [........................................XXXXXX..............] duration: 6.306689289s FAIL (2025-07-19 19:57:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.306719537s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...........................................XXX..............] duration: 5.85444585s FAIL (2025-07-19 19:56:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.854468974s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.........................................XXXXXX.............] duration: 6.102817088s FAIL (2025-07-19 19:56:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.102842676s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [............................................XXX.............] duration: 5.78777292s FAIL (2025-07-19 19:55:55 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.787815281s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [..........................................XXXXXX............] duration: 5.925931713s FAIL (2025-07-19 19:55:36 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.925959476s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [.............................................XXX............] duration: 5.624513264s FAIL (2025-07-19 19:55:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.624539815s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [...........................................XXXXXX...........] duration: 6.836457974s FAIL (2025-07-19 19:54:30 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 6.836486439s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [..............................................XXX...........] duration: 6.764524619s FAIL (2025-07-19 19:54:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.764550359s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20) antiblip: 3 out of 60 is failed (threshold=3) [...............................................XXX..........] duration: 5.770898154s FAIL (2025-07-19 19:53:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== 7848 (20h54m48.288967213s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916415812 (20h54m48.288995167s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916709062 (20h54m48.289288417s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916861384 (20h54m48.289440749s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.916935646 (20h54m48.289515001s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917160236 (20h54m48.289739591s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917175124 (20h54m48.289754479s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917373244 (20h54m48.289952599s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917481661 (20h54m48.290061016s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917615176 (20h54m48.290194531s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.917784180 (20h54m48.290363535s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918031533 (20h54m48.290610888s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918281632 (20h54m48.290860987s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918492254 (20h54m48.291071609s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918719600 (20h54m48.291298955s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.918930743 (20h54m48.291510098s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:41.919150143 (20h54m48.291729498s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Uploading piece... Jul 19 19:45:43.033526636 (20h54m49.406105991s) [upload=16638]: Next returned nil segment Jul 19 19:45:43.033548608 (20h54m49.406127963s) [upload=16638]: Waiting for error group managing segments... Jul 19 19:45:43.368051177 (20h54m49.740630572s) [noise=true , address=79.127.205.233:4621 , piece_id=35MW6WACUECIHLE3W5WWCGHX5IZ2YYLISRHMTORWNK4YM7JUUM5A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.667078612 (20h54m50.039657967s) [noise=true , address=79.127.205.245:4245 , piece_id=ABTIC4DMOPNTKUJ7YYO2T6XRKFU6YOP77SIQEWIPRI4IRDPJRWGA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.830458308 (20h54m50.203037673s) [noise=true , address=79.127.201.215:4358 , piece_id=FWGZX5AID6PFEKW75LU2CRSJ7NTF5NSKNFA2UNV57V6ZYFDRCJNQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.839119359 (20h54m50.211698754s) [noise=true , address=104.36.82.18:4400 , piece_id=FQNSBCA37YGIBG4NIUUJE4CVIFQIFUAFXG2SXI57VB6ZE5ZLSVEQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.846613288 (20h54m50.219192663s) [noise=true , address=104.36.82.40:4961 , piece_id=VPUE2QBXZUSEZ27STEH7HMUN4V5O2RZRS572QD2KLRGVO7PDATEA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.905336063 (20h54m50.277915438s) [noise=true , address=79.127.219.47:4055 , piece_id=5SCFP4WKEX327LZ6OTGOBP4L3VCDECHT7YVKZ7PRQB2LHDEVMUDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.946328965 (20h54m50.31890832s) [noise=true , address=104.36.82.44:4902 , piece_id=I53357NLQDIKXESB6MRNKLQCB2QLXQE7JXGRIKZUOQWALQXHENCA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953031279 (20h54m50.325610624s) [noise=true , address=104.36.82.12:4152 , piece_id=EQFL55U6R6ELK6AUIJTCFKPQVVVSLG3DXLZTOM56KHYXUQLXV6PA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:43.953713835 (20h54m50.32629319s) [noise=true , address=104.36.82.49:4980 , piece_id=ZHVON73WV3JXGECZ5IMAEHTKFRDHZYUROS7SL4FOLVO2IZ2YH4AQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:45:44.164465048 (20h54m50.537044403s) [noise=true , address=104.36.82.51:4071 , piece_id=QJ7TRQ7GJCTPUVJEOXKZ7KQ7IK4GKZR3MULUBYUU7FLC6JSJXDWA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: <nil> Jul 19 19:46:40.808104090 (20h55m47.180683445s) [noise=true , address=45.91.116.9:4290 , piece_id=MNL77IA22542NU543FL6UAV2XWBSR6N3TGWK2DQEN7ZZAU56J66A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1txTbjkxQ82kwxCWNkQG4aPmVvcGARaAHxXNw1CXVTVMQFCLQ9, address:45.91.116.9:4290): protocol: rpc: dial tcp 45.91.116.9:4290: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808164726 (20h55m47.180744081s) [noise=true , address=45.91.116.14:4212 , piece_id=MQ67AOPO6WCZ6TPN3LWPUJ6ODJKVZWOMD6ACOL345H7IRLKDAFRA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1RiikePNRVqnERsVeC6eFxkTpttN5bkps9fwmntzX7EsxPAW6y, address:45.91.116.14:4212): protocol: write tcp 188.166.225.170:60452->45.91.116.14:4212: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808201998 (20h55m47.180781343s) [noise=true , address=77.90.31.18:4834 , piece_id=BCPFOSJZCAMP4RNNCL64TYC5AMCWXCB5GMP6FW5EF66Y3JT6INDA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:1sfZTiYg2UFBmB45CB8FrBeMvmDoKMQ6Mt6zQHKgD5D8cwYe6q, address:77.90.31.18:4834): protocol: write tcp 188.166.225.170:39120->77.90.31.18:4834: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808320064 (20h55m47.180899419s) [noise=true , address=77.90.31.16:4740 , piece_id=FFROTB5MBFFGCM2ETZ2QK3IR6Y2Y2LB3VNZPCNQ5NCHVOWYXU56A, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:145raYQJozNw3SMbj78wDsR6EjPvJTqqcocf1SaPs3HEnNVS5p, address:77.90.31.16:4740): protocol: write tcp 188.166.225.170:48266->77.90.31.16:4740: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808338409 (20h55m47.180917764s) [noise=true , address=77.90.31.17:4774 , piece_id=PZSNV4UQDV6OLR4PV3APVEJMK3QFSAA2TFTI3Q5TVX5UX5AJXKHQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12mZTv2q5zT1Za4DTcdFbUwWs1897vv9GXuLqPcnY3FaxTdGv3Y, address:77.90.31.17:4774): protocol: write tcp 188.166.225.170:57650->77.90.31.17:4774: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808386862 (20h55m47.180966217s) [noise=true , address=77.90.31.21:4900 , piece_id=SPFBTE2TWSS6EZALXQ4HCWX67PPYNEN3IN3VZV52NVZONEOYV6EA, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:129YE2Wd6LmDABJFaGVtYDwCHmStVDnPRCagD2cKPWA5PDTtZyh, address:77.90.31.21:4900): protocol: write tcp 188.166.225.170:55570->77.90.31.21:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808398804 (20h55m47.18097816s) [noise=true , address=77.90.31.10:4530 , piece_id=YHCHPBJU4TNX4BN3UB2M5ALIWMYCBASWDEXXE2I7SVNG4ZAN5ETQ, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:127c4HGgJu3z16vxq7TYZydCUa86FhCc3TmoPZoheVeqTbnJfne, address:77.90.31.10:4530): protocol: rpc: dial tcp 77.90.31.10:4530: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808427980 (20h55m47.181007335s) [noise=true , address=77.90.31.15:4701 , piece_id=ZBOEDVZ5TJI2SXFIRFYBAN55VUD6WFBPP33VFZD4JI4ODTBKPD3Q, seg_pos={0 0}, upload=16638]: Done uploading piece. err: ecclient: upload failed (node:12u5UY7VzJcjCvHtATf2ofXugWew2JXVXBjpManmX4wSzQwR7A6, address:77.90.31.15:4701): protocol: write tcp 188.166.225.170:55032->77.90.31.15:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:40.808454491 (20h55m47.181033836s) [seg_pos={0 0}, upload=16638]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Jul 19 19:46:41.038976382 (20h55m47.411555737s) [upload=16638]: Done uploading segments... Jul 19 19:46:41.039000268 (20h55m47.411579623s) [upload=16638]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.770932629s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [............................................XXXXXX..........] duration: 5.784038734s FAIL (2025-07-19 19:53:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.784063312s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.............................................XXXXXX.........] duration: 5.909804888s FAIL (2025-07-19 19:52:48 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 8) ======LOGS====== (needed at least 13 but got 8) Jul 19 19:45:56.955947272 (20h55m3.328526617s) [upload=16640]: Starting upload Jul 19 19:45:56.956012336 (20h55m3.328591681s) [upload=16640]: Done starting upload Jul 19 19:45:56.956155420 (20h55m3.328734775s) [upload=16640]: Uploading segments... Jul 19 19:45:56.956460032 (20h55m3.329039378s) [upload=16640]: Got next segment. Inline: false Jul 19 19:45:56.956489009 (20h55m3.329068354s) [upload=16640]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 19:45:56.956494329 (20h55m3.329073684s) [upload=16640]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 19:45:57.188815182 (20h55m3.561394537s) [seg_pos={0 0}, upload=16640]: Begin upload segment... Jul 19 19:45:57.188859768 (20h55m3.561439123s) [seg_pos={0 0}, upload=16640]: Done begin upload segment. Jul 19 19:45:57.188881990 (20h55m3.561461345s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.188916005 (20h55m3.56149536s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189230216 (20h55m3.561809571s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189362509 (20h55m3.561941864s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189500263 (20h55m3.562079618s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189661682 (20h55m3.562241037s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.189944903 (20h55m3.562524258s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190136841 (20h55m3.562716196s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.190521236 (20h55m3.563100591s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191050038 (20h55m3.563629413s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191561276 (20h55m3.564140641s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.191980658 (20h55m3.564560023s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192371286 (20h55m3.564950651s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.192693993 (20h55m3.565273348s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193026399 (20h55m3.565605754s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:57.193341352 (20h55m3.565920707s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Uploading piece... Jul 19 19:45:58.904082428 (20h55m5.276661783s) [upload=16640]: Next returned nil segment Jul 19 19:45:58.904100943 (20h55m5.276680298s) [upload=16640]: Waiting for error group managing segments... Jul 19 19:45:59.240386913 (20h55m5.612966268s) [noise=true , address=104.36.82.5:4837 , piece_id=KWDKXHG7BRJBGJJA4QECLAXFNWUTBB56U4XCJXLXEI4HMNTBP7SA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.265222572 (20h55m5.637801917s) [noise=true , address=104.36.82.16:4294 , piece_id=5BUSMFOTB2PEWUEZ6A54VBTLGFYLEMCMJ3JS2K2UJDW6BAIYMNKA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.328187464 (20h55m5.700766819s) [noise=true , address=104.36.82.14:4223 , piece_id=FHL3KR2FWK3UN3RGKJR2ASC2MOLXXFZD7SUATGKN7KTRDQBKANCQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.340208313 (20h55m5.712787668s) [noise=true , address=104.36.82.15:4259 , piece_id=ZV52K26NYDFEG44J57VLLATELCMLPLXH7FQ7OBGPQMW2QGGQ42SQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.388879266 (20h55m5.761458621s) [noise=true , address=104.36.82.10:4077 , piece_id=Y2HUXNCWXIVK2KHJNXM7MSJV3I4MFGXWRWEK6GHJ3XJPCHHMO7JQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.425886182 (20h55m5.798465537s) [noise=true , address=104.36.82.8:4972 , piece_id=H4JTIHR4YWG6JWOUWLZB3XSDRQOHHUMPSVIN6BNGTZ3NQWEIDIYQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.437905088 (20h55m5.810484443s) [noise=true , address=104.36.82.7:4910 , piece_id=JFHPTGV4Z6NUH6HXFOXOT24GYVHZKNQETH2SKQWVFAG7KBOPRJ5Q, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:45:59.453460110 (20h55m5.826039455s) [noise=true , address=104.36.82.11:4115 , piece_id=TNDZSLUAJFAWM6CAVRDK35LAXDEYYNLQNEA3NKUIGQ77REKYKHTA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: <nil> Jul 19 19:46:26.316383271 (20h55m32.688962626s) [noise=true , address=45.91.116.13:4153 , piece_id=75OGJ56DQMKOKPIW4G7H5DDVXCSKNMPNFH4O6N7C2OYLR4WGIJLA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1fSFcAat264kAGFqWt6Kkc14SSpUnXACWby84NqugHqheWcu2b, address:45.91.116.13:4153): protocol: rpc: dial tcp 45.91.116.13:4153: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316453304 (20h55m32.68903266s) [noise=true , address=77.90.31.12:4590 , piece_id=UUPP7J6SD5CIWXNZJ3RSZ24OADTUIXNTBCEI5WXH3MXB2DVH5IJQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw, address:77.90.31.12:4590): protocol: write tcp 188.166.225.170:34108->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316503360 (20h55m32.689082716s) [noise=true , address=45.91.116.4:4938 , piece_id=34E2KP6C2YVJJZ5ZS2NQCGZ4EXXERK75YNDUYEURNS5ZQ5ZPQDNA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW, address:45.91.116.4:4938): protocol: write tcp 188.166.225.170:50430->45.91.116.4:4938: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316554318 (20h55m32.689133683s) [noise=true , address=45.91.116.7:4010 , piece_id=ADVPKMIIYWWZ5MLTJYDZRBELVOHSQT3USU3XVQXMVUTL46NAZKTQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316571071 (20h55m32.689150416s) [noise=true , address=77.90.31.11:4555 , piece_id=CZMSPHUHSKZBM4ZFAIXMLEFFR7GGEUXS47TDNSJ6URLKBRJHIMQA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ, address:77.90.31.11:4555): protocol: rpc: dial tcp 77.90.31.11:4555: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316611477 (20h55m32.689190832s) [noise=true , address=77.90.31.15:4698 , piece_id=P4W553EH243TPBG2VTMCUI24VJCRBCMRBIWB3AX6OC42C6Q6QTOQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: rpc: dial tcp 77.90.31.15:4698: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316617990 (20h55m32.689197335s) [noise=true , address=45.91.116.7:4009 , piece_id=IO63DS5IXXTUELN2NI6KIA2NIBOC6WHPUIQFJBKWM7DYITX6ELHQ, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE, address:45.91.116.7:4009): protocol: rpc: dial tcp 45.91.116.7:4009: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316646504 (20h55m32.689225849s) [noise=true , address=45.91.116.6:4973 , piece_id=4ZUD4MCUNWBCL6EAXSITRFCS5FMK6ZTDGLEFSEFDWXOKRPAX6NHA, seg_pos={0 0}, upload=16640]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: rpc: dial tcp 45.91.116.6:4973: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 19:46:26.316660982 (20h55m32.689240337s) [seg_pos={0 0}, upload=16640]: Done waiting for segment. successful: 8 optimal: 13 errs: <nil> Jul 19 19:46:26.546689890 (20h55m32.919269245s) [upload=16640]: Done uploading segments... Jul 19 19:46:26.546711271 (20h55m32.919290626s) [upload=16640]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 8) Duration: 5.909844474s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)