Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 6.008178973s FAIL (2025-09-30 23:58:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 6.008209712s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.899417002s FAIL (2025-09-30 23:57:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 5.899450065s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.924429552s FAIL (2025-09-30 23:57:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 5.924459649s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.964602293s FAIL (2025-09-30 23:56:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 5.964629644s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.990345916s FAIL (2025-09-30 23:56:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 5.990377726s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.911669112s FAIL (2025-09-30 23:55:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 5.91169976s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.010846413s FAIL (2025-09-30 23:54:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 6.010879825s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.980212166s FAIL (2025-09-30 23:53:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 5.980245508s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.014220331s FAIL (2025-09-30 23:52:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 6.01424644s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.978853923s FAIL (2025-09-30 23:52:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 5.978950337s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.970022375s FAIL (2025-09-30 23:51:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 5.970058684s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.082132594s FAIL (2025-09-30 23:51:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 6.082253635s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.051645147s FAIL (2025-09-30 23:50:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 6.051673642s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.931199007s FAIL (2025-09-30 23:50:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 5.931230526s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.029844062s FAIL (2025-09-30 23:49:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 6.02986435s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 6.346645534s FAIL (2025-09-30 23:49:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 6.346680121s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.985030464s FAIL (2025-09-30 23:48:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 5.985071753s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.847712708s FAIL (2025-09-30 23:47:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 6.847746291s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 6.185787322s FAIL (2025-09-30 23:46:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 6.185819734s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 6.125620908s FAIL (2025-09-30 23:45:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 6.125653981s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 7.11808199s FAIL (2025-09-30 23:44:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 7.118116608s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.131741611s FAIL (2025-09-30 23:43:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 6.131775546s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 6.172468882s FAIL (2025-09-30 23:42:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 6.172506844s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.592712534s FAIL (2025-09-30 23:42:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 6.59274723s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.007464734s FAIL (2025-09-30 23:41:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 6.007499721s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.988587386s FAIL (2025-09-30 23:40:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 5.988621671s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 6.236943985s FAIL (2025-09-30 23:39:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 6.2369728s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 6.060064689s FAIL (2025-09-30 23:38:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 6.060108773s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 5.980407014s FAIL (2025-09-30 23:37:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 5.980435819s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.068337751s FAIL (2025-09-30 23:36:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 6.068367527s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.034282105s FAIL (2025-09-30 23:36:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 6.034308856s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.952283444s FAIL (2025-09-30 23:35:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 5.952317838s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.048571552s FAIL (2025-09-30 23:35:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 6.048609232s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.980765055s FAIL (2025-09-30 23:34:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 5.980798288s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.055619883s FAIL (2025-09-30 23:34:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 6.055640543s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 6.055697966s FAIL (2025-09-30 23:33:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 6.055791133s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.242770388s FAIL (2025-09-30 23:33:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== BatchRequestItem_ObjectBegin Sep 30 23:32:30.787106666 (178h1m16.249199149s) [upload=156525]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 30 23:32:31.035880869 (178h1m16.497973342s) [seg_pos={0 0}, upload=156525]: Begin upload segment... Sep 30 23:32:31.035944881 (178h1m16.498037364s) [seg_pos={0 0}, upload=156525]: Done begin upload segment. Sep 30 23:32:31.035968565 (178h1m16.498061048s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036091099 (178h1m16.498183582s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.036948150 (178h1m16.499040633s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037394821 (178h1m16.499487304s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.037780465 (178h1m16.499872948s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038223750 (178h1m16.500316233s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.038618852 (178h1m16.500711325s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.039046266 (178h1m16.501138749s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040254075 (178h1m16.502346548s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.040723169 (178h1m16.502815652s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.041619466 (178h1m16.503711949s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.043026595 (178h1m16.505119088s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.044323945 (178h1m16.506416418s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.045414312 (178h1m16.507506795s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.049552442 (178h1m16.511644925s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:31.053682636 (178h1m16.515775109s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Uploading piece... Sep 30 23:32:32.829078526 (178h1m18.291171009s) [upload=156525]: Next returned nil segment Sep 30 23:32:32.829103844 (178h1m18.291196357s) [upload=156525]: Waiting for error group managing segments... Sep 30 23:32:33.116111149 (178h1m18.578203622s) [noise=true , address=77.90.31.13:4626 , piece_id=KCU4B4H72AMMFVP5PC3BNJDRXQ3BIADQM26MJWKZEVU6TE7NKZSA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.117574465 (178h1m18.579666938s) [noise=true , address=77.90.31.11:4553 , piece_id=TZ54D5NWARUDSRMBD7FWGBQUIIVPJROSBFT5BRWU35537UL3NQCQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.147654172 (178h1m18.609746655s) [noise=true , address=77.90.31.16:4733 , piece_id=FPQJ3XZAU6SK7PDP4OCGJ4FF7ITN277QGAEKGW4DZBEV2GPCBFSQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.169509248 (178h1m18.631601731s) [noise=true , address=77.90.31.18:4807 , piece_id=L4VSEIGBUIBL4MP34HOUVKIWMKHPELN2NDZ2O7F7EAWRMCK75HAQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.196498391 (178h1m18.658590874s) [noise=true , address=77.90.31.19:4841 , piece_id=ICKWI5NHQJVQE7JPKFTPB5C4SQQZ35LUJIREXDEOL4M2UKPK4FVQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.225847266 (178h1m18.687939749s) [noise=true , address=45.91.116.14:4191 , piece_id=VFV6L5BFGA5NQUZD66G2PG5OF25AFKC4C3UJFMO4CCG6UKBPMT7Q, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.234350858 (178h1m18.696443341s) [noise=true , address=45.91.116.13:4153 , piece_id=LQ5V7EDQTHAVJWYMCSHOQBDTV6VNGGWXYS7EZIO3KBRBMDDNVMXA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.291091744 (178h1m18.753184227s) [noise=true , address=77.90.31.21:4879 , piece_id=X24TC6JGEGM2PQD626ZD6SGRDW6HPPHJPG65WRKW3PSRSMQRANRQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.329183768 (178h1m18.791276251s) [noise=true , address=77.90.31.13:4627 , piece_id=DZNGY57CZ3N4N2L7V5B7HKTBKDKEFMVZO7VZ2DEUZJQPQH43MEIQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:32:33.464330452 (178h1m18.926422935s) [noise=true , address=77.90.31.17:4771 , piece_id=IVAUAFMGKYVER4D3GOXBZNOLB4UA37Q6JZFZ7ZKSICRBJ5C2BJFA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: <nil> Sep 30 23:33:00.088911554 (178h1m45.551004037s) [noise=true , address=145.79.227.7:4295 , piece_id=MNGTXVBIC3SGSAMO3XIZDLESUEPT3ORFRWHVSVOMSK2K4AW5GJTA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57810->145.79.227.7:4295: use of closed network connection Sep 30 23:33:00.089058865 (178h1m45.551151348s) [noise=true , address=145.79.227.4:4185 , piece_id=A3WOIVGEBMMW5OMKWHIKYKEORNRYE47UVSTOPFMSPV7JDC6T6OAA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50040->145.79.227.4:4185: use of closed network connection Sep 30 23:33:00.089173733 (178h1m45.551266236s) [noise=true , address=145.79.227.7:4293 , piece_id=OU3LD2GOKIKKJSZZH6JK3F62DM5ABLWPRY55JCUSVXKHU52DIS4A, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37476->145.79.227.7:4293: use of closed network connection Sep 30 23:33:00.089227225 (178h1m45.551319708s) [noise=true , address=145.79.227.4:4187 , piece_id=SPNSMLM7XFTUPDZ2SD3O3NKTNMH7K3YFNLHAN7KXYQLPIZYU3ZLQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33086->145.79.227.4:4187: use of closed network connection Sep 30 23:33:00.089281278 (178h1m45.551373811s) [noise=true , address=145.79.227.15:4873 , piece_id=KTLONWYEZPGJNGQT26WAS6SZIV26YOZUF5KEIUXF2UTNNM4WEKGQ, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55198->145.79.227.15:4873: use of closed network connection Sep 30 23:33:00.089477532 (178h1m45.551570015s) [noise=true , address=145.79.227.12:4767 , piece_id=LDK4AQB6MAPPIJW3JLV3OFD5DDBU76VPMSDDQOO64OOFYVH2KLUA, seg_pos={0 0}, upload=156525]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:45372->145.79.227.12:4767: use of closed network connection; context deadline exceeded Sep 30 23:33:00.089504934 (178h1m45.551597417s) [seg_pos={0 0}, upload=156525]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 30 23:33:00.330302686 (178h1m45.792395179s) [upload=156525]: Done uploading segments... Sep 30 23:33:00.330334727 (178h1m45.792427211s) [upload=156525]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration:* 30.242800066s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.298141337s FAIL (2025-09-30 23:10:00 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.298161344s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.52300801s FAIL (2025-09-30 23:09:29 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.52306068s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.416779363s FAIL (2025-09-30 23:09:04 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.416807619s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.262190611s FAIL (2025-09-30 23:08:41 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.262216921s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.348405321s FAIL (2025-09-30 23:07:37 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.348427343s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.396074568s FAIL (2025-09-30 23:07:06 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.396103024s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 7.040010575s FAIL (2025-09-30 23:05:53 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 7.040077283s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.160932379s FAIL (2025-09-30 23:04:45 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.160978385s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.31201133s FAIL (2025-09-30 23:04:11 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.312040126s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.735840256s FAIL (2025-09-30 23:03:27 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 5.735868991s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.378069727s FAIL (2025-09-30 23:02:51 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.378108329s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.170660495s FAIL (2025-09-30 23:01:44 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.170699649s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.4904028s FAIL (2025-09-30 23:00:34 UTC) orbiter warning: Post "https://auth.us-select-1.storjshare.io/v1/access": net/http: TLS handshake timeout Duration: 6.490430232s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)