Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 6.038966202s FAIL (2025-10-14 22:45:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.038997091s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 6.0373473s FAIL (2025-10-14 22:44:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.03738941s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.824895705s FAIL (2025-10-14 22:44:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.824926103s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.691309473s FAIL (2025-10-14 22:44:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.691340393s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.874433012s FAIL (2025-10-14 22:43:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.874464113s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.73530531s FAIL (2025-10-14 22:42:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.735329997s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 6.127870943s FAIL (2025-10-14 22:41:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.12790135s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.18471837s FAIL (2025-10-14 22:41:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.18474956s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.012481131s FAIL (2025-10-14 22:40:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.012531946s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.991880831s FAIL (2025-10-14 22:39:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.991914335s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.215412576s FAIL (2025-10-14 22:38:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.215455889s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.045841755s FAIL (2025-10-14 22:37:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.045869587s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.883473809s FAIL (2025-10-14 22:36:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.883507142s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.20381356s FAIL (2025-10-14 22:35:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.203867573s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.00592097s FAIL (2025-10-14 22:35:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.006309159s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.98704102s FAIL (2025-10-14 22:34:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.987071217s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.008476417s FAIL (2025-10-14 22:34:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.008510612s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.115727909s FAIL (2025-10-14 22:33:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.115759681s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.886657113s FAIL (2025-10-14 22:32:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.886688743s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.925115635s FAIL (2025-10-14 22:31:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.925152016s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 6.205169733s FAIL (2025-10-14 22:30:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.205199689s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.987424321s FAIL (2025-10-14 22:30:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.987456893s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.094529776s FAIL (2025-10-14 22:29:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.094561607s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.989582217s FAIL (2025-10-14 22:28:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.989617725s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 6.158660368s FAIL (2025-10-14 22:27:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.158685485s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 6.095804873s FAIL (2025-10-14 22:27:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.095838157s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.960274533s FAIL (2025-10-14 22:26:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.960309981s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 6.130053807s FAIL (2025-10-14 22:25:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.130093163s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.839583057s FAIL (2025-10-14 22:24:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.839619696s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.019582114s FAIL (2025-10-14 22:23:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.019613404s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 6.171015032s FAIL (2025-10-14 22:22:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.171053606s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 5.873737752s FAIL (2025-10-14 22:22:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.873768521s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 6.065859546s FAIL (2025-10-14 22:22:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.065891987s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 5.976610468s FAIL (2025-10-14 22:21:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.976634183s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.092785802s FAIL (2025-10-14 22:20:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.092828333s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.064467734s FAIL (2025-10-14 22:19:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.06450214s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.921861337s FAIL (2025-10-14 22:18:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.921886295s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.896166297s FAIL (2025-10-14 22:17:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.896195703s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.994680014s FAIL (2025-10-14 22:16:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.994774976s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.083656784s FAIL (2025-10-14 22:15:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.083691409s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 6.00300668s FAIL (2025-10-14 22:14:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.003039261s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.246440273s FAIL (2025-10-14 22:14:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 14 22:13:57.988619723 (512h42m43.450712206s) [upload=451062]: Starting upload Oct 14 22:13:57.988711849 (512h42m43.450804332s) [upload=451062]: Done starting upload Oct 14 22:13:57.988853849 (512h42m43.450946333s) [upload=451062]: Uploading segments... Oct 14 22:13:57.989058419 (512h42m43.451150892s) [upload=451062]: Got next segment. Inline: false Oct 14 22:13:57.989120538 (512h42m43.451213011s) [upload=451062]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 14 22:13:57.989201061 (512h42m43.451293534s) [upload=451062]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 14 22:13:58.272732991 (512h42m43.734825464s) [seg_pos={0 0}, upload=451062]: Begin upload segment... Oct 14 22:13:58.272814536 (512h42m43.734907009s) [seg_pos={0 0}, upload=451062]: Done begin upload segment. Oct 14 22:13:58.272850224 (512h42m43.734942697s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.273571708 (512h42m43.735664181s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.275992559 (512h42m43.738085042s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276573133 (512h42m43.738665616s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.276929172 (512h42m43.739021655s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277457849 (512h42m43.739550322s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.277790722 (512h42m43.739883205s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278144296 (512h42m43.740236779s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278491095 (512h42m43.740583579s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.278821926 (512h42m43.740914409s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279220555 (512h42m43.741313028s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279526898 (512h42m43.741619371s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.279568958 (512h42m43.741661451s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280130258 (512h42m43.742222741s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280566347 (512h42m43.742658821s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:13:58.280928838 (512h42m43.743021321s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Uploading piece... Oct 14 22:14:00.072722214 (512h42m45.534814697s) [upload=451062]: Next returned nil segment Oct 14 22:14:00.072744697 (512h42m45.53483717s) [upload=451062]: Waiting for error group managing segments... Oct 14 22:14:00.380655749 (512h42m45.842748232s) [noise=true , address=77.90.31.14:4663 , piece_id=B4NX76RMRGVAVAIH5RY3RGWWCJHO4BX5RVXRIDIRD7QDYUEIBK5A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.396435687 (512h42m45.85852816s) [noise=true , address=45.91.116.4:4939 , piece_id=2BEKHUGHMNOVFLNTXKLFQDTFHYXROMAEE2KON4ONHXQDNP4ODJAA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.424296270 (512h42m45.886388743s) [noise=true , address=77.90.31.12:4590 , piece_id=JIE4ONPDUWVGP5AOWEBYCW57WQWFWWCYR5CGDY6PGNVOL3RXCS2A, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.458690202 (512h42m45.920782675s) [noise=true , address=45.91.116.6:4974 , piece_id=PSNF7XAX3WFQV6AJ4HCNT2G7JPVRCAJGLP4DXP4OUAF3PT7UO3NQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.461377209 (512h42m45.923469682s) [noise=true , address=77.90.31.13:4627 , piece_id=T2PYDENRCOANUOQHNQVRQ4UNEFJOXXQQOPKX2J3I3UOL4ZBREAHA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:00.952188189 (512h42m46.414280673s) [noise=true , address=77.90.31.15:4697 , piece_id=FJUT37Y4KEX7I2UGIVKLZWWUH5TDWMV63BL3QZ2SBDL27QEG7ZOA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:01.765394303 (512h42m47.227486777s) [noise=true , address=45.91.116.11:4118 , piece_id=ATDNSRRFG7RPWY2OJDNCPBRFI56YZQRZSIDCU5PPTYEIM3UUFCIQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:04.626308705 (512h42m50.088401189s) [noise=true , address=145.79.227.3:4151 , piece_id=WYTLNOC3EGSXDLFNFQ7YJRBWQV7DJ4YGOPLLX7UBJIL4N2VN2O7Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:06.487123532 (512h42m51.949216005s) [noise=true , address=145.79.227.17:4972 , piece_id=RAAZEEILQHV6B5UJZIVAKTCZBBAJNPSFD34LES2ETMSTHEYC6EKA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.416143379 (512h42m54.878235862s) [noise=true , address=145.79.227.7:4295 , piece_id=6MZLVZQPI4MXRXCPROH7JE5MNS55M5YH5OL5XDJUF2HHAFBG2PXQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:09.523381782 (512h42m54.985474255s) [noise=true , address=145.79.227.13:4803 , piece_id=27MZUNX2AHIVW7JSSJOJLXU734F7ZLVR2VNPNMHZ2JCR7EN7AYVQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:14.099135616 (512h42m59.561228099s) [noise=true , address=145.79.227.4:4186 , piece_id=L7WXJ2LQHZXY4HRZVULPWZ5TJ5HZXVMV2ODNEBVMLQ5R6GJBTYUQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: <nil> Oct 14 22:14:27.281716795 (512h43m12.743809268s) [noise=true , address=145.79.227.15:4874 , piece_id=33SFW24HLKF2UOK5UPL2FSK2EZUF4LLYUWEPTHKLM343NUAESWTA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55870->145.79.227.15:4874: use of closed network connection Oct 14 22:14:27.281887540 (512h43m12.743980013s) [noise=true , address=145.79.227.5:4222 , piece_id=BHICZSI3JNVVL5PPDBXBGOQK4FJLCHZSJXIYMT3MUDHAFW5PL5DA, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46658->145.79.227.5:4222: use of closed network connection Oct 14 22:14:27.281937927 (512h43m12.74403041s) [noise=true , address=145.79.227.19:4078 , piece_id=TGRA6OVQNCL4PGHOTIXZTTGSK4ILVBKCC3BRJLXFZHPVKRVXMH6Q, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, address:145.79.227.19:4078): protocol: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:39328->145.79.227.19:4078: use of closed network connection Oct 14 22:14:27.281996588 (512h43m12.744089061s) [noise=true , address=145.79.227.2:4113 , piece_id=QSQMTTAFSPO7YMIRB7S42L6OXQQFIQMUCQBLOH5SRRJPFGMHKYBQ, seg_pos={0 0}, upload=451062]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34626->145.79.227.2:4113: use of closed network connection Oct 14 22:14:27.282027757 (512h43m12.74412024s) [seg_pos={0 0}, upload=451062]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 14 22:14:27.527051330 (512h43m12.989143813s) [upload=451062]: Done uploading segments... Oct 14 22:14:27.527077860 (512h43m12.989170343s) [upload=451062]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.246463278s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 2.250453578s FAIL (2025-10-14 18:21:41 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 2.250481941s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 2.063545213s FAIL (2025-10-14 18:20:46 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 2.063569789s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 2.14367886s FAIL (2025-10-14 18:20:22 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 2.143715581s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 2.020642093s FAIL (2025-10-14 18:19:38 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 2.020665998s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 2.085758549s FAIL (2025-10-14 18:18:36 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 2.085793886s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 2.088576764s FAIL (2025-10-14 18:18:12 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 2.088634133s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 2.035094756s FAIL (2025-10-14 18:17:43 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 2.03513448s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 2.130263338s FAIL (2025-10-14 18:17:00 UTC) orbiter warning: operation: ListObjects uplink: metainfo: metaclient: context deadline exceeded Duration: 2.130297042s core uplink us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)