Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.206388289s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 11:10:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.206416833s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.776823665s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 11:09:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.776849496s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.483121002s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 11:09:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.483185796s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.847459008s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 11:09:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.847487823s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.312857175s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 11:08:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.312887935s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 5.943891864s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 11:08:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.943919097s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.216885338s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 11:07:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.216910616s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.49175823s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 11:06:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.49178393s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 5.896014654s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 11:06:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.896040664s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.774454355s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 11:06:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.774481317s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.296254508s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 11:06:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 6.296282802s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.866838165s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 11:05:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.866864506s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.324469487s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 11:04:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.324493564s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.815448247s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 11:04:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.815514144s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 5.489188844s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 11:03:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.489217782s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 5.880385037s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 11:03:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.880409184s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 7.648987681s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 11:03:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 7.649016246s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.866942031s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 11:02:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.866967912s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.286239836s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 11:02:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.286263082s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.482331857s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 11:02:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.482361633s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.509317267s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 11:01:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.509344198s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.796021498s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 11:01:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.796048741s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.331258771s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 11:01:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.331283188s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.854156574s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 11:00:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.854172556s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.319762375s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 11:00:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.319793976s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 6.06780841s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 11:00:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 6.067845521s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.372463545s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:59:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.372480087s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.80808383s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:58:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.808143124s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.327684288s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:58:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.327708544s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.709356904s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:57:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.709389618s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.320008606s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:57:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.320032293s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.828162552s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:57:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.828190214s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.401335277s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:57:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.401363652s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.377698626s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:56:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.377715269s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.805093235s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:56:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.805120167s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.349718767s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:56:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.349783863s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.472754999s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:55:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.472780499s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.814873987s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:55:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.814898263s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.301011578s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:55:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.301039752s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.716917443s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:55:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.716938065s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.322737569s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:54:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.322757719s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.765606322s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:54:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.765632082s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.36083488s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:54:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.360896979s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.653581239s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:53:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.653607901s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.778327334s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:53:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.778342915s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.238630213s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:53:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.238644541s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.274633188s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:52:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.274655801s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.78613466s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:52:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.786165018s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.366614114s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:52:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.366645786s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.67658143s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:51:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.676607119s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)