Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.814873987s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:55:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.814898263s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.301011578s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:55:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.301039752s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.716917443s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:55:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.716938065s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.322737569s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:54:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.322757719s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.765606322s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:54:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.765632082s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.36083488s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:54:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.360896979s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.653581239s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:53:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.653607901s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 5.778327334s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:53:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.778342915s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.238630213s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:53:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.238644541s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.274633188s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:52:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.274655801s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 5.78613466s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:52:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.786165018s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.366614114s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:52:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.366645786s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.67658143s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:51:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.676607119s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.456490624s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:51:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.456517848s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.809531603s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:50:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.809560139s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.427133986s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:50:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.427158503s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.823487037s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:49:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.823510452s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 5.740632301s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:49:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.740667529s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.338016684s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:49:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.338043576s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 5.671304556s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:48:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.67132208s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.312721974s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:48:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.312751161s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 5.812583574s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:48:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.812606369s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.330293981s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:47:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.330323137s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 5.720227123s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:46:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.720257272s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 5.764893107s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:46:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.76490992s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.336099038s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:46:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.336126261s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.356670115s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:45:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.356715824s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.711545974s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:45:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.711576683s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.685377172s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:45:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.68540248s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.241650127s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:44:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.241675206s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.663867236s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:44:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.663889319s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.763381023s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:43:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.763406973s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.311587805s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:43:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.311611871s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.354292631s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:43:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.354317769s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.823863879s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:43:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.823892222s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.748152841s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:42:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.748180333s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.299455998s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:42:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.299477568s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.702489012s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:42:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.702516604s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.674104973s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:41:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.674134942s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.350500222s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:41:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.350523396s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.746256318s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:40:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.746290415s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.40851721s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:40:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.408545245s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.863088348s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:39:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.863114979s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.307261753s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:39:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.30730169s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.733029784s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:38:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.733051526s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.418980024s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:38:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.419011555s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.382593507s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:37:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.382617151s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 5.798807021s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:37:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.798830656s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 5.714066293s trace: https://jaeger.dev.storj.io/trace/5f23406c8de425aa FAIL (2026-04-05 10:37:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 5) ======LOGS====== 28:41.758966513 (719h23m24.671304144s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759267184 (719h23m24.671604815s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.759550080 (719h23m24.671887711s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.760717443 (719h23m24.673055074s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.761764013 (719h23m24.674101644s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.762960341 (719h23m24.675297972s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.763418997 (719h23m24.675756638s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.764429707 (719h23m24.676767348s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.765511835 (719h23m24.677849467s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.766581169 (719h23m24.6789188s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:41.767045916 (719h23m24.679383548s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Uploading piece... Apr 5 10:28:43.526912859 (719h23m26.43925049s) [upload=690375]: Next returned nil segment Apr 5 10:28:43.526933919 (719h23m26.43927156s) [upload=690375]: Waiting for error group managing segments... Apr 5 10:28:43.865844872 (719h23m26.778182503s) [noise=true , address=77.90.31.14:4662 , piece_id=C35343F6RHNQMFM6GNZSE4MPEHNWFJV4SDTAWSPLEG3AXQMCXQMA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.871275142 (719h23m26.783612773s) [noise=true , address=77.90.31.21:4878 , piece_id=VLLQV5I4SPXGDW4Z6RA2VQ77YGWR4YN6GX5SDYXFSOKU7LUPDJFQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.873556334 (719h23m26.785893965s) [noise=true , address=77.90.31.19:4842 , piece_id=VXTVHHUTRZGUOIKTCAFLWMH6GXJLCEHGJDYB2J45GBLUBDOSMZLA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.874842286 (719h23m26.787179928s) [noise=true , address=77.90.31.10:4521 , piece_id=DP6THRI5UQQBXRRJKVJWVN3VIXCVLQHJ3R44SQFV5BK57GHGHENA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:28:43.877507860 (719h23m26.789845492s) [noise=true , address=45.91.116.8:4048 , piece_id=3ZSQVGEN7FBLJY77QYFGMQXEU5346C5CI5AMDASKALATC3ZZ2MDQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: <nil> Apr 5 10:29:10.872997255 (719h23m53.785334896s) [noise=true , address=145.79.227.34:4910 , piece_id=IK33TSOAXL4I4YUDAXS6FUTQNX5QXXH65E4LYZU3UJHOP34VXTSQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59300->145.79.227.34:4910: use of closed network connection Apr 5 10:29:10.873130363 (719h23m53.785468004s) [noise=true , address=145.79.227.52:4446 , piece_id=77Z7QQBRTHLP57D5EJVR5HBTC5SZ4P3T47JDZLJADVJ6OV2WSH2A, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1RNJPWzogZpsQfuFzJYZiz8bbHSRSrgSE5FLyhCynSwEDfMid7, address:145.79.227.52:4446): protocol: read tcp 188.166.225.170:43724->145.79.227.52:4446: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873193304 (719h23m53.785530935s) [noise=true , address=145.79.227.5:4227 , piece_id=H4WDLSYCJWA232GA77TDL5JT5XOPPP7NCX5HTMPVPW4YXWR64BQQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12nwrhLqAKoDy7KkoPsgCAy44m7T1JcEqbgatNYXThZ7sGc84vA, address:145.79.227.5:4227): protocol: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34256->145.79.227.5:4227: use of closed network connection Apr 5 10:29:10.873269541 (719h23m53.785607172s) [noise=true , address=145.79.227.38:4980 , piece_id=XSVBJZV5AKEF3M62LGISDLBRR6JPXMFW7DYGID46YAWSL74EIBGA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1hzDqFsTcWDF5LLW9tm77hLADjUkhbLqkE3DBSHyJf3PyiHszu, address:145.79.227.38:4980): protocol: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33856->145.79.227.38:4980: use of closed network connection Apr 5 10:29:10.873376307 (719h23m53.785713938s) [noise=true , address=145.79.227.15:4873 , piece_id=FD6KPQYJUTWW5CJ5LM7VAVPAGIMN2JGR2CRPFHVG4TSYHRLSSWHA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:60226->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873463886 (719h23m53.785801517s) [noise=true , address=145.79.227.49:4338 , piece_id=DJ4ZNILUUDLVUL4UCJGGEMSSLQJ2WH2F3QKLTTTW6EW7PPWNNHRA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:37050->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873545043 (719h23m53.785882674s) [noise=true , address=145.79.227.26:4655 , piece_id=GZQF772HVQPBHPGENON5UQOGLVZWI42DP4G3GZ6R2UIWRBEFSKUA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1xpZx5VbgHmbepaS9ZXgHfp7SZGeYPEjuJddQeYPUVpDU4EGx5, address:145.79.227.26:4655): protocol: read tcp 188.166.225.170:43040->145.79.227.26:4655: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.873620678 (719h23m53.785958309s) [noise=true , address=145.79.227.45:4232 , piece_id=N6YQQUNR3HC73EQVT76P3PKCMMBDNRZ6TACULESTLHB7YUVNSQPQ, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1knYU6KSZJCmdpXTRHXZmCsRJYfNGGQCLPEPkWkBgUdyvtSgJe, address:145.79.227.45:4232): protocol: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56608->145.79.227.45:4232: use of closed network connection Apr 5 10:29:10.873692026 (719h23m53.786029657s) [noise=true , address=145.79.227.31:4801 , piece_id=WUBRZAEM4XGQCILZCTM33JHNGWHVR5INQ3W6UI4M466XKC6VUINA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12pdMen2HsTBypaNt4cUu12TWjhbKnk3V287JJus1273x28xDXM, address:145.79.227.31:4801): protocol: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56190->145.79.227.31:4801: use of closed network connection Apr 5 10:29:10.873782761 (719h23m53.786120392s) [noise=true , address=145.79.227.28:4730 , piece_id=BQ5BKEMCOG45UCUEWFG4INBSDJKRQQMS2M6CFSNKDKIIJVEMNP6Q, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:1SXQt5FugmWj1YdhsvaaR7bDoxTBkqHKZpyXSmQqGdX5ZRhKxV, address:145.79.227.28:4730): protocol: read tcp 188.166.225.170:46524->145.79.227.28:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874019218 (719h23m53.786356849s) [noise=true , address=145.79.227.10:4404 , piece_id=PI3OI5Q5BRFD65SAV3OH3B7EERSN46OFSXLAV6GZX55HYXYPBOZA, seg_pos={0 0}, upload=690375]: Done uploading piece. err: ecclient: upload failed (node:12u1v9cAH2841Zq5JieXcPKWHYuEkg18S89ykFQuhtYBmvAPS7b, address:145.79.227.10:4404): protocol: read tcp 188.166.225.170:38076->145.79.227.10:4404: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:10.874041300 (719h23m53.786378931s) [seg_pos={0 0}, upload=690375]: Done waiting for segment. successful: 5 optimal: 13 errs: <nil> Apr 5 10:29:11.101342434 (719h23m54.013680075s) [upload=690375]: Done uploading segments... Apr 5 10:29:11.101383785 (719h23m54.013721416s) [upload=690375]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 5) Duration: 5.714097423s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 5.363136638s trace: https://jaeger.dev.storj.io/trace/4c7df959bc369e5d FAIL (2026-04-05 10:36:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 673s) [seg_pos={0 0}, upload=690369]: Done begin upload segment. Apr 5 10:28:08.593935972 (719h22m51.506273603s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.593949238 (719h22m51.506286879s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594385870 (719h22m51.506723512s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594397313 (719h22m51.506734954s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594688734 (719h22m51.507026366s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594730776 (719h22m51.507068407s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.594956221 (719h22m51.507293852s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595237504 (719h22m51.507575135s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595313561 (719h22m51.507651192s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595551561 (719h22m51.507889202s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.595617788 (719h22m51.507955419s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596433032 (719h22m51.508770663s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.596878090 (719h22m51.509215731s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.597725636 (719h22m51.510063267s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.598574223 (719h22m51.510911854s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599313451 (719h22m51.511651092s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.599679778 (719h22m51.512017409s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:08.600374478 (719h22m51.51271211s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Uploading piece... Apr 5 10:28:09.940724342 (719h22m52.853061983s) [upload=690369]: Next returned nil segment Apr 5 10:28:09.940780801 (719h22m52.853118432s) [upload=690369]: Waiting for error group managing segments... Apr 5 10:28:10.334846027 (719h22m53.247183658s) [noise=true , address=167.17.72.71:4794 , piece_id=A22N4MLDEBLAXB7E457NJWS7EMTYMCSOLEVIQT6OHLOTUZBOBWRA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.388987055 (719h22m53.301324686s) [noise=true , address=167.17.72.69:4734 , piece_id=HGE5ZTIFCUZZMRQ52GJMGD5RETX2Y7EBGFXKOHPTREXHA6K4ZQJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.392271775 (719h22m53.304609406s) [noise=true , address=193.149.164.68:4043 , piece_id=3BMTXRTKHDMAJID56LW355ZAW22KGRO5FE3WQY7SR7DBD6QGQHOQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.410788310 (719h22m53.323125951s) [noise=true , address=193.149.164.143:4703 , piece_id=XXBAVBUSNRAFEUQX5EAHNTC43QXYONCRVFTKZDVJ7JZFCQWNTYXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.422920936 (719h22m53.335258567s) [noise=true , address=193.149.164.69:4096 , piece_id=YECLPL5D3DMFSRM33DPUITK25GWIL3YBGZ7HTFPPWUU2OFN67P6Q, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.438753100 (719h22m53.351090731s) [noise=true , address=193.149.164.66:4991 , piece_id=XYBDSWIAPPLIB4JV4MGOHZHBOXT4BSZNULH3XJECM2XB5B3LFLJQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.466100228 (719h22m53.378437859s) [noise=true , address=193.149.164.71:4291 , piece_id=CUQVGDAAIGHVEGEWISAP5DBXPR6OFY2EUABMYNXGI4HSGG56UVPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.480587597 (719h22m53.392925239s) [noise=true , address=45.91.116.12:4238 , piece_id=HY5ZCY7GBKAKF2R4J275H76WAQT4PX2LEOJAPGJD3G7N5DM4G7FA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.528165128 (719h22m53.440502769s) [noise=true , address=193.149.164.67:4997 , piece_id=YCASEB63XDUV2HHGRP35C5S5QTRZ33IBJNBE3TJHO3KZ6BBBRMYQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.559681773 (719h22m53.472019415s) [noise=true , address=77.90.31.12:4610 , piece_id=AFZB2DBTL64Q5ZSS3D44MMDUGHNR3QLIRLYRU6BWEAN7O2M6TGPQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.564440517 (719h22m53.476778148s) [noise=true , address=77.90.31.18:4817 , piece_id=JL7G2CQJNJFQ5VYLBJDHY4TED5NCD7RTBZROBBFUOM6HE2HVQTZQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:10.568254798 (719h22m53.480592439s) [noise=true , address=77.90.31.17:4781 , piece_id=Q55ALJQVVAVU6TFUINQURPPFQN6VDC67ZHCJKLFRRRLB6MIBFR5A, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:28:25.855767800 (719h23m8.768105421s) [noise=true , address=145.79.227.13:4836 , piece_id=QOCCR3OEZ34RQ74U6FBAGOGAB6XJAMGWIAFSBKBMYSIGJQ33UFDQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: <nil> Apr 5 10:29:07.700074174 (719h23m50.612411805s) [noise=true , address=145.79.227.34:4926 , piece_id=R53DLWHX6HDFC76CWLWVNCXC2GNJFKWIV3Z4SSZGFZJR3BR7VDNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1LkGN9UdiF163eg78WUPpXqz4FxHBxhnPin4okcdU1YPDWx4MF, address:145.79.227.34:4926): protocol: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54598->145.79.227.34:4926: use of closed network connection Apr 5 10:29:07.700436523 (719h23m50.612774154s) [noise=true , address=64.49.13.169:4706 , piece_id=L5JPIJQOS2B7HOT5E777O4P3VSYBF6KF6FOMT2AJ5VU32W4GLGXQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1xwktmeUvkUf2wfqQTazBsknPs2aC7sj1GLqR875qcW4Cj2GD7, address:64.49.13.169:4706): protocol: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56176->64.49.13.169:4706: use of closed network connection Apr 5 10:29:07.700604889 (719h23m50.61294253s) [noise=true , address=145.79.227.19:4087 , piece_id=TTCMPBSPQODYJ3UL4GSWOD6E2LNVTIAXIV5S7XGDYOGJYGB2SDLA, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1S9k5eejJcxkiG5pXjjBZm18uP8k5jCGfWyeW2QQxmuRU8ZzRA, address:145.79.227.19:4087): protocol: expected piece hash; context deadline exceeded; EOF Apr 5 10:29:07.701022564 (719h23m50.613360205s) [noise=true , address=145.79.227.11:4736 , piece_id=NIPFCM35EA2IE7MLHGBVTH6V2ZA2U4APT7PZFNBO2VDPBFGNUBNQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:1ejnWsitdrh68vSJvjR2QnRg4kva8DFRzqvvUuDkXM6zKUDWNC, address:145.79.227.11:4736): protocol: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51376->145.79.227.11:4736: use of closed network connection Apr 5 10:29:07.701260553 (719h23m50.613598194s) [noise=true , address=64.49.13.139:4480 , piece_id=2AUZAY45DUAWIGAQWSNX6XKDSHQWHFQNEPJZSI4KEBWBGI5GFPIQ, seg_pos={0 0}, upload=690369]: Done uploading piece. err: ecclient: upload failed (node:12qN4pwQ12oQxGpHgDuoFuutMUQMuLnic9syqZFsE3SKr6ZoEfT, address:64.49.13.139:4480): protocol: read tcp 188.166.225.170:50388->64.49.13.139:4480: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Apr 5 10:29:07.701290692 (719h23m50.613628323s) [seg_pos={0 0}, upload=690369]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Apr 5 10:29:07.940175502 (719h23m50.852513133s) [upload=690369]: Done uploading segments... Apr 5 10:29:07.940208335 (719h23m50.852545966s) [upload=690369]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.363156906s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)