{"id":2725,"date":"2026-06-06T03:34:23","date_gmt":"2026-06-06T03:34:23","guid":{"rendered":"https:\/\/tucumandevelopers.com\/index.php\/2026\/06\/06\/run-gemma-4-12b-on-wsl2-with-llama-cpp\/"},"modified":"2026-06-06T03:34:23","modified_gmt":"2026-06-06T03:34:23","slug":"run-gemma-4-12b-on-wsl2-with-llama-cpp","status":"publish","type":"post","link":"https:\/\/tucumandevelopers.com\/index.php\/2026\/06\/06\/run-gemma-4-12b-on-wsl2-with-llama-cpp\/","title":{"rendered":"Run Gemma-4 12B on WSL2 with llama.cpp"},"content":{"rendered":"
\n
<\/header>\n
\n

<\/a> 1. update WSL environment <\/h2>\n
\n
sudo <\/span>apt update &&<\/span> sudo <\/span>apt upgrade -y<\/span> <\/code><\/pre>\n
\n<\/p><\/div>\n<\/p><\/div>\n
  <\/a> 2. install dependencies <\/h2>\n
If you don’t use -hf<\/code> option, you don’t need to install libssl-dev in this step. <\/p>\n
\n
sudo <\/span>apt install <\/span>build-essential cmake git libssl-dev -y<\/span> <\/code><\/pre>\n
\n<\/p><\/div>\n<\/p><\/div>\n
If nvidia-smi<\/code> shows a GPU\/GPUs on your terminal, you will need to install the tooklit. This will take some time. <\/p>\n
\n
sudo <\/span>apt install <\/span>nvidia-cuda-toolkit -y<\/span> <\/code><\/pre>\n
\n<\/p><\/div>\n<\/p><\/div>\n
  <\/a> 3. clone the repo <\/h2>\n
Build llama-cli and llama-server. This step also will take some time.
 If you don’t plan to use -hf<\/code> option, you don’t need to use -DLLAMA_OPENSSL=ON<\/code>. <\/p>\n
\n
git clone https:\/\/github.com\/ggerganov\/llama.cpp cd <\/span>llama.cpp cmake -B<\/span> build -DGGML_CUDA<\/span>=<\/span>ON -DLLAMA_OPENSSL<\/span>=<\/span>ON cmake --build<\/span> build --config<\/span> Release # no GPU<\/span> git clone https:\/\/github.com\/ggerganov\/llama.cpp cd <\/span>llama.cpp cmake -B<\/span> build cmake --build<\/span> build --config<\/span> Release <\/code><\/pre>\n
\n<\/p><\/div>\n<\/p><\/div>\n
  <\/a> 4. run the model <\/h2>\n
Run gemma-4-12b-it<\/code> with cli and server.<\/p>\n
\n
 <\/a> <\/p>\n
\n
  unsloth\/gemma-4-12b-it-GGUF \u00b7 Hugging Face <\/a> <\/h2>\n
 We\u2019re on a journey to advance and democratize artificial intelligence through open source and open science. <\/p>\n
 huggingface.co <\/p>\n<\/p><\/div>\n<\/p><\/div>\n
\n
.\/build\/bin\/llama-cli -hf<\/span> unsloth\/gemma-4-12b-it-GGUF:UD-Q4_K_XL <\/code><\/pre>\n
\n<\/p><\/div>\n<\/p><\/div>\n
\n
><\/span> hello [<\/span>Start thinking] The user said \"hello\"<\/span>.<\/span> The user is initiating a conversation. Respond politely and offer assistance. *<\/span> \"Hello! How can I help you today?\"<\/span> *<\/span> \"Hi there! What's on your mind?\"<\/span> *<\/span> \"Hello! Is there anything I can assist you with?\"<\/span> [<\/span>End thinking] Hello! How can I help <\/span>you today? [<\/span> Prompt: 19.5 t\/s | Generation: 11.8 t\/s ]<\/span> <\/code><\/pre>\n
\n<\/p><\/div>\n<\/p><\/div>\n
or run web-ui<\/code> <\/p>\n
\n
.\/build\/bin\/llama-server -hf<\/span> unsloth\/gemma-4-12b-it-GGUF:UD-Q4_K_XL --port<\/span> 8080 <\/code><\/pre>\n
\n<\/p><\/div>\n<\/p><\/div>\n
  <\/a> optional download model from huggingface <\/h3>\n
\n
mkdir<\/span> -p<\/span> models wget -O<\/span> models\/gemma-4-12b-it-UD-Q4_K_XL.gguf https:\/\/huggingface.co\/unsloth\/gemma-4-12b-it-GGUF\/resolve\/main\/gemma-4-12b-it-UD-Q4_K_XL.gguf <\/code><\/pre>\n
\n<\/p><\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/article>\n
  <\/div>\n
 <\/main> <\/div>\n<\/div>\n<\/div>\n<\/div>\n
Fuente: Art\u00edculo original<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"
1. update WSL environment sudo apt update && sudo apt upgrade -y 2. install dependencies If you don’t use -hf option, you don’t need to install libssl-dev in this step. sudo apt install build-essential cmake git libssl-dev -y If nvidia-smi shows a GPU\/GPUs on your terminal, you will need to install the tooklit. This will […]<\/p>\n","protected":false},"author":1,"featured_media":2724,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","jetpack_publicize_message":"","jetpack_publicize_feature_enabled":true,"jetpack_social_post_already_shared":true,"jetpack_social_options":{"image_generator_settings":{"template":"highway","default_image_id":0,"font":"","enabled":false},"version":2}},"categories":[41],"tags":[],"class_list":["post-2725","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-devto"],"jetpack_publicize_connections":[],"_links":{"self":[{"href":"https:\/\/tucumandevelopers.com\/index.php\/wp-json\/wp\/v2\/posts\/2725","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/tucumandevelopers.com\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/tucumandevelopers.com\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/tucumandevelopers.com\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/tucumandevelopers.com\/index.php\/wp-json\/wp\/v2\/comments?post=2725"}],"version-history":[{"count":0,"href":"https:\/\/tucumandevelopers.com\/index.php\/wp-json\/wp\/v2\/posts\/2725\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/tucumandevelopers.com\/index.php\/wp-json\/wp\/v2\/media\/2724"}],"wp:attachment":[{"href":"https:\/\/tucumandevelopers.com\/index.php\/wp-json\/wp\/v2\/media?parent=2725"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/tucumandevelopers.com\/index.php\/wp-json\/wp\/v2\/categories?post=2725"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/tucumandevelopers.com\/index.php\/wp-json\/wp\/v2\/tags?post=2725"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}