How to Run Gemma 4 Locally with Ollama, Llama.cpp, and vLLM
Google's Gemma 4 is the latest frontier-level open model optimized for local reasoning. Running it locally ensures maximum privacy and allows you to use its multimodal features without a subscription.
Running with Ollama
Ollama is the easiest way to get started. After installing Ollama, simply run 'ollama run gemma4:e4b' in your terminal.
Gemma 4 supports image and audio input, making it a versatile tool for local AI workflows.
Hardware Requirements
For the compact 4B variant, 16GB of RAM is recommended. If you want to run the larger 31B dense model, you will need 16GB+ of VRAM on an RTX 3090/4090 or a Mac Studio.
Frequently Asked Questions
Is Gemma 4 free?
Yes, Gemma 4 is an open-weights model provided by Google under a permissive license.
Can I run Gemma 4 on a CPU?
Yes, using quantized GGUF models in Llama.cpp, you can run the model on standard CPUs, though speed will be lower.