Gemma 4 E2B-it GGUF β€” Quantized by BatiAI

BatiFlow Ollama Upstream

Optimized GGUF quantizations of google/gemma-4-E2B-it β€” Google's tiniest fully-multimodal Gemma 4 (text + image + audio) for on-device AI on Mac. Built directly from official Google BF16 weights by BatiAI for BatiFlow.

The "E" stands for Edge. Tiny enough to run on basically any Mac (8 GB+) while still understanding speech and images alongside text β€” the only Gemma 4 size with audio support shared with E4B.

Quick Start

# Recommended
ollama pull batiai/gemma4-e2b:q4

# Higher quality
ollama pull batiai/gemma4-e2b:q6

Available Quantizations

Tag Quant Size Recommended For
:q4 Q4_K_M ~2.1 GB balanced (recommended default)
:q6 Q6_K ~2.8 GB higher quality, near-lossless

Sub-3 B models stay sharp at Q4 β€” quantization noise is tiny relative to model capacity.

Two modes β€” text-only by default, multimodal opt-in

Upstream Gemma 4 E2B-it is fully multimodal β€” text + image + audio in one model. In the GGUF ecosystem this is delivered as two files: a main model.gguf (text tower) and a separate mmproj.gguf that holds both vision and audio encoders together (a single 1411-tensor projector covering image and speech input).

Text-only (default) Multimodal (opt-in)
Files needed main GGUF only main GGUF + mmproj-BF16.gguf
Capabilities Q&A, coding, tool calling, agents + image (OCR, captioning, visual reasoning) + audio (speech understanding)
ollama pull βœ… single command ⚠ Ollama mmproj integration is still rough β€” use llama.cpp directly
Disk / RAM smaller (no projector weights) larger (+ ~940 MB)
Recommended for most users (chat, code, agents) OCR, image / speech understanding

Multimodal usage (llama.cpp)

# Pick a main model (text tower)
wget https://huggingface.co/batiai/Gemma-4-E2B-it-GGUF/resolve/main/google-gemma-4-E2B-it-Q4_K_M.gguf

# Get the multimodal projector (vision + audio in one file)
wget https://huggingface.co/batiai/Gemma-4-E2B-it-GGUF/resolve/main/mmproj-BF16.gguf

Server mode (image input via OpenAI-compatible Vision API):

llama-server \
  -m google-gemma-4-E2B-it-Q4_K_M.gguf \
  --mmproj mmproj-BF16.gguf \
  -c 32768 --host 127.0.0.1 --port 8080

curl http://127.0.0.1:8080/v1/chat/completions -d '{
  "messages": [{
    "role": "user",
    "content": [
      {"type": "image_url", "image_url": {"url": "data:image/jpeg;base64,..."}},
      {"type": "text", "text": "What does this show?"}
    ]
  }]
}'

One-shot CLI with image:

llama-mtmd-cli \
  -m google-gemma-4-E2B-it-Q4_K_M.gguf \
  --mmproj mmproj-BF16.gguf \
  --image ~/Desktop/photo.jpg \
  -p "describe this image"

One-shot CLI with audio:

llama-mtmd-cli \
  -m google-gemma-4-E2B-it-Q4_K_M.gguf \
  --mmproj mmproj-BF16.gguf \
  --audio ~/Downloads/voice.wav \
  -p "transcribe and summarize"

mmproj quantization

File Quant Size Why this only?
mmproj-BF16.gguf BF16 ~940 MB The vision + audio projector tensors don't satisfy K-quant block-alignment requirements (108+ tensors require fallback), so Q6_K reliably aborts on this combined projector. BF16 is the only safe choice today β€” applies to every quantizer of this model. The main text GGUF is unaffected.

Why E series (E2B / E4B) vs 26B / 31B?

E2B / E4B 26B-A4B / 31B
Audio support βœ… ❌ (vision only)
Min RAM 8 GB+ 24 GB+
Speed very fast (small) slower (larger)
Reasoning depth lower higher
Use case edge / mobile-class Mac / voice + image desktop chat + agents

If you need voice + image + text in one model on a small Mac, E series is the only Gemma 4 option.

Why BatiAI?

BatiAI Third-party (unsloth, etc.)
Source Quantized directly from official Google weights Often re-quantized from other GGUFs
Compatibility βœ… Verified on Ollama 0.20+ ❌ Known issues with Ollama 0.20+
Tested on Real Mac mini M4 (16 GB) + MacBook Pro M4 Max Often untested
Tool calling βœ… Verified with BatiFlow's 57 tool functions Often broken
Korean βœ… Validated Not tested
Multimodal βœ… Vision + audio mmproj available Often missing
Signing general.author: BatiAI for provenance Varies

About BatiFlow

flow.bati.ai

BatiFlow is a macOS-native AI desktop automation app β€” just 5 MB, built with Swift.

  • Free & Unlimited β€” On-device AI via Ollama, no API costs
  • 100 % Private β€” All data stays on your Mac
  • Ultra Lightweight β€” Native macOS app, only 5 MB
  • 57 built-in tools β€” calendar, notes, reminders, files, email, browser, messaging, and more

Related models in the BatiAI Gemma 4 lineup

Model Modalities Min RAM Repo
Gemma 4 E2B-it text + image + audio 8 GB this repo
Gemma 4 E4B-it text + image + audio 8 GB batiai/Gemma-4-E4B-it-GGUF
Gemma 4 26B-A4B-it text + image / video (no audio) 24 GB batiai/Gemma-4-26B-A4B-it-GGUF
Gemma 4 31B-it text + image / video (no audio) 24 GB batiai/Gemma-4-31B-it-GGUF

Technical Details

  • Original Model: google/gemma-4-E2B-it
  • Architecture: Gemma 4 (Edge variant)
  • Modalities: Text (primary) + Image + Audio (via opt-in mmproj)
  • License: Gemma (commercial use permitted under terms)
  • Quantized with: llama.cpp
  • Quantized by: BatiAI
  • GGUF metadata: general.author: BatiAI, general.url: https://flow.bati.ai

License

Mirrors the upstream Gemma license. Commercial use permitted per Google's Gemma terms.

BatiAI's quantization pipeline is provided under MIT.

Downloads last month
391
GGUF
Model size
5B params
Architecture
gemma4
Hardware compatibility
Log In to add your hardware

4-bit

6-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for batiai/gemma-4-E2B-it-GGUF

Quantized
(474)
this model

Collections including batiai/gemma-4-E2B-it-GGUF