prism-ml/Ternary-Bonsai-2-2B-mlx-2bit

Prism ML Website  |  Demo & Examples  |  Discord

Bonsai 2 Ternary 2B: MLX

A 2B-class vision-language model with ternary transformer weights, packed for MLX on Apple Silicon.

1.20 GB on disk, language model + vision tower (FP16 language model: ~3.8 GB)

Highlights

• Ternary language weights stored in MLX's 2-bit grouped format (group 128). MLX stores a scale and a bias per group, so the container costs a little more than the GGUF packings; the weights decode to exactly the same ternary values.
• Vision input through the official vision tower, carried unrotated in FP16.
• 262K-token context, inherited from the Qwen3.5-2B hybrid-attention backbone (18 linear-attention and 6 full-attention layers).

Model Overview

Item	Specification
Base model	Derived from Qwen3.5-2B (architecture unchanged)
Parameters	~1.88B language model (24 blocks, hidden 2048) + ~0.33B vision tower
Weight format	Ternary {−1, 0, +1} in MLX affine 2-bit, group 128
Weight basis	Blockwise Hadamard rotation folded into the stored weights; the bundled runtime applies the matching transform to activations
Backends	MLX on Apple Silicon (Metal): mlx-vlm 0.7.8+ natively, or the loader in runtime/
License	Apache 2.0

Files

File	Contents	Size
model.safetensors	language model (2-bit, group 128) + vision tower (FP16)	1.20 GB
runtime/	a bundled loader for older mlx-vlm versions, plus pinned requirements.txt
quickstart.py	one-command test with the recommended sampler
PACK-RUNTIME.md	the bundled loader's contract and limits

Quickstart

PrismML-Eng/Bonsai-demo is the source of truth for running these models. Where anything here disagrees with it, it is right.

hf download prism-ml/Ternary-Bonsai-2-2B-mlx-2bit --local-dir bonsai2-2b-mlx
pip install "mlx-vlm>=0.7.8"

mlx-vlm 0.7.8 and newer load this pack natively (model_type: prism_hadamard_qwen35):

from mlx_vlm import load, generate
from mlx_vlm.prompt_utils import apply_chat_template

model, processor = load("bonsai2-2b-mlx")
prompt = apply_chat_template(processor, model.config, "What is in this image?", num_images=1)
print(generate(model, processor, prompt, ["photo.jpg"], max_tokens=4096,
               temperature=1.0, top_p=0.95, top_k=20))

For an OpenAI-compatible server, run python -m mlx_vlm.server --model bonsai2-2b-mlx and send the sampling settings in each request.

The released mlx-lm (0.32.0) cannot load this pack yet; its support is merged and arrives with the next mlx-lm release. On older mlx-vlm versions, use the loader bundled in runtime/ (pip install -r bonsai2-2b-mlx/runtime/requirements.txt, then python bonsai2-2b-mlx/quickstart.py "your prompt"); PACK-RUNTIME.md documents it. quickstart.py is a smoke test: it answers with thinking off and stops at 256 tokens.

Pass no images for text-only use.

For CUDA, CPU, and llama.cpp on Metal, use the GGUF packs of the same weights: Ternary-Bonsai-2-2B-gguf.

Best Practices

Generation Parameters

We recommend these sampling settings (the same as the GGUF packs):

• Thinking mode (default): temperature=1.0, top_p=0.95, top_k=20, min_p=0.0, presence_penalty=1.0, repetition_penalty=1.0
• Non-thinking mode: temperature=0.7, top_p=0.80, top_k=20, min_p=0.0, presence_penalty=1.5, repetition_penalty=1.0

Pass the sampler explicitly. mlx-vlm's Python generate() decodes greedily unless you pass these arguments. mlx_vlm.server takes temperature, top_p and top_k from generation_config.json when a request omits them, but not presence penalty or min_p, so send the full set per request. Apply penalties over the last 64 tokens where your client supports a penalty window. Avoid greedy decoding with thinking on; it can get stuck reasoning on some prompts. To turn thinking off, render the chat template with enable_thinking=False.

Output Length

Give thinking mode room to finish: even simple prompts can use 3,000+ thinking tokens, so allow at least 8,192 output tokens with thinking on. For short answers, turn thinking off.

System Prompt

You can use a simple system prompt such as:

You are a helpful assistant

License

Apache 2.0, following the base model.