Instructions to use protoLabsAI/Ornith-1.0-9B-MTP-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use protoLabsAI/Ornith-1.0-9B-MTP-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf protoLabsAI/Ornith-1.0-9B-MTP-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf protoLabsAI/Ornith-1.0-9B-MTP-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf protoLabsAI/Ornith-1.0-9B-MTP-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf protoLabsAI/Ornith-1.0-9B-MTP-GGUF:Q4_K_M

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf protoLabsAI/Ornith-1.0-9B-MTP-GGUF:Q4_K_M
# Run inference directly in the terminal:
./llama-cli -hf protoLabsAI/Ornith-1.0-9B-MTP-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf protoLabsAI/Ornith-1.0-9B-MTP-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama-cli -hf protoLabsAI/Ornith-1.0-9B-MTP-GGUF:Q4_K_M

Use Docker

docker model run hf.co/protoLabsAI/Ornith-1.0-9B-MTP-GGUF:Q4_K_M

LM Studio
Jan

vLLM

How to use protoLabsAI/Ornith-1.0-9B-MTP-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "protoLabsAI/Ornith-1.0-9B-MTP-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "protoLabsAI/Ornith-1.0-9B-MTP-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/protoLabsAI/Ornith-1.0-9B-MTP-GGUF:Q4_K_M

Ollama
How to use protoLabsAI/Ornith-1.0-9B-MTP-GGUF with Ollama:
```
ollama run hf.co/protoLabsAI/Ornith-1.0-9B-MTP-GGUF:Q4_K_M
```
Unsloth Desktop

How to use protoLabsAI/Ornith-1.0-9B-MTP-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf protoLabsAI/Ornith-1.0-9B-MTP-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "protoLabsAI/Ornith-1.0-9B-MTP-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use protoLabsAI/Ornith-1.0-9B-MTP-GGUF with Docker Model Runner:
```
docker model run hf.co/protoLabsAI/Ornith-1.0-9B-MTP-GGUF:Q4_K_M
```

Lemonade

How to use protoLabsAI/Ornith-1.0-9B-MTP-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull protoLabsAI/Ornith-1.0-9B-MTP-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.Ornith-1.0-9B-MTP-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use protoLabsAI/Ornith-1.0-9B-MTP-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf protoLabsAI/Ornith-1.0-9B-MTP-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default protoLabsAI/Ornith-1.0-9B-MTP-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use protoLabsAI/Ornith-1.0-9B-MTP-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf protoLabsAI/Ornith-1.0-9B-MTP-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "protoLabsAI/Ornith-1.0-9B-MTP-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

protoLabs commited on Jul 3

Commit

7bcb8f0

1 Parent(s): 47b2452

rename files to resolvable <Model>-<QUANT>.gguf convention

Browse files

llama.cpp's -hf resolver (gguf_filename_is_model) excludes any filename
containing 'mtp-' — the old '-mtp-kl-' infix made every ladder rung
invisible to 'Use this model' / -hf repo:TAG. Upstream bug report for the
substring match filed separately.

Files changed (11) hide show

ornith-9b-mtp-kl-BF16.gguf → Ornith-1.0-9B-MTP-BF16.gguf +0 -0
ornith-9b-mtp-kl-IQ2_M.gguf → Ornith-1.0-9B-MTP-IQ2_M.gguf +0 -0
ornith-9b-mtp-kl-IQ3_M.gguf → Ornith-1.0-9B-MTP-IQ3_M.gguf +0 -0
ornith-9b-mtp-kl-IQ4_XS.gguf → Ornith-1.0-9B-MTP-IQ4_XS.gguf +0 -0
Ornith-1.0-9B-NVFP4-MTP.gguf → Ornith-1.0-9B-MTP-NVFP4.gguf +0 -0
ornith-9b-mtp-kl-Q4_K_M.gguf → Ornith-1.0-9B-MTP-Q4_K_M.gguf +0 -0
ornith-9b-mtp-kl-Q5_K_M.gguf → Ornith-1.0-9B-MTP-Q5_K_M.gguf +0 -0
ornith-9b-mtp-kl-Q6_K.gguf → Ornith-1.0-9B-MTP-Q6_K.gguf +0 -0
ornith-9b-mtp-kl-Q8_0.gguf → Ornith-1.0-9B-MTP-Q8_0.gguf +0 -0
README.md +23 -13
mtp-ornith-9b-mtp-kl-Q8_0.gguf → mtp-Ornith-1.0-9B-head-Q8_0.gguf +0 -0

@@ -28,7 +28,7 @@ head's per-token acceptance on llama.cpp matches the vLLM reference (**0.766** h
 ## NEW: NVFP4 build — 306 tok/s on Blackwell, the fastest rung in this repo
-`Ornith-1.0-9B-NVFP4-MTP.gguf` (6.6 GB) — calibrated NVFP4 (`GGML_TYPE_NVFP4`) on all
 attention/MLP GEMMs, Q8_0 on the DeltaNet trunk/embeddings, MTP `nextn` head included.
 Converted directly from our gate-verified
 [vLLM NVFP4 quant](https://huggingface.co/protoLabsAI/Ornith-1.0-9B-NVFP4) — same calibrated
@@ -62,15 +62,15 @@ MTP support (both merged spring 2026).
 | File | Form | Size | Use |
 |---|---|---:|---|
-| `ornith-9b-mtp-kl-Q8_0.gguf` | bundled (trunk + head) | 9.8 GB | highest quality / biggest relative speedup |
-| `ornith-9b-mtp-kl-Q6_K.gguf` | bundled | 7.6 GB | near-lossless quant |
-| `ornith-9b-mtp-kl-Q5_K_M.gguf` | bundled | 6.6 GB | balanced |
-| `ornith-9b-mtp-kl-Q4_K_M.gguf` | bundled | 5.8 GB | fastest k-quant |
-| `ornith-9b-mtp-kl-IQ4_XS.gguf` | bundled (imatrix) | 5.5 GB | low VRAM, near-Q4 quality |
-| `ornith-9b-mtp-kl-IQ3_M.gguf` | bundled (imatrix) | 4.7 GB | lower VRAM |
-| `ornith-9b-mtp-kl-IQ2_M.gguf` | bundled (imatrix) | 3.9 GB | **very low VRAM** (~5 GB to serve) |
-| `ornith-9b-mtp-kl-BF16.gguf` | bundled (full precision) | 18.4 GB | the master; re-quantize from this |
-| `mtp-ornith-9b-mtp-kl-Q8_0.gguf` | standalone draft head | 2.4 GB | attach to a base GGUF via `--model-draft` |
 The **IQ** quants are i-quants built with an importance matrix (calibrated on the trunk) for
 quality at low bit-rates, with the **MTP `nextn` head pinned to Q8_0** so speculative-decode
@@ -84,7 +84,7 @@ Requires **llama.cpp ≥ b9616** (Qwen3.5 `qwen35` arch + `--spec-type draft-mtp
 **Bundled (recommended)** — the head travels in the file:
 ```bash
-llama-server --model ornith-9b-mtp-kl-Q4_K_M.gguf \
   --n-gpu-layers 99 --ctx-size 8192 --flash-attn on --jinja \
   --spec-type draft-mtp --spec-draft-n-max 3
 ```
@@ -93,7 +93,7 @@ llama-server --model ornith-9b-mtp-kl-Q4_K_M.gguf \
 ```bash
 llama-server --model ornith-1.0-9b-Q4_K_M.gguf \
-  --model-draft mtp-ornith-9b-mtp-kl-Q8_0.gguf \
   --spec-type draft-mtp --spec-draft-n-max 3 \
   --n-gpu-layers 99 --ctx-size 8192 --flash-attn on --jinja
 ```
@@ -165,7 +165,7 @@ then expects 442 tensors and finds 427 → load fails.
 (`attn_*`, `ffn_*`, the norms) — so grepping for `nextn` shows only 4, but the head is complete.
 Don't want to graft? You don't have to build the bundled file at all — run the base GGUF with
-`--model-draft mtp-ornith-9b-mtp-kl-Q8_0.gguf --spec-type draft-mtp`. Functionally identical.
 ## Provenance & license
@@ -181,3 +181,13 @@ Open a Community discussion — requests usually ship within 48h. That's how mos
 quants in this repo got here. All benchmark rows behind these numbers:
 [`protoLabsAI/lab-benchmarks`](https://huggingface.co/datasets/protoLabsAI/lab-benchmarks) ·
 charts at [protolabs.studio/lab](https://protolabs.studio/lab).

 ## NEW: NVFP4 build — 306 tok/s on Blackwell, the fastest rung in this repo
+`Ornith-1.0-9B-MTP-NVFP4.gguf` (6.6 GB) — calibrated NVFP4 (`GGML_TYPE_NVFP4`) on all
 attention/MLP GEMMs, Q8_0 on the DeltaNet trunk/embeddings, MTP `nextn` head included.
 Converted directly from our gate-verified
 [vLLM NVFP4 quant](https://huggingface.co/protoLabsAI/Ornith-1.0-9B-NVFP4) — same calibrated
 | File | Form | Size | Use |
 |---|---|---:|---|
+| `Ornith-1.0-9B-MTP-Q8_0.gguf` | bundled (trunk + head) | 9.8 GB | highest quality / biggest relative speedup |
+| `Ornith-1.0-9B-MTP-Q6_K.gguf` | bundled | 7.6 GB | near-lossless quant |
+| `Ornith-1.0-9B-MTP-Q5_K_M.gguf` | bundled | 6.6 GB | balanced |
+| `Ornith-1.0-9B-MTP-Q4_K_M.gguf` | bundled | 5.8 GB | fastest k-quant |
+| `Ornith-1.0-9B-MTP-IQ4_XS.gguf` | bundled (imatrix) | 5.5 GB | low VRAM, near-Q4 quality |
+| `Ornith-1.0-9B-MTP-IQ3_M.gguf` | bundled (imatrix) | 4.7 GB | lower VRAM |
+| `Ornith-1.0-9B-MTP-IQ2_M.gguf` | bundled (imatrix) | 3.9 GB | **very low VRAM** (~5 GB to serve) |
+| `Ornith-1.0-9B-MTP-BF16.gguf` | bundled (full precision) | 18.4 GB | the master; re-quantize from this |
+| `mtp-Ornith-1.0-9B-MTP-Q8_0.gguf` | standalone draft head | 2.4 GB | attach to a base GGUF via `--model-draft` |
 The **IQ** quants are i-quants built with an importance matrix (calibrated on the trunk) for
 quality at low bit-rates, with the **MTP `nextn` head pinned to Q8_0** so speculative-decode
 **Bundled (recommended)** — the head travels in the file:
 ```bash
+llama-server --model Ornith-1.0-9B-MTP-Q4_K_M.gguf \
   --n-gpu-layers 99 --ctx-size 8192 --flash-attn on --jinja \
   --spec-type draft-mtp --spec-draft-n-max 3
 ```
 ```bash
 llama-server --model ornith-1.0-9b-Q4_K_M.gguf \
+  --model-draft mtp-Ornith-1.0-9B-MTP-Q8_0.gguf \
   --spec-type draft-mtp --spec-draft-n-max 3 \
   --n-gpu-layers 99 --ctx-size 8192 --flash-attn on --jinja
 ```
 (`attn_*`, `ffn_*`, the norms) — so grepping for `nextn` shows only 4, but the head is complete.
 Don't want to graft? You don't have to build the bundled file at all — run the base GGUF with
+`--model-draft mtp-Ornith-1.0-9B-MTP-Q8_0.gguf --spec-type draft-mtp`. Functionally identical.
 ## Provenance & license
 quants in this repo got here. All benchmark rows behind these numbers:
 [`protoLabsAI/lab-benchmarks`](https://huggingface.co/datasets/protoLabsAI/lab-benchmarks) ·
 charts at [protolabs.studio/lab](https://protolabs.studio/lab).
+## One-command usage
+```bash
+llama-server -hf protoLabsAI/Ornith-1.0-9B-MTP-GGUF:NVFP4 --spec-type draft-mtp -ngl 99   # Blackwell: fastest
+llama-server -hf protoLabsAI/Ornith-1.0-9B-MTP-GGUF:Q4_K_M --spec-type draft-mtp -ngl 99  # everything else
+```
+> 2026-07-03: files renamed to the standard `<Model>-<QUANT>.gguf` convention — older llama.cpp
+> `-hf` invocations with the previous `ornith-9b-mtp-kl-*` filenames should switch to tags.