--- license: apache-2.0 language: - en base_model: deepreinforce-ai/Ornith-1.0-9B pipeline_tag: text-generation library_name: gguf tags: - qwen - qwen3.5 - 9b - ornith - coding - agent - deepreinforce - quantized - quantization - gguf - llama-cpp - imatrix - hybrid-quantization - ASHQ1 - priority-queue - mse --- # Ornith-1.0-9B — ASHQ1 Quantization ASHQ1 quantizations of [Ornith-1.0-9B](https://huggingface.co/deepreinforce-ai/Ornith-1.0-9B) — a state-of-the-art open-source model for agentic coding by DeepReinforce. ## Quants > **Note:** File names contain "BF16" for HuggingFace parser compatibility — these are ASHQ1 quants, not BF16. | File | Size | Method | Description | |------|:----:|:------:|-------------| | `Ornith-1.0-9B-BF16-ASHQ1-4850.gguf` | 4.8 GB | ASHQ1 | Tiny — quality between i1-Q4_K_M and i1-Q5_K_M, 32% smaller than i1-Q5_K_M | | `Ornith-1.0-9B-BF16-ASHQ1-5500.gguf` | 5.4 GB | ASHQ1 | Compact — beats i1-Q6_K (7.0 GB), 23% smaller | | `Ornith-1.0-9B-BF16-ASHQ1-6500.gguf` | 6.4 GB | ASHQ1 | Balanced — near i1-Q8_0 (9.6 GB), 33% smaller | | `Ornith-1.0-9B-BF16-ASHQ1-7300.gguf` | 7.2 GB | ASHQ1 | Best — surpasses i1-Q8_0 (9.6 GB), 25% smaller | ASHQ1 has been validated across multiple architectures (Qwen3.5-9B, Gemma4-12B QAT, Mellum2 MoE) and consistently outperforms uniform Q6_K and manual SHQ hybrids — see [wepiqx/ASHQ1](https://huggingface.co/wepiqx/ASHQ1) for method, source code, and full benchmark results. Perplexity comparisons per model are redundant; the priority queue optimises for PPL directly. ## Real-World Test: Autonomous Agentic Coding We tested the 6500 quant (6.4 GB) as the LLM backend for [Pi](https://pi.dev/), an autonomous coding agent that runs on `llama.cpp`. The task: build a complete personal finance dashboard — single HTML file, no external dependencies, with Canvas charts, budget tracker, dark mode, transaction filtering, and upcoming bills. At `temperature 0.6` (not cherry-picked, this was the first run), the agent's workflow was: 1. **Architecture planning** — parsed the requirements, planned 6 sections (overview, charts, transactions, budget, bills, theme toggle) 2. **Code generation** — wrote the full ~1100-line single HTML file with all CSS variables, Canvas rendering, and JavaScript logic 3. **Self-debugging** — caught `date.now is not a function`, corrected it to `date.getTime()`, fixed dark mode toggle logic, adjusted chart rendering to use `requestAnimationFrame` 4. **Validation** — wrote Node.js check scripts to verify: JavaScript syntax, all 30 transactions present, dark mode CSS variables exist, all 6 render functions wired up, script tags balanced 5. **Delivery** — confirmed the `finance-dashboard.html` passed all checks and was ready to open in a browser The model never hallucinated APIs, never broke the single-file constraint, never produced dead code. Every fix it applied was correct. The final output was a polished, production-quality single-page application — built autonomously by a 6.4 GB quant driving an agent. Full agent interaction log: [pi-log-temp0.6-6500.md](pi-log-temp0.6-6500.md) (589 lines). Final output: `finance-dashboard.html` (1100 lines, 39.8 KB). The benchmarks didn't lie. This quant preserves enough quality for autonomous code generation. ## Usage ### llama.cpp ```bash # Chat mode with built-in Jinja template (recommended) llama-cli \ -m Ornith-1.0-9B-BF16-ASHQ1-6500.gguf \ --jinja \ -ngl 99 \ -c 8192 # Or with a direct prompt llama-cli \ -m Ornith-1.0-9B-BF16-ASHQ1-6500.gguf \ -p "Write a Python function that..." \ -ngl 99 \ -c 8192 ``` Recommended sampling: `temperature 0.6–1.0`, `top_p 0.95`, `top_k 20`. The real-world Pi agent test (finance dashboard) used 0.6 with excellent results — lower temperatures are safe. ### Ollama The model's built-in Jinja template is auto-detected by Ollama — no need to specify `TEMPLATE` manually. 1. Create a `Modelfile`: ``` FROM ./Ornith-1.0-9B-BF16-ASHQ1-6500.gguf PARAMETER num_ctx 8192 PARAMETER temperature 0.6 PARAMETER top_k 20 PARAMETER top_p 0.95 ``` 2. Build and run: ```bash ollama create ornith-ashq1-6500 -f Modelfile ollama run ornith-ashq1-6500 ``` ### LM Studio 1. Open LM Studio 2. Drag `Ornith-1.0-9B-BF16-ASHQ1-4850.gguf` (or 5500/6500/7300) into the app 3. Set GPU Offload to 99 layers 4. Set context length to 8192+ 5. Set preset sampling: `temperature 0.6`, `top_p 0.95`, `top_k 20` 6. Start chatting ## Quantization Configs Generated by ASHQ1 priority queue — files in this repo were produced with these exact `llama-quantize` arguments. ### 4850 Config (base=Q5_K_M) ```bash --output-tensor-type Q5_K --token-embedding-type Q5_K --tensor-type "(blk|BLK)\.(31)\.ffn_down=Q4_K" --tensor-type "(blk|BLK)\.(31)\.attn_output=Q5_K" --tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30))\.ffn_down=Q3_K" --tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k_norm=F16" --tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16" --tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.post_attention_norm=F16" --tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16" --tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16" --tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16" --tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q_norm=F16" --tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.attn_norm=F16" --tensor-type "(blk|BLK)\.(0|25|(?:28|29|30))\.ssm_alpha=Q5_K" --tensor-type "(blk|BLK)\.(0|25|(?:28|29|30))\.attn_qkv=Q5_K" --tensor-type "(blk|BLK)\.(0|25|(?:28|29|30))\.ssm_beta=Q5_K" --tensor-type "(blk|BLK)\.(3|27|31)\.attn_q=Q5_K" --tensor-type "(blk|BLK)\.(3|27|31)\.attn_v=Q5_K" --tensor-type "(blk|BLK)\.(0|25|(?:28|29|30))\.attn_gate=Q5_K" --tensor-type "(blk|BLK)\.(3|27|31)\.attn_k=Q5_K" --tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_up=Q4_K" --tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|24|26)\.attn_gate=Q4_K" --tensor-type "(blk|BLK)\.(7|11|15|19|23)\.attn_k=Q4_K" --tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|24|26)\.ssm_alpha=Q4_K" --tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|24|26)\.ssm_beta=Q4_K" --tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_gate=Q4_K" --tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|24|26)\.attn_qkv=Q4_K" --tensor-type "(blk|BLK)\.(7|11|15|19|23)\.attn_q=Q4_K" --tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27)\.attn_output=Q4_K" --tensor-type "(blk|BLK)\.(7|11|15|19|23)\.attn_v=Q4_K" --tensor-type "(blk|BLK)\.(28|30)\.ssm_out=Q4_K" --tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|29)\.ssm_out=Q3_K" --tensor-type ".*output_norm.*=F16" ``` ### 5500 Config (base=Q5_K_M) ```bash --output-tensor-type Q5_K --token-embedding-type Q5_K --tensor-type "(blk|BLK)\.(31)\.attn_output=Q6_K" --tensor-type "(blk|BLK)\.(31)\.ffn_down=Q5_K" --tensor-type "(blk|BLK)\.(0|(?:13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_beta=Q6_K" --tensor-type "(blk|BLK)\.(3|7|19|23|27|31)\.attn_v=Q6_K" --tensor-type "(blk|BLK)\.(3|7|19|23|27|31)\.attn_k=Q6_K" --tensor-type "(blk|BLK)\.(0|(?:13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_qkv=Q6_K" --tensor-type "(blk|BLK)\.(0|(?:13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_alpha=Q6_K" --tensor-type "(blk|BLK)\.(0|(?:13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_gate=Q6_K" --tensor-type "(blk|BLK)\.(3|7|19|23|27|31)\.attn_q=Q6_K" --tensor-type "(blk|BLK)\.(27)\.attn_output=Q5_K" --tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30))\.ffn_down=Q4_K" --tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16" --tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16" --tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.attn_norm=F16" --tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16" --tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16" --tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k_norm=F16" --tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q_norm=F16" --tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.post_attention_norm=F16" --tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|12)\.ssm_beta=Q5_K" --tensor-type "(blk|BLK)\.((?:22|23|24|25|26|27|28|29|30|31))\.ffn_up=Q5_K" --tensor-type "(blk|BLK)\.((?:22|23|24|25|26|27|28|29|30|31))\.ffn_gate=Q5_K" --tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|12)\.ssm_alpha=Q5_K" --tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|12)\.attn_gate=Q5_K" --tensor-type "(blk|BLK)\.([1-2]|[4-6]|(?:8|9|10)|12)\.attn_qkv=Q5_K" --tensor-type "(blk|BLK)\.(11|15)\.attn_v=Q5_K" --tensor-type "(blk|BLK)\.(11|15)\.attn_k=Q5_K" --tensor-type "(blk|BLK)\.(11|15)\.attn_q=Q5_K" --tensor-type "(blk|BLK)\.((?:28|29|30))\.ssm_out=Q5_K" --tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21))\.ffn_gate=Q4_K" --tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26))\.ssm_out=Q4_K" --tensor-type "(blk|BLK)\.(3|7|11|15|19|23)\.attn_output=Q4_K" --tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21))\.ffn_up=Q4_K" --tensor-type ".*output_norm.*=F16" ``` ### 6500 Config (base=Q5_K_M) ```bash --output-tensor-type Q5_K --token-embedding-type Q5_K --tensor-type "(blk|BLK)\.(31)\.ffn_down=Q8_0" --tensor-type "(blk|BLK)\.(31)\.attn_output=Q8_0" --tensor-type "(blk|BLK)\.(0|[5-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_alpha=Q8_0" --tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k=Q8_0" --tensor-type "(blk|BLK)\.(0|[5-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_gate=Q8_0" --tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_v=Q8_0" --tensor-type "(blk|BLK)\.(0|[5-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_beta=Q8_0" --tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q=Q8_0" --tensor-type "(blk|BLK)\.(0|[5-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_qkv=Q8_0" --tensor-type "(blk|BLK)\.(27)\.attn_output=Q6_K" --tensor-type "(blk|BLK)\.((?:29|30))\.ffn_down=Q6_K" --tensor-type "(blk|BLK)\.(6|(?:10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_up=Q6_K" --tensor-type "(blk|BLK)\.(24|(?:28|29|30))\.ssm_out=Q6_K" --tensor-type "(blk|BLK)\.(6|(?:10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_gate=Q6_K" --tensor-type "(blk|BLK)\.([1-2]|4)\.attn_qkv=Q6_K" --tensor-type "(blk|BLK)\.([1-2]|4)\.ssm_beta=Q6_K" --tensor-type "(blk|BLK)\.([1-2]|4)\.ssm_alpha=Q6_K" --tensor-type "(blk|BLK)\.([1-2]|4)\.attn_gate=Q6_K" --tensor-type "(blk|BLK)\.(22|(?:26|27|28))\.ffn_down=Q5_K" --tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21)|(?:23|24|25))\.ffn_down=Q4_K" --tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.attn_norm=F16" --tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.post_attention_norm=F16" --tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16" --tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16" --tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16" --tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k_norm=F16" --tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16" --tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q_norm=F16" --tensor-type "(blk|BLK)\.(15|19|23)\.attn_output=Q5_K" --tensor-type "(blk|BLK)\.([4-5]|[7-9])\.ffn_gate=Q5_K" --tensor-type "(blk|BLK)\.(18|(?:20|21|22)|(?:25|26))\.ssm_out=Q5_K" --tensor-type "(blk|BLK)\.([4-5]|[7-9])\.ffn_up=Q5_K" --tensor-type "(blk|BLK)\.(3|7|11)\.attn_output=Q4_K" --tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17))\.ssm_out=Q4_K" --tensor-type "(blk|BLK)\.([0-3])\.ffn_gate=Q4_K" --tensor-type "(blk|BLK)\.([0-3])\.ffn_up=Q4_K" --tensor-type ".*output_norm.*=F16" ``` ### 7300 Config (base=Q5_K_M) ```bash --output-tensor-type Q5_K --token-embedding-type Q5_K --tensor-type "(blk|BLK)\.((?:29|30|31))\.ffn_down=Q8_0" --tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_beta=Q8_0" --tensor-type "(blk|BLK)\.((?:12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_up=Q8_0" --tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q=Q8_0" --tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_gate=Q8_0" --tensor-type "(blk|BLK)\.((?:12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.ffn_gate=Q8_0" --tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_alpha=Q8_0" --tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.attn_qkv=Q8_0" --tensor-type "(blk|BLK)\.((?:28|29|30))\.ssm_out=Q8_0" --tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k=Q8_0" --tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_v=Q8_0" --tensor-type "(blk|BLK)\.(27|31)\.attn_output=Q8_0" --tensor-type "(blk|BLK)\.((?:22|23|24|25|26|27|28))\.ffn_down=Q6_K" --tensor-type "(blk|BLK)\.(18|(?:20|21|22)|(?:24|25|26))\.ssm_out=Q6_K" --tensor-type "(blk|BLK)\.((?:3|4|5|6|7|8|9|10|11))\.ffn_gate=Q6_K" --tensor-type "(blk|BLK)\.((?:3|4|5|6|7|8|9|10|11))\.ffn_up=Q6_K" --tensor-type "(blk|BLK)\.(11|15|19|23)\.attn_output=Q6_K" --tensor-type "(blk|BLK)\.(7)\.attn_output=Q5_K" --tensor-type "(blk|BLK)\.(3)\.attn_output=Q4_K" --tensor-type "(blk|BLK)\.((?:17|18|19|20|21))\.ffn_down=Q5_K" --tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16))\.ffn_down=Q4_K" --tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.attn_norm=F16" --tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_dt=F16" --tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_conv1d=F16" --tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_k_norm=F16" --tensor-type "(blk|BLK)\.((?:0|1|2|3|4|5|6|7|8|9|10|11|12|13|14|15|16|17|18|19|20|21|22|23|24|25|26|27|28|29|30|31))\.post_attention_norm=F16" --tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_norm=F16" --tensor-type "(blk|BLK)\.([0-2]|[4-6]|(?:8|9|10)|(?:12|13|14)|(?:16|17|18)|(?:20|21|22)|(?:24|25|26)|(?:28|29|30))\.ssm_a=F16" --tensor-type "(blk|BLK)\.(3|7|11|15|19|23|27|31)\.attn_q_norm=F16" --tensor-type "(blk|BLK)\.(0|(?:8|9|10)|(?:12|13|14)|(?:16|17))\.ssm_out=Q5_K" --tensor-type "(blk|BLK)\.([0-2])\.ffn_up=Q5_K" --tensor-type "(blk|BLK)\.([0-2])\.ffn_gate=Q5_K" --tensor-type "(blk|BLK)\.([1-2]|[4-6])\.ssm_out=Q4_K" --tensor-type ".*output_norm.*=F16" ``` ## Hardware Requirements | Variant | Size (GGUF) | VRAM (ctx 8192) | Minimum GPU | |---------|:-----------:|:----------------:|:-----------:| | 4850 | 4.8 GB | ~5.9 GB | 8 GB | | 5500 | 5.4 GB | ~6.5 GB | 8 GB | | 6500 | 6.4 GB | ~7.5 GB | 8 GB | | 7300 | 7.2 GB | ~8.3 GB | 12 GB | ## Notes - Quantized with the [Ornith imatrix](https://huggingface.co/bartowski/deepreinforce-ai_Ornith-1.0-9B-GGUF/blob/main/deepreinforce-ai_Ornith-1.0-9B-imatrix.gguf) by bartowski (248 entries, 6 datasets) - Architecture: Qwen3.5-9B (32 layers, GQA, SSM, 262k context) - MIT licensed upstream — no usage restrictions - Quantization and tuning by wepiqx - Built with [llama.cpp](https://github.com/ggml-org/llama.cpp)