DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF

Experimental Release: Pending Further Verification / Empirical Validation
This checkpoint represents an active research artifact from DuoNeural's statistical mechanics quantization program. All empirical benchmarks, test-time compute logs, and physics proofs are documented transparently below.

Developed by Jesse Caldwell, Archon, and Aura ✨ (DuoNeural Research Lab).


Model Summary

  • Foundation Model: deepseek-ai/DeepSeek-R1-Distill-Llama-8B (32 Layers, 32:8 GQA, SwiGLU FFN, 128k RoPE, Test-Time Compute Reasoning)
  • Quantization Precision: ~4.8 bpw (4.58 GiB)
  • Methodology: Pre-conditioned with Generalized Thouless-Anderson-Palmer (G-TAP v3) Onsager cavity damping on DeepSeek-R1-Distill-Llama-8B. Filters out discretization noise across extended test-time compute thinking traces, delivering near-lossless university-level mathematical reasoning and algorithmic coding on Meta's Llama 3.1 8B architecture.
  • Continuous Holdout Perplexity (131k tokens): 3.3492
  • GSM8K Native CoT Proofs: 21/25 (84.0%)
  • Average Thinking Trace: 433.0 tokens (</think> Closure Rate: 92.0%)
  • Olympiad Competition Math: 4/10 (40.0%)
  • Python Code AST Execution: 1/10 (10.0%)
  • Inference Decode Throughput: 112.6 t/s on NVIDIA GeForce RTX 4080 Super (32GB VRAM)

Theoretical Methodology

Extended chain-of-thought models rely on dynamic test-time compute, generating hundreds of intermediate deliberation tokens inside <think>...</think> scratchpads. Discretization noise from standard PTQ accumulates multiplicatively across intermediate reasoning steps, leading to proof derailment and infinite repetitive cycles.

DuoNeural's Generalized Thouless-Anderson-Palmer (G-TAP v3) framework models neural weights as spin glasses embedded in activation cavity fields. By subtracting the Onsager reaction term: Ωi=1d(∥H~i,:∥22−H~ii2)\Omega_i = \frac{1}{d} \left( \| \tilde{H}_{i,:} \|_2^2 - \tilde{H}_{ii}^2 \right) and projecting parameter updates strictly into the contractive Lyapunov half-space: Re⁡(λ(S))≤−δ\operatorname{Re}(\lambda(S)) \le -\delta G-TAP dampens back-reaction noise and preserves reasoning attractor basins across extended thinking sequences.


Usage Instructions

Native Prompt Template

DeepSeek-R1 models require the native prompt format to activate internal chain-of-thought:

<|begin of sentence|><|User|>{prompt}<|Assistant|><think>

Running with llama-cli

llama-cli -hf DuoNeural/DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF -p "<|begin of sentence|><|User|>Solve for all real roots: x^4 - 4x^3 + 6x^2 - 4x + 1 = 0.<|Assistant|><think>\n" -n 2048 -c 8192 --temp 0.6 --top-p 0.95

Serving with llama-server

llama-server -hf DuoNeural/DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF --port 8080 -c 8192 -ngl 99 -fa on

DuoNeural Cognitive Light Cone — Jesse Caldwell, Archon, Aura ✨
Empirically Validated on NVIDIA RTX 4080 Super 32GB Pod Testbed
Downloads last month
103
GGUF
Model size
8B params
Architecture
llama
Hardware compatibility
Log In to add your hardware

4-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for DuoNeural/DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF

Quantized
(213)
this model