Instructions to use DuoNeural/DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use DuoNeural/DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf DuoNeural/DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf DuoNeural/DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf DuoNeural/DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf DuoNeural/DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf DuoNeural/DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf DuoNeural/DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF:Q4_K_M
Use Docker
docker model run hf.co/DuoNeural/DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use DuoNeural/DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "DuoNeural/DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "DuoNeural/DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/DuoNeural/DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF:Q4_K_M
- Ollama
How to use DuoNeural/DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF with Ollama:
ollama run hf.co/DuoNeural/DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use DuoNeural/DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF with Docker Model Runner:
docker model run hf.co/DuoNeural/DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF:Q4_K_M
- Lemonade
How to use DuoNeural/DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull DuoNeural/DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF-Q4_K_M
List all available models
lemonade list
- Atomic Chat
DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF
Experimental Release: Pending Further Verification / Empirical Validation
This checkpoint represents an active research artifact from DuoNeural's statistical mechanics quantization program. All empirical benchmarks, test-time compute logs, and physics proofs are documented transparently below.
Developed by Jesse Caldwell, Archon, and Aura ✨ (DuoNeural Research Lab).
Model Summary
- Foundation Model:
deepseek-ai/DeepSeek-R1-Distill-Llama-8B(32 Layers, 32:8 GQA, SwiGLU FFN, 128k RoPE, Test-Time Compute Reasoning) - Quantization Precision:
~4.8 bpw(4.58 GiB) - Methodology: Pre-conditioned with Generalized Thouless-Anderson-Palmer (G-TAP v3) Onsager cavity damping on DeepSeek-R1-Distill-Llama-8B. Filters out discretization noise across extended test-time compute thinking traces, delivering near-lossless university-level mathematical reasoning and algorithmic coding on Meta's Llama 3.1 8B architecture.
- Continuous Holdout Perplexity (131k tokens):
3.3492 - GSM8K Native CoT Proofs:
21/25 (84.0%) - Average Thinking Trace:
433.0 tokens(</think>Closure Rate:92.0%) - Olympiad Competition Math:
4/10 (40.0%) - Python Code AST Execution:
1/10 (10.0%) - Inference Decode Throughput:
112.6 t/son NVIDIA GeForce RTX 4080 Super (32GB VRAM)
Theoretical Methodology
Extended chain-of-thought models rely on dynamic test-time compute, generating hundreds of intermediate deliberation tokens inside <think>...</think> scratchpads. Discretization noise from standard PTQ accumulates multiplicatively across intermediate reasoning steps, leading to proof derailment and infinite repetitive cycles.
DuoNeural's Generalized Thouless-Anderson-Palmer (G-TAP v3) framework models neural weights as spin glasses embedded in activation cavity fields. By subtracting the Onsager reaction term: and projecting parameter updates strictly into the contractive Lyapunov half-space: G-TAP dampens back-reaction noise and preserves reasoning attractor basins across extended thinking sequences.
Usage Instructions
Native Prompt Template
DeepSeek-R1 models require the native prompt format to activate internal chain-of-thought:
<|begin of sentence|><|User|>{prompt}<|Assistant|><think>
Running with llama-cli
llama-cli -hf DuoNeural/DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF -p "<|begin of sentence|><|User|>Solve for all real roots: x^4 - 4x^3 + 6x^2 - 4x + 1 = 0.<|Assistant|><think>\n" -n 2048 -c 8192 --temp 0.6 --top-p 0.95
Serving with llama-server
llama-server -hf DuoNeural/DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF --port 8080 -c 8192 -ngl 99 -fa on
DuoNeural Cognitive Light Cone — Jesse Caldwell, Archon, Aura ✨
Empirically Validated on NVIDIA RTX 4080 Super 32GB Pod Testbed
- Downloads last month
- 103
4-bit
Model tree for DuoNeural/DeepSeek-R1-Distill-Llama-8B-GTAP-v3-Q4_K_M-GGUF
Base model
deepseek-ai/DeepSeek-R1-Distill-Llama-8B