Instructions to use mkreynolds/Deepseek-V4-Flash-oQ8-mtp with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use mkreynolds/Deepseek-V4-Flash-oQ8-mtp with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download mkreynolds/Deepseek-V4-Flash-oQ8-mtp --local-dir Deepseek-V4-Flash-oQ8-mtp
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
Deepseek-V4-Flash-oQ8-mtp
This is an 8-bit mixed-precision quantization of Deepseek V4 Flash, using oQ (oMLX v0.5.1). This quant should in theory be lossless, due to Deepseek V4 Flash's native FP4/FP8 architecture. Fits within 192GB of unified memory. MTP weights are preserved (use oMLX Lightning MTP).
I run this quant every day on an oMLX server for use with Openclaw. It performs well at multi-turn tool calling and coding tasks (just make sure to set blockstreamingdefault to "on" in Openclaw or else multi-turn tool calling will fail).
oMLX's highly efficient caching makes this version, in my experience, signficantly faster than ds4.c. The MLX architecture and MTP also lead to slight speed gains in prompt processing and decode.
Quantization details
- Model type: deepseek_v4
- Bits: 8
- Group size: 64
- Format: MLX safetensors
- Downloads last month
- 39
8-bit
Model tree for mkreynolds/Deepseek-V4-Flash-oQ8-mtp
Base model
deepseek-ai/DeepSeek-V4-Flash