Edit Models filters

Model Tree

Apps

Docker Model Runner

Inference Providers

OVHcloud AI Endpoints

HF Inference API

One-click Deployment

Inference Endpoints

Microsoft Foundry

Amazon SageMaker AI

Misc

text-generation-inference

Eval Results (legacy)

text-embeddings-inference

4-bit precision

8-bit precision

Mixture of Experts

Carbon Emissions

Models

1,311

Base only

Active filters: int4

donghyunli/Llama-2-13b-KronQ-W4A16-g128

Text Generation • 7B • Updated Jul 13 • 98

donghyunli/Llama-2-70b-KronQ-W4A16-g128

Text Generation • 36B • Updated Jul 14 • 127

donghyunli/Meta-Llama-3-70B-KronQ-W4A16-g128

Text Generation • 37B • Updated Jul 14 • 131

aufklarer/Qwen3-4B-Instruct-2507-MLX-4bit

Text Generation • 0.6B • Updated Jun 24 • 762 • 1

machine-made-Fibre/Qwen3-14B-Heretic-OpenVINO-INT4-NPU

Text Generation • Updated Jun 24 • 23

machine-made-Fibre/qwen3-8b-heretic-OpenVINO-INT4-NPU

Text Generation • Updated Jun 24 • 10

machine-made-Fibre/Qwen3-4B-Thinking-2507-heretic-OpenVINO-INT4-NPU

Text Generation • Updated Jun 24 • 7

machine-made-Fibre/Huihui-Qwen3-14B-abliterated-v2-OpenVINO-INT4-NPU

Text Generation • Updated Jun 24 • 18

machine-made-Fibre/Qwen3-4B-Instruct-2507-heretic-OpenVINO-INT4-NPU

Text Generation • Updated Jun 25 • 12

exzile/Qwythos-9B-Claude-Mythos-5-1M-text-int4-ov

Text Generation • Updated Jun 25 • 9 • 2

Patil/krea-turbo-svdquant

Text-to-Image • Updated Jun 26 • 45 • 7

gonuit/Qwen3-VL-Embedding-8B-AWQ-4bit

Sentence Similarity • 2B • Updated Jun 26 • 2.2k

Sebesky/MiniMax-M3-W4A16-GPTQ

Image-Text-to-Text • 430B • Updated Jun 26 • 773 • 3

Prog-up/gemma-4-E2B-it-int4-ov

Text Generation • Updated Jun 26 • 11

Prog-up/gemma-4-E4B-it-int4-ov

Text Generation • Updated Jun 26 • 9

Prog-up/Qwen2.5-3B-Instruct-int4-ov

Text Generation • Updated Jun 26 • 12

srv-sngh/Ornith-1.0-9B-MLX-4bit

Image-Text-to-Text • 2B • Updated Jun 27 • 59 • 1

AnishRacherla/aya-expanse-8b-compressed-final

Translation • 7B • Updated Jun 27 • 4

cyburn/Ornith-1.0-35B-int4-AutoRound

Text Generation • 1B • Updated Jun 28 • 368 • 1

Pilcothink/Ornith-1.0-397B-W4A16-AutoRound

Text Generation • Updated Jun 28 • 1.14k • 3

canada-quant/GLM-5.2-W4A16-MTP

Text Generation • 116B • Updated Jul 3 • 2.65k • 21

GRRNMAKE/SYNAXIM

Text Generation • Updated 28 days ago • 1

almernzh/Gemma-4-12B-it-AWQ-INT4-Multimodal-LongContext

Image-Text-to-Text • 12B • Updated Jun 28 • 521

XReyRobert/Ornith-1.0-35B-GPTQ-Pro-FOEM-4bit-g128-ns256

Text Generation • 35B • Updated Jul 2 • 5.6k • 5

TrNi/efficient-cube3d

Text-to-3D • Updated Jul 13 • 8

toxzak/gemma4-e2b-exp-quant

Text Generation • Updated Jun 29

amnibro/granite41-8b-gf17-atex

5B • Updated Jun 29 • 9

litert-community/Qwen3-0.6B-int4

Text Generation • Updated Jun 30 • 1.62k • 1

pyromind/qwen3.5-4b-debug-mlx-int4

Text Generation • 2B • Updated Jun 30 • 22

Teora/qwen3.5-4b-awq-v128k

4B • Updated Jun 30 • 27