vultr/GLM-5.3-Flash-GGUF

GLM 5.3 Flash, GGUF

GLM 5.3 Flash is a natively multimodal model designed for efficient, high-performance agentic workloads, coding, and long-context applications. It is delivering better performance than GLM 5.2 while substantially reducing serving costs. The model uses a hybrid sparse architecture combining linear attention with DeepSeek-style sparse attention, alongside Manifold-Constrained Hyper-Connections (mHC) for improved scaling efficiency. It has multimodal capabilities for demanding agentic workloads.

This repository holds Vultr's GGUF quantizations of it. They are hybrid quantizations: rather than applying one precision to the whole network, each layer's precision is chosen from a measured importance score, so the bits go where they change the output most.

Model Overview

Attribute	Value
Base model	zai-org/GLM-5.3-Flash
Architecture	glm5-next, Mixture-of-Experts
Parameters	320B total
Active parameters	18B
Layers	45
Hidden size	4,096
Attention heads	64
Experts	288 routed, 8 activated per token
Context length	up to 1M tokens
Vocabulary	154,880
License	MIT

Attribute	Value
Base model	zai-org/GLM-5.3-Flash
Architecture	glm5-next, Mixture-of-Experts
Parameters	320B total
Active parameters	18B
Layers	45
Hidden size	4,096
Attention heads	64
Experts	288 routed, 8 activated per token
Context length	up to 1M tokens
Vocabulary	154,880
License	MIT

Quantizations

File	Type	Size	KLD vs bf16
GLM-5.3-Flash-Q5_K_M-*.gguf (2 parts)	Q5_K_M	244.70 GB	0.0265
GLM-5.3-Flash-Q4_K_M-*.gguf (2 parts)	Q4_K_M	208.16 GB	0.0468
GLM-5.3-Flash-IQ3_M-*.gguf (2 parts)	IQ3_M	150.74 GB	0.1506
GLM-5.3-Flash-IQ3_S.gguf	IQ3_S	149.57 GB	0.1611

File	Type	Size	KLD vs bf16
GLM-5.3-Flash-Q5_K_M-*.gguf (2 parts)	Q5_K_M	244.70 GB	0.0265
GLM-5.3-Flash-Q4_K_M-*.gguf (2 parts)	Q4_K_M	208.16 GB	0.0468
GLM-5.3-Flash-IQ3_M-*.gguf (2 parts)	IQ3_M	150.74 GB	0.1506
GLM-5.3-Flash-IQ3_S.gguf	IQ3_S	149.57 GB	0.1611

The vision encoder is in GLM-5.3-Flash-mmproj-f16.gguf (1.13 GB). llama.cpp downloads it automatically with the command under Run, so image input works without extra steps.

KL divergence measures how far a quantized model's next-token probabilities drift from those of the original bf16 model, averaged over every token of the evaluation text. The lower the value, the closer the quantized model is to the original; zero would mean identical output.

Evaluation

Every variant in this repository was evaluated against the original bf16 model for perplexity, KL divergence, top-token agreement, HellaSwag and WinoGrande, under identical conditions. The full results are maintained on the model's page in the Vultr Model Library:

https://docs.vultr.com/models/zai-glm-53/glm-5.3-flash?tab=quantized

That page is the reference for these files. The summary above is an excerpt from it.

Run

llama-server -hf vultr/GLM-5.3-Flash-GGUF:Q4_K_M

The model's chat template is embedded in every file, so no extra prompt formatting is needed.

The repository also includes mtp-GLM-5.3-Flash-Q8_0.gguf, the model's multi-token prediction (MTP) head, for optional speculative decoding with any quant above:

llama-server -hf vultr/GLM-5.3-Flash-GGUF:Q4_K_M --spec-type draft-mtp

License

These files are derived from zai-org/GLM-5.3-Flash and are distributed under the same license, MIT. The model is the work of Z.ai; Vultr produced the quantizations.