csoares31/AMALIA-9B-0626-DPO-GGUF

AMALIA-9B-0626-DPO — GGUF

GGUF quantizations of amalia-llm/AMALIA-9B-0626-DPO, the Portuguese open LLM (European Portuguese) developed by a consortium of Portuguese universities and research centers, released under Apache 2.0.

Converted with llama.cpp b9859 (convert_hf_to_gguf.py to F16, then quantized with llama-quantize). Verified locally before upload.

⚠️ Important: use a recent llama.cpp build

Converting or running this model with older llama.cpp builds (e.g. b8681, as packaged in Ubuntu 26.04) may produce corrupted/degenerate output: input tokenization works, but generation yields invisible/garbage tokens. Use a recent build (tested with b9859) for both conversion and inference.

Files

File	Quant	Size	Notes
AMALIA-9B-0626-DPO-Q8_0.gguf	Q8_0	~9.7 GB	Near-lossless, recommended if you have the RAM
AMALIA-9B-0626-DPO-Q4_K_M.gguf	Q4_K_M	~5.5 GB	Good quality/size trade-off
AMALIA-9B-0626-DPO-F16.gguf	F16	~17 GB	Full precision, base for requantization

Usage

Ollama

ollama run hf.co/csoares31/AMALIA-9B-0626-DPO-GGUF:Q8_0

The model uses a ChatML template (<|im_start|> / <|im_end|>, with non-standard BOS=3 / EOS=4). If responses come out malformed or drift into English, create the model with an explicit template (Modelfile):

FROM ./AMALIA-9B-0626-DPO-Q8_0.gguf

TEMPLATE """{{- range .Messages }}<|im_start|>{{ .Role }}
{{ .Content }}<|im_end|>
{{ end }}<|im_start|>assistant
"""

SYSTEM """És o AMALIA, um assistente de IA que responde sempre em português europeu."""

PARAMETER stop <|im_end|>

Then:

ollama create amalia -f Modelfile
ollama run amalia

llama.cpp

llama-server -hf csoares31/AMALIA-9B-0626-DPO-GGUF:Q8_0 --jinja -c 8192

Sobre o modelo (PT-PT)

O AMALIA é o modelo de linguagem aberto criado para o português de Portugal e para a cultura portuguesa, desenvolvido por um consórcio de universidades e centros de investigação portugueses, com financiamento público. Todos os créditos do modelo original pertencem ao consórcio AMALIA.

Documentação oficial: https://amalia-llm.github.io/

Quantizações não oficiais, publicadas ao abrigo da licença Apache 2.0.