Ozankj/MoganBERT-Embed-ONNX

MoganBERT-Embed ONNX (Unofficial)

Unofficial ONNX mirror of moganai/MoganBERT-Embed. The FP16 export preserves the original weights; model_quantized.onnx is an int8 quantized derivative intended for compact inference.

• Original model: https://huggingface.co/moganai/MoganBERT-Embed
• Paper: https://huggingface.co/papers/2608.25768
• Original authors / copyright: MoganAI (Furkan Yilmaz, Habibe Aleyna Tasdemir, Muhammed Faruk Gozay)
• License: Apache License 2.0 (see LICENSE and NOTICE)
• ONNX conversion by: Ozankj

The original mean pooling and L2 normalization are retained. FP32 and FP16 exports preserve the original weights (FP16 values are represented in half precision). onnx/model_quantized.onnx is a compact custom int8/uint8 graph: MatMul weights use 8-bit MatMulNBits; the embedding table uses group-wise asymmetric uint8 quantization with exception rows dequantized at inference time. It requires a compatible ONNX Runtime build. See QUANTIZATION.md for custom quantization notes.

Accuracy

Comparison	Result
FP32 ONNX vs PyTorch	cosine ~1.0 (max abs diff ~1e-05)
FP16 ONNX vs PyTorch	cosine ~1.0 (max abs diff 8.3e-04)
Custom int8 vs FP32 ONNX, Wiki40B Turkish validation (120 independent examples, 512-token truncation)	avg cosine .999806, min .999694, P1 .999702, ≥0.999 100%
Retrieval, same 120 examples	Top-1 98.3%, Top-5 98.5% (best rank vs FP32)
Pairwise similarity drift	max .00824, mean ≈**+.00078**
Special/edge vocabulary IDs (IDs 0–21, 50047, 42, 7 paired with special IDs)	all outputs finite; limited edge-ID checks must be interpreted separately

These are fixed-set ONNX Runtime CPU checks, not a full MTEB evaluation and not an end-to-end production usage test. Real-world Turkish retrieval applications, domain-specific corpora, latency under your serving stack, and runtime parity remain the user's responsibility. sentence_embedding is mean-pooled and L2-normalized. FP16 remains the maximum-compatibility option. See QUANTIZATION.md.

Files

File	Format	Size	Notes
onnx/model.onnx	FP32	~568 MB	Exact export; outputs match PyTorch
onnx/model_fp16.onnx	FP16	~298 MB	Recommended for compatibility; negligible output difference
onnx/model_quantized.onnx	Custom int8 / uint8 graph	~165 MB	Compact custom export; requires MatMulNBits support

Outputs: token_embeddings (batch, seq_len, 768) and sentence_embedding (batch, 768). The latter is mean-pooled and L2-normalized.

Usage

from sentence_transformers import SentenceTransformer

# FP16 (recommended for compatibility)
model = SentenceTransformer(
    "Ozankj/MoganBERT-Embed-ONNX",
    backend="onnx",
    model_kwargs={"file_name": "onnx/model_fp16.onnx"},
)

# Compact quantized variant; requires MatMulNBits support
quantized_model = SentenceTransformer(
    "Ozankj/MoganBERT-Embed-ONNX",
    backend="onnx",
    model_kwargs={"file_name": "onnx/model_quantized.onnx"},
)

sentences = ["Türkiye'nin başkenti Ankara'dır.", "Bugün hava çok güzel."]
embeddings = model.encode(sentences)

For direct ONNX Runtime use, load onnx/model_fp16.onnx and read the sentence_embedding output.

For direct ONNX Runtime use of the quantized model, load onnx/model_quantized.onnx and read sentence_embedding. It is a custom mixed int8/uint8 graph, not a standard QDQ / dynamic int8 export. It requires an ONNX Runtime build and execution provider that support MatMulNBits; the embedding table uses group-wise asymmetric uint8 quantization with in-graph dequantization and an exception-row fallback. Do not assume compatibility with web/mobile runtimes without testing.

Not affiliated

This repository is not affiliated with, endorsed by, or maintained by MoganAI. For the official model, weights and benchmarks, use the original link above.