Instructions to use Xalphinions/MARS-Qwen2.5-7B-blk16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Xalphinions/MARS-Qwen2.5-7B-blk16 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Xalphinions/MARS-Qwen2.5-7B-blk16")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelWithLMHead
model = AutoModelWithLMHead.from_pretrained("Xalphinions/MARS-Qwen2.5-7B-blk16", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Xalphinions/MARS-Qwen2.5-7B-blk16 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Xalphinions/MARS-Qwen2.5-7B-blk16"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Xalphinions/MARS-Qwen2.5-7B-blk16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/Xalphinions/MARS-Qwen2.5-7B-blk16

SGLang

How to use Xalphinions/MARS-Qwen2.5-7B-blk16 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Xalphinions/MARS-Qwen2.5-7B-blk16" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Xalphinions/MARS-Qwen2.5-7B-blk16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Xalphinions/MARS-Qwen2.5-7B-blk16" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Xalphinions/MARS-Qwen2.5-7B-blk16",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use Xalphinions/MARS-Qwen2.5-7B-blk16 with Docker Model Runner:
```
docker model run hf.co/Xalphinions/MARS-Qwen2.5-7B-blk16
```

MARS-Qwen2.5-7B-blk16

Qwen2.5-7B-Instruct fine-tuned with MARS (MARS: Enabling Autoregressive Models Multi-Token Generation; Jin, Wang, Luo, Sun, 2026), block size 16. One forward pass predicts a block of 16 tokens from 16 placeholder tokens; decoded ordinarily (no placeholders) the same weights are a normal autoregressive (AR) model. No extra parameters, heads or modules.

It is also the 7B proposer used in the quick start of ProVe: Proposer-Verifier Decoding (Jin, Xu, Li, Zhong, She, Sun, 2026). There a verifier reads the proposed block in one more forward pass and commits the longest prefix it accepts:

ProVe-lossless – the proposer verifies its own block. The output equals greedy AR decoding of this checkpoint, token for token; no second model, no draft head.
ProVe – a smaller stock model (Qwen2.5-3B/7B-Instruct) verifies, with one threshold trading speed for accuracy.
ProVe project page: https://xalp.net/prove/
ProVe code (serving stack, benchmarks, evaluation): https://github.com/Xalp/ProVe
MARS code and training recipe: https://github.com/Xalp/MARS · paper: https://arxiv.org/abs/2604.07023
Stage-1 checkpoint (AR SFT only, the matched AR control): Xalphinions/MARS-Qwen2.5-7B-AR-SFT

Model details


Architecture	Qwen2 (unchanged: no extra parameters, heads or modules)
Parameters	7.6B
Base model	Qwen/Qwen2.5-7B-Instruct (Apache-2.0)
Block size	16 (`block_size: 16` must be used at serving time)
Placeholder token	`<
Precision	bf16 safetensors, 4 shards
Context length	as the base model (32k)
Checkpoint format	`config.json` carries `model_type: a2d-qwen2` / `architectures: [A2DQwen2LMHeadModel]` as written by the MARS trainer; rewrite it to `qwen2` / `Qwen2ForCausalLM` for serving (ProVe's `serve.sh` does this for you). The weights are plain Qwen2 weights.
License	Apache-2.0

Training

Two stages on the same public instruction data, both 5 epochs on allenai/Dolci-Instruct-SFT (about 2M rows), learning rate 5e-6, warmup ratio 0.03, effective batch 384, sequence length 512:

	Stage 1: AR SFT	Stage 2: MARS block training
Init	Qwen2.5-7B-Instruct	stage-1 checkpoint (MARS-Qwen2.5-7B-AR-SFT)
Objective	next-token cross-entropy	next-token loss on the clean stream + block-prediction loss on the placeholder stream (block size 16)

Stage 1 exists so that the effect of the block objective can be measured against a matched AR control. Recipe, scripts and the MARS paper's own evaluation: https://github.com/Xalp/MARS.

How to use

ProVe / ProVe-lossless serving

git clone https://github.com/Xalp/ProVe.git && cd ProVe/inference && bash setup_env.sh
hf download Xalphinions/MARS-Qwen2.5-7B-blk16 --local-dir models/MARS-Qwen2.5-7B-blk16

# ProVe-lossless: the proposer verifies its own block; output == greedy AR; one GPU
bash serve.sh --proposer models/MARS-Qwen2.5-7B-blk16 --port 30000

# ProVe with a 3B verifier (theta 0.10 / 0.05 / 0.01: lower = faster, lossier)
hf download Qwen/Qwen2.5-3B-Instruct --local-dir models/Qwen2.5-3B-Instruct
bash serve.sh --proposer models/MARS-Qwen2.5-7B-blk16 \
              --verifier models/Qwen2.5-3B-Instruct --theta 0.05 --port 30001

serve.sh rewrites the config (weights are linked, not copied), writes the algorithm yaml and starts an SGLang server with the usual /generate and OpenAI-compatible /v1/chat/completions endpoints. --algorithm mars-tau serves the original MARS confidence-gated decoding (threshold τ, no verifier model) on the same stack; the MARS repository has the reference implementation.

Plain AR use with transformers

The checkpoint also works as an ordinary causal LM once the config names the Qwen2 architecture:

from transformers import AutoConfig, AutoModelForCausalLM, AutoTokenizer
path = "models/MARS-Qwen2.5-7B-blk16"
cfg = AutoConfig.from_pretrained("Qwen/Qwen2.5-7B-Instruct")      # same architecture
model = AutoModelForCausalLM.from_pretrained(path, config=cfg, torch_dtype="bfloat16", device_map="auto")
tok = AutoTokenizer.from_pretrained(path)

(or run python inference/tools/prepare_checkpoint.py <dir> once and load the <dir>-qwen2 copy).

Measured with the ProVe stack

GSM8K (gsm8k_cot), greedy, one request at a time, up to 1024 new tokens, one GPU, relative to greedy AR decoding of this checkpoint on the same stack. This 7B experiment is not in the ProVe paper, whose tables use the 32B proposer Xalphinions/ProVe-Qwen2.5-32B-blk16.

configuration	tok/s	speed-up	accepted tokens per cycle	GSM8K accuracy
greedy AR	234	1×	–	74.4
ProVe-lossless	433	1.85×	6.13	75.1

The two accuracies differ although the method is lossless by construction: the block path and the AR path run different kernels, and at a near-tie token their bf16 logits can pick different winners (in a 40-prompt, 512-token check 28 outputs were byte-identical). The ProVe paper therefore compares task accuracy, not byte identity.

Intended use and limitations

Research on multi-token generation and inference acceleration, and as a drop-in for Qwen2.5-7B-Instruct-class AR decoding where single-request throughput matters.
Greedy decoding only in the released ProVe serving code; sampling is not implemented.
The block size is fixed at training time (16); serving with another block size is not supported.
Fine-tuned on English instruction data; behaviour on other languages follows the base model but was not evaluated.
Safety behaviour is inherited from Qwen2.5-7B-Instruct and the public instruction data; no additional alignment or red-teaming was done. Apply the same safeguards you would apply to the base model.

Citation

@misc{jin2026mars,
  title         = {MARS: Enabling Autoregressive Models Multi-Token Generation},
  author        = {Ziqi Jin and Lei Wang and Ziwei Luo and Aixin Sun},
  year          = {2026},
  eprint        = {2604.07023},
  archivePrefix = {arXiv},
  primaryClass  = {cs.CL},
  url           = {https://arxiv.org/abs/2604.07023}
}

@article{jin2026prove,
  title   = {ProVe: Proposer-Verifier Decoding},
  author  = {Jin, Ziqi and Xu, Weiwen and Li, Jiaxi and Zhong, Yujie and She, Qi and Sun, Aixin},
  journal = {arXiv preprint},
  year    = {2026}
}