Instructions to use reach-vb/FastVLM-0.5B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use reach-vb/FastVLM-0.5B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="reach-vb/FastVLM-0.5B", trust_remote_code=True)
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("reach-vb/FastVLM-0.5B", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use reach-vb/FastVLM-0.5B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "reach-vb/FastVLM-0.5B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "reach-vb/FastVLM-0.5B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/reach-vb/FastVLM-0.5B

SGLang

How to use reach-vb/FastVLM-0.5B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "reach-vb/FastVLM-0.5B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "reach-vb/FastVLM-0.5B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "reach-vb/FastVLM-0.5B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "reach-vb/FastVLM-0.5B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use reach-vb/FastVLM-0.5B with Docker Model Runner:
```
docker model run hf.co/reach-vb/FastVLM-0.5B
```

FastVLM: Efficient Vision Encoding for Vision Language Models

FastVLM was introduced in FastVLM: Efficient Vision Encoding for Vision Language Models. (CVPR 2025)

Accuracy vs latency figure.

Highlights

We introduce FastViTHD, a novel hybrid vision encoder designed to output fewer tokens and significantly reduce encoding time for high-resolution images.
Our smallest variant outperforms LLaVA-OneVision-0.5B with 85x faster Time-to-First-Token (TTFT) and 3.4x smaller vision encoder.
Our larger variants using Qwen2-7B LLM outperform recent works like Cambrian-1-8B while using a single image encoder with a 7.9x faster TTFT.

Evaluations

Benchmark	FastVLM-0.5B	FastVLM-1.5B	FastVLM-7B
Ai2D	68.0	77.4	83.6
ScienceQA	85.2	94.4	96.7
MMMU	33.9	37.8	45.4
VQAv2	76.3	79.1	80.8
ChartQA	76.0	80.1	85.0
TextVQA	64.5	70.4	74.9
InfoVQA	46.4	59.7	75.8
DocVQA	82.5	88.3	93.2
OCRBench	63.9	70.2	73.1
RealWorldQA	56.1	61.2	67.2
SeedBench-Img	71.0	74.2	75.4

Usage Example

To run inference of PyTorch checkpoint, follow the instruction in the official repo:

Download the model

huggingface-cli download apple/FastVLM-0.5B

Run inference using predict.py from the official repo.

python predict.py --model-path /path/to/checkpoint-dir \
                  --image-file /path/to/image.png \
                  --prompt "Describe the image."

Citation

If you found this model useful, please cite the following paper:

@InProceedings{fastvlm2025,
  author = {Pavan Kumar Anasosalu Vasu, Fartash Faghri, Chun-Liang Li, Cem Koc, Nate True, Albert Antony, Gokul Santhanam, James Gabriel, Peter Grasch, Oncel Tuzel, Hadi Pouransari},
  title = {FastVLM: Efficient Vision Encoding for Vision Language Models},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  month = {June},
  year = {2025},
}

Downloads last month: 1

Safetensors

Model size

0.8B params

Tensor type

BF16

Paper for reach-vb/FastVLM-0.5B

FastVLM: Efficient Vision Encoding for Vision Language Models

Paper • 2412.13303 • Published Dec 17, 2024 • 77