Instructions to use bunny0417/antares-1b-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use bunny0417/antares-1b-GGUF with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf bunny0417/antares-1b-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf bunny0417/antares-1b-GGUF:Q4_K_M

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf bunny0417/antares-1b-GGUF:Q4_K_M
# Run inference directly in the terminal:
llama cli -hf bunny0417/antares-1b-GGUF:Q4_K_M

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf bunny0417/antares-1b-GGUF:Q4_K_M
# Run inference directly in the terminal:
./build/bin/llama cli -hf bunny0417/antares-1b-GGUF:Q4_K_M

Use Docker

docker model run hf.co/bunny0417/antares-1b-GGUF:Q4_K_M

Llama new
LM Studio
Jan

vLLM

How to use bunny0417/antares-1b-GGUF with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "bunny0417/antares-1b-GGUF"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "bunny0417/antares-1b-GGUF",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/bunny0417/antares-1b-GGUF:Q4_K_M

Ollama
How to use bunny0417/antares-1b-GGUF with Ollama:
```
ollama run hf.co/bunny0417/antares-1b-GGUF:Q4_K_M
```
Unsloth Desktop

How to use bunny0417/antares-1b-GGUF with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf bunny0417/antares-1b-GGUF:Q4_K_M

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "bunny0417/antares-1b-GGUF:Q4_K_M"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use bunny0417/antares-1b-GGUF with Docker Model Runner:
```
docker model run hf.co/bunny0417/antares-1b-GGUF:Q4_K_M
```

Lemonade

How to use bunny0417/antares-1b-GGUF with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull bunny0417/antares-1b-GGUF:Q4_K_M

Run and chat with the model

lemonade run user.antares-1b-GGUF-Q4_K_M

List all available models

lemonade list

Hermes Agent

How to use bunny0417/antares-1b-GGUF with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf bunny0417/antares-1b-GGUF:Q4_K_M

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default bunny0417/antares-1b-GGUF:Q4_K_M

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use bunny0417/antares-1b-GGUF with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf bunny0417/antares-1b-GGUF:Q4_K_M

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "bunny0417/antares-1b-GGUF:Q4_K_M" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

This is a community GGUF conversion of fdtn-ai/antares-1b. By requesting access, you acknowledge the original model's license, access terms, intended use, and safety limitations. Do not use it for malicious activity, and use human oversight and a sandboxed environment for security workflows.

Antares-1B GGUF — Q4_K_M

This repository provides a community conversion of fdtn-ai/antares-1b to the Q4_K_M GGUF format for use with llama.cpp-compatible applications, including LM Studio. It is not an official Cisco or Foundation AI release, and it contains a format quantization only—no training, fine-tuning, or evaluation was performed by this repository's publisher.

The original model is access-gated. This derivative repository is also manually gated so that the source model's controlled-access intent is not bypassed.

Source and license

Source model: fdtn-ai/antares-1b
Pinned source revision: 10417eb35641b32e7141157db19c76eb545193b6
Source license: Apache-2.0, as declared by the source model card
Upstream model: Antares-1B, a terminal-agent model for vulnerability localization, built on IBM Granite 4.0 1B

The source model card, license, and access terms remain authoritative. This repository does not grant rights beyond those terms.

Files

File	Quantization	Size	SHA-256
`antares-1b-Q4_K_M.gguf`	Q4_K_M	1,139,251,168 bytes (1.06 GiB)	`e28fbe997146f84ccb680863321d2e14cdbe4b51986a28ac73b608fe858a958b`

The GGUF includes the converted tokenizer and chat template. No original checkpoint, Antares CLI ZIP, or unrelated files are included here.

Conversion provenance

Converted on Windows 11 with llama.cpp b10091 (b4d6c7d8ff69c2e05e4e8ee7e6e710a08abd7b45).
The pinned BF16 source checkpoint was converted to F16 GGUF with convert_hf_to_gguf.py --outtype f16.
The F16 artifact was quantized directly to Q4_K_M with llama-quantize. It was not quantized from another low-precision model.
The final GGUF was checked for the GGUF header and 363 non-empty tensors.
The file hash above was calculated after conversion.

Use with llama.cpp

Use the embedded chat template (-cnv) and start with the sampling values from the upstream model card:

llama-cli -m antares-1b-Q4_K_M.gguf -cnv -ngl 99 -c 4096 -n 512 \
  --temp 0.3 --top-p 1.0 \
  -p "You are a security vulnerability localization agent."

-ngl 99 offloads all compatible layers to the selected GPU. Reduce the context length or GPU layers if your available VRAM is limited.

LM Studio

Download the .gguf, then load it as a local GGUF model in LM Studio. Keep the model's embedded chat template enabled. The model was smoke-tested with full Vulkan offload on an AMD Radeon RX 6500 XT at a 4K context; this is only a loading/inference check, not a quality benchmark.

Intended use and safety

Antares-1B is intended for authorized vulnerability-localization work in a terminal-agent loop. It is not designed as a general-purpose chat model. Follow the upstream guidance: execute generated tool calls only inside an isolated, network-disabled sandbox; apply resource limits; and require qualified human review before making security or remediation decisions.

Do not use this model to develop malware, phishing, targeted attacks, or unauthorized exploitation. Respect applicable law, the upstream model terms, and the Apache-2.0 license.

Limitations

Quantization can change generation quality and performance. This conversion has not been benchmarked against the upstream checkpoint. Antares' vulnerability localization outputs are hypotheses for analyst review, not proof of a vulnerability or a replacement for testing and professional judgment.

Citation and acknowledgments

Please cite and credit the upstream Antares project when using this conversion:

Vijay, Priyanshu, et al. Antares: Foundation Models for Agentic Vulnerability Localization. Foundation AI, Cisco, 2026.

See the upstream model card for the full technical report, intended-use guidance, evaluation details, and contact information.

Downloads last month: -

GGUF

Model size

2B params

Architecture

granite

Hardware compatibility

4-bit

Model tree for bunny0417/antares-1b-GGUF

Base model

ibm-granite/granite-4.0-1b-base

Finetuned

ibm-granite/granite-4.0-1b

Finetuned

fdtn-ai/antares-1b

Quantized

(16)

this model