How to use from
llama.cppInstall from WinGet (Windows)
winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf tzervas/lab-bitnet-b1.58-2B-4T-i2s# Run inference directly in the terminal:
llama cli -hf tzervas/lab-bitnet-b1.58-2B-4T-i2sBuild from source code
git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf tzervas/lab-bitnet-b1.58-2B-4T-i2s# Run inference directly in the terminal:
./build/bin/llama cli -hf tzervas/lab-bitnet-b1.58-2B-4T-i2sUse Docker
docker model run hf.co/tzervas/lab-bitnet-b1.58-2B-4T-i2sQuick Links
lab-bitnet BitNet-b1.58-2B-4T I2_S
Official Microsoft b1.58 I2_S GGUF (ggml-model-i2_s.gguf), not GGUF Q4.
Card targets: gpu5080 sm120, akula-prime sm86, gpu-1080ti sm61 (file is ~1.2 GB).
Layer-group ternary convert (scripts/layer_group_quant.py) checkpointed on gpu5080 against the bf16 safetensors.
See ggml-model-i2_s.quant.json.
- Downloads last month
- 48
Hardware compatibility
Log In to add your hardware
We're not able to determine the quantization variants.
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Install (macOS, Linux)
# Start a local OpenAI-compatible server with a web UI: llama serve -hf tzervas/lab-bitnet-b1.58-2B-4T-i2s# Run inference directly in the terminal: llama cli -hf tzervas/lab-bitnet-b1.58-2B-4T-i2s