Instructions to use JonasGeiping/stream-qwen3.5-27b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use JonasGeiping/stream-qwen3.5-27b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="JonasGeiping/stream-qwen3.5-27b", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("JonasGeiping/stream-qwen3.5-27b", trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained("JonasGeiping/stream-qwen3.5-27b", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use JonasGeiping/stream-qwen3.5-27b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "JonasGeiping/stream-qwen3.5-27b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JonasGeiping/stream-qwen3.5-27b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/JonasGeiping/stream-qwen3.5-27b

SGLang

How to use JonasGeiping/stream-qwen3.5-27b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "JonasGeiping/stream-qwen3.5-27b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JonasGeiping/stream-qwen3.5-27b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "JonasGeiping/stream-qwen3.5-27b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "JonasGeiping/stream-qwen3.5-27b",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use JonasGeiping/stream-qwen3.5-27b with Docker Model Runner:
```
docker model run hf.co/JonasGeiping/stream-qwen3.5-27b
```

nielsr HF Staff commited on May 14

Commit

ec2ff2f

verified ·

1 Parent(s): 8811223

Improve model card: add paper/code links and citation

Browse files

This pull request improves the model card by adding links to the official research paper "Multi-Stream LLMs: Unblocking Language Models with Parallel Streams of Thoughts, Inputs and Outputs" and the corresponding GitHub repository. It also includes a BibTeX citation section for academic reference.

Files changed (1) hide show

README.md +18 -6

README.md CHANGED Viewed

@@ -1,8 +1,12 @@
 ---
-license: apache-2.0
 language:
 - en
 library_name: transformers
 tags:
 - stream-llm
 - multi-stream
@@ -10,21 +14,19 @@ tags:
 - monitorability
 - qwen3.5
 - deltanet
-base_model: Qwen/Qwen3.5-27B
-datasets:
-- JonasGeiping/stream-data
-pipeline_tag: text-generation
 ---
 # Stream-Qwen3.5-27B
 A multi-stream variant of Qwen3.5-27B (DeltaNet hybrid) that generates in ten
 parallel streams (1 input, 1 visible output, 8 thinking channels) simultaneously per timestep. One forward pass
 produces the next-row token for each stream; tokens within a row cannot see
 each other (block-causal attention), but every stream can attend to every
 prior row's tokens.
-This model was trained for the monitoring experiments in Section 7 and to see whether whether we can train a generic instruction-tuned model with 8 internal streams, and still have it make sense.
 As such, the internal streams are not always helpful, but they are coherent, and do (in the best case) respond to each other and the user stream. Nevertheless this is still a research prototype model.
 ## Architecture
@@ -247,3 +249,13 @@ Trained on `JonasGeiping/stream-data` (see dataset card). Recipe details:
 - 2 epochs, packing depth 7, lr 2e-5, weight decay 1e-3, attention dropout 0.2
 - DeepSpeed ZeRO-2 across 4×80 GB GPUs
 - Per-group lr: 1e-5

 ---
+base_model: Qwen/Qwen3.5-27B
+datasets:
+- JonasGeiping/stream-data
 language:
 - en
 library_name: transformers
+license: apache-2.0
+pipeline_tag: text-generation
 tags:
 - stream-llm
 - multi-stream
 - monitorability
 - qwen3.5
 - deltanet
 ---
 # Stream-Qwen3.5-27B
+[[Paper](https://huggingface.co/papers/2605.12460)] [[Code](https://github.com/seal-rg/streaming)]
 A multi-stream variant of Qwen3.5-27B (DeltaNet hybrid) that generates in ten
 parallel streams (1 input, 1 visible output, 8 thinking channels) simultaneously per timestep. One forward pass
 produces the next-row token for each stream; tokens within a row cannot see
 each other (block-causal attention), but every stream can attend to every
 prior row's tokens.
+This model was trained for the monitoring experiments in Section 7 of the paper [Multi-Stream LLMs: Unblocking Language Models with Parallel Streams of Thoughts, Inputs and Outputs](https://huggingface.co/papers/2605.12460) and to see whether whether we can train a generic instruction-tuned model with 8 internal streams, and still have it make sense.
 As such, the internal streams are not always helpful, but they are coherent, and do (in the best case) respond to each other and the user stream. Nevertheless this is still a research prototype model.
 ## Architecture
 - 2 epochs, packing depth 7, lr 2e-5, weight decay 1e-3, attention dropout 0.2
 - DeepSpeed ZeRO-2 across 4×80 GB GPUs
 - Per-group lr: 1e-5
+## Citation
+```bibtex
+@article{su_2026_multi-stream,
+  title={Multi-Stream LLMs: Unblocking Language Models with Parallel Streams of Thoughts, Inputs and Outputs},
+  author={Su, Guinan and Yang, Yanwu and Li, Xueyan and Geiping, Jonas},
+  year={2026}
+}
+```