Qwen2.5-1.5B — Slips IDS Security Summarization

Model Description

A fine-tuned version of Qwen2.5-1.5B-Instruct specialized for translating technical network security events from Slips IDS into clear, human-readable incident summaries with severity assessments.

Slips is a network intrusion detection system that generates DAG-structured alert logs — chains of related security events per source IP per time window. Raw Slips output is highly technical and difficult to interpret quickly. This model translates those logs into structured, concise summaries grouped by event type, with per-event severity labels (CRITICAL / HIGH / MEDIUM / LOW / INFO) and an overall severity breakdown.

The model was fine-tuned using SFT (Supervised Fine-Tuning) with best-of-N response selection: for each training incident, the highest-scoring response (judged by an LLM-as-judge) among GPT-4o, GPT-4o-mini, Qwen2.5 3B, and Qwen2.5 1.5B was selected as ground truth.

A smaller version with the same training data, stratosphere/qwen3.5-0.8b-slips-immune-summarization, scores on par with this model at about half the size and runs faster on CPU-only devices such as the Raspberry Pi 5.

Quick Start

Ollama (recommended for local deployment)

ollama run stratosphere/qwen2.5-1.5b-slips-immune-summarization
# or a specific quantization:
ollama run stratosphere/qwen2.5-1.5b-slips-immune-summarization:q5_k_m
ollama run stratosphere/qwen2.5-1.5b-slips-immune-summarization:q8_0

latest is the Q8_0 build. The Ollama tags set the default Qwen system prompt, an 8,192-token context (num_ctx) and an output limit of 1024 tokens (num_predict), matching the evaluation setup. Tags pulled before November 2026 lack these settings; pull again to update.

On CPU-only devices, llama.cpp (llama-server -c 8192 --temp 0 -fa off) runs the same GGUF quantizations about 1.5–2.5× faster than Ollama (see Speed on a Raspberry Pi 5).

Python (Transformers)

This model uses a merged prompt format: instructions and the DAG are combined into a single user message, with the incident metadata (taken from the DAG header) filled in both in the metadata block and in the output-format header. The chat template adds the default Qwen system prompt (You are Qwen, created by Alibaba Cloud. You are a helpful assistant.), as during training and evaluation.

from transformers import AutoTokenizer, AutoModelForCausalLM
import torch

model_id = "stratosphere/qwen2.5-1.5b-slips-immune-summarization"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype=torch.float16, device_map="auto")

dag_input = """
============================================================
Incident: abc123
Source IP: 192.168.1.100 | Timewindow: 5
Timeline: 2024-01-15 14:00:00 to 2024-01-15 15:00:00
Threat Level: 8.5 | Events: 42
...
"""  # paste your Slips DAG analysis here

user_message = f"""You are a security analyst. Your task is to translate technical security events into clear, concise, human-readable summaries and assess their severity.

INCIDENT METADATA:
- Incident ID: abc123
- Source IP: 192.168.1.100
- Timewindow: 5
- Accumulated Threat Level: 8.5
- Time Range: 2024-01-15 14:00:00 to 2024-01-15 15:00:00
- Total Events: 42

RAW EVENTS (Time | Description):
{dag_input}

YOUR TASK:
1. Transform the technical event descriptions into clear, readable summaries using plain language
2. Group identical or very similar events (e.g., 24 identical connections → one summary line)
3. Assess the severity of each event/group based on security impact:
   - CRITICAL: Active exploitation, data exfiltration, confirmed malware C2
   - HIGH: Scanning, suspicious connections, potential threats
   - MEDIUM: Anomalous but potentially benign behavior
   - LOW: Minor issues, likely false positives
   - INFO: Informational events, normal network behavior
4. Calculate the overall severity breakdown based on your assessments

OUTPUT FORMAT (match this structure exactly):

============================================================
Incident: abc123
Source IP: 192.168.1.100 | Timewindow: 5
Timeline: 2024-01-15 14:00:00 to 2024-01-15 15:00:00
Threat Level: 8.5 | Events: 42

• HH:MM-HH:MM - [Your clear grouped summary] [YOUR_ASSESSED_SEVERITY]
• HH:MM - [Your clear summary] [YOUR_ASSESSED_SEVERITY]

Total Evidence: 42 events
Severity breakdown: [Your calculated breakdown, e.g., "High: 5, Medium: 3, Info: 2"]

EXAMPLES OF GOOD SUMMARIZATION WITH SEVERITY ASSESSMENT:
- "Connection on port 0 from 0.0.0.0:0 to 224.0.0.1:0" → "IGMP multicast traffic to group address [INFO]"
- "Detected a horizontal port scan to port 443/TCP. 50 unique dst IPs" → "Port scanning 50 hosts on HTTPS port [HIGH]"
- "Connection to known C2 server 185.29.135.234:443" → "Direct connection to command & control server [CRITICAL]"
- "Connection without DNS resolution to CDN IP" → "Direct IP connection (likely CDN/API) [LOW]"

RULES:
- Group identical events into ONE line (don't list the same event 24 times)
- Use time ranges (HH:MM-HH:MM) when showing grouped events
- Assess severity based on security impact, not just event type
- Use severity levels: CRITICAL, HIGH, MEDIUM, LOW, INFO
- Keep descriptions clear and concise
- Just output the structured summary - no explanations or meta-commentary
- Do not include token counts or performance statistics"""

messages = [{"role": "user", "content": user_message}]

inputs = tokenizer.apply_chat_template(messages, return_tensors="pt", add_generation_prompt=True, return_dict=True).to(model.device)
output = model.generate(**inputs, max_new_tokens=1024, do_sample=False)
print(tokenizer.decode(output[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True))

LoRA Adapter

The LoRA adapter itself (r=64, alpha 64, RSLoRA) is in lora_adapter/, for inspection or further fine-tuning. It was trained on unsloth/qwen2.5-1.5b-instruct-unsloth-bnb-4bit (QLoRA). The published weights are this adapter merged into the 16-bit base; for this model, merging into the dequantized 4-bit base instead gives the same judge scores within noise.

Training Details

Dataset

The training data is publicly available at stratosphere/immune-summary-sft-dataset.

  • Source: 532 incidents from real Slips IDS network captures
  • Responses: 4 model responses per incident (GPT-4o, GPT-4o-mini, Qwen2.5 3B, Qwen2.5 1.5B) used as candidate labels, scored by an LLM-as-judge
  • Selection: Best-of-N — highest-scoring response per incident used as training target
  • Filtering: Responses with judge score < 4 or summary token length outside [50, 400] discarded
  • Split: 90% train / 10% eval (stratified, seed=42)

Training Procedure

Parameter Value
Base model unsloth/Qwen2.5-1.5B-Instruct
Training method SFT (Supervised Fine-Tuning)
Framework Unsloth + TRL SFTTrainer
LoRA rank (r) 64
LoRA alpha 64
LoRA dropout 0.0
RSLoRA enabled (required at r=64)
LoRA targets q_proj, k_proj, v_proj, o_proj, gate_proj, up_proj, down_proj
Sequence length 4096
Batch size 1 (effective: 16 via gradient accumulation)
Gradient accumulation steps 16
Learning rate 2e-5
LR scheduler cosine
Warmup steps 20
Weight decay 0.01
Epochs 3
Optimizer adamw_8bit
Precision BF16
Quantization (training) 4bit
Hardware A100 80GB MiG 20GB slice (e-infra.cz cloud)
Response masking train_on_responses_only — loss computed on assistant turns only

Framework Versions

  • Unsloth: 2026.3.18
  • Transformers: (auto-detected)
  • PyTorch: (auto-detected)

Evaluation

Overall Results

Evaluated on 47 held-out Slips IDS incidents (45 Malware, 2 Normal) with gpt-oss-120b as an independent LLM-as-judge, temperature 0. The judge scores each answer on its own (pointwise) against four criteria of 1–10 each: evidence coverage, severity assessment, grouping and conciseness, translation clarity (total out of 40). Summaries that leave template text unfilled (e.g. [YOUR_ASSESSED_SEVERITY]) are penalized. The quantized rows are GGUF builds of the published weights (the same quantization types as the Ollama tags, rebuilt with a current llama.cpp), run with the default system prompt, an 8,192-token context and a 1,024-token output limit, as the Ollama tags now do.

Model Summary score (/40)
GPT-4o-mini 20.74
GPT-4o 19.53
Qwen2.5-1.5B finetuned, Q8_0 18.19
Qwen3.5-0.8B finetuned, Q8_0 17.91
Qwen2.5-1.5B finetuned (4-bit adapter, as trained) 17.49
Qwen2.5-1.5B finetuned, Q5_K_M 16.79
Qwen2.5-1.5B finetuned, Q4_K_M 16.06
Qwen2.5-1.5B (base, untuned) 11.19

Fine-tuning raises the score from 11.2 to 16–18, close to GPT-4o. Q8_0 is the best quantization; at Q4_K_M, 5 of the 47 summaries left some template text unfilled.

Speed on a Raspberry Pi 5

Raspberry Pi 5 (8 GB RAM, 4 Cortex-A76 cores, no GPU), 5 median-length incidents (~2,260 input tokens), 8,192-token context, temperature 0. Same GGUF files and identical prompts in both runtimes.

Quantization llama.cpp Ollama 0.19 Peak RAM (llama.cpp / Ollama)
Q4_K_M 106 s per summary (prompt 49 tok/s, generation 7.2 tok/s) 181 s 2.5 / 1.3 GB
Q5_K_M 103 s 217 s 2.7 / 1.4 GB
Q8_0 124 s (prompt 35 tok/s, generation 5.0 tok/s) 194 s 3.7 / 1.9 GB

The Pi reached 83–84 °C and throttled during the Ollama Q4_K_M and Q5_K_M runs, so those two times are somewhat pessimistic. Most of the time goes into reading the prompt; llama.cpp does this 3× faster than Ollama on the Pi.

Known Limitations

  • Long incidents: training used a 4096-token sequence length; incidents with thousands of events produce DAGs that exceed this, and quality drops on them. Inference works up to 8,192 tokens, but very long incidents take several minutes on a Raspberry Pi.
  • Quantization: Q4_K_M scores about 2 points lower than Q8_0 and sometimes leaves template text unfilled; use Q8_0 when memory allows.
  • Normal traffic: only 2 Normal incidents in the test set — results for that category are not statistically reliable.

Intended Use

  • Automated triage of Slips IDS alerts for security analysts
  • First-pass summarization of network incident logs
  • Input to downstream reporting or ticketing workflows

Out-of-Scope Use

  • General-purpose chat or instruction following
  • Security domains outside network IDS (malware analysis, vulnerability scanning, etc.)
  • Non-English inputs

Acknowledgments

This work was supported by the NLnet Foundation as part of the IMMUNE project. NLnet Foundation promotes open internet standards and open source software.

Citation

@misc{qwen2.5-1.5b-slips-immune,
  title        = {Qwen2.5-1.5B fine-tuned for Slips IDS security summarization},
  author       = {Stratosphere Laboratory, CTU Prague},
  year         = {2026},
  howpublished = {\url{https://huggingface.co/stratosphere/qwen2.5-1.5b-slips-immune-summarization}}
}

Model Details

  • Model size: 1.5B params
  • Tensor type: FP16
  • License: Apache-2.0
  • Tags: Text Generation, Transformers, Safetensors, Network Security, IDS, SLIPS, Summarization, Cybersecurity, LoRA, SFT, TRL, Unsloth
Downloads last month
156
Safetensors
Model size
2B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for stratosphere/qwen2.5-1.5b-slips-immune-summarization

Adapter
(510)
this model

Dataset used to train stratosphere/qwen2.5-1.5b-slips-immune-summarization