Duplicate from IFM/K2-Horizon-3.7B

Co-authored-by: Kevin Zhou <zhou94539@users.noreply.huggingface.co>

.gitattributes

@@ -0,0 +1,38 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+assets/k2-horizon-3-7b-training-loss-vs-tokens.png filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text
+k2-horizon-3.7b-benchmarks.png filter=lfs diff=lfs merge=lfs -text

APPENDIX.md

@@ -0,0 +1,483 @@
+# K2-Horizon-3.7B Technical Appendix
+
+This appendix contains the detailed architecture, checkpoint, training, deployment, hardware, evaluation, dataset, source-code, intended-use, and safety material moved from the full model card. The short [README](README.md) follows the common K2-Horizon release format.
+
+## Model Series Overview
+
+K2-Horizon is released as a family of checkpoints that share the same post-training recipe, release process, and user-facing deployment guidance while covering dense, MoE, and MoVA configurations. The table below summarizes the public family view; individual repositories contain the files and branches for one model size.
+
+| Model | Architecture | Stored parameters | Active parameters/token | Context length | Vocab size | Intended use |
+| --- | --- | --- | --- | --- | --- | --- |
+| K2-Horizon-0.9B | K2HorizonForCausalLM; dense decoder-only | 0.9B | 0.9B | 8,192 | 64,256 | Lightweight experimentation, local development, evaluation dry runs |
+| K2-Horizon-3.7B | XllmForCausalLM; dense decoder-only | 5.06B | 5.06B | 524,288 | 250,624 | Efficient research, evaluation, and single-node serving |
+| K2-Horizon-7B | XllmForCausalLM; dense decoder-only | 9.00B | 9.00B | 524,288 | 250,624 | General research, fine-tuning, and cost-conscious deployment |
+| K2-Horizon-32B | K2HorizonForCausalLM; dense decoder-only | 34.78B | 34.78B | 524,288 | 250,624 | Stronger long-context and reasoning experiments |
+| K2-Horizon-36B | K2HorizonForCausalLM; MoVA + MoE | 37.44B | 5.95B | 524,288 | 250,624 | Multistage research and production-style serving experiments |
+| K2-Horizon-375B | XllmForCausalLM; sparse MoE | 379.17B | 26.67B | 524,288 | 250,624 | Frontier-scale open research, long-context evaluation, and high-capacity serving |
+
+Tokenizer files are distributed with each model repository. The compact K2-Horizon-0.9B model comes from the IFM Mobile line and uses a 64,256-token vocabulary. The 3.7B, 7B, 32B, 36B, and 375B models use a 250,624-token vocabulary.
+
+The native 0.9B `main`, 32B `mid_4`, and 36B `main`/`mid_4` revisions publish `K2HorizonForCausalLM`, `model_type: k2_horizon`, and matching `configuration_k2_horizon.py` and `modeling_k2_horizon.py` modules. The deployment preflights below validate that public contract before loading weights.
+
+## This Repository
+
+| Field | Value |
+| --- | --- |
+| Model name | K2-Horizon-3.7B |
+| Repository | IFM/K2-Horizon-3.7B |
+| Model family | K2-Horizon |
+| Architecture class | XllmForCausalLM |
+| Classification | Dense |
+| Stored parameters | 3.78B core; 5.06B including embeddings |
+| Active parameters per token | 3.78B core; 5.06B including embeddings |
+| Context length | 512K tokens |
+| Transformer layers | 36 |
+| Hidden size | 2,560 |
+| Dense / expert intermediate size | 10,240 / 0 |
+| Attention heads / KV heads | 32 / 8 |
+| Dense attention / sparse attention blocks | 36 / 0 |
+| Dense FFN / sparse FFN blocks | 36 / 0 |
+| MoE experts / active experts | 0 / 0 |
+| MoVA count / active values | 0 / 0 |
+| Vocab size | 250,624 |
+| Tokenizer metadata | Hugging Face tokenizer files; vocab size 250,624 |
+| Checkpoint storage dtype | float32 (FP32) |
+| Checkpoint config dtype metadata | float32 |
+| Evaluation and sample dtype | bfloat16 (BF16) |
+| Shard/index summary | 36 shards |
+
+## Checkpoint Revisions
+
+Each repository uses branches for model stages and stable tags for specific checkpoints. Pin a revision tag when reproducibility matters; use `main` for the default model checkpoint.
+
+| Stage | Branch | Revision tag | Max context | Description |
+| --- | --- | --- | --- | --- |
+| Base / pretraining | `pretrain` | `base_final` | 8K tokens | Base model checkpoint |
+| Midtraining 1 | `mid_1` | `mid_1_final` | 32K tokens | Midtraining 1 checkpoint |
+| Midtraining 2 | `mid_2` | `mid_2_final` | 128K tokens | Midtraining 2 checkpoint |
+| Midtraining 3 | `mid_3` | `mid_3_final` | 512K tokens | Midtraining 3 checkpoint |
+| Midtraining 4 | `mid_4` | `mid_4_final` | 512K tokens | Midtraining 4 checkpoint |
+| Default | `main` | `main` | 512K tokens | Default release view of `mid_4_final` |
+
+Example revision pinning:
+
+<details>
+<summary>Show revision pinning example</summary>
+
+```python
+# Validated with Python 3.14,
+# Transformers 4.57.6, and
+# Safetensors 0.8.0.
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "IFM/K2-Horizon-3.7B"
+revision = "mid_4_final"
+
+tokenizer = AutoTokenizer.from_pretrained(model_id, revision=revision, trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id,
+    revision=revision,
+    device_map="auto",
+    dtype="bfloat16",
+    low_cpu_mem_usage=True,
+    trust_remote_code=True,
+)
+```
+
+</details>
+
+
+
+## Training Provenance
+
+Each row identifies the last W&B process record associated with the selected checkpoint, together with its Slurm job and exact xLLM source revision. A training stage can span multiple W&B process records when checkpointed training is resumed.
+
+| Stage | Revision | Checkpoint context | xLLM source | W&B last run | Slurm job | Training nodes |
+| --- | --- | --- | --- | --- | --- | --- |
+| Base / pretraining | `base_final` | 8K tokens | [`ad6dac16`](https://github.com/LLM360/xllm/commit/ad6dac16031ae74d3e02c10bf3f37384815503a2) | [`k2v3-4b/aici7hvs`](https://wandb.ai/mbzuai-llm/k2v3-4b/runs/aici7hvs) | `423852` | 38 |
+| Midtraining 1 | `mid_1_final` | 32K tokens | [`b69a6b2e`](https://github.com/LLM360/xllm/commit/b69a6b2ed17091dfee8af68a570f78816f765bad) | [`k2v3_midtrain/um64dwov`](https://wandb.ai/mbzuai-llm/k2v3_midtrain/runs/um64dwov) | `498823` | 38 |
+| Midtraining 2 | `mid_2_final` | 128K tokens | [`b69a6b2e`](https://github.com/LLM360/xllm/commit/b69a6b2ed17091dfee8af68a570f78816f765bad) | [`k2v3_midtrain/axxd0ylh`](https://wandb.ai/mbzuai-llm/k2v3_midtrain/runs/axxd0ylh) | `504283` | 38 |
+| Midtraining 3 | `mid_3_final` | 512K tokens | [`198c2fb6`](https://github.com/LLM360/xllm/commit/198c2fb61457529d7d790550acd0f937da931112) | [`k2v3_midtrain/9qm67nzz`](https://wandb.ai/mbzuai-llm/k2v3_midtrain/runs/9qm67nzz) | `966354` | 38 |
+| Midtraining 4 | `mid_4_final` | 512K tokens | [`5494c84c`](https://github.com/LLM360/xllm/commit/5494c84cc7a2a66e5d6439bf9aa07454097ce9be) | [`k2v3_midtrain/0fhhm8wn`](https://wandb.ai/mbzuai-llm/k2v3_midtrain/runs/0fhhm8wn) | `980739` | 38 |
+
+
+
+## Training Loss
+
+The chart reconstructs the release checkpoint lineage for Pretraining, Midtraining 1, Midtraining 2, Midtraining 3, Midtraining 4. Training loss is the next-token prediction objective; lower values indicate that the model assigns greater probability to the observed next token. Training loss is not a validation score and should not be compared across different tokenizers or data mixtures without additional controls.
+
+![K2-Horizon-3.7B training loss versus cumulative training tokens](./assets/k2-horizon-3-7b-training-loss-vs-tokens.png)
+
+Each panel combines the W&B process records that contribute to the release checkpoint. Faint lines show sampled raw training loss; bold lines show an unpadded centered moving average with a raised-cosine taper over the final half-window. The final sample retains 20% of its base weight, limiting right-edge bias without hiding the raw tail. Checkpoint restart overlap is replaced by the later process record.
+
+
+## Deployment Guide
+
+These are base checkpoints. They can be served for research, evaluation, retrieval-augmented generation experiments, tool-use research, and downstream adaptation. They are not instruction-aligned assistants by default, so production deployments should add application-level safety filters, system prompting, and task-specific validation.
+
+All reported K2-Horizon evaluation used BF16. Some released checkpoint shards and configuration metadata remain FP32; the examples below explicitly load or serve them as BF16 to match the evaluation runtime and reduce accelerator memory usage.
+
+### Validated Independent Dependency Versions
+
+Use a separate Python environment for every workflow. Direct Transformers loading, xLLM conversion, vLLM, and the pinned SGLang production revision have independent dependency contracts. Do not install one row into another row's environment. Versions intentionally differ when a framework or remote-model contract requires them.
+
+| Workflow | Python | PyTorch | CUDA | Transformers | Safetensors | Attention kernel | Framework revision |
+| --- | --- | --- | --- | --- | --- | --- | --- |
+| Direct Transformers loading | 3.14 | 2.13.0 | 12.6 | 4.57.6 | 0.8.0 | PyTorch SDPA 2.13.0 | Transformers 4.57.6 remote code |
+| xLLM checkpoint conversion | 3.12 | 2.8.0 | 12.8 | 4.57.3 | 0.8.0 | FlashAttention 2.8.3.post1 | xLLM `7cfa3b2c` |
+| vLLM serving | 3.12 | 2.13.0 | 12.9 | 5.16.1 | 0.8.0 | FlashInfer 0.6.17 | vLLM 0.26.1rc1.dev1212 source `d9fd5f11` (PR #53806) |
+| SGLang serving | 3.12 | 2.9.1 | 12.8 (CUDA Python 12.9) | 5.3.0 | 0.8.0 | FA3 (sglang-kernel 0.4.1); FlashInfer 0.6.7.post2 available | SGLang `ffb41d77` (stable-only metadata) |
+
+### Inference Parameter Starting Points
+
+K2-Horizon does not use one family-wide sampling preset. The values below are operational baselines: temperature 0 is the validated deterministic serving comparison, while sampled settings are conservative starting points that should be tuned on application prompts. Benchmark reproduction must use the decoding configuration attached to that benchmark.
+
+| Use case | Temperature | Top-p | Max new tokens | Notes |
+| --- | --- | --- | --- | --- |
+| Checkpoint smoke test and deterministic evaluation | 0.0 | 1.0 | 512-4,096 | Greedy decoding is the reproducible baseline used by serving comparisons. |
+| Base-model continuation | 0.6 | 0.95 | 1,024-4,096 | A conservative sampling baseline; tune on a held-out prompt set. |
+| Long-context continuation or summarization | 0.2 | 0.9 | 2,048-8,192 | Start with one request and increase context only after KV-cache capacity checks. |
+| Reasoning and tool-call validation | 0.0 | 1.0 | 4,096 | Use a compatible chat template and the K2Horizon parser stack below. |
+
+### Transformers
+
+The selected default checkpoint is loaded with Transformers 4.57.6 and PyTorch 2.13.0 on CUDA 12.6. Select the corresponding PyTorch build for your accelerator when that CUDA runtime is unavailable. The selected checkpoint uses the self-contained legacy `XllmForCausalLM` remote implementation. Its saved Transformers 4.57.3 contract is compatible with stable 4.57.6, PyTorch SDPA, and no external xLLM or FlashAttention package. xLLM conversion remains isolated below because its native build has a different runtime contract.
+
+<details>
+<summary>Show Transformers installation commands</summary>
+
+```bash
+python -m pip install "torch==2.13.0" \
+  --index-url "https://download.pytorch.org/whl/cu126"
+python -m pip install \
+  "transformers==4.57.6" \
+  "safetensors==0.8.0" \
+  "accelerate==1.14.0"
+```
+
+</details>
+
+<details>
+<summary>Show Transformers inference example</summary>
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+MODEL_ID = "IFM/K2-Horizon-3.7B"
+REVISION = "mid_4"
+
+tokenizer = AutoTokenizer.from_pretrained(
+    MODEL_ID,
+    revision=REVISION,
+    trust_remote_code=True,
+)
+model = AutoModelForCausalLM.from_pretrained(
+    MODEL_ID,
+    revision=REVISION,
+    device_map="auto",
+    dtype="bfloat16",
+    low_cpu_mem_usage=True,
+    trust_remote_code=True,
+)
+
+prompt = "Explain why long-context evaluation is difficult."
+inputs = tokenizer(prompt, return_tensors="pt").to(model.device)
+inputs.pop("token_type_ids", None)
+outputs = model.generate(
+    **inputs,
+    max_new_tokens=512,
+    temperature=0.7,
+    top_p=0.9,
+    do_sample=True,
+)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+</details>
+
+### xLLM
+
+xLLM provides the training framework, checkpoint converter, serving bridges, and runtime kernels used by the XLLM-based K2-Horizon repositories. For checkpoint conversion, parity work, or loading a legacy 36B `XllmForCausalLM` stage that imports xLLM MoVA kernels, use the isolated legacy contract and reviewed bridge revision [`7cfa3b2c`](https://github.com/LLM360/xllm/commit/7cfa3b2c61fad3b2ef48f7f7b497ef360087c72f):
+
+<details>
+<summary>Show xLLM setup and conversion commands</summary>
+
+```bash
+git clone https://github.com/LLM360/xllm.git
+cd xllm
+git checkout 7cfa3b2c61fad3b2ef48f7f7b497ef360087c72f
+
+python -m pip install "torch==2.8.0" \
+  --index-url "https://download.pytorch.org/whl/cu128"
+export XLLM_BUILD_ROOT="${XLLM_BUILD_ROOT:-$PWD/.xllm-build}"
+export TMPDIR="$XLLM_BUILD_ROOT/tmp"
+export PIP_CACHE_DIR="$XLLM_BUILD_ROOT/pip-cache"
+mkdir -p "$TMPDIR" "$PIP_CACHE_DIR"
+python -m pip install packaging ninja
+python -m pip install "https://github.com/Dao-AILab/flash-attention/releases/download/v2.8.3.post1/flash_attn-2.8.3.post1+cu12torch2.8cxx11abiTRUE-cp312-cp312-linux_x86_64.whl"
+python -m pip install -r requirements.txt
+python -m pip install -e . --no-build-isolation \
+  --config-settings editable_mode=compat
+python -m pip install \
+  "transformers==4.57.3" \
+  "safetensors==0.8.0" \
+  accelerate fire
+
+python xllm_bridges/huggingface/xllm_to_hf_main.py --help
+```
+
+</details>
+
+The converter requires an original xLLM distributed checkpoint and tokenizer. Released Hugging Face revisions are already converted and use the pure direct Transformers/SDPA contract above. Install xLLM only for conversion, parity work, or a model-specific native-kernel dependency. Pinning a revision does not make runtime contracts interchangeable.
+
+### vLLM
+
+The example pins [vLLM PR #53806](https://github.com/vllm-project/vllm/pull/53806) at [`d9fd5f11`](https://github.com/vllm-project/vllm/commit/d9fd5f11423a1a5628fe29e7296ceb9de91aac3c). That source registers `K2HorizonForCausalLM` plus the built-in `k2_horizon` reasoning and tool parsers. The command uses vLLM's `transformers` model implementation for this checkpoint contract.
+
+<details>
+<summary>Show vLLM setup and serving commands</summary>
+
+```bash
+git clone --filter=blob:none --no-checkout \
+  https://github.com/vllm-project/vllm.git
+cd vllm
+git fetch origin \
+  pull/53806/head:refs/remotes/origin/pr-53806
+git checkout --detach d9fd5f11423a1a5628fe29e7296ceb9de91aac3c
+test "$(git rev-parse HEAD)" = "d9fd5f11423a1a5628fe29e7296ceb9de91aac3c"
+
+python3.12 -m venv .venv
+source .venv/bin/activate
+python -m pip install --upgrade pip uv
+export UV_LINK_MODE=copy
+VLLM_USE_PRECOMPILED=1 uv pip install --upgrade --editable . \
+  --torch-backend=auto
+uv pip install \
+  "transformers==5.16.1" \
+  "safetensors==0.8.0"
+python -m pip check
+
+python - <<'PY'
+from vllm import ModelRegistry
+from vllm.reasoning import ReasoningParserManager
+from vllm.tool_parsers import ToolParserManager
+
+assert "K2HorizonForCausalLM" in ModelRegistry.get_supported_archs()
+assert ReasoningParserManager.get_reasoning_parser("k2_horizon") is not None
+assert ToolParserManager.get_tool_parser("k2_horizon") is not None
+PY
+
+export MODEL_ID="IFM/K2-Horizon-3.7B"
+export MODEL_REVISION="mid_4"
+export TENSOR_PARALLEL_SIZE=1
+export MAX_MODEL_LEN=131072
+
+vllm serve "$MODEL_ID" \
+  --revision "$MODEL_REVISION" \
+  --model-impl transformers \
+  --tensor-parallel-size "$TENSOR_PARALLEL_SIZE" \
+  --trust-remote-code \
+  --dtype bfloat16 \
+  --served-model-name "$MODEL_ID" \
+  --max-model-len "$MAX_MODEL_LEN" \
+  --reasoning-parser k2_horizon \
+  --enable-auto-tool-choice \
+  --tool-call-parser k2_horizon
+```
+
+</details>
+
+The source checkout is immutable at `d9fd5f11423a1a5628fe29e7296ceb9de91aac3c`. The command serves `mid_4`; the repository's longest checkpoint configuration reports 524,288 tokens, while this example starts at 131,072 tokens. Increase context and concurrency only after measuring KV-cache capacity. The preflight fails immediately if the native architecture or either Horizon parser registration is absent.
+
+### SGLang
+
+Native K2 Horizon support is provided by
+[sgl-project/sglang#37654](https://github.com/sgl-project/sglang/pull/37654).
+Use a `lmsysorg/sglang:dev` image built after that PR is merged. Once support is
+included in a tagged SGLang release, use the corresponding versioned image.
+
+<details>
+<summary>Show the SGLang serving command</summary>
+
+```bash
+docker run --gpus all \
+  --shm-size 32g \
+  -p 30000:30000 \
+  -v ~/.cache/huggingface:/root/.cache/huggingface \
+  --ipc=host \
+  lmsysorg/sglang:dev \
+  python3 -m sglang.launch_server \
+    --model-path "IFM/K2-Horizon-3.7B" \
+    --revision main \
+    --tp 1 \
+    --dtype bfloat16 \
+    --context-length 131072 \
+    --attention-backend fa3 \
+    --reasoning-parser k2_horizon \
+    --tool-call-parser k2_horizon \
+    --mem-fraction-static 0.85 \
+    --host 0.0.0.0 \
+    --port 30000
+```
+
+</details>
+
+This uses SGLang's native `K2HorizonForCausalLM` implementation; no
+`--trust-remote-code`, source patch, or external parser plugin is required. The
+131,072-token limit is a conservative starting point; increase it only after
+measuring KV-cache capacity.
+
+### Reasoning and Tool Parsers
+
+Parser configuration is part of the deployment contract. The canonical K2-Horizon format uses `<ifm|...>` reasoning and tool tokens; test streaming and non-streaming responses before exposing an OpenAI-compatible endpoint.
+
+| Goal | vLLM guidance | SGLang guidance | When to use |
+| --- | --- | --- | --- |
+| Plain text generation | Do not enable a reasoning or tool parser. | Do not enable a reasoning or tool parser. | Base-model evaluation, completion-style generation, and checkpoint smoke tests. |
+| Canonical K2-Horizon reasoning | Use the built-in `--reasoning-parser k2_horizon`. | Use `--reasoning-parser k2_horizon`. | Checkpoints and chat templates using `<ifm|think>`, `<ifm|think_fast>`, or `<ifm|think_faster>`. |
+| Canonical K2-Horizon tool calls | Use `--enable-auto-tool-choice --tool-call-parser k2_horizon`. | Use `--tool-call-parser k2_horizon`. | XML, typed-XML, or JSON calls wrapped in `<ifm|tool_call>`. |
+| Legacy K2 reasoning traces | Use the legacy parser only in a compatibility image that registers it. | Use `--reasoning-parser k2_horizon_legacy`. | Older traces using `<think>`, `<think_fast>`, or `<think_faster>` instead of canonical IFM tokens. |
+| Mixed tool-call corpora | Use `multi_format` only in a build that includes and tests that parser. | Use `--tool-call-parser multi_format`. | Evaluation corpora that intentionally mix K2, GLM, or JSON tool-call dialects. |
+
+PR #53806 registers both Horizon parsers in-tree; no external parser plugin is required.
+
+For OpenAI-compatible requests, select reasoning effort through chat-template kwargs:
+
+<details>
+<summary>Show OpenAI-compatible request example</summary>
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")
+response = client.chat.completions.create(
+    model="IFM/K2-Horizon-3.7B",
+    messages=[{"role": "user", "content": "Explain the result step by step."}],
+    temperature=0,
+    max_tokens=4096,
+    extra_body={"chat_template_kwargs": {"reasoning_effort": "high"}},
+)
+print(response.choices[0].message)
+```
+
+</details>
+
+### Other Serving Frameworks
+
+Text Generation Inference, TensorRT-LLM, llama.cpp, Ollama, and LM Studio may require model-specific conversion, quantization, or custom architecture support. Validate tokenizer behavior, remote-code requirements, attention implementation, and long-context behavior before publishing service results.
+
+
+## Hardware Planning
+
+The table below gives capacity-planning starting points for self-hosted inference. Weight memory uses the parameter count including input and output embeddings. On-disk FP32 shards and dtype conversion can require substantially more host RAM during loading than the final BF16 GPU footprint. KV-cache estimates use BF16 keys and values for one request at the smaller of the model maximum and 128K tokens; batching, allocator overhead, activations, CUDA graphs, and framework workspaces require additional memory.
+
+| Model | Stored params | Active params/token | BF16 weight memory | FP32 weight memory | BF16 KV cache / request | Starting TP | Recommended starting point |
+| --- | --- | --- | --- | --- | --- | --- | --- |
+| K2-Horizon-0.9B | 0.9B | 0.9B | 2.0 GiB | 4.0 GiB | 0.4 GiB @ 8K tokens | 1 | 1 GPU with 16 GiB is a practical starting point for the model's 8K context; allow additional memory for batching. |
+| K2-Horizon-3.7B | 5.06B | 5.06B | 9.4 GiB | 18.8 GiB | 18.0 GiB @ 128K tokens | 1 | 1 GPU with 24 GiB can serve short contexts in BF16; use 48-80 GiB for larger KV caches and 128K-class requests. |
+| K2-Horizon-7B | 9.00B | 9.00B | 16.8 GiB | 33.5 GiB | 18.0 GiB @ 128K tokens | 1 | 1 GPU with 40-48 GiB is suitable for short-context BF16 serving; 80 GiB is the practical starting point for 128K-class requests. |
+| K2-Horizon-32B | 34.78B | 34.78B | 64.8 GiB | 129.6 GiB | 32.0 GiB @ 128K tokens | 2-4 | 1x 80 GiB fits BF16 weights only at short context; start with 2x 80 GiB for 128K requests and 4x 80 GiB for larger context or throughput. |
+| K2-Horizon-36B | 37.44B | 5.95B | 69.7 GiB | 139.5 GiB | 24.0 GiB @ 128K tokens | 2-4 | Use at least 256 GiB of host RAM to load the current 139.5 GiB FP32 shard set. Start with 2x 80 GiB GPUs for BF16 weight sharding, or 1x H200 141 GiB for short single-request validation; use 4x 80 GiB for 128K-class requests. |
+| K2-Horizon-375B | 379.17B | 26.67B | 706.3 GiB | 1.38 TiB | 30.5 GiB @ 128K tokens | 8 H200 or 16x 80 GiB | The full 512K profile is validated on 8x H200 141 GiB GPUs at TP=8 and 0.92 memory utilization; use at least 16x 80 GiB GPUs on lower-memory hardware. |
+
+## Evaluation
+
+Benchmark values are reported as percentages unless otherwise noted. A dash means the metric was not reported for that model or revision in the release evaluation records. The family tables below use each model's latest listed midtraining release checkpoint.
+
+### Family Evaluation: Knowledge and Instruction Following
+
+| Model | Revision | IFEval | MMLU-Pro | MMLU | BBH |
+| --- | --- | --- | --- | --- | --- |
+| K2-Horizon-0.9B | `mid_2_final` | 43.1 | 40.3 | 44.4 | 43.8 |
+| K2-Horizon-3.7B | `mid_4_final` | 85.7 | 73.0 | 80.7 | 84.4 |
+| K2-Horizon-7B | `mid_4_final` | 89.9 | 75.7 | 83.3 | 87.8 |
+| K2-Horizon-32B | `mid_4_final` | 88.0 | 79.5 | 87.5 | 89.1 |
+| K2-Horizon-36B | `mid_4_final` | 89.6 | 79.2 | 87.3 | 88.6 |
+| K2-Horizon-375B | `mid_4_final` | 92.1 | 82.6 | 89.5 | 87.5 |
+
+### Family Evaluation: STEM, Reasoning, and Coding
+
+| Model | Revision | AIME26 | GPQA-Diamond | GSM8K | MBPP |
+| --- | --- | --- | --- | --- | --- |
+| K2-Horizon-0.9B | `mid_2_final` | 12.3 | 27.8 | 53.1 | 53.2 |
+| K2-Horizon-3.7B | `mid_4_final` | 88.2 | 73.3 | 92.3 | 86.4 |
+| K2-Horizon-7B | `mid_4_final` | 87.6 | 73.9 | 95.1 | 91.0 |
+| K2-Horizon-32B | `mid_4_final` | 91.7 | 75.4 | 95.5 | 87.8 |
+| K2-Horizon-36B | `mid_4_final` | 92.7 | 75.4 | 95.2 | 87.2 |
+| K2-Horizon-375B | `mid_4_final` | 92.5 | 84.9 | 95.1 | 93.8 |
+
+### Family Evaluation: Agentic and Long-Context Tasks
+
+| Model | Revision | SWE-Bench Verified (avg@3) | Terminal-Bench 2.1 (avg@3) | GDPval (acc@75%) | Tau3 Banking | BrowseComp+ (acc@1) | Apex Agents (pass@1) | WildClawBench | LCR |
+| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
+| K2-Horizon-7B | `mid_4_final` | 57.5 | 28.1 | 34.5 | 10.4 | 52.2 | 6.0 | 23.9 | - |
+| K2-Horizon-375B | `mid_4_final` | 67.8 | 38.2 | 39.6 | 11.4 | 58.1 | 13.8 | 34.8 | 56.0 |
+
+Agentic results are joined only to exact selected release checkpoints. SWE-Bench Verified and Terminal-Bench 2.1 report three-run averages where available; GDPval reports accuracy at 75 percent; BrowseComp+ reports accuracy at one; Apex Agents reports pass at one. Fractional workbook values are converted to percentages. When a source cell contains multiple reported aggregates, the first aggregate is shown unless the cell contains only three slash-separated trial values, in which case their mean is reported. A dash means the exact release checkpoint has no reported result for that benchmark.
+
+### Family Evaluation: Advanced Research Benchmarks
+
+| Model | Revision | MCP Atlas | HLE | SciCode | OmniScience Accuracy | OmniScience Non-Hallucination | CritPT |
+| --- | --- | --- | --- | --- | --- | --- | --- |
+| K2-Horizon-7B | `mid_4_final` | 53.6 | 13.8 | - | - | 43.7 | - |
+| K2-Horizon-375B | `mid_4_final` | 59.0 | 21.2 | 40.3 | 20.6 | 49.2 | 5.1 |
+
+These results use exact final-stage checkpoint records. Benchmark availability differs by model, so unreported cells remain `-`.
+
+### This Repository: Base / Pretraining Snapshot
+
+| Revision | ArabicMMLU | ARC-C | BBH | GPQA-Diamond-CoT | GSM8K | MMLU | MMLU-Pro | HumanEval | MBPP | IFEval |
+| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
+| `base_final` | 61.4 | 60.8 | 71.4 | 13.6 | 70.8 | 66.4 | 41.1 | 51.8 | 48.6 | 23.8 |
+
+### This Repository: Midtraining Snapshot
+
+| Revision | IFEval avg | MMLU-Pro | GSM8K | MMLU | BBH | MBPP | AIME26 | GPQA-Diamond | LongBench-v2 | LCR | SWE-Bench Verified | Terminal Bench 2 |
+| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
+| `mid_1_final` | 84.6 | 72.2 | 95.2 | 80.7 | 84.9 | 88.8 | 71.5 | 52.0 | - | - | - | - |
+| `mid_2_final` | 85.9 | 71.9 | 93.7 | 80.9 | 84.5 | 84.6 | 87.8 | 60.1 | - | - | - | - |
+| `mid_3_final` | 83.3 | 71.9 | 93.9 | 81.0 | 84.5 | 84.6 | 87.9 | 61.9 | - | - | - | - |
+| `mid_4_final` | 85.7 | 73.0 | 92.3 | 80.7 | 84.4 | 86.4 | 88.2 | 73.3 | - | - | - | - |
+
+
+
+### This Repository: RULER Long-Context Results
+
+| Revision | 4K | 8K | 16K | 32K | 64K | 128K |
+| --- | --- | --- | --- | --- | --- | --- |
+| `mid_1_final` | 94.5 (13) | 92.7 (13) | 92.7 (13) | - | - | - |
+
+RULER scores are macro-averages over the non-empty task columns in the exact checkpoint row. The number of contributing tasks is shown in parentheses.
+
+
+## Datasets and Mixtures
+
+K2-Horizon training is organized around transparent data releases:
+
+- Pretraining data: [IFM/K2-Horizon-Pretrain-Data](https://huggingface.co/datasets/IFM/K2-Horizon-Pretrain-Data), derived from the TxT360 lineage.
+- Midtraining data: [IFM/K2-Horizon-Midtrain-Data](https://huggingface.co/datasets/IFM/K2-Horizon-Midtrain-Data), following the TxT360-Midas reasoning and long-context release pattern.
+- The public data-preparation lineage is maintained in [LLM360/TxT360](https://github.com/LLM360/TxT360).
+
+## Source Code
+
+- [LLM360/xllm](https://github.com/LLM360/xllm) - Training framework and Hugging Face/vLLM bridges. Per-stage training commits are linked in the provenance table.
+- [LLM360/Eval360-V2 at `f5081bfb`](https://github.com/LLM360/Eval360-V2/commit/f5081bfba9ef8e7b380f5bd85ed1cf61a91f90b0) - Evaluation harness revision reviewed against the release benchmark workbooks.
+- [LLM360/agent-evals at `f3788399`](https://github.com/LLM360/agent-evals/commit/f3788399a974e49121984f11556138f23f907cee) - Agentic benchmark harness, Slurm launchers, and serving profiles used to reconcile the agentic release results.
+- [vLLM PR #53806 at `d9fd5f11`](https://github.com/vllm-project/vllm/commit/d9fd5f11423a1a5628fe29e7296ceb9de91aac3c) - Native K2-Horizon architecture plus built-in reasoning and tool parsers used by the serving example.
+- [LLM360/Inference360 at `a7ee94f9`](https://github.com/LLM360/Inference360/commit/a7ee94f9ad9a561e65a67ede22d4e7a2c8428b57) - Serving-image and deployment-validation references.
+- [LLM360/sglang at `ffb41d77`](https://github.com/LLM360/sglang/commit/ffb41d77d8b96b8a953351cb4d66d6bf4ed061a7) - SGLang serving base used by the hash-guarded K2-Horizon namespace migration.
+- [LLM360/TxT360](https://github.com/LLM360/TxT360) - Public data-preparation and source-data lineage reference.
+
+## Intended Use
+
+K2-Horizon-3.7B is intended for open research on foundation models, transparent evaluation, staged checkpoint analysis, downstream fine-tuning, and reproducibility studies. It is suitable for researchers and developers who want to inspect training-stage behavior, serve a model privately, compare model sizes, or build downstream adaptations.
+
+## Limitations and Safety
+
+- This is a base checkpoint family and may produce inaccurate, biased, unsafe, or hallucinated content.
+- The model is not optimized for moderation, refusal behavior, or production assistant safety.
+- Long-context behavior depends on serving stack, prompt construction, retrieval quality, and KV-cache configuration.
+- Tool-use and agentic results depend heavily on parser format, prompt template, tool schema, and execution harness.
+- The model family is documented primarily with English-language evaluations; additional multilingual evaluation should be performed before broad multilingual claims.
+- Users are responsible for complying with the model license and for validating deployments in their own risk domain.
+

README.md

@@ -0,0 +1,224 @@
+---
+pipeline_tag: text-generation
+library_name: transformers
+model_name: K2-Horizon-3.7B
+language:
+  - en
+license: apache-2.0
+datasets:
+  - IFM/K2-Horizon-Pretrain-Data
+  - IFM/K2-Horizon-Midtrain-Data
+tags:
+  - k2-horizon
+  - 3.7b
+  - dense
+  - open-weights
+  - ifm
+---
+
+# K2-Horizon-3.7B
+
+
+K2-Horizon-3.7B is the small dense member of the K2-Horizon family: a 3.7B-core decoder-only model with a 512K context window.
+
+<p align="center">
+  <img src="assets/k2-horizon-3.7b-benchmarks.png" alt="K2-Horizon-3.7B benchmark results" width="100%">
+</p>
+
+## K2-Horizon-3.7B Highlights
+
+- **Strong small-model baseline.** A dense model evaluated on the same agentic, coding, and reasoning benchmarks as the rest of the family.
+- **512K context.** Native 524,288-token context from the midtraining stages onward.
+- **Intermediate checkpoints.** Intermediate checkpoints are released so capability changes can be studied across training rather than at a single checkpoint.
+- **Fully open.** Training data and recipe, training code, and evaluation resources are public.
+
+## Benchmark Results
+
+The chart at the top of this card shows K2-Horizon-3.7B against selected reference models. The table below lists every comparison model used in the figure.
+
+### Full Results
+
+<!-- TABLE:START -->
+<div style="font-family:-apple-system,BlinkMacSystemFont,'Segoe UI',Roboto,sans-serif;margin:0 auto;padding:8px 0 16px;overflow-x:auto"><table style="display:table;width:100%;table-layout:fixed;border-collapse:collapse;font-size:12px;margin:0"><thead><tr><th style="width:30%;border-bottom:none"></th><th colspan="5" style="padding:6px 4px 2px;text-align:center;font-size:11px;font-weight:600;letter-spacing:0.04em;text-transform:uppercase;opacity:0.65;border-bottom:1px solid rgba(128,128,128,0.25)">Open-weight dense models</th></tr><tr><th style="padding:10px 6px;text-align:left;border-bottom:2px solid #2450D6"></th><th style="padding:10px 3px;text-align:center;font-weight:600;border-bottom:2px solid #2450D6;color:#2450D6;font-size:12.5px;line-height:1.2;width:14%;overflow-wrap:anywhere;background:rgba(36,80,214,0.08);">K2-Horizon-3.7B</th><th style="padding:10px 3px;text-align:center;font-weight:600;border-bottom:2px solid #2450D6;color:#2450D6;font-size:12.5px;line-height:1.2;width:14%;overflow-wrap:anywhere;">Qwen3.5-4B</th><th style="padding:10px 3px;text-align:center;font-weight:600;border-bottom:2px solid #2450D6;color:#2450D6;font-size:12.5px;line-height:1.2;width:14%;overflow-wrap:anywhere;">G9v3-3B</th><th style="padding:10px 3px;text-align:center;font-weight:600;border-bottom:2px solid #2450D6;color:#2450D6;font-size:12.5px;line-height:1.2;width:14%;overflow-wrap:anywhere;">Granite 4.2-3B</th><th style="padding:10px 3px;text-align:center;font-weight:600;border-bottom:2px solid #2450D6;color:#2450D6;font-size:12.5px;line-height:1.2;width:14%;overflow-wrap:anywhere;">Nemotron 3 Nano-4B</th></tr></thead><tbody><tr><td style="padding:7px 4px 7px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;font-weight:600"># Params</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">3.7B</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">4B</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">3B</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">3B</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">4B</td></tr><tr><td style="padding:7px 4px 7px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;font-weight:600"># Activated params</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">3.7B</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">4B</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">3B</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">3B</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">4B</td></tr><tr><td style="padding:7px 4px 7px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;font-weight:600">Architecture</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);">Dense</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">Dense</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">Dense</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">Dense</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">Dense</td></tr><tr><td colspan="6" style="padding:6px 10px;font-weight:600;font-size:12.5px;color:#2450D6;border-bottom:1px solid rgba(36,80,214,0.25);background:rgba(36,80,214,0.12)">Math</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2">HMMT Feb 2026</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Competition mathematics</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);"><strong>70.5</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">61.6</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">34.1</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">57.2</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">34.7</td></tr><tr><td colspan="6" style="padding:6px 10px;font-weight:600;font-size:12.5px;color:#2450D6;border-bottom:1px solid rgba(36,80,214,0.25);background:rgba(36,80,214,0.12)">Coding</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2">SWE-bench Verified</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Software engineering</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);"><strong>68.6</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">41.2</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">16.4</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">32.2</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">1.8</td></tr><tr><td colspan="6" style="padding:6px 10px;font-weight:600;font-size:12.5px;color:#2450D6;border-bottom:1px solid rgba(36,80,214,0.25);background:rgba(36,80,214,0.12)">Scientific Reasoning</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2">GPQA Diamond</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Graduate-level science QA</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);"><strong>65.4</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">77.1</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">43.8</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">55.9</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">51.3</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2">HLE</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Expert-level reasoning</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);"><strong>12.9</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">9.9</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">4.5</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">6.6</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">4.9</td></tr><tr><td colspan="6" style="padding:6px 10px;font-weight:600;font-size:12.5px;color:#2450D6;border-bottom:1px solid rgba(36,80,214,0.25);background:rgba(36,80,214,0.12)">Coding</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2">SciCode</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Scientific coding</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);"><strong>25.9</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">16.1</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">17.7</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">24.9</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">16.4</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2">Terminal-Bench 2.1</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Agentic terminal use</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);"><strong>25.1</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">25.8</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">6.0</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">13.9</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">3.7</td></tr><tr><td colspan="6" style="padding:6px 10px;font-weight:600;font-size:12.5px;color:#2450D6;border-bottom:1px solid rgba(36,80,214,0.25);background:rgba(36,80,214,0.12)">Agents</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2">tau3-Banking</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Agentic tool use</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);"><strong>17.7</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">6.8</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">—</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">5.6</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">—</td></tr><tr><td style="padding:6px 4px 6px 10px;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle"><div style="font-size:12.5px;font-weight:600;line-height:1.2">BFCL v4</div><div style="margin-top:2px;font-size:10px;opacity:0.65">Function calling</div></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;background:rgba(36,80,214,0.08);"><strong>50.9</strong></td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">55.7</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">47.9</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">50.8</td><td style="padding:6px 2px;text-align:center;border-bottom:1px solid rgba(128,128,128,0.15);vertical-align:middle;font-size:12.5px;line-height:1.2;">36.8</td></tr></tbody></table></div>
+<!-- TABLE:END -->
+
+Scores in %. Bold marks the best score in each row. Baseline protocols may differ;
+
+## Quickstart
+
+### Serving
+
+vLLM, recipe at [recipes.vllm.ai/IFM](https://recipes.vllm.ai/IFM):
+
+```shell
+vllm serve IFM/K2-Horizon-3.7B \
+  --trust-remote-code \
+  --dtype bfloat16 \
+  --tensor-parallel-size 1 \
+  --reasoning-parser k2_horizon \
+  --enable-auto-tool-choice \
+  --tool-call-parser k2_horizon
+```
+
+Use an exact branch name from the inventory with vLLM's `--revision` option. For example, `--revision pretrain_1100000` selects the final checkpoint of Pretraining, at step 1,100,000.
+
+SGLang, this is the recipe validated in the [SGLang K2 Horizon cookbook](https://docs.sglang.io/cookbook/autoregressive/IFM/K2-Horizon):
+
+```shell
+sglang serve \
+  --model-path IFM/K2-Horizon-3.7B \
+  --revision 09568bab0d7ee4152a12ee13c21902ca910d15b2 \
+  --tp 1 \
+  --dtype bfloat16 \
+  --attention-backend fa3 \
+  --reasoning-parser k2_horizon \
+  --host 0.0.0.0 \
+  --port 30000
+```
+
+### API Usage
+
+> [!Tip]
+> Recommended settings: `reasoning_effort="high"`, `temperature=1.0`, `top_p=0.95`, and at least 32,768 output tokens.
+> Reasoning depth is selected per request through `chat_template_kwargs`. Thinking is returned in `reasoning_content` and the answer in `content`.
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://localhost:30000/v1", api_key="EMPTY")
+response = client.chat.completions.create(
+    model="IFM/K2-Horizon-3.7B",
+    messages=[{"role": "user", "content": "Explain the result step by step."}],
+    temperature=1.0,
+    top_p=0.95,
+    max_tokens=32768,
+    extra_body={"chat_template_kwargs": {"reasoning_effort": "high"}},
+)
+message = response.choices[0].message
+print("Reasoning:", getattr(message, "reasoning_content", None))
+print("Answer:", message.content)
+```
+
+### Transformers
+
+Validated with Transformers 5.15.0, PyTorch 2.13.0, Safetensors 0.8.0.
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+
+model_id = "IFM/K2-Horizon-3.7B"
+tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True)
+model = AutoModelForCausalLM.from_pretrained(
+    model_id, device_map="auto", dtype="bfloat16", low_cpu_mem_usage=True, trust_remote_code=True
+)
+
+inputs = tokenizer("Explain why long-context evaluation is difficult.", return_tensors="pt").to(model.device)
+inputs.pop("token_type_ids", None)
+outputs = model.generate(**inputs, max_new_tokens=32768, temperature=1.0, top_p=0.95, do_sample=True)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
+```
+
+## Training Overview
+
+The table below lists the training stages in order and the purpose of each stage.
+
+Training steps are counted within each stage or phase. Token budgets cover only the additional training in that stage or phase. For example, the 50B tokens listed for SFT Phase 2 are additional to the 219B tokens in Phase 1, bringing the cumulative budget to 269B tokens by the end of Phase 2. Here, B and T denote billion and trillion tokens, respectively.
+
+Each stage or phase continues from the final checkpoint of the preceding stage or phase.
+
+Some stages, such as SFT, have multiple phases with slight changes to the data mix while retaining the same overall purpose. During RL, training branches into multiple expert models, which are then merged, as described below.
+
+| Training stage | Training steps | Training tokens | Sequence length | Purpose |
+| --- | --- | --- | --- | --- |
+| Pretraining | 1100000 | 22.9T | 8K | Pretraining. |
+| Midtraining — Stage 1 | 55000 | 1.1T | 32K | Context extension. |
+| Midtraining — Stage 2 | 25000 | 498B | 128K | Context extension. |
+| Midtraining — Stage 3 | 5500 | 110B | 512K | Context extension. |
+| Midtraining — Stage 4 | 10000 | 199B | 512K | Continued context extension from Stage 3, with the data mix shifted toward agentic and reasoning SFT data. |
+| RL — Math expert | 2549 | 28.9B | 64K | Math RL from Midtraining Stage 4; common ancestor of the Code and STEM-Code experts. |
+| RL — Code expert | 949 | 15.5B | 64K | Code RL from the Math expert. |
+| RL — STEM-Code expert | 451 | 1.3B | 64K | STEM and code RL from the Math expert. |
+| RL — Merge | — | — | — | ISO merge on self-attention, RAM on the remaining weights; inputs: Midtraining Stage 4 base + Math, Code, STEM-Code experts. |
+| SFT — Phase 1 | 10000 | 199B | 512K | SFT for better domain coverage, starting from the merged RL checkpoint. |
+| SFT — Phase 2 | 2500 | 50B | 512K | SFT on a high-quality subset of the data used in Phase 1, with learning rate decay. |
+
+## Release Artifacts
+
+The tables below list the release artifacts for **K2-Horizon-3.7B**, their availability, and the expected release dates for remaining items.
+
+**Last updated:** 2026-09-11
+
+**Status:**
+
+- **Available** — fully released for the scope listed;
+- **Partial** — some items are available, with remaining items listed in the notes;
+- **In Progress** — being prepared for release but not yet available.
+
+### Artifact Index
+
+| Artifact | Link | Status | Remaining items / expected availability |
+| --- | --- | --- | --- |
+| Model card | [Hugging Face](https://huggingface.co/IFM/K2-Horizon-3.7B) | Available | N/A |
+| Training logs | [W&B](https://wandb.ai/llm360/K2-Horizon-3.7B) | Available | N/A |
+| Blog post | [Blog post](https://ifm.ai/blog/k2/) | Available | N/A |
+| Checkpoints | [Checkpoint inventory](#checkpoint-inventory) | Available | N/A |
+| Technical report | Not yet available | In Progress | End of September 2026 |
+| Code repository | [GitHub](https://github.com/ifm-ai/xllm) | In Progress | End of September 2026 |
+
+### Checkpoint Inventory
+
+**Model repository:** [IFM/K2-Horizon-3.7B](https://huggingface.co/IFM/K2-Horizon-3.7B)
+
+Branch names below refer to this repository. Patterns containing `*` group branches by training stage or phase. The `*` is a placeholder for a training-step number, not a literal branch name. Intermediate checkpoint groups exclude the final checkpoint listed separately; a pattern does not imply that a checkpoint is available at every step.
+
+For example, `sft_1_10000` is the checkpoint saved at training step 11,000 within SFT Phase 1, and is the final checkpoint of that phase. The numeric suffix is the step within the named stage or phase, not the cumulative step across all training. Thus, `sft_2_2500` refers to step 2,500 within SFT Phase 2.
+
+For a partially released group, the available checkpoints and the remaining checkpoints are listed in the notes.
+
+| Checkpoint | Branch / repository | Status | Remaining items / expected availability |
+| --- | --- | --- | --- |
+| Pretrain Intermediate Checkpoints | `pretrain_*` | Available | N/A |
+| Pretrain Final Checkpoint | `pretrain_1100000` | Available | N/A |
+| Midtrain Stage 1 Intermediate Checkpoints | `mid_1_*` | Available | N/A |
+| Midtrain Stage 1 Final Checkpoint | `mid_1_55000` | Available | N/A |
+| Midtrain Stage 2 Intermediate Checkpoints | `mid_2_*` | Available | N/A |
+| Midtrain Stage 2 Final Checkpoint | `mid_2_25000` | Available | N/A |
+| Midtrain Stage 3 Intermediate Checkpoints | `mid_3_*` | Available | N/A |
+| Midtrain Stage 3 Final Checkpoint | `mid_3_5500` | Available | N/A |
+| Midtrain Stage 4 Intermediate Checkpoints | `mid_4_*` | Available | N/A |
+| Midtrain Stage 4 Final Checkpoint | `mid_4_10000` | Available | N/A |
+| RL Math Expert Checkpoint | `rl_math` | Available | N/A |
+| RL Code Expert Checkpoint | `rl_code` | Available | N/A |
+| RL Stem Code Expert Checkpoint | `rl_stemcode` | Available | N/A |
+| RL Merged Final Checkpoint | `rl_merged` | Available | N/A |
+| SFT Phase 1 Intermediate Checkpoints | `sft_1_*` | Available | N/A |
+| SFT Phase 1 Final Checkpoint | `sft_1_10000` | Available | N/A |
+| SFT Phase 2 Intermediate Checkpoints | `sft_2_*` | Available | N/A |
+| SFT Phase 2 Final Checkpoint | `sft_2_2500` | Available | N/A |
+
+> [!IMPORTANT]
+> Note:
+> - Released K2-Horizon Hugging Face checkpoints (e.g. [Huggingface](https://huggingface.co/IFM/K2-Horizon-375B-A23B/tree/main)) can be used for inference, evaluation, and downstream fine-tuning (including SFT).
+> Training behavior in the bundled Hugging Face implementation may differ from native xLLM, including the auxiliary load-balancing loss.
+> To continue the original pretraining with xLLM's training behavior, use the native xLLM checkpoint and XLLM runtime.
+
+## Best Practices
+
+1. **Reasoning effort: always `high`.** All reported results use high reasoning effort. Pass `{"chat_template_kwargs": {"reasoning_effort": "high"}}` on every request; `medium` and `low` trade accuracy for speed and are not recommended for evaluation.
+2. **Sampling parameters.** `temperature=1.0`, `top_p=0.95`.
+3. **Output length.** Allow at least 32,768 output tokens so reasoning is never cut off. Truncated reasoning is a failed response, not a shorter one.
+4. **Serving.** Use the validated SGLang recipe above: BF16, TP=1, FlashAttention-3. Full recipes for every K2-Horizon size, with measured H200 latency and throughput, are in the [SGLang cookbook](https://docs.sglang.io/cookbook/autoregressive/IFM/K2-Horizon).
+5. **Parsers.** Enable the `k2_horizon` reasoning parser for chat, and add the `k2_horizon` tool-call parser for agent use. Leave both off for plain completion-style generation.
+6. **Revisions.** Pin a revision tag when reproducibility matters. `main` is the default checkpoint; `base_final` and the `mid_*_final` tags identify training stages.
+
+## Citation
+
+```bibtex
+@misc{k2horizon2026,
+  title  = {Introducing K2 Horizon: Frontier Performance, Radically Open},
+  author = {{IFM Team}},
+  year   = {2026},
+  url    = {https://ifm.ai/blog/k2/},
+}
+```
+

chat_template.jinja

@@ -0,0 +1,994 @@
+{{- bos_token }}
+{%- if tool_presentation is defined -%}
+    {{- raise_exception("Unsupported argument: tool_presentation. Use tool_presentation_format with one of: json, xml, markdown.") -}}
+{%- endif -%}
+{%- if tool_calling_format is defined -%}
+    {{- raise_exception("Unsupported argument: tool_calling_format. Use tool_call_format with one of: json, xml, xml_typed.") -}}
+{%- endif -%}
+{%- if tool_format is defined -%}
+    {{- raise_exception("Unsupported argument: tool_format. Use tool_call_format with one of: json, xml, xml_typed.") -}}
+{%- endif -%}
+{%- set tool_presentation_fmt = tool_presentation_format | default('markdown') -%}
+{%- set tool_call_fmt = tool_call_format | default('xml') -%}
+{%- if tool_presentation_fmt != 'json' and tool_presentation_fmt != 'xml' and tool_presentation_fmt != 'markdown' -%}
+    {{- raise_exception("Unsupported tool_presentation_format: '" ~ tool_presentation_fmt ~ "'. Supported formats: json, xml, markdown.") -}}
+{%- endif -%}
+{%- if tool_call_fmt != 'json' and tool_call_fmt != 'xml' and tool_call_fmt != 'xml_typed' -%}
+    {{- raise_exception("Unsupported tool_call_format: '" ~ tool_call_fmt ~ "'. Supported formats: json, xml, xml_typed.") -}}
+{%- endif -%}
+
+{#- Renderability state, computed during validate_tools (single walk, no extra -#}
+{#- traversal at render time): ok = working flag for the tool being validated; -#}
+{#- bad = pipe-delimited indices of tools that must render as verbatim JSON. -#}
+{%- set RB = namespace(ok=true, bad='|') -%}
+
+{%- macro value_contains_mapping(v) -%}
+{%- if v is mapping -%}
+true
+{%- elif v is sequence and v is not string -%}
+{%- set f = namespace(x='false') -%}
+{%- for c in v -%}{%- if value_contains_mapping(c) == 'true' -%}{%- set f.x = 'true' -%}{%- endif -%}{%- endfor -%}
+{{- f.x -}}
+{%- else -%}
+false
+{%- endif -%}
+{%- endmacro -%}
+
+{#- $ref inlining state: defs = local $defs of the tool being rendered; seen = -#}
+{#- pipe-delimited names already expanded for this tool (each def inlines at most -#}
+{#- once; later references render by def name; cycles terminate immediately). -#}
+{#- $ref-sibling annotations (description/default/...) merge OVER the def at -#}
+{#- the inline site, so use-site annotations win and are never dropped. -#}
+{%- set REFS = namespace(defs={}, seen='|') -%}
+
+{%- macro render_compact_type_name(type_name, spec) -%}
+{%- if type_name == "array" -%}
+array[{%- if 'items' in spec -%}{{ render_compact_type(spec['items']) }}{%- else -%}any{%- endif -%}]
+{%- elif type_name -%}
+{{- type_name -}}
+{%- else -%}
+any
+{%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_compact_type(spec) -%}
+{%- if spec is not mapping -%}
+any
+{%- elif spec.type is defined and spec.type is sequence and spec.type is not string and spec.type | length > 0 -%}
+{%- for type_name in spec.type -%}{{ render_compact_type_name(type_name, spec) }}{%- if not loop.last -%}|{%- endif -%}{%- endfor -%}
+{%- elif spec.type is defined and spec.type is sequence and spec.type is not string -%}
+any
+{%- elif spec.type -%}
+{{- render_compact_type_name(spec.type, spec) -}}
+{%- elif spec['$ref'] is string -%}
+{{- spec['$ref'].split('/') | last -}}
+{%- elif spec.oneOf -%}
+oneOf[{%- for variant in spec.oneOf -%}{{ render_compact_type(variant) }}{%- if not loop.last -%}|{%- endif -%}{%- endfor -%}]
+{%- elif spec.anyOf -%}
+anyOf[{%- for variant in spec.anyOf -%}{{ render_compact_type(variant) }}{%- if not loop.last -%}|{%- endif -%}{%- endfor -%}]
+{%- elif spec.properties -%}
+object
+{%- elif 'items' in spec -%}
+array[{{ render_compact_type(spec['items']) }}]
+{%- else -%}
+any
+{%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_markdown_type_name(type_name, spec) -%}
+{%- if type_name == "array" -%}
+array of {% if 'items' in spec %}{{ render_markdown_type(spec['items']) }}{% else %}any{% endif %}
+{%- elif type_name -%}
+{{- type_name -}}
+{%- else -%}
+any
+{%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_markdown_type(spec) -%}
+{%- if spec is sameas true -%}
+True
+{%- elif spec is sameas false -%}
+False
+{%- elif spec is not mapping -%}
+any
+{%- elif spec.type is defined and spec.type is sequence and spec.type is not string and spec.type | length > 0 -%}
+{%- for type_name in spec.type -%}{{ render_markdown_type_name(type_name, spec) }}{% if not loop.last %} or {% endif %}{%- endfor -%}
+{%- elif spec.type is defined and spec.type is sequence and spec.type is not string -%}
+any
+{%- elif spec.type -%}
+{{- render_markdown_type_name(spec.type, spec) -}}
+{%- elif spec['$ref'] is string -%}
+{{- spec['$ref'].split('/') | last -}}
+{%- elif spec.oneOf -%}
+oneOf[{%- for variant in spec.oneOf -%}{{ render_markdown_type(variant) }}{% if not loop.last %} or {% endif %}{%- endfor -%}]
+{%- elif spec.anyOf -%}
+anyOf[{%- for variant in spec.anyOf -%}{{ render_markdown_type(variant) }}{% if not loop.last %} or {% endif %}{%- endfor -%}]
+{%- elif spec.properties -%}
+object
+{%- elif 'items' in spec -%}
+array of {{ render_markdown_type(spec['items']) }}
+{%- else -%}
+any
+{%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_xml_text(value) -%}
+{{- value.split() | join(" ") -}}
+{%- endmacro -%}
+
+{%- macro render_python_string(value) -%}
+'{{- value.split() | join(" ") | replace("\\", "\\\\") | replace("'", "\\'") -}}'
+{%- endmacro -%}
+
+{%- macro render_python_repr(value) -%}
+{%- if value is string -%}
+{{ render_python_string(value) }}
+{%- elif value is sameas true -%}
+True
+{%- elif value is sameas false -%}
+False
+{%- elif value is none -%}
+None
+{%- elif value is mapping -%}
+{{- "{" -}}
+{%- for key, child in value | items -%}
+{{ render_python_repr(key) }}: {{ render_python_repr(child) }}{%- if not loop.last -%}, {% endif -%}
+{%- endfor -%}
+{{- "}" -}}
+{%- elif value is sequence -%}
+{{- "[" -}}
+{%- for child in value -%}
+{{ render_python_repr(child) }}{%- if not loop.last -%}, {% endif -%}
+{%- endfor -%}
+{{- "]" -}}
+{%- else -%}
+{{- value -}}
+{%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_xml_value(value) -%}
+{%- if value is string -%}{{ render_xml_text(value) }}{%- else -%}{{ render_python_repr(value) }}{%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_xml_enum_value(value) -%}
+{%- if value is string -%}"{{- value | replace("\\", "\\\\") | replace("\"", "\\\"") -}}"{%- else -%}"{{- render_python_repr(value) | replace("\\", "\\\\") | replace("\"", "\\\"") -}}"{%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_xml_enum(values) -%}
+{%- for value in values -%}{{ render_xml_enum_value(value) }}{%- if not loop.last -%}|{%- endif -%}{%- endfor -%}
+{%- endmacro -%}
+
+{%- macro render_xml_default_attr(value) -%}
+{{- " default=" }}{%- if value is string -%}"{{- value | replace("\\", "\\\\") | replace("\"", "\\\"") -}}"{%- else -%}{{ render_xml_value(value) }}{%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_xml_attr(name, value) -%}
+{{- " " + name + "=" }}{%- if value == "" -%}""{%- else -%}{{ render_xml_value(value) }}{%- endif -%}
+{%- endmacro -%}
+
+{%- macro validate_schema(spec, path, lenient=false, classify=true, in_variant=false) -%}
+{%- if spec is mapping -%}
+    {%- if not lenient -%}
+    {%- if spec.required is defined -%}
+        {%- if spec.required is string or spec.required is not sequence -%}
+            {{- raise_exception("Schema '" + path + "' has 'required' but it is not a list.") -}}
+        {%- endif -%}
+        {%- if spec.required | length > 0 and not spec.properties and not in_variant -%}
+            {{- raise_exception("Schema '" + path + "' has required fields but no properties object to define them.") -}}
+        {%- endif -%}
+        {%- if spec.properties -%}
+            {%- for required_name in spec.required -%}
+                {%- if required_name not in spec.properties -%}
+                    {{- raise_exception("Schema '" + path + "' marks '" + required_name + "' as required, but that property is not defined in properties.") -}}
+                {%- endif -%}
+            {%- endfor -%}
+        {%- endif -%}
+    {%- endif -%}
+    {%- endif -%}
+    {#- renderability classification, piggybacking on this walk (no raises here): -#}
+    {#- constructs the pretty renderer does not fully handle flip RB.ok so the -#}
+    {#- tool falls back to verbatim JSON. Skipped entirely for json presentation. -#}
+    {%- if classify -%}
+    {%- for key, value in spec | items -%}
+        {%- if key == '$ref' -%}
+            {%- if value is not string -%}{%- set RB.ok = false -%}
+            {%- elif not (value.startswith('#/$defs/') or value.startswith('#/definitions/')) -%}{%- set RB.ok = false -%}{%- endif -%}
+        {%- elif key == '$defs' or key == 'definitions' -%}
+            {%- if value is mapping -%}
+                {%- for dk, dv in value | items -%}
+                    {{- validate_schema(dv, path + ".$defs." + dk, true) -}}
+                {%- endfor -%}
+            {%- else -%}{%- set RB.ok = false -%}{%- endif -%}
+        {%- elif key == 'type' -%}
+            {%- if value is mapping -%}{%- set RB.ok = false -%}{%- endif -%}
+        {%- elif key == 'enum' -%}
+            {%- if value is string or value is mapping or value is not sequence -%}{%- set RB.ok = false -%}{%- endif -%}
+        {%- elif key == 'items' -%}
+            {#- any items shape renders: mapping structurally, others via repr detail -#}
+        {%- elif key == 'oneOf' or key == 'anyOf' -%}
+            {%- if value is mapping or value is string or value is not sequence -%}{%- set RB.ok = false -%}{%- endif -%}
+        {%- elif key == 'required' -%}
+            {%- if value and not spec.properties -%}{%- set RB.ok = false -%}{%- endif -%}
+        {%- elif ('|' ~ key ~ '|') in '|description|default|title|examples|properties|patternProperties|additionalProperties|returns|' -%}
+        {%- elif value is mapping -%}
+            {%- for uk, uv in value | items -%}
+                {%- if value_contains_mapping(uv) == 'true' -%}{%- set RB.ok = false -%}{%- endif -%}
+            {%- endfor -%}
+        {%- elif value is sequence and value is not string -%}
+            {%- if value_contains_mapping(value) == 'true' -%}{%- set RB.ok = false -%}{%- endif -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {%- endif -%}
+    {%- if spec.properties -%}
+        {%- for child_name, child_spec in spec.properties | items -%}
+            {{- validate_schema(child_spec, path + "." + child_name, lenient, classify) -}}
+        {%- endfor -%}
+    {%- endif -%}
+    {%- if 'items' in spec -%}{{- validate_schema(spec['items'], path + "[]", lenient, classify) -}}{%- endif -%}
+    {%- if spec.oneOf -%}
+        {%- for variant in spec.oneOf -%}{{- validate_schema(variant, path + ".oneOf[" + (loop.index0 | string) + "]", lenient, classify, true) -}}{%- endfor -%}
+    {%- endif -%}
+    {%- if spec.anyOf -%}
+        {%- for variant in spec.anyOf -%}{{- validate_schema(variant, path + ".anyOf[" + (loop.index0 | string) + "]", lenient, classify, true) -}}{%- endfor -%}
+    {%- endif -%}
+    {%- if spec.additionalProperties is mapping -%}{{- validate_schema(spec.additionalProperties, path + ".additionalProperties", lenient, classify) -}}{%- endif -%}
+    {%- if spec.patternProperties is mapping -%}
+        {%- for pattern, pattern_spec in spec.patternProperties | items -%}
+            {{- validate_schema(pattern_spec, path + ".patternProperties[" + pattern + "]", lenient, classify) -}}
+        {%- endfor -%}
+    {%- endif -%}
+    {%- if spec.returns is mapping -%}{{- validate_schema(spec.returns, path + ".returns", lenient, classify) -}}{%- endif -%}
+{%- endif -%}
+{%- endmacro -%}
+
+{%- macro validate_tools(tools_list, classify=true) -%}
+{%- set RB.bad = '|' -%}
+{%- for tool in tools_list -%}
+    {%- set fn = tool.function if tool.function is defined else tool -%}
+    {%- set RB.ok = true -%}
+    {%- if fn.parameters is defined and fn.parameters is string -%}
+        {{- raise_exception("tool.function.parameters must be a dict, not a JSON string. Parse it before passing to the template.") -}}
+    {%- endif -%}
+    {%- if fn.parameters is not defined or fn.parameters is none -%}
+        {%- if fn.arguments is defined -%}
+            {{- raise_exception("Tool '" + fn.name + "' has 'arguments' instead of 'parameters'. Rename 'arguments' to 'parameters'.") -}}
+        {%- else -%}
+            {{- raise_exception("Tool '" + fn.name + "' is missing required 'parameters' field. Each tool must have a 'parameters' dict with 'type', 'properties', and 'required' keys.") -}}
+        {%- endif -%}
+    {%- endif -%}
+    {{- validate_schema(fn.parameters, "tool." + fn.name + ".parameters", false, classify) -}}
+    {%- if classify -%}
+    {%- if fn.parameters is mapping -%}
+        {#- unknown container-valued keys at the parameters ROOT are never rendered -#}
+        {#- by the pretty path (root extras are dropped) -> verbatim fallback. -#}
+        {%- for rk, rv in fn.parameters | items -%}
+            {%- if rk not in ['type', 'description', 'enum', 'default', 'properties', 'required', 'optional', 'title', 'items', 'oneOf', 'anyOf', 'additionalProperties', 'patternProperties', 'returns', 'examples', '$defs', 'definitions', '$ref'] -%}
+                {%- if rv is mapping or (rv is sequence and rv is not string) -%}{%- set RB.ok = false -%}{%- endif -%}
+            {%- endif -%}
+        {%- endfor -%}
+    {%- else -%}
+        {%- set RB.ok = false -%}
+    {%- endif -%}
+    {%- endif -%}
+    {%- if fn.returns is mapping -%}{{- validate_schema(fn.returns, "tool." + fn.name + ".returns", false, classify) -}}{%- endif -%}
+    {%- if classify and fn.returns is not defined and fn.response is mapping -%}{{- validate_schema(fn.response, "tool." + fn.name + ".response", true) -}}{%- endif -%}
+    {#- unknown container-valued keys at the FUNCTION level are never rendered -> fallback. -#}
+    {%- if classify -%}
+    {%- for fk, fv in fn | items -%}
+        {%- if fk not in ['name', 'description', 'parameters', 'returns', 'response', 'type', 'function'] -%}
+            {%- if fv is mapping or (fv is sequence and fv is not string) -%}{%- set RB.ok = false -%}{%- endif -%}
+        {%- endif -%}
+    {%- endfor -%}
+    {%- endif -%}
+    {%- if not RB.ok -%}{%- set RB.bad = RB.bad ~ loop.index0 ~ '|' -%}{%- endif -%}
+{%- endfor -%}
+{%- endmacro -%}
+
+{%- macro render_tools_json(tools_list) -%}
+{{- "<ifm|tools>" }}
+{%- for tool in tools_list %}
+{{- "\n" }}
+{{- tool | tojson }}
+{%- endfor %}
+{{- "\n</ifm|tools>" }}
+{%- endmacro -%}
+
+{%- macro render_xml_schema_attrs(spec, include_value_attrs) -%}
+{%- if spec is mapping -%}
+{%- set structural_keys = ["type", "description", "enum", "default", "properties", "required", "items", "oneOf", "anyOf", "additionalProperties", "patternProperties", "returns"] -%}
+{%- if include_value_attrs and spec.enum -%}{{- " enum=" }}{{ render_xml_enum(spec.enum) }}{%- endif -%}
+{%- if include_value_attrs and spec.default is defined -%}{{ render_xml_default_attr(spec.default) }}{%- endif -%}
+{%- if spec.additionalProperties is defined and spec.additionalProperties is not mapping -%}{{ render_xml_attr("additionalProperties", spec.additionalProperties) }}{%- endif -%}
+{%- if spec.patternProperties is defined and spec.patternProperties is not mapping -%}{{ render_xml_attr("patternProperties", spec.patternProperties) }}{%- endif -%}
+{%- for key, value in spec | items -%}
+    {%- if key not in structural_keys -%}
+{{ render_xml_attr(key, value) }}
+    {%- endif -%}
+{%- endfor -%}
+{%- endif -%}
+{%- endmacro -%}
+
+{%- macro xml_schema_has_children(spec, include_properties, include_description) -%}
+{%- if spec is not mapping -%}
+false
+{%- elif (include_description and spec.description is defined) or (include_properties and spec.properties) or 'items' in spec or spec.oneOf or spec.anyOf or spec.additionalProperties is mapping or spec.patternProperties is mapping or spec.returns is defined -%}
+true
+{%- else -%}
+false
+{%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_xml_schema_node(tag, spec, include_properties) -%}
+{%- if spec is mapping and spec['$ref'] is string -%}
+    {%- set _r = spec['$ref'] -%}
+    {%- set _k = _r[8:] if _r.startswith('#/$defs/') else (_r[14:] if _r.startswith('#/definitions/') else none) -%}
+    {%- if _k is not none and ('|' + _k + '|') not in REFS.seen and REFS.defs[_k] is mapping -%}
+        {%- set spec = dict((REFS.defs[_k] | items | list) + (spec | items | rejectattr('0', 'equalto', '$ref') | list)) -%}
+        {%- set REFS.seen = REFS.seen + _k + '|' -%}
+        {%- if spec['$ref'] is string -%}
+            {%- set _r2 = spec['$ref'] -%}
+            {%- set _k2 = _r2[8:] if _r2.startswith('#/$defs/') else (_r2[14:] if _r2.startswith('#/definitions/') else none) -%}
+            {%- if _k2 is not none and REFS.defs[_k2] is mapping -%}
+                {%- set spec = dict((REFS.defs[_k2] | items | list) + (spec | items | rejectattr('0', 'equalto', '$ref') | list)) -%}
+                {%- set REFS.seen = REFS.seen + _k2 + '|' -%}
+            {%- endif -%}
+        {%- endif -%}
+    {%- endif -%}
+{%- endif -%}
+{%- if spec is mapping -%}
+{{- "<" + tag + " type=" + render_compact_type(spec) }}{{ render_xml_schema_attrs(spec, true) }}
+{%- if xml_schema_has_children(spec, include_properties, true) == 'true' -%}
+{{- ">" }}{{ render_xml_schema_children(spec, include_properties, true) }}{{- "</" + tag + ">" }}
+{%- else -%}
+{{- "/>" }}
+{%- endif -%}
+{%- else -%}
+{{- "<" + tag + ">" }}{{ render_xml_value(spec) }}{{- "</" + tag + ">" }}
+{%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_xml_pattern_property(pattern, spec) -%}
+{%- if spec is mapping and spec['$ref'] is string -%}
+    {%- set _r = spec['$ref'] -%}
+    {%- set _k = _r[8:] if _r.startswith('#/$defs/') else (_r[14:] if _r.startswith('#/definitions/') else none) -%}
+    {%- if _k is not none and ('|' + _k + '|') not in REFS.seen and REFS.defs[_k] is mapping -%}
+        {%- set spec = dict((REFS.defs[_k] | items | list) + (spec | items | rejectattr('0', 'equalto', '$ref') | list)) -%}
+        {%- set REFS.seen = REFS.seen + _k + '|' -%}
+        {%- if spec['$ref'] is string -%}
+            {%- set _r2 = spec['$ref'] -%}
+            {%- set _k2 = _r2[8:] if _r2.startswith('#/$defs/') else (_r2[14:] if _r2.startswith('#/definitions/') else none) -%}
+            {%- if _k2 is not none and REFS.defs[_k2] is mapping -%}
+                {%- set spec = dict((REFS.defs[_k2] | items | list) + (spec | items | rejectattr('0', 'equalto', '$ref') | list)) -%}
+                {%- set REFS.seen = REFS.seen + _k2 + '|' -%}
+            {%- endif -%}
+        {%- endif -%}
+    {%- endif -%}
+{%- endif -%}
+{%- if spec is mapping -%}
+{{- "<patternProperty" }}{{ render_xml_attr("pattern", pattern) }}{{- " type=" + render_compact_type(spec) }}{{ render_xml_schema_attrs(spec, true) }}
+{%- if xml_schema_has_children(spec, true, true) == 'true' -%}
+{{- ">" }}{{ render_xml_schema_children(spec, true, true) }}{{- "</patternProperty>" }}
+{%- else -%}
+{{- "/>" }}
+{%- endif -%}
+{%- else -%}
+{{- "<patternProperty" }}{{ render_xml_attr("pattern", pattern) }}{{- ">" }}{{ render_xml_value(spec) }}{{- "</patternProperty>" }}
+{%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_xml_schema_children(spec, include_properties, include_description) -%}
+{%- if include_description and spec.description is defined -%}{{- "<description>" }}{{ spec.description }}{{- "</description>" }}{%- endif -%}
+{%- if include_properties and spec.properties -%}
+{%- for child_name, child_spec in spec.properties | items -%}
+{{- render_xml_param(child_name, child_spec, spec.required or []) }}
+{%- endfor -%}
+{%- endif -%}
+{%- if 'items' in spec -%}{{ render_xml_schema_node("items", spec['items'], true) }}{%- endif -%}
+{%- if spec.oneOf -%}
+{{- "<oneOf>" }}
+{%- for variant in spec.oneOf -%}{{ render_xml_schema_node("variant", variant, true) }}{%- endfor -%}
+{{- "</oneOf>" }}
+{%- endif -%}
+{%- if spec.anyOf -%}
+{{- "<anyOf>" }}
+{%- for variant in spec.anyOf -%}{{ render_xml_schema_node("variant", variant, true) }}{%- endfor -%}
+{{- "</anyOf>" }}
+{%- endif -%}
+{%- if spec.additionalProperties is mapping -%}{{ render_xml_schema_node("additionalProperties", spec.additionalProperties, true) }}{%- endif -%}
+{%- if spec.patternProperties is mapping -%}
+{{- "<patternProperties>" }}
+{%- for pattern, pattern_spec in spec.patternProperties | items -%}{{ render_xml_pattern_property(pattern, pattern_spec) }}{%- endfor -%}
+{{- "</patternProperties>" }}
+{%- elif spec.patternProperties is defined -%}<patternProperties>{{ render_xml_value(spec.patternProperties) }}</patternProperties>{%- endif -%}
+{%- if spec.returns is mapping -%}{{ render_xml_schema_node("returns", spec.returns, true) }}{%- elif spec.returns is defined -%}<returns>{{ render_xml_value(spec.returns) }}</returns>{%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_xml_param(name, spec, required_list) -%}
+{%- if spec is mapping and spec['$ref'] is string -%}
+    {%- set _r = spec['$ref'] -%}
+    {%- set _k = _r[8:] if _r.startswith('#/$defs/') else (_r[14:] if _r.startswith('#/definitions/') else none) -%}
+    {%- if _k is not none and ('|' + _k + '|') not in REFS.seen and REFS.defs[_k] is mapping -%}
+        {%- set spec = dict((REFS.defs[_k] | items | list) + (spec | items | rejectattr('0', 'equalto', '$ref') | list)) -%}
+        {%- set REFS.seen = REFS.seen + _k + '|' -%}
+        {%- if spec['$ref'] is string -%}
+            {%- set _r2 = spec['$ref'] -%}
+            {%- set _k2 = _r2[8:] if _r2.startswith('#/$defs/') else (_r2[14:] if _r2.startswith('#/definitions/') else none) -%}
+            {%- if _k2 is not none and REFS.defs[_k2] is mapping -%}
+                {%- set spec = dict((REFS.defs[_k2] | items | list) + (spec | items | rejectattr('0', 'equalto', '$ref') | list)) -%}
+                {%- set REFS.seen = REFS.seen + _k2 + '|' -%}
+            {%- endif -%}
+        {%- endif -%}
+    {%- endif -%}
+{%- endif -%}
+{{- "<param name=" + name + " type=" + render_compact_type(spec) }}
+{%- if name in (required_list or []) -%}{{- " required=true" }}{%- endif -%}
+{%- if spec.enum -%}{{- " enum=" }}{{ render_xml_enum(spec.enum) }}{%- endif -%}
+{%- if spec.default is defined -%}{{ render_xml_default_attr(spec.default) }}{%- endif -%}
+{{- render_xml_schema_attrs(spec, false) }}
+{%- if spec.description or xml_schema_has_children(spec, true, false) == 'true' -%}
+{{- ">" }}
+{%- if spec.description -%}{{ spec.description }}{%- endif -%}
+{{- render_xml_schema_children(spec, true, false) }}
+{{- "</param>" }}
+{%- else -%}
+{{- "/>" }}
+{%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_tools_xml(tools_list) -%}
+{{- "<ifm|tools>" }}
+{%- for tool in tools_list -%}
+    {%- set fn = tool.function if tool.function is defined else tool -%}
+    {%- set REFS.defs = fn.parameters['$defs'] if (fn.parameters is mapping and fn.parameters['$defs'] is mapping) else (fn.parameters['definitions'] if (fn.parameters is mapping and fn.parameters['definitions'] is mapping) else {}) -%}
+    {%- set REFS.seen = '|' -%}
+    {%- set fnp = namespace(p=fn.parameters) -%}
+    {%- if fnp.p is mapping and fnp.p['$ref'] is string -%}
+        {%- set _r = fnp.p['$ref'] -%}
+        {%- set _k = _r[8:] if _r.startswith('#/$defs/') else (_r[14:] if _r.startswith('#/definitions/') else none) -%}
+        {%- if _k is not none and REFS.defs[_k] is mapping -%}
+            {%- set fnp.p = dict((REFS.defs[_k] | items | list) + (fnp.p | items | rejectattr('0', 'equalto', '$ref') | list)) -%}
+            {%- set REFS.seen = REFS.seen + _k + '|' -%}
+        {%- endif -%}
+    {%- endif -%}
+{{- "\n<function name=" + fn.name + ">" }}
+{%- if fn.description -%}
+{{- "<description>" }}{{ fn.description }}{{- "</description>" }}
+{%- endif -%}
+{{- "<parameters>" }}
+{%- if fnp.p and fnp.p.properties -%}
+    {%- for pname, pspec in fnp.p.properties | items -%}
+{{- render_xml_param(pname, pspec, fnp.p.required or []) }}
+    {%- endfor -%}
+{%- elif fnp.p is mapping and (fnp.p.oneOf or fnp.p.anyOf or 'items' in fnp.p) -%}
+{{- render_xml_schema_children(fnp.p, true, false) }}
+{%- endif -%}
+{{- "</parameters>" }}
+{%- set fn_ret = fn.returns if fn.returns is defined else fn.response -%}
+{%- if fn_ret is mapping -%}{{ render_xml_schema_node("returns", fn_ret, true) }}{%- elif fn_ret is defined -%}<returns>{{ render_xml_value(fn_ret) }}</returns>{%- endif -%}
+{{- "</function>" }}
+{%- endfor -%}
+{{- "\n</ifm|tools>" }}
+{%- endmacro -%}
+
+{%- macro render_markdown_literal(value) -%}
+{%- if value is string and value == "" -%}""
+{%- elif value is string -%}`{{ value | replace("\n", "\\n") }}`
+{%- else -%}`{{ render_python_repr(value) }}`
+{%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_allowed_values(values) -%}
+{%- for value in values -%}{{ render_markdown_literal(value) }}{% if not loop.last %}, {% endif %}{%- endfor -%}
+{%- endmacro -%}
+
+{%- macro render_markdown_value(value) -%}
+{%- if value is string and value == "" -%}""{%- elif value is string -%}{{ value }}{%- else -%}{{ render_python_repr(value) }}{%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_markdown_detail(indent, label, value) -%}
+{{- "\n" + indent + "  - " + label + ": " }}{{ render_markdown_value(value) }}
+{%- endmacro -%}
+
+{%- macro render_markdown_metadata_detail(label, value) -%}
+{{- "\n- " + label + ": " }}{{ render_markdown_value(value) }}
+{%- endmacro -%}
+
+{%- macro render_markdown_schema_annotations(spec, indent, include_value_details) -%}
+{%- if include_value_details and spec.description is defined -%}{{ render_markdown_detail(indent, "Description", spec.description | replace("\n", "\n" + indent + "    ")) }}{%- endif -%}
+{%- if include_value_details and spec.enum is defined -%}{{- "\n" + indent + "  - Allowed values: " }}{{ render_allowed_values(spec.enum) }}{%- endif -%}
+{%- if include_value_details and spec.default is defined -%}{{- "\n" + indent + "  - Default: " }}{{ render_markdown_literal(spec.default) }}{%- endif -%}
+{%- if spec.additionalProperties is defined -%}
+    {%- if spec.additionalProperties is mapping -%}
+{{- "\n" + indent + "  - Additional properties *(" + render_markdown_type(spec.additionalProperties) + ")*" }}
+{{- render_markdown_schema_details(spec.additionalProperties, indent + "  ", true) }}
+    {%- else -%}
+{{ render_markdown_detail(indent, "Additional properties", spec.additionalProperties) }}
+    {%- endif -%}
+{%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_markdown_metadata_annotations(spec) -%}
+{%- if spec.description is defined -%}{{ render_markdown_metadata_detail("Description", spec.description | replace("\n", "\n    ")) }}{%- endif -%}
+{%- if spec.enum is defined -%}{{- "\n- Allowed values: " }}{{ render_allowed_values(spec.enum) }}{%- endif -%}
+{%- if spec.default is defined -%}{{- "\n- Default: " }}{{ render_markdown_literal(spec.default) }}{%- endif -%}
+{%- if spec.additionalProperties is defined -%}
+    {%- if spec.additionalProperties is mapping -%}
+{{- "\n- Additional properties *(" + render_markdown_type(spec.additionalProperties) + ")*" }}
+{{- render_markdown_schema_details(spec.additionalProperties, "", true) }}
+    {%- else -%}
+{{ render_markdown_metadata_detail("Additional properties", spec.additionalProperties) }}
+    {%- endif -%}
+{%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_markdown_schema_extras(spec, indent) -%}
+{%- set rendered_keys = ["type", "description", "enum", "default", "properties", "required", "items", "oneOf", "anyOf", "additionalProperties", "patternProperties", "returns"] -%}
+{%- for key, value in spec | items -%}
+    {%- if key not in rendered_keys -%}
+{{- "\n" + indent + "  - " + key + ": " }}{{ render_markdown_value(value) }}
+    {%- endif -%}
+{%- endfor -%}
+{%- endmacro -%}
+
+{%- macro render_markdown_metadata_extras(spec) -%}
+{%- set rendered_keys = ["type", "description", "enum", "default", "properties", "required", "items", "oneOf", "anyOf", "additionalProperties", "patternProperties", "returns"] -%}
+{%- for key, value in spec | items -%}
+    {%- if key not in rendered_keys -%}
+{{- "\n- " + key + ": " }}{{ render_markdown_value(value) }}
+    {%- endif -%}
+{%- endfor -%}
+{%- endmacro -%}
+
+{%- macro markdown_schema_has_extra(spec) -%}
+{%- set rendered_keys = ["type", "description", "enum", "default", "properties", "required", "items", "oneOf", "anyOf", "additionalProperties", "patternProperties", "returns"] -%}
+{%- set found = namespace(value='false') -%}
+{%- for key, value in spec | items -%}
+    {%- if key not in rendered_keys -%}{%- set found.value = 'true' -%}{%- endif -%}
+{%- endfor -%}
+{{- found.value -}}
+{%- endmacro -%}
+
+{%- macro markdown_parameter_schema_has_details(spec) -%}
+{%- if spec.description is defined or spec.enum is defined or spec.default is defined or spec.additionalProperties is defined or spec.patternProperties is defined or 'items' in spec or spec.oneOf or spec.anyOf or spec.returns is defined or markdown_schema_has_extra(spec) == 'true' -%}
+true
+{%- else -%}
+false
+{%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_markdown_schema_structure(spec, indent, include_properties) -%}
+{%- if include_properties and spec.properties -%}
+    {%- for child_name, child_spec in spec.properties | items -%}
+{{- render_markdown_param(child_name, child_spec, spec.required or [], indent + "  ") }}
+    {%- endfor -%}
+{%- endif -%}
+{%- if 'items' in spec and spec['items'] is mapping -%}
+{{- "\n" + indent + "  - Items *(" + render_markdown_type(spec['items']) + ")*" }}
+{{- render_markdown_schema_details(spec['items'], indent + "  ", true) }}
+{%- elif 'items' in spec -%}
+{{ render_markdown_detail(indent, "Items", spec['items']) }}
+{%- endif -%}
+{%- if spec.oneOf -%}
+{{- "\n" + indent + "  - oneOf:" }}
+    {%- for variant in spec.oneOf -%}
+{{- "\n" + indent + "    - Variant " }}{{ loop.index }}{{- " *(" + render_markdown_type(variant) + ")*" }}
+{{- render_markdown_schema_details(variant, indent + "    ", true) }}
+    {%- endfor -%}
+{%- endif -%}
+{%- if spec.anyOf -%}
+{{- "\n" + indent + "  - anyOf:" }}
+    {%- for variant in spec.anyOf -%}
+{{- "\n" + indent + "    - Variant " }}{{ loop.index }}{{- " *(" + render_markdown_type(variant) + ")*" }}
+{{- render_markdown_schema_details(variant, indent + "    ", true) }}
+    {%- endfor -%}
+{%- endif -%}
+{%- if spec.patternProperties is mapping -%}
+{{- "\n" + indent + "  - Pattern properties:" }}
+    {%- for pattern, pattern_spec in spec.patternProperties | items -%}
+        {%- if pattern_spec is mapping -%}
+{{- "\n" + indent + "    - `" + pattern + "` *(" + render_markdown_type(pattern_spec) + ")*" }}
+{{- render_markdown_schema_details(pattern_spec, indent + "    ", true) }}
+        {%- else -%}
+{{- "\n" + indent + "    - `" + pattern + "`: " }}{{ render_markdown_value(pattern_spec) }}
+        {%- endif -%}
+    {%- endfor -%}
+{%- elif spec.patternProperties is defined -%}
+{{ render_markdown_detail(indent, "Pattern properties", spec.patternProperties) }}
+{%- endif -%}
+{%- if spec.returns is mapping -%}
+{{- "\n" + indent + "  - Returns *(" + render_markdown_type(spec.returns) + ")*" }}
+{{- render_markdown_schema_details(spec.returns, indent + "  ", true) }}
+{%- elif spec.returns is defined -%}
+{{ render_markdown_detail(indent, "Returns", spec.returns) }}
+{%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_markdown_schema_details(spec, indent, include_value_details) -%}
+{%- if spec is mapping and spec['$ref'] is string -%}
+    {%- set _r = spec['$ref'] -%}
+    {%- set _k = _r[8:] if _r.startswith('#/$defs/') else (_r[14:] if _r.startswith('#/definitions/') else none) -%}
+    {%- if _k is not none and ('|' + _k + '|') not in REFS.seen and REFS.defs[_k] is mapping -%}
+        {%- set spec = dict((REFS.defs[_k] | items | list) + (spec | items | rejectattr('0', 'equalto', '$ref') | list)) -%}
+        {%- set REFS.seen = REFS.seen + _k + '|' -%}
+        {%- if spec['$ref'] is string -%}
+            {%- set _r2 = spec['$ref'] -%}
+            {%- set _k2 = _r2[8:] if _r2.startswith('#/$defs/') else (_r2[14:] if _r2.startswith('#/definitions/') else none) -%}
+            {%- if _k2 is not none and REFS.defs[_k2] is mapping -%}
+                {%- set spec = dict((REFS.defs[_k2] | items | list) + (spec | items | rejectattr('0', 'equalto', '$ref') | list)) -%}
+                {%- set REFS.seen = REFS.seen + _k2 + '|' -%}
+            {%- endif -%}
+        {%- endif -%}
+    {%- endif -%}
+{%- endif -%}
+{%- if spec is mapping -%}
+{{- render_markdown_schema_annotations(spec, indent, include_value_details) }}
+{{- render_markdown_schema_structure(spec, indent, true) }}
+{{- render_markdown_schema_extras(spec, indent) }}
+{%- elif spec is not sameas true and spec is not sameas false -%}
+{{- "\n" + indent + "  - Value: " }}{{ render_markdown_literal(spec) }}
+{%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_markdown_parameter_schema(spec) -%}
+{%- if spec is mapping and spec['$ref'] is string -%}
+    {%- set _r = spec['$ref'] -%}
+    {%- set _k = _r[8:] if _r.startswith('#/$defs/') else (_r[14:] if _r.startswith('#/definitions/') else none) -%}
+    {%- if _k is not none and ('|' + _k + '|') not in REFS.seen and REFS.defs[_k] is mapping -%}
+        {%- set spec = dict((REFS.defs[_k] | items | list) + (spec | items | rejectattr('0', 'equalto', '$ref') | list)) -%}
+        {%- set REFS.seen = REFS.seen + _k + '|' -%}
+        {%- if spec['$ref'] is string -%}
+            {%- set _r2 = spec['$ref'] -%}
+            {%- set _k2 = _r2[8:] if _r2.startswith('#/$defs/') else (_r2[14:] if _r2.startswith('#/definitions/') else none) -%}
+            {%- if _k2 is not none and REFS.defs[_k2] is mapping -%}
+                {%- set spec = dict((REFS.defs[_k2] | items | list) + (spec | items | rejectattr('0', 'equalto', '$ref') | list)) -%}
+                {%- set REFS.seen = REFS.seen + _k2 + '|' -%}
+            {%- endif -%}
+        {%- endif -%}
+    {%- endif -%}
+{%- endif -%}
+{%- if spec is mapping -%}
+{{- render_markdown_metadata_annotations(spec) }}
+{%- if 'items' in spec and spec['items'] is mapping -%}
+{{- "\n- Items *(" + render_markdown_type(spec['items']) + ")*" }}
+{{- render_markdown_schema_details(spec['items'], "", true) }}
+{%- elif 'items' in spec -%}
+{{ render_markdown_metadata_detail("Items", spec['items']) }}
+{%- endif -%}
+{%- if spec.oneOf -%}
+{{- "\n- oneOf:" }}
+    {%- for variant in spec.oneOf -%}
+{{- "\n  - Variant " }}{{ loop.index }}{{- " *(" + render_markdown_type(variant) + ")*" }}
+{{- render_markdown_schema_details(variant, "  ", true) }}
+    {%- endfor -%}
+{%- endif -%}
+{%- if spec.anyOf -%}
+{{- "\n- anyOf:" }}
+    {%- for variant in spec.anyOf -%}
+{{- "\n  - Variant " }}{{ loop.index }}{{- " *(" + render_markdown_type(variant) + ")*" }}
+{{- render_markdown_schema_details(variant, "  ", true) }}
+    {%- endfor -%}
+{%- endif -%}
+{%- if spec.patternProperties is mapping -%}
+{{- "\n- Pattern properties:" }}
+    {%- for pattern, pattern_spec in spec.patternProperties | items -%}
+        {%- if pattern_spec is mapping -%}
+{{- "\n  - `" + pattern + "` *(" + render_markdown_type(pattern_spec) + ")*" }}
+{{- render_markdown_schema_details(pattern_spec, "  ", true) }}
+        {%- else -%}
+{{- "\n  - `" + pattern + "`: " }}{{ render_markdown_value(pattern_spec) }}
+        {%- endif -%}
+    {%- endfor -%}
+{%- elif spec.patternProperties is defined -%}
+{{ render_markdown_metadata_detail("Pattern properties", spec.patternProperties) }}
+{%- endif -%}
+{%- if spec.returns is mapping -%}
+{{- "\n- Returns *(" + render_markdown_type(spec.returns) + ")*" }}
+{{- render_markdown_schema_details(spec.returns, "", true) }}
+{%- elif spec.returns is defined -%}
+{{ render_markdown_metadata_detail("Returns", spec.returns) }}
+{%- endif -%}
+{{- render_markdown_metadata_extras(spec) }}
+{%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_markdown_param(name, spec, required_list, indent) -%}
+{%- if spec is mapping and spec['$ref'] is string -%}
+    {%- set _r = spec['$ref'] -%}
+    {%- set _k = _r[8:] if _r.startswith('#/$defs/') else (_r[14:] if _r.startswith('#/definitions/') else none) -%}
+    {%- if _k is not none and ('|' + _k + '|') not in REFS.seen and REFS.defs[_k] is mapping -%}
+        {%- set spec = dict((REFS.defs[_k] | items | list) + (spec | items | rejectattr('0', 'equalto', '$ref') | list)) -%}
+        {%- set REFS.seen = REFS.seen + _k + '|' -%}
+        {%- if spec['$ref'] is string -%}
+            {%- set _r2 = spec['$ref'] -%}
+            {%- set _k2 = _r2[8:] if _r2.startswith('#/$defs/') else (_r2[14:] if _r2.startswith('#/definitions/') else none) -%}
+            {%- if _k2 is not none and REFS.defs[_k2] is mapping -%}
+                {%- set spec = dict((REFS.defs[_k2] | items | list) + (spec | items | rejectattr('0', 'equalto', '$ref') | list)) -%}
+                {%- set REFS.seen = REFS.seen + _k2 + '|' -%}
+            {%- endif -%}
+        {%- endif -%}
+    {%- endif -%}
+{%- endif -%}
+{{- "\n" + indent + "- `" + name + "` *(" + render_markdown_type(spec) }}
+{%- if name in (required_list or []) -%}{{- ", required" }}{%- endif -%}
+{{- ")*" }}
+{%- if spec.description -%}{{- " - " + spec.description | replace("\n", "\n" + indent + "  ") }}{%- endif -%}
+{%- if spec.enum -%}
+{{- "\n" + indent + "  - Allowed values: " }}{{ render_allowed_values(spec.enum) }}
+{%- endif -%}
+{%- if spec.default is defined -%}
+{{- "\n" + indent + "  - Default: " }}{{ render_markdown_literal(spec.default) }}
+{%- endif -%}
+{{- render_markdown_schema_details(spec, indent, false) }}
+{%- endmacro -%}
+
+{%- macro render_tools_markdown(tools_list) -%}
+{{- "<ifm|tools>" }}
+{%- for tool in tools_list -%}
+    {%- set fn = tool.function if tool.function is defined else tool -%}
+    {%- set REFS.defs = fn.parameters['$defs'] if (fn.parameters is mapping and fn.parameters['$defs'] is mapping) else (fn.parameters['definitions'] if (fn.parameters is mapping and fn.parameters['definitions'] is mapping) else {}) -%}
+    {%- set REFS.seen = '|' -%}
+    {%- set fnp = namespace(p=fn.parameters) -%}
+    {%- if fnp.p is mapping and fnp.p['$ref'] is string -%}
+        {%- set _r = fnp.p['$ref'] -%}
+        {%- set _k = _r[8:] if _r.startswith('#/$defs/') else (_r[14:] if _r.startswith('#/definitions/') else none) -%}
+        {%- if _k is not none and REFS.defs[_k] is mapping -%}
+            {%- set fnp.p = dict((REFS.defs[_k] | items | list) + (fnp.p | items | rejectattr('0', 'equalto', '$ref') | list)) -%}
+            {%- set REFS.seen = REFS.seen + _k + '|' -%}
+        {%- endif -%}
+    {%- endif -%}
+{{- "\n## " + fn.name }}
+{%- if fn.description -%}
+{{- "\n" + fn.description }}
+{%- endif -%}
+{{- "\n\n**Parameters**" }}
+{%- if fnp.p and fnp.p.properties -%}
+    {%- for pname, pspec in fnp.p.properties | items -%}
+{{- render_markdown_param(pname, pspec, fnp.p.required or [], "") }}
+    {%- endfor -%}
+{%- elif fnp.p is mapping and (fnp.p.oneOf or fnp.p.anyOf or 'items' in fnp.p) -%}
+{{- render_markdown_parameter_schema(fnp.p) }}
+{%- else -%}
+{{- "\n- None" }}
+{%- endif -%}
+{%- set fn_ret = fn.returns if fn.returns is defined else fn.response -%}
+{%- if fn_ret is mapping -%}
+{{- "\n\n**Returns**" }}
+{{- "\n- Return *(" + render_markdown_type(fn_ret) + ")*" }}
+{{- render_markdown_schema_details(fn_ret, "", true) }}
+{%- elif fn_ret is defined -%}
+{{- "\n\n**Returns**\n- " }}{{ render_markdown_value(fn_ret) }}
+{%- endif -%}
+{%- if not loop.last -%}{{- "\n" }}{%- endif -%}
+{%- endfor -%}
+{{- "\n</ifm|tools>" }}
+{%- endmacro -%}
+
+{%- macro render_tool_presentation(tools_list, fmt) -%}
+{%- if fmt == 'json' -%}
+{{- render_tools_json(tools_list) }}
+{%- elif RB.bad != '|' -%}
+{#- some tool uses constructs the pretty renderers cannot represent (verdicts -#}
+{#- computed during validate_tools): render the WHOLE toolset exactly as the -#}
+{#- json presentation would, so the block stays uniform and model-familiar. -#}
+{{- render_tools_json(tools_list) }}
+{%- elif fmt == 'xml' -%}
+{{- render_tools_xml(tools_list) }}
+{%- elif fmt == 'markdown' -%}
+{{- render_tools_markdown(tools_list) }}
+{%- else -%}
+{{- raise_exception("Unsupported tool_presentation_format: '" + fmt + "'. Supported formats: json, xml, markdown.") }}
+{%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_call_instructions(fmt) -%}
+{%- if fmt == 'json' -%}
+{{- "Wrap all tool calls in a single <ifm|tool_calls></ifm|tool_calls> block. For each call, emit one JSON object with the function name and arguments on the same line inside <ifm|tool_call></ifm|tool_call> tags:\n\n<ifm|tool_calls>\n<ifm|tool_call>{\"name\": <function-name>, \"arguments\": <args-json-object>}</ifm|tool_call>\n</ifm|tool_calls>" }}
+{%- elif fmt == 'xml' -%}
+{{- "Wrap all tool calls in a single <ifm|tool_calls></ifm|tool_calls> block. For each call, write the function name at the start of <ifm|tool_call>, followed by paired <ifm|arg_key> and <ifm|arg_value> tags for each argument:\n\n<ifm|tool_calls>\n<ifm|tool_call>$FUNCTION_NAME\n<ifm|arg_key>$PARAMETER_NAME</ifm|arg_key>\n<ifm|arg_value>$PARAMETER_VALUE</ifm|arg_value>\n...\n</ifm|tool_call>\n</ifm|tool_calls>\n\nString and scalar parameters should be written as plain text. Array and object parameters should be written as JSON literals." }}
+{%- elif fmt == 'xml_typed' -%}
+{{- "Wrap all tool calls in a single <ifm|tool_calls></ifm|tool_calls> block. For each call, write the function name at the start of <ifm|tool_call>, followed by <ifm|arg_key>, <ifm|arg_type>, and <ifm|arg_value> tags for each argument:\n\n<ifm|tool_calls>\n<ifm|tool_call>$FUNCTION_NAME\n<ifm|arg_key>$PARAMETER_NAME</ifm|arg_key>\n<ifm|arg_type>$ARGUMENT_TYPE</ifm|arg_type>\n<ifm|arg_value>$PARAMETER_VALUE</ifm|arg_value>\n...\n</ifm|tool_call>\n</ifm|tool_calls>\n\nUse the parameter type shown in the tool definition. If that type contains anyOf or oneOf, use the actual argument value type instead. String and scalar parameters should be written as plain text. Array and object parameters should be written as JSON literals." }}
+{%- else -%}
+{{- raise_exception("Unsupported tool_call_format: '" + fmt + "'. Supported formats: json, xml, xml_typed.") }}
+{%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_system_with_tools(tools_list, system_content, presentation_fmt, call_fmt) -%}
+{{- "<|ifm|im_start|>system\n# Tools\nYou may call one or more tools to assist with the user query.\n\nAvailable tools are:\n\n" }}
+{{- render_tool_presentation(tools_list, presentation_fmt) }}
+{{- "\n\nWhen calling tools, you MUST follow the tool-call format below:\n\n" }}
+{{- render_call_instructions(call_fmt) }}
+{%- if system_content -%}
+{{- "\n\n" + system_content }}
+{%- endif -%}
+{{- "<|ifm|im_end|>" }}
+{%- endmacro -%}
+
+{%- macro render_argument_value(value) -%}
+{%- if value is string -%}{{- value -}}{%- else -%}{{- value | tojson -}}{%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_value_type(value) -%}
+{%- if value is none -%}null
+{%- elif value is boolean -%}boolean
+{%- elif value is integer -%}integer
+{%- elif value is number -%}number
+{%- elif value is string -%}string
+{%- elif value is mapping -%}object
+{%- elif value is sequence -%}array
+{%- else -%}any
+{%- endif -%}
+{%- endmacro -%}
+
+{%- macro schema_has_combinator(spec) -%}
+{%- if spec.oneOf or spec.anyOf -%}
+true
+{%- elif spec.type is defined and spec.type is sequence and spec.type is not string and spec.type | length > 1 -%}
+true
+{%- elif spec.type == "array" and 'items' in spec -%}
+{{- schema_has_combinator(spec['items']) -}}
+{%- elif spec.properties -%}
+    {%- set found = namespace(value='false') -%}
+    {%- for child_name, child_spec in spec.properties | items -%}
+        {%- if schema_has_combinator(child_spec) == 'true' -%}
+            {%- set found.value = 'true' -%}
+        {%- endif -%}
+    {%- endfor -%}
+{{- found.value -}}
+{%- else -%}
+false
+{%- endif -%}
+{%- endmacro -%}
+
+{%- macro render_arg_type(tools_list, tool_name, arg_name, value) -%}
+{%- set found = namespace(type='any') -%}
+{%- for tool in tools_list -%}
+    {%- set fn = tool.function if tool.function is defined else tool -%}
+    {%- if fn.name == tool_name and fn.parameters and fn.parameters.properties and arg_name in fn.parameters.properties -%}
+        {%- set spec = fn.parameters.properties[arg_name] -%}
+        {%- if spec is mapping and spec['$ref'] is string -%}
+            {%- set _r = spec['$ref'] -%}
+            {%- set _k = _r[8:] if _r.startswith('#/$defs/') else (_r[14:] if _r.startswith('#/definitions/') else none) -%}
+            {%- set _d = fn.parameters['$defs'] if fn.parameters['$defs'] is mapping else fn.parameters['definitions'] -%}
+            {%- set spec = dict((_d[_k] | items | list) + (spec | items | rejectattr('0', 'equalto', '$ref') | list)) if (_k is not none and _d is mapping and _d[_k] is mapping) else spec -%}
+        {%- endif -%}
+        {%- if schema_has_combinator(spec) == 'true' -%}
+            {%- set found.type = render_value_type(value) -%}
+        {%- else -%}
+            {%- set found.type = render_compact_type(spec) -%}
+        {%- endif -%}
+    {%- endif -%}
+{%- endfor -%}
+{{- found.type -}}
+{%- endmacro -%}
+
+{%- macro render_tool_calls_block(tool_calls, fmt, tools_list) -%}
+{{- "<ifm|tool_calls>" }}
+{%- for raw_tool_call in tool_calls -%}
+    {%- set tool_call = raw_tool_call.function if raw_tool_call.function else raw_tool_call -%}
+    {%- if tool_call.arguments is string -%}
+        {{- raise_exception("tool_call.arguments must be a dict, not a JSON string. Parse it before passing to the template.") -}}
+    {%- endif -%}
+    {%- if fmt == 'json' -%}
+{{- "\n<ifm|tool_call>{\"name\": \"" + tool_call.name + "\", \"arguments\": " }}{{ tool_call.arguments | tojson }}{{- "}</ifm|tool_call>" }}
+    {%- elif fmt == 'xml' or fmt == 'xml_typed' -%}
+{{- "\n<ifm|tool_call>" + tool_call.name + "\n" }}
+        {%- for key, value in tool_call.arguments | items -%}
+{{- "<ifm|arg_key>" + key + "</ifm|arg_key>\n" }}
+{%- if fmt == 'xml_typed' -%}
+{{- "<ifm|arg_type>" + render_arg_type(tools_list, tool_call.name, key, value) + "</ifm|arg_type>\n" }}
+{%- endif -%}
+{{- "<ifm|arg_value>" }}{{ render_argument_value(value) }}{{- "</ifm|arg_value>\n" }}
+        {%- endfor -%}
+{{- "</ifm|tool_call>" }}
+    {%- else -%}
+        {{- raise_exception("Unsupported tool_call_format: '" + fmt + "'. Supported formats: json, xml, xml_typed.") -}}
+    {%- endif -%}
+{%- endfor -%}
+{{- "\n</ifm|tool_calls>" }}
+{%- endmacro -%}
+
+{%- macro render_tool_response_messages(raw_content) -%}
+{%- if raw_content is string -%}
+{{- '<|ifm|im_start|>tool\n' + raw_content + '<|ifm|im_end|>' }}
+{%- elif raw_content is sequence and raw_content is not string and raw_content is not mapping -%}
+    {%- if raw_content | length == 0 -%}
+        {{- raise_exception("tool message content list must not be empty.") -}}
+    {%- endif -%}
+{{- '<|ifm|im_start|>tool\n' -}}
+    {%- for item in raw_content -%}
+        {%- if not loop.first -%}{{- '\n' -}}{%- endif -%}
+        {%- if item is string -%}
+{{- item -}}
+        {%- elif item is mapping and item.text is string -%}
+{{- item.text -}}
+        {%- else -%}
+{{- (item | tojson) -}}
+        {%- endif -%}
+    {%- endfor -%}
+{{- '<|ifm|im_end|>' -}}
+{%- else -%}
+{{- '<|ifm|im_start|>tool\n' }}{{ raw_content | tojson }}{{- '<|ifm|im_end|>' }}
+{%- endif -%}
+{%- endmacro -%}
+
+{%- set available_tools = tools if tools else [] -%}
+{%- if (not available_tools) and messages[0].role == 'system' and messages[0].get('tools') -%}
+    {%- set available_tools = messages[0]['tools'] -%}
+{%- endif -%}
+{%- if available_tools -%}
+    {{- validate_tools(available_tools, tool_presentation_fmt != 'json') }}
+    {%- set system_content = '' -%}
+    {%- if messages[0].role == 'system' and messages[0].content -%}
+        {%- set system_content = messages[0].content -%}
+    {%- endif -%}
+    {{- render_system_with_tools(available_tools, system_content, tool_presentation_fmt, tool_call_fmt) }}
+{%- else -%}
+    {%- if messages[0].role == 'system' -%}
+        {{- '<|ifm|im_start|>system\n' + messages[0].content + '<|ifm|im_end|>' }}
+    {%- endif -%}
+{%- endif -%}
+
+{%- for message in messages -%}
+    {%- if message.content is string -%}
+        {%- set content = message.content -%}
+    {%- else -%}
+        {%- set content = '' -%}
+    {%- endif -%}
+    {%- if (message.role == "user") or (message.role == "system" and not loop.first) -%}
+        {{- '<|ifm|im_start|>' + message.role + '\n' + content + '<|ifm|im_end|>' }}
+    {%- elif message.role == "assistant" -%}
+        {%- set thinking_content = '' -%}
+        {%- set think_tag = 'ifm|think' -%}
+        {%- if message.think is defined and message.think is string -%}
+            {%- set thinking_content = message.think -%}
+            {%- set think_tag = 'ifm|think' -%}
+        {%- elif message.think_fast is defined and message.think_fast is string -%}
+            {%- set thinking_content = message.think_fast -%}
+            {%- set think_tag = 'ifm|think_fast' -%}
+        {%- elif message.think_faster is defined and message.think_faster is string -%}
+            {%- set thinking_content = message.think_faster -%}
+            {%- set think_tag = 'ifm|think_faster' -%}
+        {%- elif message.reasoning_content is defined and message.reasoning_content is string -%}
+            {%- set thinking_content = message.reasoning_content -%}
+            {%- set think_tag = 'ifm|think' -%}
+        {%- elif message.reasoning is defined and message.reasoning is string -%}
+            {%- set thinking_content = message.reasoning -%}
+            {%- set think_tag = 'ifm|think' -%}
+        {%- elif message.think is not defined and message.reasoning is not defined and message.reasoning_content is not defined and message.think_fast is not defined and message.think_faster is not defined -%}
+            {{- raise_exception("Assistant message is missing a thinking field. Provide one of: think, reasoning, reasoning_content, think_fast, think_faster.") -}}
+        {%- else -%}
+            {{- raise_exception("Assistant thinking fields must be strings. Provide one of: think, reasoning, reasoning_content, think_fast, think_faster as a string.") -}}
+        {%- endif -%}
+        {{- '<|ifm|im_start|>' + message.role }}
+        {% generation %}
+        {%- if think_tag -%}
+            {%- if thinking_content -%}
+                {{- '<' + think_tag + '>\n' + thinking_content + '</' + think_tag + '>' + content }}
+            {%- else -%}
+                {{- '<' + think_tag + '>\n</' + think_tag + '>' + content }}
+            {%- endif -%}
+        {%- else -%}
+            {{- content }}
+        {%- endif -%}
+        {%- if message.tool_calls -%}
+            {{- render_tool_calls_block(message.tool_calls, tool_call_fmt, available_tools) }}
+        {%- endif -%}
+        {{- '<|ifm|im_end|>' -}}
+        {%- endgeneration -%}
+    {%- elif message.role == "tool" -%}
+        {{- render_tool_response_messages(message.content) }}
+    {%- endif -%}
+{%- endfor -%}
+{%- if add_generation_prompt -%}
+    {%- set effort = reasoning_effort | default('high') -%}
+    {%- if effort == 'high' -%}
+        {{- '<|ifm|im_start|>assistant\n<ifm|think>\n' }}
+    {%- elif effort == 'medium' -%}
+        {{- '<|ifm|im_start|>assistant\n<ifm|think_fast>\n' }}
+    {%- elif effort == 'low' -%}
+        {{- '<|ifm|im_start|>assistant\n<ifm|think_faster>\n' }}
+    {%- else -%}
+        {{- raise_exception("Unsupported reasoning_effort: '" + effort + "'. Supported values: high, medium, low.") -}}
+    {%- endif -%}
+{%- endif -%}

config.json

@@ -0,0 +1,92 @@
+{
+  "architectures": [
+    "K2HorizonForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "attention_gate_func": null,
+  "auto_map": {
+    "AutoConfig": "configuration_k2_horizon.K2HorizonConfig",
+    "AutoModel": "modeling_k2_horizon.K2HorizonModel",
+    "AutoModelForCausalLM": "modeling_k2_horizon.K2HorizonForCausalLM"
+  },
+  "bos_token_id": 0,
+  "decoder_sparse_step": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": 1,
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 2560,
+  "initializer_range": 0.02,
+  "intermediate_size": 10240,
+  "layernorm_num_groups": 2,
+  "max_position_embeddings": 524288,
+  "mlp_only_layers": [
+    0,
+    1,
+    2,
+    3,
+    4,
+    5,
+    6,
+    7,
+    8,
+    9,
+    10,
+    11,
+    12,
+    13,
+    14,
+    15,
+    16,
+    17,
+    18,
+    19,
+    20,
+    21,
+    22,
+    23,
+    24,
+    25,
+    26,
+    27,
+    28,
+    29,
+    30,
+    31,
+    32,
+    33,
+    34,
+    35
+  ],
+  "model_type": "k2_horizon",
+  "moe_gate_bias": false,
+  "moe_intermediate_size": 0,
+  "mova_num_experts": 0,
+  "mova_num_experts_per_tok": 0,
+  "norm_topk_prob": true,
+  "num_attention_heads": 32,
+  "num_experts": 0,
+  "num_experts_per_tok": 0,
+  "num_hidden_layers": 36,
+  "num_key_value_heads": 8,
+  "num_shared_experts": 0,
+  "output_router_logits": false,
+  "pad_token_id": null,
+  "query_key_norm": false,
+  "rms_norm_eps": 1e-06,
+  "rope_head_dim": 128,
+  "rope_parameters": {
+    "rope_theta": 10000000.0,
+    "rope_type": "default"
+  },
+  "router_aux_loss_coef": 0.001,
+  "router_scaling_factor": 1.0,
+  "router_score_func": "sigmoid",
+  "sliding_window": null,
+  "tie_word_embeddings": false,
+  "transformers_version": "5.17.0",
+  "use_cache": true,
+  "use_sliding_window": false,
+  "vocab_size": 250624
+}

configuration_k2_horizon.py

@@ -0,0 +1,96 @@
+# Copyright 2024 The Qwen team, Alibaba Group and the HuggingFace Inc. team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+"""K2Horizon model configuration"""
+
+from huggingface_hub.dataclasses import strict
+
+from transformers.configuration_utils import PreTrainedConfig
+from transformers.modeling_rope_utils import RopeParameters
+
+
+@strict
+class K2HorizonConfig(PreTrainedConfig):
+    r"""
+    decoder_sparse_step (`int`, *optional*, defaults to 1):
+        The frequency of the MoE layer.
+    mlp_only_layers (`list[int]`, *optional*, defaults to `[]`):
+        Indicate which layers use K2HorizonMLP rather than K2HorizonSparseMoeBlock
+        The list contains layer index, from 0 to num_layers-1 if we have num_layers layers
+        If `mlp_only_layers` is empty, `decoder_sparse_step` is used to determine the sparsity.
+
+    ```python
+    >>> from transformers import K2HorizonModel, K2HorizonConfig
+
+    >>> # Initializing a K2Horizon style configuration
+    >>> configuration = K2HorizonConfig()
+    >>> model = K2HorizonModel(configuration)
+
+    >>> # Accessing the model configuration
+    >>> configuration = model.config
+    ```
+    """
+
+    model_type = "k2_horizon"
+    keys_to_ignore_at_inference = ["past_key_values"]
+
+    vocab_size: int = 151936
+    hidden_size: int = 2048
+    intermediate_size: int = 6144
+    num_hidden_layers: int = 24
+    num_attention_heads: int = 32
+    num_key_value_heads: int = 4
+    hidden_act: str = "silu"
+    max_position_embeddings: int = 32768
+    initializer_range: float = 0.02
+    rms_norm_eps: float = 1e-6
+    use_cache: bool = True
+    tie_word_embeddings: bool = False
+    rope_parameters: RopeParameters | dict | None = None
+    attention_bias: bool = False
+    use_sliding_window: bool = False
+    sliding_window: int | None = 4096
+    attention_dropout: float | int = 0.0
+    decoder_sparse_step: int = 1
+    moe_intermediate_size: int = 768
+    num_experts_per_tok: int = 8
+    num_experts: int = 128
+    norm_topk_prob: bool = False
+    output_router_logits: bool = False
+    router_aux_loss_coef: float = 0.001
+    mlp_only_layers: list[int] | None = None
+    pad_token_id: int | None = None
+    bos_token_id: int | None = None
+    eos_token_id: int | list[int] | None = None
+
+    head_dim: int = 128
+    query_key_norm: bool = True
+    moe_gate_bias: bool = False
+    layernorm_num_groups: int = 1
+    num_shared_experts: int = 0
+    router_score_func: str = "softmax"
+    router_scaling_factor: float | None = 1.0
+    rope_head_dim: int | None = None
+    attention_gate_func: str | None = None
+    mova_num_experts: int = 0
+    mova_num_experts_per_tok: int = 0
+
+    def __post_init__(self, **kwargs):
+        self.sliding_window = self.sliding_window if self.use_sliding_window else None
+        self.mlp_only_layers = [] if self.mlp_only_layers is None else self.mlp_only_layers
+        if self.router_scaling_factor is None:
+            self.router_scaling_factor = 1.0
+        super().__post_init__(**kwargs)
+
+
+__all__ = ["K2HorizonConfig"]

generation_config.json

@@ -0,0 +1,7 @@
+{
+    "bos_token_id": 0,
+    "eos_token_id": [
+        1,
+        250019
+    ]
+}

model.safetensors.index.json

@@ -0,0 +1,334 @@
+{
+  "metadata": {
+    "total_size": 10116510720
+  },
+  "weight_map": {
+    "model.layers.0.self_attn.q_proj.weight": "pytorch_model-00001-of-00036.safetensors",
+    "model.layers.0.self_attn.k_proj.weight": "pytorch_model-00001-of-00036.safetensors",
+    "model.layers.0.self_attn.v_proj.weight": "pytorch_model-00001-of-00036.safetensors",
+    "model.layers.0.self_attn.o_proj.weight": "pytorch_model-00001-of-00036.safetensors",
+    "model.layers.0.input_layernorm.weight": "pytorch_model-00001-of-00036.safetensors",
+    "model.layers.0.post_attention_layernorm.weight": "pytorch_model-00001-of-00036.safetensors",
+    "model.layers.0.mlp.gate_proj.weight": "pytorch_model-00001-of-00036.safetensors",
+    "model.layers.0.mlp.down_proj.weight": "pytorch_model-00001-of-00036.safetensors",
+    "model.layers.0.mlp.up_proj.weight": "pytorch_model-00001-of-00036.safetensors",
+    "model.layers.1.self_attn.q_proj.weight": "pytorch_model-00002-of-00036.safetensors",
+    "model.layers.1.self_attn.k_proj.weight": "pytorch_model-00002-of-00036.safetensors",
+    "model.layers.1.self_attn.v_proj.weight": "pytorch_model-00002-of-00036.safetensors",
+    "model.layers.1.self_attn.o_proj.weight": "pytorch_model-00002-of-00036.safetensors",
+    "model.layers.1.input_layernorm.weight": "pytorch_model-00002-of-00036.safetensors",
+    "model.layers.1.post_attention_layernorm.weight": "pytorch_model-00002-of-00036.safetensors",
+    "model.layers.1.mlp.gate_proj.weight": "pytorch_model-00002-of-00036.safetensors",
+    "model.layers.1.mlp.down_proj.weight": "pytorch_model-00002-of-00036.safetensors",
+    "model.layers.1.mlp.up_proj.weight": "pytorch_model-00002-of-00036.safetensors",
+    "model.layers.2.self_attn.q_proj.weight": "pytorch_model-00003-of-00036.safetensors",
+    "model.layers.2.self_attn.k_proj.weight": "pytorch_model-00003-of-00036.safetensors",
+    "model.layers.2.self_attn.v_proj.weight": "pytorch_model-00003-of-00036.safetensors",
+    "model.layers.2.self_attn.o_proj.weight": "pytorch_model-00003-of-00036.safetensors",
+    "model.layers.2.input_layernorm.weight": "pytorch_model-00003-of-00036.safetensors",
+    "model.layers.2.post_attention_layernorm.weight": "pytorch_model-00003-of-00036.safetensors",
+    "model.layers.2.mlp.gate_proj.weight": "pytorch_model-00003-of-00036.safetensors",
+    "model.layers.2.mlp.down_proj.weight": "pytorch_model-00003-of-00036.safetensors",
+    "model.layers.2.mlp.up_proj.weight": "pytorch_model-00003-of-00036.safetensors",
+    "model.layers.3.self_attn.q_proj.weight": "pytorch_model-00004-of-00036.safetensors",
+    "model.layers.3.self_attn.k_proj.weight": "pytorch_model-00004-of-00036.safetensors",
+    "model.layers.3.self_attn.v_proj.weight": "pytorch_model-00004-of-00036.safetensors",
+    "model.layers.3.self_attn.o_proj.weight": "pytorch_model-00004-of-00036.safetensors",
+    "model.layers.3.input_layernorm.weight": "pytorch_model-00004-of-00036.safetensors",
+    "model.layers.3.post_attention_layernorm.weight": "pytorch_model-00004-of-00036.safetensors",
+    "model.layers.3.mlp.gate_proj.weight": "pytorch_model-00004-of-00036.safetensors",
+    "model.layers.3.mlp.down_proj.weight": "pytorch_model-00004-of-00036.safetensors",
+    "model.layers.3.mlp.up_proj.weight": "pytorch_model-00004-of-00036.safetensors",
+    "model.layers.4.self_attn.q_proj.weight": "pytorch_model-00005-of-00036.safetensors",
+    "model.layers.4.self_attn.k_proj.weight": "pytorch_model-00005-of-00036.safetensors",
+    "model.layers.4.self_attn.v_proj.weight": "pytorch_model-00005-of-00036.safetensors",
+    "model.layers.4.self_attn.o_proj.weight": "pytorch_model-00005-of-00036.safetensors",
+    "model.layers.4.input_layernorm.weight": "pytorch_model-00005-of-00036.safetensors",
+    "model.layers.4.post_attention_layernorm.weight": "pytorch_model-00005-of-00036.safetensors",
+    "model.layers.4.mlp.gate_proj.weight": "pytorch_model-00005-of-00036.safetensors",
+    "model.layers.4.mlp.down_proj.weight": "pytorch_model-00005-of-00036.safetensors",
+    "model.layers.4.mlp.up_proj.weight": "pytorch_model-00005-of-00036.safetensors",
+    "model.layers.5.self_attn.q_proj.weight": "pytorch_model-00006-of-00036.safetensors",
+    "model.layers.5.self_attn.k_proj.weight": "pytorch_model-00006-of-00036.safetensors",
+    "model.layers.5.self_attn.v_proj.weight": "pytorch_model-00006-of-00036.safetensors",
+    "model.layers.5.self_attn.o_proj.weight": "pytorch_model-00006-of-00036.safetensors",
+    "model.layers.5.input_layernorm.weight": "pytorch_model-00006-of-00036.safetensors",
+    "model.layers.5.post_attention_layernorm.weight": "pytorch_model-00006-of-00036.safetensors",
+    "model.layers.5.mlp.gate_proj.weight": "pytorch_model-00006-of-00036.safetensors",
+    "model.layers.5.mlp.down_proj.weight": "pytorch_model-00006-of-00036.safetensors",
+    "model.layers.5.mlp.up_proj.weight": "pytorch_model-00006-of-00036.safetensors",
+    "model.layers.6.self_attn.q_proj.weight": "pytorch_model-00007-of-00036.safetensors",
+    "model.layers.6.self_attn.k_proj.weight": "pytorch_model-00007-of-00036.safetensors",
+    "model.layers.6.self_attn.v_proj.weight": "pytorch_model-00007-of-00036.safetensors",
+    "model.layers.6.self_attn.o_proj.weight": "pytorch_model-00007-of-00036.safetensors",
+    "model.layers.6.input_layernorm.weight": "pytorch_model-00007-of-00036.safetensors",
+    "model.layers.6.post_attention_layernorm.weight": "pytorch_model-00007-of-00036.safetensors",
+    "model.layers.6.mlp.gate_proj.weight": "pytorch_model-00007-of-00036.safetensors",
+    "model.layers.6.mlp.down_proj.weight": "pytorch_model-00007-of-00036.safetensors",
+    "model.layers.6.mlp.up_proj.weight": "pytorch_model-00007-of-00036.safetensors",
+    "model.layers.7.self_attn.q_proj.weight": "pytorch_model-00008-of-00036.safetensors",
+    "model.layers.7.self_attn.k_proj.weight": "pytorch_model-00008-of-00036.safetensors",
+    "model.layers.7.self_attn.v_proj.weight": "pytorch_model-00008-of-00036.safetensors",
+    "model.layers.7.self_attn.o_proj.weight": "pytorch_model-00008-of-00036.safetensors",
+    "model.layers.7.input_layernorm.weight": "pytorch_model-00008-of-00036.safetensors",
+    "model.layers.7.post_attention_layernorm.weight": "pytorch_model-00008-of-00036.safetensors",
+    "model.layers.7.mlp.gate_proj.weight": "pytorch_model-00008-of-00036.safetensors",
+    "model.layers.7.mlp.down_proj.weight": "pytorch_model-00008-of-00036.safetensors",
+    "model.layers.7.mlp.up_proj.weight": "pytorch_model-00008-of-00036.safetensors",
+    "model.layers.8.self_attn.q_proj.weight": "pytorch_model-00009-of-00036.safetensors",
+    "model.layers.8.self_attn.k_proj.weight": "pytorch_model-00009-of-00036.safetensors",
+    "model.layers.8.self_attn.v_proj.weight": "pytorch_model-00009-of-00036.safetensors",
+    "model.layers.8.self_attn.o_proj.weight": "pytorch_model-00009-of-00036.safetensors",
+    "model.layers.8.input_layernorm.weight": "pytorch_model-00009-of-00036.safetensors",
+    "model.layers.8.post_attention_layernorm.weight": "pytorch_model-00009-of-00036.safetensors",
+    "model.layers.8.mlp.gate_proj.weight": "pytorch_model-00009-of-00036.safetensors",
+    "model.layers.8.mlp.down_proj.weight": "pytorch_model-00009-of-00036.safetensors",
+    "model.layers.8.mlp.up_proj.weight": "pytorch_model-00009-of-00036.safetensors",
+    "model.layers.9.self_attn.q_proj.weight": "pytorch_model-00010-of-00036.safetensors",
+    "model.layers.9.self_attn.k_proj.weight": "pytorch_model-00010-of-00036.safetensors",
+    "model.layers.9.self_attn.v_proj.weight": "pytorch_model-00010-of-00036.safetensors",
+    "model.layers.9.self_attn.o_proj.weight": "pytorch_model-00010-of-00036.safetensors",
+    "model.layers.9.input_layernorm.weight": "pytorch_model-00010-of-00036.safetensors",
+    "model.layers.9.post_attention_layernorm.weight": "pytorch_model-00010-of-00036.safetensors",
+    "model.layers.9.mlp.gate_proj.weight": "pytorch_model-00010-of-00036.safetensors",
+    "model.layers.9.mlp.down_proj.weight": "pytorch_model-00010-of-00036.safetensors",
+    "model.layers.9.mlp.up_proj.weight": "pytorch_model-00010-of-00036.safetensors",
+    "model.layers.10.self_attn.q_proj.weight": "pytorch_model-00011-of-00036.safetensors",
+    "model.layers.10.self_attn.k_proj.weight": "pytorch_model-00011-of-00036.safetensors",
+    "model.layers.10.self_attn.v_proj.weight": "pytorch_model-00011-of-00036.safetensors",
+    "model.layers.10.self_attn.o_proj.weight": "pytorch_model-00011-of-00036.safetensors",
+    "model.layers.10.input_layernorm.weight": "pytorch_model-00011-of-00036.safetensors",
+    "model.layers.10.post_attention_layernorm.weight": "pytorch_model-00011-of-00036.safetensors",
+    "model.layers.10.mlp.gate_proj.weight": "pytorch_model-00011-of-00036.safetensors",
+    "model.layers.10.mlp.down_proj.weight": "pytorch_model-00011-of-00036.safetensors",
+    "model.layers.10.mlp.up_proj.weight": "pytorch_model-00011-of-00036.safetensors",
+    "model.layers.11.self_attn.q_proj.weight": "pytorch_model-00012-of-00036.safetensors",
+    "model.layers.11.self_attn.k_proj.weight": "pytorch_model-00012-of-00036.safetensors",
+    "model.layers.11.self_attn.v_proj.weight": "pytorch_model-00012-of-00036.safetensors",
+    "model.layers.11.self_attn.o_proj.weight": "pytorch_model-00012-of-00036.safetensors",
+    "model.layers.11.input_layernorm.weight": "pytorch_model-00012-of-00036.safetensors",
+    "model.layers.11.post_attention_layernorm.weight": "pytorch_model-00012-of-00036.safetensors",
+    "model.layers.11.mlp.gate_proj.weight": "pytorch_model-00012-of-00036.safetensors",
+    "model.layers.11.mlp.down_proj.weight": "pytorch_model-00012-of-00036.safetensors",
+    "model.layers.11.mlp.up_proj.weight": "pytorch_model-00012-of-00036.safetensors",
+    "model.layers.12.self_attn.q_proj.weight": "pytorch_model-00013-of-00036.safetensors",
+    "model.layers.12.self_attn.k_proj.weight": "pytorch_model-00013-of-00036.safetensors",
+    "model.layers.12.self_attn.v_proj.weight": "pytorch_model-00013-of-00036.safetensors",
+    "model.layers.12.self_attn.o_proj.weight": "pytorch_model-00013-of-00036.safetensors",
+    "model.layers.12.input_layernorm.weight": "pytorch_model-00013-of-00036.safetensors",
+    "model.layers.12.post_attention_layernorm.weight": "pytorch_model-00013-of-00036.safetensors",
+    "model.layers.12.mlp.gate_proj.weight": "pytorch_model-00013-of-00036.safetensors",
+    "model.layers.12.mlp.down_proj.weight": "pytorch_model-00013-of-00036.safetensors",
+    "model.layers.12.mlp.up_proj.weight": "pytorch_model-00013-of-00036.safetensors",
+    "model.layers.13.self_attn.q_proj.weight": "pytorch_model-00014-of-00036.safetensors",
+    "model.layers.13.self_attn.k_proj.weight": "pytorch_model-00014-of-00036.safetensors",
+    "model.layers.13.self_attn.v_proj.weight": "pytorch_model-00014-of-00036.safetensors",
+    "model.layers.13.self_attn.o_proj.weight": "pytorch_model-00014-of-00036.safetensors",
+    "model.layers.13.input_layernorm.weight": "pytorch_model-00014-of-00036.safetensors",
+    "model.layers.13.post_attention_layernorm.weight": "pytorch_model-00014-of-00036.safetensors",
+    "model.layers.13.mlp.gate_proj.weight": "pytorch_model-00014-of-00036.safetensors",
+    "model.layers.13.mlp.down_proj.weight": "pytorch_model-00014-of-00036.safetensors",
+    "model.layers.13.mlp.up_proj.weight": "pytorch_model-00014-of-00036.safetensors",
+    "model.layers.14.self_attn.q_proj.weight": "pytorch_model-00015-of-00036.safetensors",
+    "model.layers.14.self_attn.k_proj.weight": "pytorch_model-00015-of-00036.safetensors",
+    "model.layers.14.self_attn.v_proj.weight": "pytorch_model-00015-of-00036.safetensors",
+    "model.layers.14.self_attn.o_proj.weight": "pytorch_model-00015-of-00036.safetensors",
+    "model.layers.14.input_layernorm.weight": "pytorch_model-00015-of-00036.safetensors",
+    "model.layers.14.post_attention_layernorm.weight": "pytorch_model-00015-of-00036.safetensors",
+    "model.layers.14.mlp.gate_proj.weight": "pytorch_model-00015-of-00036.safetensors",
+    "model.layers.14.mlp.down_proj.weight": "pytorch_model-00015-of-00036.safetensors",
+    "model.layers.14.mlp.up_proj.weight": "pytorch_model-00015-of-00036.safetensors",
+    "model.layers.15.self_attn.q_proj.weight": "pytorch_model-00016-of-00036.safetensors",
+    "model.layers.15.self_attn.k_proj.weight": "pytorch_model-00016-of-00036.safetensors",
+    "model.layers.15.self_attn.v_proj.weight": "pytorch_model-00016-of-00036.safetensors",
+    "model.layers.15.self_attn.o_proj.weight": "pytorch_model-00016-of-00036.safetensors",
+    "model.layers.15.input_layernorm.weight": "pytorch_model-00016-of-00036.safetensors",
+    "model.layers.15.post_attention_layernorm.weight": "pytorch_model-00016-of-00036.safetensors",
+    "model.layers.15.mlp.gate_proj.weight": "pytorch_model-00016-of-00036.safetensors",
+    "model.layers.15.mlp.down_proj.weight": "pytorch_model-00016-of-00036.safetensors",
+    "model.layers.15.mlp.up_proj.weight": "pytorch_model-00016-of-00036.safetensors",
+    "model.layers.16.self_attn.q_proj.weight": "pytorch_model-00017-of-00036.safetensors",
+    "model.layers.16.self_attn.k_proj.weight": "pytorch_model-00017-of-00036.safetensors",
+    "model.layers.16.self_attn.v_proj.weight": "pytorch_model-00017-of-00036.safetensors",
+    "model.layers.16.self_attn.o_proj.weight": "pytorch_model-00017-of-00036.safetensors",
+    "model.layers.16.input_layernorm.weight": "pytorch_model-00017-of-00036.safetensors",
+    "model.layers.16.post_attention_layernorm.weight": "pytorch_model-00017-of-00036.safetensors",
+    "model.layers.16.mlp.gate_proj.weight": "pytorch_model-00017-of-00036.safetensors",
+    "model.layers.16.mlp.down_proj.weight": "pytorch_model-00017-of-00036.safetensors",
+    "model.layers.16.mlp.up_proj.weight": "pytorch_model-00017-of-00036.safetensors",
+    "model.layers.17.self_attn.q_proj.weight": "pytorch_model-00018-of-00036.safetensors",
+    "model.layers.17.self_attn.k_proj.weight": "pytorch_model-00018-of-00036.safetensors",
+    "model.layers.17.self_attn.v_proj.weight": "pytorch_model-00018-of-00036.safetensors",
+    "model.layers.17.self_attn.o_proj.weight": "pytorch_model-00018-of-00036.safetensors",
+    "model.layers.17.input_layernorm.weight": "pytorch_model-00018-of-00036.safetensors",
+    "model.layers.17.post_attention_layernorm.weight": "pytorch_model-00018-of-00036.safetensors",
+    "model.layers.17.mlp.gate_proj.weight": "pytorch_model-00018-of-00036.safetensors",
+    "model.layers.17.mlp.down_proj.weight": "pytorch_model-00018-of-00036.safetensors",
+    "model.layers.17.mlp.up_proj.weight": "pytorch_model-00018-of-00036.safetensors",
+    "model.layers.18.self_attn.q_proj.weight": "pytorch_model-00019-of-00036.safetensors",
+    "model.layers.18.self_attn.k_proj.weight": "pytorch_model-00019-of-00036.safetensors",
+    "model.layers.18.self_attn.v_proj.weight": "pytorch_model-00019-of-00036.safetensors",
+    "model.layers.18.self_attn.o_proj.weight": "pytorch_model-00019-of-00036.safetensors",
+    "model.layers.18.input_layernorm.weight": "pytorch_model-00019-of-00036.safetensors",
+    "model.layers.18.post_attention_layernorm.weight": "pytorch_model-00019-of-00036.safetensors",
+    "model.layers.18.mlp.gate_proj.weight": "pytorch_model-00019-of-00036.safetensors",
+    "model.layers.18.mlp.down_proj.weight": "pytorch_model-00019-of-00036.safetensors",
+    "model.layers.18.mlp.up_proj.weight": "pytorch_model-00019-of-00036.safetensors",
+    "model.layers.19.self_attn.q_proj.weight": "pytorch_model-00020-of-00036.safetensors",
+    "model.layers.19.self_attn.k_proj.weight": "pytorch_model-00020-of-00036.safetensors",
+    "model.layers.19.self_attn.v_proj.weight": "pytorch_model-00020-of-00036.safetensors",
+    "model.layers.19.self_attn.o_proj.weight": "pytorch_model-00020-of-00036.safetensors",
+    "model.layers.19.input_layernorm.weight": "pytorch_model-00020-of-00036.safetensors",
+    "model.layers.19.post_attention_layernorm.weight": "pytorch_model-00020-of-00036.safetensors",
+    "model.layers.19.mlp.gate_proj.weight": "pytorch_model-00020-of-00036.safetensors",
+    "model.layers.19.mlp.down_proj.weight": "pytorch_model-00020-of-00036.safetensors",
+    "model.layers.19.mlp.up_proj.weight": "pytorch_model-00020-of-00036.safetensors",
+    "model.layers.20.self_attn.q_proj.weight": "pytorch_model-00021-of-00036.safetensors",
+    "model.layers.20.self_attn.k_proj.weight": "pytorch_model-00021-of-00036.safetensors",
+    "model.layers.20.self_attn.v_proj.weight": "pytorch_model-00021-of-00036.safetensors",
+    "model.layers.20.self_attn.o_proj.weight": "pytorch_model-00021-of-00036.safetensors",
+    "model.layers.20.input_layernorm.weight": "pytorch_model-00021-of-00036.safetensors",
+    "model.layers.20.post_attention_layernorm.weight": "pytorch_model-00021-of-00036.safetensors",
+    "model.layers.20.mlp.gate_proj.weight": "pytorch_model-00021-of-00036.safetensors",
+    "model.layers.20.mlp.down_proj.weight": "pytorch_model-00021-of-00036.safetensors",
+    "model.layers.20.mlp.up_proj.weight": "pytorch_model-00021-of-00036.safetensors",
+    "model.layers.21.self_attn.q_proj.weight": "pytorch_model-00022-of-00036.safetensors",
+    "model.layers.21.self_attn.k_proj.weight": "pytorch_model-00022-of-00036.safetensors",
+    "model.layers.21.self_attn.v_proj.weight": "pytorch_model-00022-of-00036.safetensors",
+    "model.layers.21.self_attn.o_proj.weight": "pytorch_model-00022-of-00036.safetensors",
+    "model.layers.21.input_layernorm.weight": "pytorch_model-00022-of-00036.safetensors",
+    "model.layers.21.post_attention_layernorm.weight": "pytorch_model-00022-of-00036.safetensors",
+    "model.layers.21.mlp.gate_proj.weight": "pytorch_model-00022-of-00036.safetensors",
+    "model.layers.21.mlp.down_proj.weight": "pytorch_model-00022-of-00036.safetensors",
+    "model.layers.21.mlp.up_proj.weight": "pytorch_model-00022-of-00036.safetensors",
+    "model.layers.22.self_attn.q_proj.weight": "pytorch_model-00023-of-00036.safetensors",
+    "model.layers.22.self_attn.k_proj.weight": "pytorch_model-00023-of-00036.safetensors",
+    "model.layers.22.self_attn.v_proj.weight": "pytorch_model-00023-of-00036.safetensors",
+    "model.layers.22.self_attn.o_proj.weight": "pytorch_model-00023-of-00036.safetensors",
+    "model.layers.22.input_layernorm.weight": "pytorch_model-00023-of-00036.safetensors",
+    "model.layers.22.post_attention_layernorm.weight": "pytorch_model-00023-of-00036.safetensors",
+    "model.layers.22.mlp.gate_proj.weight": "pytorch_model-00023-of-00036.safetensors",
+    "model.layers.22.mlp.down_proj.weight": "pytorch_model-00023-of-00036.safetensors",
+    "model.layers.22.mlp.up_proj.weight": "pytorch_model-00023-of-00036.safetensors",
+    "model.layers.23.self_attn.q_proj.weight": "pytorch_model-00024-of-00036.safetensors",
+    "model.layers.23.self_attn.k_proj.weight": "pytorch_model-00024-of-00036.safetensors",
+    "model.layers.23.self_attn.v_proj.weight": "pytorch_model-00024-of-00036.safetensors",
+    "model.layers.23.self_attn.o_proj.weight": "pytorch_model-00024-of-00036.safetensors",
+    "model.layers.23.input_layernorm.weight": "pytorch_model-00024-of-00036.safetensors",
+    "model.layers.23.post_attention_layernorm.weight": "pytorch_model-00024-of-00036.safetensors",
+    "model.layers.23.mlp.gate_proj.weight": "pytorch_model-00024-of-00036.safetensors",
+    "model.layers.23.mlp.down_proj.weight": "pytorch_model-00024-of-00036.safetensors",
+    "model.layers.23.mlp.up_proj.weight": "pytorch_model-00024-of-00036.safetensors",
+    "model.layers.24.self_attn.q_proj.weight": "pytorch_model-00025-of-00036.safetensors",
+    "model.layers.24.self_attn.k_proj.weight": "pytorch_model-00025-of-00036.safetensors",
+    "model.layers.24.self_attn.v_proj.weight": "pytorch_model-00025-of-00036.safetensors",
+    "model.layers.24.self_attn.o_proj.weight": "pytorch_model-00025-of-00036.safetensors",
+    "model.layers.24.input_layernorm.weight": "pytorch_model-00025-of-00036.safetensors",
+    "model.layers.24.post_attention_layernorm.weight": "pytorch_model-00025-of-00036.safetensors",
+    "model.layers.24.mlp.gate_proj.weight": "pytorch_model-00025-of-00036.safetensors",
+    "model.layers.24.mlp.down_proj.weight": "pytorch_model-00025-of-00036.safetensors",
+    "model.layers.24.mlp.up_proj.weight": "pytorch_model-00025-of-00036.safetensors",
+    "model.layers.25.self_attn.q_proj.weight": "pytorch_model-00026-of-00036.safetensors",
+    "model.layers.25.self_attn.k_proj.weight": "pytorch_model-00026-of-00036.safetensors",
+    "model.layers.25.self_attn.v_proj.weight": "pytorch_model-00026-of-00036.safetensors",
+    "model.layers.25.self_attn.o_proj.weight": "pytorch_model-00026-of-00036.safetensors",
+    "model.layers.25.input_layernorm.weight": "pytorch_model-00026-of-00036.safetensors",
+    "model.layers.25.post_attention_layernorm.weight": "pytorch_model-00026-of-00036.safetensors",
+    "model.layers.25.mlp.gate_proj.weight": "pytorch_model-00026-of-00036.safetensors",
+    "model.layers.25.mlp.down_proj.weight": "pytorch_model-00026-of-00036.safetensors",
+    "model.layers.25.mlp.up_proj.weight": "pytorch_model-00026-of-00036.safetensors",
+    "model.layers.26.self_attn.q_proj.weight": "pytorch_model-00027-of-00036.safetensors",
+    "model.layers.26.self_attn.k_proj.weight": "pytorch_model-00027-of-00036.safetensors",
+    "model.layers.26.self_attn.v_proj.weight": "pytorch_model-00027-of-00036.safetensors",
+    "model.layers.26.self_attn.o_proj.weight": "pytorch_model-00027-of-00036.safetensors",
+    "model.layers.26.input_layernorm.weight": "pytorch_model-00027-of-00036.safetensors",
+    "model.layers.26.post_attention_layernorm.weight": "pytorch_model-00027-of-00036.safetensors",
+    "model.layers.26.mlp.gate_proj.weight": "pytorch_model-00027-of-00036.safetensors",
+    "model.layers.26.mlp.down_proj.weight": "pytorch_model-00027-of-00036.safetensors",
+    "model.layers.26.mlp.up_proj.weight": "pytorch_model-00027-of-00036.safetensors",
+    "model.layers.27.self_attn.q_proj.weight": "pytorch_model-00028-of-00036.safetensors",
+    "model.layers.27.self_attn.k_proj.weight": "pytorch_model-00028-of-00036.safetensors",
+    "model.layers.27.self_attn.v_proj.weight": "pytorch_model-00028-of-00036.safetensors",
+    "model.layers.27.self_attn.o_proj.weight": "pytorch_model-00028-of-00036.safetensors",
+    "model.layers.27.input_layernorm.weight": "pytorch_model-00028-of-00036.safetensors",
+    "model.layers.27.post_attention_layernorm.weight": "pytorch_model-00028-of-00036.safetensors",
+    "model.layers.27.mlp.gate_proj.weight": "pytorch_model-00028-of-00036.safetensors",
+    "model.layers.27.mlp.down_proj.weight": "pytorch_model-00028-of-00036.safetensors",
+    "model.layers.27.mlp.up_proj.weight": "pytorch_model-00028-of-00036.safetensors",
+    "model.layers.28.self_attn.q_proj.weight": "pytorch_model-00029-of-00036.safetensors",
+    "model.layers.28.self_attn.k_proj.weight": "pytorch_model-00029-of-00036.safetensors",
+    "model.layers.28.self_attn.v_proj.weight": "pytorch_model-00029-of-00036.safetensors",
+    "model.layers.28.self_attn.o_proj.weight": "pytorch_model-00029-of-00036.safetensors",
+    "model.layers.28.input_layernorm.weight": "pytorch_model-00029-of-00036.safetensors",
+    "model.layers.28.post_attention_layernorm.weight": "pytorch_model-00029-of-00036.safetensors",
+    "model.layers.28.mlp.gate_proj.weight": "pytorch_model-00029-of-00036.safetensors",
+    "model.layers.28.mlp.down_proj.weight": "pytorch_model-00029-of-00036.safetensors",
+    "model.layers.28.mlp.up_proj.weight": "pytorch_model-00029-of-00036.safetensors",
+    "model.layers.29.self_attn.q_proj.weight": "pytorch_model-00030-of-00036.safetensors",
+    "model.layers.29.self_attn.k_proj.weight": "pytorch_model-00030-of-00036.safetensors",
+    "model.layers.29.self_attn.v_proj.weight": "pytorch_model-00030-of-00036.safetensors",
+    "model.layers.29.self_attn.o_proj.weight": "pytorch_model-00030-of-00036.safetensors",
+    "model.layers.29.input_layernorm.weight": "pytorch_model-00030-of-00036.safetensors",
+    "model.layers.29.post_attention_layernorm.weight": "pytorch_model-00030-of-00036.safetensors",
+    "model.layers.29.mlp.gate_proj.weight": "pytorch_model-00030-of-00036.safetensors",
+    "model.layers.29.mlp.down_proj.weight": "pytorch_model-00030-of-00036.safetensors",
+    "model.layers.29.mlp.up_proj.weight": "pytorch_model-00030-of-00036.safetensors",
+    "model.layers.30.self_attn.q_proj.weight": "pytorch_model-00031-of-00036.safetensors",
+    "model.layers.30.self_attn.k_proj.weight": "pytorch_model-00031-of-00036.safetensors",
+    "model.layers.30.self_attn.v_proj.weight": "pytorch_model-00031-of-00036.safetensors",
+    "model.layers.30.self_attn.o_proj.weight": "pytorch_model-00031-of-00036.safetensors",
+    "model.layers.30.input_layernorm.weight": "pytorch_model-00031-of-00036.safetensors",
+    "model.layers.30.post_attention_layernorm.weight": "pytorch_model-00031-of-00036.safetensors",
+    "model.layers.30.mlp.gate_proj.weight": "pytorch_model-00031-of-00036.safetensors",
+    "model.layers.30.mlp.down_proj.weight": "pytorch_model-00031-of-00036.safetensors",
+    "model.layers.30.mlp.up_proj.weight": "pytorch_model-00031-of-00036.safetensors",
+    "model.layers.31.self_attn.q_proj.weight": "pytorch_model-00032-of-00036.safetensors",
+    "model.layers.31.self_attn.k_proj.weight": "pytorch_model-00032-of-00036.safetensors",
+    "model.layers.31.self_attn.v_proj.weight": "pytorch_model-00032-of-00036.safetensors",
+    "model.layers.31.self_attn.o_proj.weight": "pytorch_model-00032-of-00036.safetensors",
+    "model.layers.31.input_layernorm.weight": "pytorch_model-00032-of-00036.safetensors",
+    "model.layers.31.post_attention_layernorm.weight": "pytorch_model-00032-of-00036.safetensors",
+    "model.layers.31.mlp.gate_proj.weight": "pytorch_model-00032-of-00036.safetensors",
+    "model.layers.31.mlp.down_proj.weight": "pytorch_model-00032-of-00036.safetensors",
+    "model.layers.31.mlp.up_proj.weight": "pytorch_model-00032-of-00036.safetensors",
+    "model.layers.32.self_attn.q_proj.weight": "pytorch_model-00033-of-00036.safetensors",
+    "model.layers.32.self_attn.k_proj.weight": "pytorch_model-00033-of-00036.safetensors",
+    "model.layers.32.self_attn.v_proj.weight": "pytorch_model-00033-of-00036.safetensors",
+    "model.layers.32.self_attn.o_proj.weight": "pytorch_model-00033-of-00036.safetensors",
+    "model.layers.32.input_layernorm.weight": "pytorch_model-00033-of-00036.safetensors",
+    "model.layers.32.post_attention_layernorm.weight": "pytorch_model-00033-of-00036.safetensors",
+    "model.layers.32.mlp.gate_proj.weight": "pytorch_model-00033-of-00036.safetensors",
+    "model.layers.32.mlp.down_proj.weight": "pytorch_model-00033-of-00036.safetensors",
+    "model.layers.32.mlp.up_proj.weight": "pytorch_model-00033-of-00036.safetensors",
+    "model.layers.33.self_attn.q_proj.weight": "pytorch_model-00034-of-00036.safetensors",
+    "model.layers.33.self_attn.k_proj.weight": "pytorch_model-00034-of-00036.safetensors",
+    "model.layers.33.self_attn.v_proj.weight": "pytorch_model-00034-of-00036.safetensors",
+    "model.layers.33.self_attn.o_proj.weight": "pytorch_model-00034-of-00036.safetensors",
+    "model.layers.33.input_layernorm.weight": "pytorch_model-00034-of-00036.safetensors",
+    "model.layers.33.post_attention_layernorm.weight": "pytorch_model-00034-of-00036.safetensors",
+    "model.layers.33.mlp.gate_proj.weight": "pytorch_model-00034-of-00036.safetensors",
+    "model.layers.33.mlp.down_proj.weight": "pytorch_model-00034-of-00036.safetensors",
+    "model.layers.33.mlp.up_proj.weight": "pytorch_model-00034-of-00036.safetensors",
+    "model.layers.34.self_attn.q_proj.weight": "pytorch_model-00035-of-00036.safetensors",
+    "model.layers.34.self_attn.k_proj.weight": "pytorch_model-00035-of-00036.safetensors",
+    "model.layers.34.self_attn.v_proj.weight": "pytorch_model-00035-of-00036.safetensors",
+    "model.layers.34.self_attn.o_proj.weight": "pytorch_model-00035-of-00036.safetensors",
+    "model.layers.34.input_layernorm.weight": "pytorch_model-00035-of-00036.safetensors",
+    "model.layers.34.post_attention_layernorm.weight": "pytorch_model-00035-of-00036.safetensors",
+    "model.layers.34.mlp.gate_proj.weight": "pytorch_model-00035-of-00036.safetensors",
+    "model.layers.34.mlp.down_proj.weight": "pytorch_model-00035-of-00036.safetensors",
+    "model.layers.34.mlp.up_proj.weight": "pytorch_model-00035-of-00036.safetensors",
+    "model.layers.35.self_attn.q_proj.weight": "pytorch_model-00036-of-00036.safetensors",
+    "model.layers.35.self_attn.k_proj.weight": "pytorch_model-00036-of-00036.safetensors",
+    "model.layers.35.self_attn.v_proj.weight": "pytorch_model-00036-of-00036.safetensors",
+    "model.layers.35.self_attn.o_proj.weight": "pytorch_model-00036-of-00036.safetensors",
+    "model.layers.35.input_layernorm.weight": "pytorch_model-00036-of-00036.safetensors",
+    "model.layers.35.post_attention_layernorm.weight": "pytorch_model-00036-of-00036.safetensors",
+    "model.layers.35.mlp.gate_proj.weight": "pytorch_model-00036-of-00036.safetensors",
+    "model.layers.35.mlp.down_proj.weight": "pytorch_model-00036-of-00036.safetensors",
+    "model.layers.35.mlp.up_proj.weight": "pytorch_model-00036-of-00036.safetensors",
+    "model.embed_tokens.weight": "pytorch_model-00036-of-00036.safetensors",
+    "model.norm.weight": "pytorch_model-00036-of-00036.safetensors",
+    "lm_head.weight": "pytorch_model-00036-of-00036.safetensors"
+  }
+}

modeling_k2_horizon.py

@@ -0,0 +1,1116 @@
+#                🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨
+#           This file was automatically generated from src/transformers/models/qwen3_moe/modular_qwen3_moe.py.
+#               Do NOT edit this file manually as any edits will be overwritten by the generation of
+#             the file from the modular. If any change should be done, please apply the change to the
+#                          modular_qwen3_moe.py file directly. One of our CI enforces this.
+#                🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨🚨
+# coding=utf-8
+# Copyright 2025 The Qwen team, Alibaba Group and the HuggingFace Inc. team. All rights reserved.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+from typing import Callable, Optional, Union
+
+import math
+import torch
+import torch.nn.functional as F
+from torch import nn
+
+from transformers.activations import ACT2FN
+from transformers.cache_utils import Cache, DynamicCache
+from transformers.generation import GenerationMixin
+from transformers.masking_utils import create_causal_mask, create_sliding_window_causal_mask
+from transformers.modeling_flash_attention_utils import FlashAttentionKwargs
+from transformers.modeling_layers import (
+    GenericForQuestionAnswering,
+    GenericForSequenceClassification,
+    GenericForTokenClassification,
+    GradientCheckpointingLayer,
+)
+from transformers.modeling_outputs import MoeCausalLMOutputWithPast, MoeModelOutputWithPast
+from transformers.modeling_rope_utils import ROPE_INIT_FUNCTIONS, dynamic_rope_update
+from transformers.modeling_utils import ALL_ATTENTION_FUNCTIONS, PreTrainedModel
+from transformers.processing_utils import Unpack
+from transformers.utils import TransformersKwargs, auto_docstring, can_return_tuple
+from transformers.utils.generic import maybe_autocast
+from transformers.utils.deprecation import deprecate_kwarg
+from transformers.utils.output_capturing import OutputRecorder
+
+from .configuration_k2_horizon import K2HorizonConfig
+
+
+def rotate_half(x):
+    """Rotates half the hidden dims of the input."""
+    x1 = x[..., : x.shape[-1] // 2]
+    x2 = x[..., x.shape[-1] // 2:]
+    return torch.cat((-x2, x1), dim=-1)
+
+
+def apply_rotary_pos_emb(q, k, cos, sin, position_ids=None, unsqueeze_dim=1):
+    """Applies Rotary Position Embedding to the query and key tensors.
+
+    Args:
+        q (`torch.Tensor`): The query tensor.
+        k (`torch.Tensor`): The key tensor.
+        cos (`torch.Tensor`): The cosine part of the rotary embedding.
+        sin (`torch.Tensor`): The sine part of the rotary embedding.
+        position_ids (`torch.Tensor`, *optional*):
+            Deprecated and unused.
+        unsqueeze_dim (`int`, *optional*, defaults to 1):
+            The 'unsqueeze_dim' argument specifies the dimension along which to unsqueeze cos[position_ids] and
+            sin[position_ids] so that they can be properly broadcasted to the dimensions of q and k. For example, note
+            that cos[position_ids] and sin[position_ids] have the shape [batch_size, seq_len, head_dim]. Then, if q and
+            k have the shape [batch_size, heads, seq_len, head_dim], then setting unsqueeze_dim=1 makes
+            cos[position_ids] and sin[position_ids] broadcastable to the shapes of q and k. Similarly, if q and k have
+            the shape [batch_size, seq_len, heads, head_dim], then set unsqueeze_dim=2.
+    Returns:
+        `tuple(torch.Tensor)` comprising of the query and key tensors rotated using the Rotary Position Embedding.
+    """
+    cos = cos.unsqueeze(unsqueeze_dim)
+    sin = sin.unsqueeze(unsqueeze_dim)
+    q_embed = (q * cos) + (rotate_half(q) * sin)
+    k_embed = (k * cos) + (rotate_half(k) * sin)
+    return q_embed, k_embed
+
+
+def repeat_kv(hidden_states: torch.Tensor, n_rep: int) -> torch.Tensor:
+    """
+    This is the equivalent of torch.repeat_interleave(x, dim=1, repeats=n_rep). The hidden states go from (batch,
+    num_key_value_heads, seqlen, head_dim) to (batch, num_attention_heads, seqlen, head_dim)
+    """
+    batch, num_key_value_heads, slen, head_dim = hidden_states.shape
+    if n_rep == 1:
+        return hidden_states
+    hidden_states = hidden_states[:, :, None, :, :].expand(batch, num_key_value_heads, n_rep, slen, head_dim)
+    return hidden_states.reshape(batch, num_key_value_heads * n_rep, slen, head_dim)
+
+
+def split_to_interleaved(x):
+    # Split halves:  x0 x1 x2 x3 ... y0 y1 y2 y3 ...
+    # Interleaved:   x0 y0 x1 y1 x2 y2 x3 y3 ...
+    return x.reshape(*x.shape[:-1], 2, -1).transpose(-1, -2).reshape(*x.shape[:-1], -1)
+
+
+def interleaved_to_split(x):
+    # Interleaved:   x0 y0 x1 y1 x2 y2 x3 y3 ...
+    # Split halves:  x0 x1 x2 x3 ... y0 y1 y2 y3 ...
+    return x.reshape(*x.shape[:-1], -1, 2).transpose(-1, -2).reshape(*x.shape[:-1], -1)
+
+
+def eager_attention_forward(
+        module: nn.Module,
+        query: torch.Tensor,
+        key: torch.Tensor,
+        value: torch.Tensor,
+        attention_mask: Optional[torch.Tensor],
+        scaling: float,
+        dropout: float = 0.0,
+        **kwargs: Unpack[TransformersKwargs],
+):
+    key_states = repeat_kv(key, module.num_key_value_groups)
+    value_states = repeat_kv(value, module.num_key_value_groups)
+
+    attn_weights = torch.matmul(query, key_states.transpose(2, 3)) * scaling
+    if attention_mask is not None:
+        causal_mask = attention_mask[:, :, :, : key_states.shape[-2]]
+        attn_weights = attn_weights + causal_mask
+
+    attn_weights = nn.functional.softmax(attn_weights, dim=-1, dtype=torch.float32).to(query.dtype)
+    attn_weights = nn.functional.dropout(attn_weights, p=dropout, training=module.training)
+    attn_output = torch.matmul(attn_weights, value_states)
+    attn_output = attn_output.transpose(1, 2).contiguous()
+
+    return attn_output, attn_weights
+
+
+def calc_router_weights(
+    router_logits: torch.Tensor,
+    router_bias: Optional[torch.Tensor],
+    score_func: str,
+    top_k: int,
+    scaling_factor: Optional[float],
+) -> tuple[torch.Tensor, torch.Tensor]:
+    """Return native-XLLM-compatible routing weights and selected experts.
+
+    XLLM applies router bias only to the values used for top-k selection. The
+    selected routes are still weighted by the original router probabilities,
+    then optionally normalized and scaled.
+    """
+    if score_func == "softmax":
+        routing_scores = F.softmax(router_logits, dim=-1, dtype=torch.float32)
+    elif score_func == "sigmoid":
+        routing_scores = torch.sigmoid(router_logits.to(torch.float32))
+    else:
+        raise ValueError(f"Unsupported router score function: {score_func}")
+
+    selection_scores = routing_scores
+    if router_bias is not None:
+        selection_scores = selection_scores + router_bias.to(selection_scores)
+
+    selected_indices = torch.topk(selection_scores, top_k, dim=-1).indices
+    routing_weights = torch.gather(routing_scores, dim=-1, index=selected_indices)
+    if top_k > 1:
+        routing_weights = routing_weights / routing_weights.sum(dim=-1, keepdim=True)
+    if scaling_factor is not None:
+        routing_weights = routing_weights * scaling_factor
+    return routing_weights, selected_indices
+
+
+def combine_routed_experts(
+    hidden_states: torch.Tensor,
+    routing_weights: torch.Tensor,
+    selected_indices: torch.Tensor,
+    experts: nn.ModuleList,
+    activation: Optional[Callable[[torch.Tensor], torch.Tensor]] = None,
+) -> torch.Tensor:
+    num_tokens, hidden_dim = hidden_states.shape
+    final_hidden_states = torch.zeros(
+        (num_tokens, experts[0].out_features),
+        dtype=hidden_states.dtype,
+        device=hidden_states.device,
+    )
+    expert_mask = torch.nn.functional.one_hot(
+        selected_indices, num_classes=len(experts)
+    ).permute(2, 1, 0)
+
+    for expert_idx in torch.nonzero(expert_mask.sum(dim=(-1, -2)), as_tuple=False).flatten():
+        topk_positions, token_positions = torch.where(expert_mask[int(expert_idx)])
+        expert_states = experts[int(expert_idx)](hidden_states[token_positions])
+        if activation is not None:
+            expert_states = activation(expert_states)
+        expert_states = expert_states * routing_weights[token_positions, topk_positions, None].to(expert_states.dtype)
+        final_hidden_states.index_add_(0, token_positions, expert_states.to(hidden_states.dtype))
+
+    return final_hidden_states
+
+
+class K2HorizonAttention(nn.Module):
+    """Multi-headed attention from 'Attention Is All You Need' paper"""
+
+    def __init__(self, config: K2HorizonConfig, layer_idx: int):
+        super().__init__()
+        self.config = config
+        self.layer_idx = layer_idx
+        self.head_dim = getattr(config, "head_dim", config.hidden_size // config.num_attention_heads)
+        self.num_key_value_groups = config.num_attention_heads // config.num_key_value_heads
+        self.scaling = self.head_dim ** -0.5
+        self.attention_dropout = config.attention_dropout
+        self.is_causal = True
+
+        self.rope_head_dim = self.head_dim if config.rope_head_dim is None else config.rope_head_dim
+
+        self.q_proj = nn.Linear(
+            config.hidden_size, config.num_attention_heads * self.head_dim, bias=config.attention_bias
+        )
+        self.k_proj = nn.Linear(
+            config.hidden_size, config.num_key_value_heads * self.head_dim, bias=config.attention_bias
+        )
+        self.v_proj = nn.Linear(
+            config.hidden_size, config.num_key_value_heads * self.head_dim, bias=config.attention_bias
+        )
+        self.o_proj = nn.Linear(
+            config.num_attention_heads * self.head_dim, config.hidden_size, bias=config.attention_bias
+        )
+
+        self.gate_func = config.attention_gate_func
+        if self.gate_func is not None:
+            self.gate_proj = nn.Linear(
+                config.hidden_size,
+                config.num_attention_heads * self.head_dim,
+                bias=False)
+
+        if config.query_key_norm:
+            self.q_norm = K2HorizonRMSNorm(
+                hidden_size=config.num_attention_heads * self.head_dim,
+                n_groups=config.num_attention_heads,
+                eps=config.rms_norm_eps)
+            self.k_norm = K2HorizonRMSNorm(
+                hidden_size=config.num_key_value_heads * self.head_dim,
+                n_groups=config.num_key_value_heads,
+                eps=config.rms_norm_eps)
+
+        self.sliding_window = getattr(config, "sliding_window", None)
+
+    @deprecate_kwarg("past_key_value", new_name="past_key_values", version="4.58")
+    def forward(
+            self,
+            hidden_states: torch.Tensor,
+            position_embeddings: tuple[torch.Tensor, torch.Tensor],
+            attention_mask: Optional[torch.Tensor],
+            past_key_values: Optional[Cache] = None,
+            cache_position: Optional[torch.LongTensor] = None,
+            **kwargs: Unpack[FlashAttentionKwargs],
+    ) -> tuple[torch.Tensor, Optional[torch.Tensor]]:
+        input_shape = hidden_states.shape[:-1]
+        hidden_shape = (*input_shape, -1, self.head_dim)
+
+        if self.config.query_key_norm:
+            query_states = self.q_norm(self.q_proj(hidden_states)).view(hidden_shape).transpose(1, 2)
+            key_states = self.k_norm(self.k_proj(hidden_states)).view(hidden_shape).transpose(1, 2)
+        else:
+            query_states = self.q_proj(hidden_states).view(hidden_shape).transpose(1, 2)
+            key_states = self.k_proj(hidden_states).view(hidden_shape).transpose(1, 2)
+
+        value_states = self.v_proj(hidden_states).view(hidden_shape).transpose(1, 2)
+
+        cos, sin = position_embeddings
+        if self.rope_head_dim == self.head_dim:
+            query_states, key_states = apply_rotary_pos_emb(query_states, key_states, cos, sin)
+        else:
+            query_states, query_states_ = torch.split(
+                split_to_interleaved(query_states),
+                split_size_or_sections=[self.rope_head_dim, self.head_dim - self.rope_head_dim],
+                dim=-1)
+
+            key_states, key_states_ = torch.split(
+                split_to_interleaved(key_states),
+                split_size_or_sections=[self.rope_head_dim, self.head_dim - self.rope_head_dim],
+                dim=-1)
+
+            query_states, key_states = apply_rotary_pos_emb(
+                interleaved_to_split(query_states),
+                interleaved_to_split(key_states),
+                cos,
+                sin)
+
+            query_states = interleaved_to_split(torch.cat(
+                [split_to_interleaved(query_states), query_states_], dim=-1))
+            key_states = interleaved_to_split(torch.cat(
+                [split_to_interleaved(key_states), key_states_], dim=-1))
+
+        if past_key_values is not None:
+            # sin and cos are specific to RoPE models; cache_position needed for the static cache
+            cache_kwargs = {"sin": sin, "cos": cos, "cache_position": cache_position}
+            key_states, value_states = past_key_values.update(key_states, value_states, self.layer_idx, cache_kwargs)
+
+        attention_interface: Callable = eager_attention_forward
+        if self.config._attn_implementation != "eager":
+            attention_interface = ALL_ATTENTION_FUNCTIONS[self.config._attn_implementation]
+
+        attn_output, attn_weights = attention_interface(
+            self,
+            query_states,
+            key_states,
+            value_states,
+            attention_mask,
+            dropout=0.0 if not self.training else self.attention_dropout,
+            scaling=self.scaling,
+            sliding_window=self.sliding_window,  # diff with Llama
+            **kwargs,
+        )
+
+        if self.gate_func is not None:
+            gate = self.gate_proj(hidden_states).view(
+                input_shape + (-1, self.head_dim))
+            if self.gate_func == 'silu':
+                gate = F.silu(gate)
+            else:
+                assert self.gate_func == 'softplus'
+                gate = F.softplus(gate, beta=math.log(2))
+
+            attn_output = attn_output * gate
+
+        attn_output = attn_output.reshape(*input_shape, -1).contiguous()
+        attn_output = self.o_proj(attn_output)
+        return attn_output, attn_weights
+
+
+def apply_rotary_pos_emb_xllm(q, k, freqs_cis):
+    if q.shape[-1] % 2 != 0 or k.shape[-1] % 2 != 0:
+        raise ValueError(f"RoPE dimensions must be even, got q={q.shape[-1]} and k={k.shape[-1]}")
+    q_ = torch.view_as_complex(q.float().reshape(*q.shape[:-1], -1, 2))
+    k_ = torch.view_as_complex(k.float().reshape(*k.shape[:-1], -1, 2))
+    if freqs_cis.ndim == 2:
+        freqs_cis = freqs_cis.unsqueeze(1)
+    elif freqs_cis.ndim == 3:
+        freqs_cis = freqs_cis.unsqueeze(2)
+    else:
+        raise ValueError(f"Unsupported freqs_cis shape: {tuple(freqs_cis.shape)}")
+    if freqs_cis.shape[-1] != q_.shape[-1] or freqs_cis.shape[-1] != k_.shape[-1]:
+        raise ValueError(
+            "RoPE frequency dimension mismatch: "
+            f"q_rope_dim={q.shape[-1]}, k_rope_dim={k.shape[-1]}, "
+            f"q_complex_dim={q_.shape[-1]}, k_complex_dim={k_.shape[-1]}, "
+            f"freqs_complex_dim={freqs_cis.shape[-1]}, freqs_shape={tuple(freqs_cis.shape)}"
+        )
+    q_embed = torch.view_as_real(q_ * freqs_cis).flatten(-2).to(q.dtype)
+    k_embed = torch.view_as_real(k_ * freqs_cis).flatten(-2).to(k.dtype)
+    return q_embed, k_embed
+
+
+class K2HorizonMoVAAttention(nn.Module):
+    """MoVA attention with routed value experts and optional post-attention gate."""
+
+    def __init__(self, config: K2HorizonConfig, layer_idx: int):
+        super().__init__()
+        self.config = config
+        self.layer_idx = layer_idx
+        self.head_dim = getattr(config, "head_dim", config.hidden_size // config.num_attention_heads)
+        self.num_key_value_groups = config.num_attention_heads // config.num_key_value_heads
+        self.scaling = self.head_dim**-0.5
+        self.attention_dropout = config.attention_dropout
+        self.is_causal = True
+
+        self.num_experts_per_tok = config.mova_num_experts_per_tok
+        self.router_score_func = config.router_score_func
+        self.router_scaling_factor = config.router_scaling_factor
+        self.gate_func = config.attention_gate_func
+
+        self.rope_head_dim = self.head_dim if config.rope_head_dim is None else config.rope_head_dim
+
+        self.q_proj = nn.Linear(
+            config.hidden_size, config.num_attention_heads * self.head_dim, bias=config.attention_bias
+        )
+        self.k_proj = nn.Linear(
+            config.hidden_size, config.num_key_value_heads * self.head_dim, bias=config.attention_bias
+        )
+        self.o_proj = nn.Linear(
+            config.num_attention_heads * self.head_dim, config.hidden_size, bias=config.attention_bias
+        )
+        self.v_router = nn.Linear(
+            config.hidden_size,
+            config.mova_num_experts,
+            bias=config.moe_gate_bias)
+        self.v_experts = nn.ModuleList([
+            nn.Linear(
+                config.hidden_size,
+                config.num_key_value_heads * self.head_dim,
+                bias=False
+            ) for _ in range(config.mova_num_experts)
+        ])
+
+        if self.gate_func is not None:
+            self.gate_proj = nn.Linear(
+                config.hidden_size,
+                config.num_attention_heads * self.head_dim,
+                bias=False)
+
+        if config.query_key_norm:
+            self.q_norm = K2HorizonRMSNorm(
+                hidden_size=config.num_attention_heads * self.head_dim,
+                n_groups=config.num_attention_heads,
+                eps=config.rms_norm_eps,
+            )
+            self.k_norm = K2HorizonRMSNorm(
+                hidden_size=config.num_key_value_heads * self.head_dim,
+                n_groups=config.num_key_value_heads,
+                eps=config.rms_norm_eps,
+            )
+
+        self.sliding_window = getattr(config, "sliding_window", None)
+
+    @deprecate_kwarg("past_key_value", new_name="past_key_values", version="4.58")
+    def forward(
+        self,
+        hidden_states: torch.Tensor,
+        position_embeddings: torch.Tensor,
+        attention_mask: Optional[torch.Tensor],
+        past_key_values: Optional[Cache] = None,
+        cache_position: Optional[torch.LongTensor] = None,
+        **kwargs: Unpack[FlashAttentionKwargs],
+    ) -> tuple[torch.Tensor, Optional[torch.Tensor]]:
+        input_shape = hidden_states.shape[:-1]
+        hidden_shape = (*input_shape, -1, self.head_dim)
+        flat_hidden_states = hidden_states.reshape(-1, hidden_states.shape[-1])
+
+        # Match native MOVAttention router semantics exactly: compute logits with
+        # the weight-only linear and apply router bias only to selection scores.
+        router_logits = F.linear(flat_hidden_states, self.v_router.weight)
+        routing_weights, selected_values = calc_router_weights(
+            router_logits=router_logits,
+            router_bias=self.v_router.bias,
+            score_func=self.router_score_func,
+            top_k=self.num_experts_per_tok,
+            scaling_factor=self.router_scaling_factor,
+        )
+
+        mixed_value_states = combine_routed_experts(
+            hidden_states=flat_hidden_states,
+            routing_weights=routing_weights,
+            selected_indices=selected_values,
+            experts=self.v_experts,
+            activation=F.silu)
+
+        value_states = mixed_value_states.view(hidden_shape).transpose(1, 2)
+
+        if self.config.query_key_norm:
+            query_states = self.q_norm(self.q_proj(hidden_states)).view(hidden_shape).transpose(1, 2)
+            key_states = self.k_norm(self.k_proj(hidden_states)).view(hidden_shape).transpose(1, 2)
+        else:
+            query_states = self.q_proj(hidden_states).view(hidden_shape).transpose(1, 2)
+            key_states = self.k_proj(hidden_states).view(hidden_shape).transpose(1, 2)
+
+        cos, sin = position_embeddings
+        if self.rope_head_dim == self.head_dim:
+            query_states, key_states = apply_rotary_pos_emb(query_states, key_states, cos, sin)
+        else:
+            query_states, query_states_ = torch.split(
+                split_to_interleaved(query_states),
+                split_size_or_sections=[self.rope_head_dim, self.head_dim - self.rope_head_dim],
+                dim=-1)
+
+            key_states, key_states_ = torch.split(
+                split_to_interleaved(key_states),
+                split_size_or_sections=[self.rope_head_dim, self.head_dim - self.rope_head_dim],
+                dim=-1)
+
+            query_states, key_states = apply_rotary_pos_emb(
+                interleaved_to_split(query_states),
+                interleaved_to_split(key_states),
+                cos,
+                sin)
+
+            query_states = interleaved_to_split(torch.cat(
+                [split_to_interleaved(query_states), query_states_], dim=-1))
+            key_states = interleaved_to_split(torch.cat(
+                [split_to_interleaved(key_states), key_states_], dim=-1))
+
+        if past_key_values is not None:
+            cache_kwargs = {"cache_position": cache_position}
+            key_states, value_states = past_key_values.update(
+                key_states, value_states, self.layer_idx, cache_kwargs
+            )
+
+        attention_interface: Callable = eager_attention_forward
+        if self.config._attn_implementation != "eager":
+            attention_interface = ALL_ATTENTION_FUNCTIONS[self.config._attn_implementation]
+
+        attn_output, attn_weights = attention_interface(
+            self,
+            query_states,
+            key_states,
+            value_states,
+            attention_mask,
+            dropout=0.0 if not self.training else self.attention_dropout,
+            scaling=self.scaling,
+            sliding_window=self.sliding_window,
+            **kwargs,
+        )
+
+        if self.gate_func is not None:
+            gate = self.gate_proj(hidden_states).view(input_shape + (-1, self.head_dim))
+            if self.gate_func == 'silu':
+                gate = F.silu(gate)
+            else:
+                assert self.gate_func == 'softplus'
+                gate = F.softplus(gate, beta=math.log(2))
+
+            attn_output = attn_output * gate
+
+        attn_output = attn_output.reshape(*input_shape, -1).contiguous()
+        attn_output = self.o_proj(attn_output)
+        return attn_output, attn_weights
+
+
+class K2HorizonMLP(nn.Module):
+    def __init__(self, config, intermediate_size=None):
+        super().__init__()
+        self.config = config
+        self.hidden_size = config.hidden_size
+        self.intermediate_size = intermediate_size if intermediate_size is not None else config.intermediate_size
+        self.gate_proj = nn.Linear(self.hidden_size, self.intermediate_size, bias=False)
+        self.up_proj = nn.Linear(self.hidden_size, self.intermediate_size, bias=False)
+        self.down_proj = nn.Linear(self.intermediate_size, self.hidden_size, bias=False)
+        self.act_fn = ACT2FN[config.hidden_act]
+
+    def forward(self, x):
+        down_proj = self.down_proj(self.act_fn(self.gate_proj(x)) * self.up_proj(x))
+        return down_proj
+
+
+class K2HorizonSparseMoeBlock(nn.Module):
+    def __init__(self, config):
+        super().__init__()
+        self.num_experts = config.num_experts
+        self.top_k = config.num_experts_per_tok
+        self.norm_topk_prob = config.norm_topk_prob
+        self.num_shared_experts = config.num_shared_experts
+        self.router_score_func = config.router_score_func
+        self.router_scaling_factor = config.router_scaling_factor
+
+        # gating
+        self.gate = nn.Linear(config.hidden_size, config.num_experts, bias=config.moe_gate_bias)
+        self.experts = nn.ModuleList(
+            [K2HorizonMLP(config, intermediate_size=config.moe_intermediate_size) for _ in range(self.num_experts)]
+        )
+
+        if config.num_shared_experts > 0:
+            self.shared_experts = K2HorizonMLP(
+                config=config,
+                intermediate_size=config.moe_intermediate_size * config.num_shared_experts)
+
+    def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:
+        """ """
+        residuals = hidden_states
+
+        batch_size, sequence_length, hidden_dim = hidden_states.shape
+        hidden_states = hidden_states.view(-1, hidden_dim)
+        # router_logits: (batch * sequence_length, n_experts)
+        # router_logits = self.gate(hidden_states)
+        router_logits = F.linear(hidden_states, self.gate.weight)
+
+        if self.router_score_func == "softmax":
+            routing_weights = F.softmax(router_logits, dim=1, dtype=torch.float)
+        else:
+            assert self.router_score_func == "sigmoid"
+            routing_weights = F.sigmoid(router_logits.to(torch.float32))
+
+        routing_weights_for_choice = routing_weights
+        if self.gate.bias is not None:
+            routing_weights_for_choice = routing_weights + self.gate.bias.to(routing_weights.dtype)
+
+        _, selected_experts = torch.topk(routing_weights_for_choice, self.top_k, dim=-1)
+        routing_weights = torch.gather(routing_weights, dim=-1, index=selected_experts)
+
+        if self.norm_topk_prob:  # only diff with mixtral sparse moe block!
+            routing_weights /= routing_weights.sum(dim=-1, keepdim=True)
+        routing_weights = routing_weights * self.router_scaling_factor
+        # we cast back to the input dtype
+        routing_weights = routing_weights.to(hidden_states.dtype)
+
+        final_hidden_states = torch.zeros(
+            (batch_size * sequence_length, hidden_dim), dtype=hidden_states.dtype, device=hidden_states.device
+        )
+
+        # One hot encode the selected experts to create an expert mask
+        # this will be used to easily index which expert is going to be sollicitated
+        expert_mask = torch.nn.functional.one_hot(selected_experts, num_classes=self.num_experts).permute(2, 1, 0)
+
+        # Loop over all available experts in the model and perform the computation on each expert
+        expert_hit = torch.greater(expert_mask.sum(dim=(-1, -2)), 0).nonzero()
+        for expert_idx in expert_hit:
+            expert_layer = self.experts[expert_idx]
+            idx, top_x = torch.where(expert_mask[expert_idx].squeeze(0))
+
+            # Index the correct hidden states and compute the expert hidden state for
+            # the current expert. We need to make sure to multiply the output hidden
+            # states by `routing_weights` on the corresponding tokens (top-1 and top-2)
+            current_state = hidden_states[None, top_x].reshape(-1, hidden_dim)
+            current_hidden_states = expert_layer(current_state) * routing_weights[top_x, idx, None]
+
+            # However `index_add_` only support torch tensors for indexing so we'll use
+            # the `top_x` tensor here.
+            final_hidden_states.index_add_(0, top_x, current_hidden_states.to(hidden_states.dtype))
+        final_hidden_states = final_hidden_states.reshape(batch_size, sequence_length, hidden_dim)
+
+        if self.num_shared_experts > 0:
+            final_hidden_states = final_hidden_states + self.shared_experts(residuals)
+
+        return final_hidden_states, router_logits
+
+
+# @use_kernel_forward_from_hub("RMSNorm")
+class K2HorizonRMSNorm(nn.Module):
+    def __init__(self, hidden_size: int, n_groups: int, eps=1e-6):
+        """
+        K2HorizonRMSNorm is equivalent to T5LayerNorm
+        """
+        super().__init__()
+        self.n_groups = n_groups
+        self.hidden_size = hidden_size
+        assert hidden_size % n_groups == 0
+        self.weight = nn.Parameter(torch.ones(hidden_size))
+        self.variance_epsilon = eps
+
+    def forward(self, hidden_states):
+        input_dtype = hidden_states.dtype
+        hidden_states = hidden_states.to(torch.float32)
+
+        hidden_states = hidden_states.reshape(*hidden_states.shape[:-1], self.n_groups, -1)
+        variance = hidden_states.pow(2).mean(-1, keepdim=True)
+
+        hidden_states = hidden_states * torch.rsqrt(variance + self.variance_epsilon)
+        hidden_states = hidden_states.reshape(*hidden_states.shape[:-2], -1)
+        hidden_states = self.weight * hidden_states
+
+        return hidden_states.to(input_dtype)
+
+    def extra_repr(self):
+        return f"{tuple(self.weight.shape)}, eps={self.variance_epsilon}"
+
+
+class K2HorizonDecoderLayer(GradientCheckpointingLayer):
+    def __init__(self, config: K2HorizonConfig, layer_idx: int):
+        super().__init__()
+        self.hidden_size = config.hidden_size
+
+        is_sparse_layer = (layer_idx not in config.mlp_only_layers) and (
+                config.num_experts > 0 and (layer_idx + 1) % config.decoder_sparse_step == 0)
+
+        if is_sparse_layer and config.mova_num_experts > 0:
+            self.self_attn = K2HorizonMoVAAttention(config=config, layer_idx=layer_idx)
+        else:
+            self.self_attn = K2HorizonAttention(config, layer_idx)
+
+        if is_sparse_layer:
+            self.mlp = K2HorizonSparseMoeBlock(config)
+        else:
+            self.mlp = K2HorizonMLP(config, intermediate_size=config.intermediate_size)
+
+        assert config.hidden_size % config.layernorm_num_groups == 0
+        self.input_layernorm = K2HorizonRMSNorm(
+            hidden_size=config.hidden_size,
+            n_groups=config.layernorm_num_groups,
+            eps=config.rms_norm_eps)
+        self.post_attention_layernorm = K2HorizonRMSNorm(
+            hidden_size=config.hidden_size,
+            n_groups=config.layernorm_num_groups,
+            eps=config.rms_norm_eps)
+
+    @deprecate_kwarg("past_key_value", new_name="past_key_values", version="4.58")
+    def forward(
+            self,
+            hidden_states: torch.Tensor,
+            position_embeddings: tuple[torch.Tensor, torch.Tensor],
+            attention_mask: Optional[torch.Tensor] = None,
+            position_ids: Optional[torch.LongTensor] = None,
+            past_key_values: Optional[Cache] = None,
+            cache_position: Optional[torch.LongTensor] = None,
+            **kwargs: Unpack[FlashAttentionKwargs],
+    ) -> torch.FloatTensor:
+        """
+        Args:
+            hidden_states (`torch.FloatTensor`): input to the layer of shape `(batch, seq_len, embed_dim)`
+            attention_mask (`torch.FloatTensor`, *optional*): attention mask of size
+                `(batch, sequence_length)` where padding elements are indicated by 0.
+            output_attentions (`bool`, *optional*):
+                Whether or not to return the attentions tensors of all attention layers. See `attentions` under
+                returned tensors for more detail.
+            output_router_logits (`bool`, *optional*):
+                Whether or not to return the logits of all the routers. They are useful for computing the router loss,
+                and should not be returned during inference.
+            use_cache (`bool`, *optional*):
+                If set to `True`, `past_key_values` key value states are returned and can be used to speed up decoding
+                (see `past_key_values`).
+            past_key_values (`Cache`, *optional*): cached past key and value projection states
+            cache_position (`torch.LongTensor` of shape `(sequence_length)`, *optional*):
+                Indices depicting the position of the input sequence tokens in the sequence.
+            position_embeddings (`tuple[torch.FloatTensor, torch.FloatTensor]`, *optional*):
+                Tuple containing the cosine and sine positional embeddings of shape `(batch_size, seq_len, head_dim)`,
+                with `head_dim` being the embedding dimension of each attention head.
+            kwargs (`dict`, *optional*):
+                Arbitrary kwargs to be ignored, used for FSDP and other methods that injects code
+                into the model
+        """
+        residual = hidden_states
+        hidden_states = self.input_layernorm(hidden_states)
+
+        # Self Attention
+        hidden_states, _ = self.self_attn(
+            hidden_states=hidden_states,
+            position_embeddings=position_embeddings,
+            attention_mask=attention_mask,
+            position_ids=position_ids,
+            past_key_values=past_key_values,
+            cache_position=cache_position,
+            **kwargs,
+        )
+
+        hidden_states = residual + hidden_states
+
+        # Fully Connected
+        residual = hidden_states
+        hidden_states = self.post_attention_layernorm(hidden_states)
+        hidden_states = self.mlp(hidden_states)
+        # For the MoE layers, we need to unpack
+        if isinstance(hidden_states, tuple):
+            hidden_states, _ = hidden_states
+        hidden_states = residual + hidden_states
+
+        return hidden_states
+
+
+class K2HorizonRotaryEmbedding(nn.Module):
+    inv_freq: torch.Tensor  # fix linting for `register_buffer`
+
+    def __init__(self, config: K2HorizonConfig, device=None):
+        super().__init__()
+        self.max_seq_len_cached = config.max_position_embeddings
+        self.original_max_seq_len = config.max_position_embeddings
+
+        self.config = config
+
+        self.rope_type = self.config.rope_parameters["rope_type"]
+        rope_init_fn: Callable = self.compute_default_rope_parameters
+        if self.rope_type != "default":
+            rope_init_fn = ROPE_INIT_FUNCTIONS[self.rope_type]
+        inv_freq, self.attention_scaling = rope_init_fn(self.config, device)
+
+        self.register_buffer("inv_freq", inv_freq, persistent=False)
+        self.register_buffer("original_inv_freq", inv_freq.clone(), persistent=False)
+
+    @staticmethod
+    def compute_default_rope_parameters(
+        config: K2HorizonConfig | None = None,
+        device: Optional["torch.device"] = None,
+        seq_len: int | None = None,
+    ) -> tuple["torch.Tensor", float]:
+        """
+        Computes the inverse frequencies according to the original RoPE implementation
+        Args:
+            config ([`~transformers.PreTrainedConfig`]):
+                The model configuration.
+            device (`torch.device`):
+                The device to use for initialization of the inverse frequencies.
+            seq_len (`int`, *optional*):
+                The current sequence length. Unused for this type of RoPE.
+        Returns:
+            Tuple of (`torch.Tensor`, `float`), containing the inverse frequencies for the RoPE embeddings and the
+            post-processing scaling factor applied to the computed cos/sin (unused in this type of RoPE).
+        """
+        base = config.rope_parameters["rope_theta"]
+        # dim = getattr(config, "head_dim", None) or config.hidden_size // config.num_attention_heads
+        dim = (
+            config.rope_head_dim
+            if config.rope_head_dim is not None
+            else getattr(config, "head_dim", None) or config.hidden_size // config.num_attention_heads
+        )
+
+        attention_factor = 1.0  # Unused in this type of RoPE
+
+        # Compute the inverse frequencies
+        inv_freq = 1.0 / (
+            base ** (torch.arange(0, dim, 2, dtype=torch.int64).to(device=device, dtype=torch.float) / dim)
+        )
+        return inv_freq, attention_factor
+
+    @torch.no_grad()
+    @dynamic_rope_update  # power user: used with advanced RoPE types (e.g. dynamic rope)
+    def forward(self, x, position_ids):
+        inv_freq_expanded = self.inv_freq[None, :, None].float().expand(position_ids.shape[0], -1, 1).to(x.device)
+        position_ids_expanded = position_ids[:, None, :].float()
+
+        device_type = x.device.type if isinstance(x.device.type, str) and x.device.type != "mps" else "cpu"
+        with maybe_autocast(device_type=device_type, enabled=False):  # Force float32
+            freqs = (inv_freq_expanded.float() @ position_ids_expanded.float()).transpose(1, 2)
+            emb = torch.cat((freqs, freqs), dim=-1)
+            cos = emb.cos() * self.attention_scaling
+            sin = emb.sin() * self.attention_scaling
+
+        return cos.to(dtype=x.dtype), sin.to(dtype=x.dtype)
+
+
+@auto_docstring
+class K2HorizonPreTrainedModel(PreTrainedModel):
+    config: K2HorizonConfig
+    base_model_prefix = "model"
+    supports_gradient_checkpointing = True
+    _no_split_modules = ["K2HorizonDecoderLayer"]
+    _skip_keys_device_placement = ["past_key_values"]
+    _supports_flash_attn = True
+    _supports_sdpa = True
+    _supports_flex_attn = True
+    _can_compile_fullgraph = False  # MoE models don't work with torch.compile (`torch.where(condition)` not supported)
+    _supports_attention_backend = True
+    _can_record_outputs = {
+        "router_logits": OutputRecorder(K2HorizonSparseMoeBlock, index=1),
+        "hidden_states": K2HorizonDecoderLayer,
+        "attentions": K2HorizonAttention,
+    }
+
+
+@auto_docstring
+class K2HorizonModel(K2HorizonPreTrainedModel):
+    def __init__(self, config: K2HorizonConfig):
+        super().__init__(config)
+        # self.padding_idx = config.pad_token_id
+        self.padding_idx = getattr(config, "padding_idx", None)
+        self.vocab_size = config.vocab_size
+
+        self.embed_tokens = nn.Embedding(config.vocab_size, config.hidden_size, self.padding_idx)
+        self.layers = nn.ModuleList(
+            [K2HorizonDecoderLayer(config, layer_idx) for layer_idx in range(config.num_hidden_layers)]
+        )
+
+        assert config.hidden_size % config.layernorm_num_groups == 0
+        self.norm = K2HorizonRMSNorm(
+            hidden_size=config.hidden_size,
+            n_groups=config.layernorm_num_groups,
+            eps=config.rms_norm_eps)
+
+        self.rotary_emb = K2HorizonRotaryEmbedding(config=config)
+        self.gradient_checkpointing = False
+
+        # Initialize weights and apply final processing
+        self.post_init()
+
+    @auto_docstring
+    def forward(
+            self,
+            input_ids: Optional[torch.LongTensor] = None,
+            attention_mask: Optional[torch.Tensor] = None,
+            position_ids: Optional[torch.LongTensor] = None,
+            past_key_values: Optional[Cache] = None,
+            inputs_embeds: Optional[torch.FloatTensor] = None,
+            use_cache: Optional[bool] = None,
+            cache_position: Optional[torch.LongTensor] = None,
+            **kwargs: Unpack[TransformersKwargs],
+    ) -> MoeModelOutputWithPast:
+        if (input_ids is None) ^ (inputs_embeds is not None):
+            raise ValueError("You must specify exactly one of input_ids or inputs_embeds")
+
+        if use_cache and past_key_values is None:
+            past_key_values = DynamicCache(config=self.config)
+
+        if inputs_embeds is None:
+            inputs_embeds = self.embed_tokens(input_ids)
+
+        if cache_position is None:
+            past_seen_tokens = past_key_values.get_seq_length() if past_key_values is not None else 0
+            cache_position = torch.arange(
+                past_seen_tokens, past_seen_tokens + inputs_embeds.shape[1], device=inputs_embeds.device
+            )
+        if position_ids is None:
+            position_ids = cache_position.unsqueeze(0)
+
+        mask_function = create_causal_mask if self.config.sliding_window is None else create_sliding_window_causal_mask
+        causal_mask = mask_function(
+            config=self.config,
+            inputs_embeds=inputs_embeds,
+            attention_mask=attention_mask,
+            past_key_values=past_key_values,
+            position_ids=position_ids,
+        )
+
+        hidden_states = inputs_embeds
+
+        # create position embeddings to be shared across the decoder layers
+        position_embeddings = self.rotary_emb(hidden_states, position_ids)
+
+        for layer_idx, decoder_layer in enumerate(self.layers[: self.config.num_hidden_layers]):
+            hidden_states = decoder_layer(
+                hidden_states,
+                position_embeddings=position_embeddings,
+                attention_mask=causal_mask,
+                position_ids=position_ids,
+                past_key_values=past_key_values,
+                use_cache=use_cache,
+                cache_position=cache_position,
+                **kwargs,
+            )
+
+        hidden_states = self.norm(hidden_states)
+
+        return MoeModelOutputWithPast(  # only diff with Mistral is the output type, we need MoE
+            last_hidden_state=hidden_states,
+            past_key_values=past_key_values,
+        )
+
+
+def load_balancing_loss_func(
+        gate_logits: Union[torch.Tensor, tuple[torch.Tensor], None],
+        num_experts: Optional[int] = None,
+        top_k=2,
+        attention_mask: Optional[torch.Tensor] = None,
+) -> Union[torch.Tensor, int]:
+    r"""
+    Computes auxiliary load balancing loss as in Switch Transformer - implemented in Pytorch.
+
+    See Switch Transformer (https://huggingface.co/papers/2101.03961) for more details. This function implements the loss
+    function presented in equations (4) - (6) of the paper. It aims at penalizing cases where the routing between
+    experts is too unbalanced.
+
+    Args:
+        gate_logits:
+            Logits from the `gate`, should be a tuple of model.config.num_hidden_layers tensors of
+            shape [batch_size X sequence_length, num_experts].
+        num_experts:
+            Number of experts
+        top_k:
+            The number of experts to route per-token, can be also interpreted as the `top-k` routing
+            parameter.
+        attention_mask (`torch.Tensor`, *optional*):
+            The attention_mask used in forward function
+            shape [batch_size X sequence_length] if not None.
+
+    Returns:
+        The auxiliary loss.
+    """
+    if gate_logits is None or not isinstance(gate_logits, tuple):
+        return 0
+
+    if isinstance(gate_logits, tuple):
+        compute_device = gate_logits[0].device
+        concatenated_gate_logits = torch.cat([layer_gate.to(compute_device) for layer_gate in gate_logits], dim=0)
+
+    routing_weights = torch.nn.functional.softmax(concatenated_gate_logits, dim=-1)
+
+    _, selected_experts = torch.topk(routing_weights, top_k, dim=-1)
+
+    expert_mask = torch.nn.functional.one_hot(selected_experts, num_experts)
+
+    if attention_mask is None:
+        # Compute the percentage of tokens routed to each experts
+        tokens_per_expert = torch.mean(expert_mask.float(), dim=0)
+
+        # Compute the average probability of routing to these experts
+        router_prob_per_expert = torch.mean(routing_weights, dim=0)
+    else:
+        batch_size, sequence_length = attention_mask.shape
+        num_hidden_layers = concatenated_gate_logits.shape[0] // (batch_size * sequence_length)
+
+        # Compute the mask that masks all padding tokens as 0 with the same shape of expert_mask
+        expert_attention_mask = (
+            attention_mask[None, :, :, None, None]
+            .expand((num_hidden_layers, batch_size, sequence_length, top_k, num_experts))
+            .reshape(-1, top_k, num_experts)
+            .to(compute_device)
+        )
+
+        # Compute the percentage of tokens routed to each experts
+        tokens_per_expert = torch.sum(expert_mask.float() * expert_attention_mask, dim=0) / torch.sum(
+            expert_attention_mask, dim=0
+        )
+
+        # Compute the mask that masks all padding tokens as 0 with the same shape of tokens_per_expert
+        router_per_expert_attention_mask = (
+            attention_mask[None, :, :, None]
+            .expand((num_hidden_layers, batch_size, sequence_length, num_experts))
+            .reshape(-1, num_experts)
+            .to(compute_device)
+        )
+
+        # Compute the average probability of routing to these experts
+        router_prob_per_expert = torch.sum(routing_weights * router_per_expert_attention_mask, dim=0) / torch.sum(
+            router_per_expert_attention_mask, dim=0
+        )
+
+    overall_loss = torch.sum(tokens_per_expert * router_prob_per_expert.unsqueeze(0))
+    return overall_loss * num_experts
+
+
+@auto_docstring
+class K2HorizonForCausalLM(K2HorizonPreTrainedModel, GenerationMixin):
+    # _tied_weights_keys = ["lm_head.weight"]
+    # _tp_plan = {"lm_head": "colwise_rep"}
+    # _pp_plan = {"lm_head": (["hidden_states"], ["logits"])}
+
+    def __init__(self, config):
+        super().__init__(config)
+        self.model = K2HorizonModel(config)
+        self.vocab_size = config.vocab_size
+        self.lm_head = nn.Linear(config.hidden_size, config.vocab_size, bias=False)
+        self.router_aux_loss_coef = config.router_aux_loss_coef
+        self.num_experts = config.num_experts
+        self.num_experts_per_tok = config.num_experts_per_tok
+
+        # Initialize weights and apply final processing
+        self.post_init()
+
+    @can_return_tuple
+    @auto_docstring
+    def forward(
+            self,
+            input_ids: Optional[torch.LongTensor] = None,
+            attention_mask: Optional[torch.Tensor] = None,
+            position_ids: Optional[torch.LongTensor] = None,
+            past_key_values: Optional[Cache] = None,
+            inputs_embeds: Optional[torch.FloatTensor] = None,
+            labels: Optional[torch.LongTensor] = None,
+            use_cache: Optional[bool] = None,
+            output_router_logits: Optional[bool] = None,
+            cache_position: Optional[torch.LongTensor] = None,
+            logits_to_keep: Union[int, torch.Tensor] = 0,
+            **kwargs: Unpack[TransformersKwargs],
+    ) -> MoeCausalLMOutputWithPast:
+        r"""
+        labels (`torch.LongTensor` of shape `(batch_size, sequence_length)`, *optional*):
+            Labels for computing the masked language modeling loss. Indices should either be in `[0, ...,
+            config.vocab_size]` or -100 (see `input_ids` docstring). Tokens with indices set to `-100` are ignored
+            (masked), the loss is only computed for the tokens with labels in `[0, ..., config.vocab_size]`.
+
+        Example:
+
+        ```python
+        >>> from transformers import AutoTokenizer, K2HorizonForCausalLM
+
+        >>> model = K2HorizonForCausalLM.from_pretrained("Qwen/Qwen3-MoE-15B-A2B")
+        >>> tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-MoE-15B-A2B")
+
+        >>> prompt = "Hey, are you conscious? Can you talk to me?"
+        >>> inputs = tokenizer(prompt, return_tensors="pt")
+
+        >>> # Generate
+        >>> generate_ids = model.generate(inputs.input_ids, max_length=30)
+        >>> tokenizer.batch_decode(generate_ids, skip_special_tokens=True, clean_up_tokenization_spaces=False)[0]
+        "Hey, are you conscious? Can you talk to me?\nI'm not conscious, but I can talk to you."
+        ```"""
+
+        output_router_logits = (
+            output_router_logits if output_router_logits is not None else self.config.output_router_logits
+        )
+
+        # decoder outputs consists of (dec_features, layer_state, dec_hidden, dec_attn)
+        outputs: MoeModelOutputWithPast = self.model(
+            input_ids=input_ids,
+            attention_mask=attention_mask,
+            position_ids=position_ids,
+            past_key_values=past_key_values,
+            inputs_embeds=inputs_embeds,
+            use_cache=use_cache,
+            output_router_logits=output_router_logits,
+            cache_position=cache_position,
+            **kwargs,
+        )
+
+        hidden_states = outputs.last_hidden_state
+        # Only compute necessary logits, and do not upcast them to float if we are not computing the loss
+        slice_indices = slice(-logits_to_keep, None) if isinstance(logits_to_keep, int) else logits_to_keep
+        logits = self.lm_head(hidden_states[:, slice_indices, :])
+
+        loss = None
+        if labels is not None:
+            loss = self.loss_function(logits, labels, self.vocab_size, **kwargs)
+
+        aux_loss = None
+        if output_router_logits:
+            aux_loss = load_balancing_loss_func(
+                outputs.router_logits,
+                self.num_experts,
+                self.num_experts_per_tok,
+                attention_mask,
+            )
+            if labels is not None:
+                loss += self.router_aux_loss_coef * aux_loss.to(loss.device)  # make sure to reside in the same device
+
+        return MoeCausalLMOutputWithPast(
+            loss=loss,
+            aux_loss=aux_loss,
+            logits=logits,
+            past_key_values=outputs.past_key_values,
+            hidden_states=outputs.hidden_states,
+            attentions=outputs.attentions,
+            router_logits=outputs.router_logits,
+        )
+
+
+class K2HorizonForSequenceClassification(GenericForSequenceClassification, K2HorizonPreTrainedModel):
+    pass
+
+
+class K2HorizonForTokenClassification(GenericForTokenClassification, K2HorizonPreTrainedModel):
+    pass
+
+
+class K2HorizonForQuestionAnswering(GenericForQuestionAnswering, K2HorizonPreTrainedModel):
+    base_model_prefix = "transformer"  # For BC, where `transformer` was used instead of `model`
+
+
+__all__ = [
+    "K2HorizonForCausalLM",
+    "K2HorizonForQuestionAnswering",
+    "K2HorizonModel",
+    "K2HorizonPreTrainedModel",
+    "K2HorizonForSequenceClassification",
+    "K2HorizonForTokenClassification",
+]

pytorch_model-00001-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d110e6762cf85f674fafa94cadbf3fb07b970fff3d2253eeddcfd9fa679c9b10
+size 209726456

pytorch_model-00002-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:219394d503b95ea4973c92b97afb24ab76dcb18911edccf824fd84bd166657fe
+size 209726456

pytorch_model-00003-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b1ddbe4ff8c50178a50babf808a662e2b5062161e368d1fcd6445814de0dcbc4
+size 209726456

pytorch_model-00004-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ef32e974a0a8ac08c2e42dd729a29a539fcedbcd54fc3b5d20ec8016c061fcd1
+size 209726456

pytorch_model-00005-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ab173ae7f0e15552083a4357337b6db221c8b39368896cf87954a6ca8112834e
+size 209726456

pytorch_model-00006-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:04800ffb44ae333db31acebabbd3b6479d922e70815d2a980095c692802bfd3b
+size 209726456

pytorch_model-00007-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:58283718bb60434d34558195aaf8a500102ea8b6900313a526bb2000e39fec5a
+size 209726456

pytorch_model-00008-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:07ea6dcc528a1c8aa40629b1ac82f1f4709af27c7bfbba463e09322ac54ffbbd
+size 209726456

pytorch_model-00009-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1c7ca45ab209e907cd0870434ce250a45d2f2fcabee172863acb7f4347d591f7
+size 209726456

pytorch_model-00010-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:72630d27e7473b133a4d0bfd06698c38d3e59e91d966ff33faf0ee0dfed45aac
+size 209726456

pytorch_model-00011-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b79b48f5cb1ba86dab068c136b6bdba5a89227ea8da77efd9b11cefee4daa042
+size 209726464

pytorch_model-00012-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:555adf66a051ed26ff2f62b7225e66c66e337d8d17883215467c50b3067a01c0
+size 209726464

pytorch_model-00013-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:88c0309e67230e0b46969c03ef1d38f7c5a1c64338618c948a03cb88f5fc1125
+size 209726464

pytorch_model-00014-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:741024e7349d3f7da3830daaba44652b29bc376d5a1b798e544251ce08f750b6
+size 209726464

pytorch_model-00015-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:109a4b90c9196f1efc85a6c37ca493b51417af36fbf5c04435a8efb1caeb0721
+size 209726464

pytorch_model-00016-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:233aba62372d793e5fe0049c8bafb5e84030a177e90db7c7870dd49c523ed55a
+size 209726464

pytorch_model-00017-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:1c7a72eb7ab4ec07fb50a1f39e6ed1699ed9d4d8eee1a97fef67f915cb860a6c
+size 209726464

pytorch_model-00018-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a6f037b8abb77be356c85c84135c35dd6e638c92d1cf6b99851301d44dc5b266
+size 209726464

pytorch_model-00019-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c93b7417584fe5f0df966a7e35c2d81eee2c3bd3a2211a3581fdd9330ece1033
+size 209726464

pytorch_model-00020-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:42f887f37c4568d975d84d64d15ecf8b811c3e06fd4c5642ab083b77c6079400
+size 209726464

pytorch_model-00021-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9180c84c2765b1195a93fcbf2b91842c3aa48b5a93d9c21a5f5186743e84b85c
+size 209726464

pytorch_model-00022-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:779922fd3ddd1d521637897f989b896fe088da27e84f8de41cbadfabc45559bf
+size 209726464

pytorch_model-00023-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7ec024d9567112f7c678176f6a7ff096d7e6adc680dcb9943c719ca1c40d0697
+size 209726464

pytorch_model-00024-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c1f47068a96f74316ad78d1f61a747af075fc551e46affd4565e32735649018f
+size 209726464

pytorch_model-00025-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:69cb586f3870597d9b9406813d3b114bb597139ed7e4290ca4727d385f308ae8
+size 209726464

pytorch_model-00026-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:db3c9ca402ce5641a0931f54e2708310b4def9b5339fc9f2498731ba49f90559
+size 209726464

pytorch_model-00027-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a213216658c5f60061b05f68c28ef2c2ad8b707e512d2ca59dcc83cbebd6c855
+size 209726464

pytorch_model-00028-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ca2eecbadf94ef6823166cb5621c016f6924c132d038feda2f98603dc43c64a9
+size 209726464

pytorch_model-00029-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:18014f073dfffc8778dee79ebdb4a10fe6a17e19ac5ce165f42ee6d9a7241422
+size 209726464

pytorch_model-00030-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:52017c4e7a7622caf41b5638c9f5670cdbe683ca158c650c102dee7db77a21a0
+size 209726464

pytorch_model-00031-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:595d6832ae7c0bc602f09db5085d1ffb6d3d0eefde5334b038174d9a679e14c6
+size 209726464

pytorch_model-00032-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ce282ec1948eae29b6766b717b5ef61f9e70ce67b4f3eb941f801b2a59526b65
+size 209726464

pytorch_model-00033-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:d54819763862df5da76be25d1e25a8366407add31bf749a098ca9265c0a6e4bc
+size 209726464

pytorch_model-00034-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:5db1fc8429dd8b14759068c0026ea84e3476ce44f2d7f715792f28da4a4e59d7
+size 209726464

pytorch_model-00035-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9e151f279a2b3fcfc13dd8a5b52609dbbb3d35c454b1371d044ed7df08d3bef7
+size 209726464

pytorch_model-00036-of-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:80ed9da7581f3f6226e9dfd701210b6d61fd7461343b45883eeb0271d39eca06
+size 2776121664

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:838d767b7c9925ff257feb20eaa4299a8e3cc35bb3d805589c373f51d2cc3cb6
+size 20637260

tokenizer_config.json

@@ -0,0 +1,10 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "<|ifm|begin_of_text|>",
+  "clean_up_tokenization_spaces": false,
+  "eos_token": "<|ifm|endoftext|>",
+  "is_local": true,
+  "local_files_only": false,
+  "model_max_length": 1000000000000000019884624838656,
+  "tokenizer_class": "TokenizersBackend"
+}