Text Generation
PEFT
English
Chinese
hypernetwork
hyper-lora
lora
role-play
character-impersonation
persona
dialogue
phase-tree
Instructions to use IAAR-Shanghai/phase_tree_models with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- PEFT
How to use IAAR-Shanghai/phase_tree_models with PEFT:
Task type is invalid.
- Notebooks
- Google Colab
- Kaggle
Add files using upload-large-folder tool
Browse files- README.md +124 -0
- phase_tree_pretrained/README.md +108 -0
- phase_tree_pretrained/adapter_config.json +36 -0
- phase_tree_pretrained/args.yaml +63 -0
- phase_tree_pretrained/hypermod.pt +3 -0
- sft/hyper_lora/README.md +137 -0
- sft/hyper_lora/adapter_config.json +36 -0
- sft/hyper_lora/args.yaml +115 -0
- sft/hyper_lora/hypermod.pt +3 -0
- sft/hyper_lora/timing_stats.json +6 -0
README.md
CHANGED
|
@@ -1,3 +1,127 @@
|
|
| 1 |
---
|
| 2 |
license: cc-by-nc-4.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: cc-by-nc-4.0
|
| 3 |
+
base_model: Qwen/Qwen2.5-7B-Instruct
|
| 4 |
+
language:
|
| 5 |
+
- en
|
| 6 |
+
- zh
|
| 7 |
+
pipeline_tag: text-generation
|
| 8 |
+
library_name: peft
|
| 9 |
+
tags:
|
| 10 |
+
- hypernetwork
|
| 11 |
+
- hyper-lora
|
| 12 |
+
- lora
|
| 13 |
+
- role-play
|
| 14 |
+
- character-impersonation
|
| 15 |
+
- persona
|
| 16 |
+
- dialogue
|
| 17 |
+
- phase-tree
|
| 18 |
+
datasets:
|
| 19 |
+
- IAAR-Shanghai/phase_tree_data
|
| 20 |
---
|
| 21 |
+
|
| 22 |
+
# PHASE-Tree Models
|
| 23 |
+
|
| 24 |
+
Released model checkpoints for the **PHASE-Tree** project
|
| 25 |
+
(*Psychology-grounded Hierarchical Attribute-Structured Evolving Tree*).
|
| 26 |
+
|
| 27 |
+
## Download
|
| 28 |
+
|
| 29 |
+
The PHASE-Tree codebase expects these checkpoints under
|
| 30 |
+
`PHASE-Tree/phase_tree_models/`. The recommended way to obtain a working copy
|
| 31 |
+
is therefore:
|
| 32 |
+
|
| 33 |
+
```bash
|
| 34 |
+
# From the repository root (i.e. inside the cloned PHASE-Tree project):
|
| 35 |
+
cd PHASE-Tree
|
| 36 |
+
hf download IAAR-Shanghai/phase_tree_models --local-dir phase_tree_models
|
| 37 |
+
```
|
| 38 |
+
|
| 39 |
+
This places every file under `PHASE-Tree/phase_tree_models/`, matching the
|
| 40 |
+
relative paths used by every script in the codebase (e.g.
|
| 41 |
+
`phase_tree_models/sft/hyper_lora/hypermod.pt`).
|
| 42 |
+
|
| 43 |
+
Alternative methods:
|
| 44 |
+
|
| 45 |
+
- `git clone https://huggingface.co/IAAR-Shanghai/phase_tree_models`
|
| 46 |
+
(run from the `PHASE-Tree/` root; clones into `phase_tree_models/`
|
| 47 |
+
automatically).
|
| 48 |
+
- Programmatic via `huggingface_hub.snapshot_download(...)` with
|
| 49 |
+
`local_dir="phase_tree_models"`.
|
| 50 |
+
|
| 51 |
+
This release contains the **single recommended checkpoint** for each of the
|
| 52 |
+
two stages in the PHASE-Tree training pipeline. During development we ran a
|
| 53 |
+
larger ablation grid (six hyper-LoRA SFT runs covering warm-start vs
|
| 54 |
+
cold-start initialisation, two learning rates, and trainable vs frozen
|
| 55 |
+
hypernet output heads, plus a separate One-PEFT-Per-User / OPPU baseline
|
| 56 |
+
sweep). Only the checkpoints reported in the paper are bundled here; the
|
| 57 |
+
ablations are kept locally for reproducibility but are not part of the
|
| 58 |
+
release.
|
| 59 |
+
|
| 60 |
+
## Layout
|
| 61 |
+
|
| 62 |
+
| Path | Description |
|
| 63 |
+
|------|-------------|
|
| 64 |
+
| `phase_tree_pretrained/` | Hypernetwork pretrained on the PHASE-Tree character corpus. Used as the warm-start initialisation for the SFT run below. |
|
| 65 |
+
| `sft/hyper_lora/` | The **anchor** hyper-LoRA SFT run (warm-start, lr=5e-6, trainable heads). This is the checkpoint reported in the PHASE-Tree paper. |
|
| 66 |
+
|
| 67 |
+
Each leaf folder is self-describing via its own `README.md`.
|
| 68 |
+
|
| 69 |
+
## Recommended Checkpoint
|
| 70 |
+
|
| 71 |
+
For character-conditioned generation, load:
|
| 72 |
+
|
| 73 |
+
```
|
| 74 |
+
sft/hyper_lora/hypermod.pt
|
| 75 |
+
```
|
| 76 |
+
|
| 77 |
+
The pretrained hypermod (`phase_tree_pretrained/hypermod.pt`) is the upstream
|
| 78 |
+
warm-start dependency of this anchor run, not an independently usable
|
| 79 |
+
inference model. It is included so the training pipeline can be reproduced
|
| 80 |
+
end-to-end.
|
| 81 |
+
|
| 82 |
+
```
|
| 83 |
+
phase_tree_pretrained/hypermod.pt ββwarm-startβββΆ sft/hyper_lora/hypermod.pt
|
| 84 |
+
(pretraining stage) (anchor SFT, recommended)
|
| 85 |
+
```
|
| 86 |
+
|
| 87 |
+
## Why a single SFT checkpoint?
|
| 88 |
+
|
| 89 |
+
Six hyper-LoRA SFT runs were trained during development, varying
|
| 90 |
+
initialisation, learning rate, and whether the hypernet output heads are
|
| 91 |
+
trainable. The released `sft/hyper_lora/` is the cell selected by the
|
| 92 |
+
LLM-as-judge `character` and `semantic` ratings together with
|
| 93 |
+
Qwen3-Embedding-4B response-vs-reference cosine similarity on a held-out
|
| 94 |
+
evaluation set; the other five cells are ablations and are not bundled.
|
| 95 |
+
|
| 96 |
+
Per-step intermediate checkpoints (`it_5000` β¦ `it_40000`) and the full
|
| 97 |
+
post-hoc evaluation artefacts (`eval_ckpt_judge_scores/`,
|
| 98 |
+
`eval_ckpt_val_loss/`) are likewise not bundled. To regenerate them you
|
| 99 |
+
would need to re-run training (`src/scripts/train_phase_tree_qwen_7b.sh`)
|
| 100 |
+
followed by the evaluation scripts under `src/scripts/`.
|
| 101 |
+
|
| 102 |
+
## Base Model
|
| 103 |
+
|
| 104 |
+
All checkpoints are trained on top of
|
| 105 |
+
[`Qwen/Qwen2.5-7B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct).
|
| 106 |
+
The hyper-LoRA and pretrained-hypermod checkpoints additionally use
|
| 107 |
+
[`Qwen/Qwen3-Embedding-4B`](https://huggingface.co/Qwen/Qwen3-Embedding-4B)
|
| 108 |
+
as the task-embedding encoder.
|
| 109 |
+
|
| 110 |
+
## Loading
|
| 111 |
+
|
| 112 |
+
| Checkpoint | Loader |
|
| 113 |
+
|------------|--------|
|
| 114 |
+
| Pretrained hypermod (`phase_tree_pretrained/`) | `hyper_llm_modulator.hyper_modulator.load_hypermod_checkpoint(...)` |
|
| 115 |
+
| Hyper-LoRA SFT (`sft/hyper_lora/`) | `hyper_llm_modulator.hyper_modulator.load_hypermod_checkpoint(...)` |
|
| 116 |
+
|
| 117 |
+
The hypermod loader expects a checkpoint directory layout identical to the
|
| 118 |
+
one used here (`hypermod.pt` + sibling `args.yaml` + `adapter_config.json`).
|
| 119 |
+
It reads the architecture from `args.yaml` automatically; no extra
|
| 120 |
+
configuration is required at inference time.
|
| 121 |
+
|
| 122 |
+
## Intended Use
|
| 123 |
+
|
| 124 |
+
These checkpoints are released as research artefacts for evaluating
|
| 125 |
+
personalised and hypernetwork-based approaches to character-grounded dialogue
|
| 126 |
+
generation. They are **not** intended for production user-facing
|
| 127 |
+
applications without additional safety filtering.
|
phase_tree_pretrained/README.md
ADDED
|
@@ -0,0 +1,108 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc-by-nc-4.0
|
| 3 |
+
base_model: Qwen/Qwen2.5-7B-Instruct
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
+
library_name: peft
|
| 6 |
+
language:
|
| 7 |
+
- en
|
| 8 |
+
- zh
|
| 9 |
+
tags:
|
| 10 |
+
- hypernetwork
|
| 11 |
+
- hyper-lora
|
| 12 |
+
- lora
|
| 13 |
+
- role-play
|
| 14 |
+
- character-impersonation
|
| 15 |
+
- pretraining
|
| 16 |
+
- phase-tree
|
| 17 |
+
datasets:
|
| 18 |
+
- IAAR-Shanghai/phase_tree_data
|
| 19 |
+
---
|
| 20 |
+
|
| 21 |
+
# PHASE-Tree Pretrained Hypermod
|
| 22 |
+
|
| 23 |
+
Hypernetwork pretrained on the PHASE-Tree character-dialogue corpus on top of
|
| 24 |
+
[`Qwen/Qwen2.5-7B-Instruct`](https://huggingface.co/Qwen/Qwen2.5-7B-Instruct).
|
| 25 |
+
|
| 26 |
+
This is the **warm-start checkpoint** consumed by the SFT runs released under
|
| 27 |
+
`phase_tree_models/sft/hyper_lora/`. It is *not* intended as a stand-alone
|
| 28 |
+
inference checkpoint β for character-conditioned generation, the SFT runs are
|
| 29 |
+
recommended.
|
| 30 |
+
|
| 31 |
+
> The pretraining-stage training schedule (full dataset list, optimizer
|
| 32 |
+
> schedule, etc.) is not bundled with this release. Only the fields required
|
| 33 |
+
> by `load_hypermod_checkpoint` (path resolution + hypermod architecture) are
|
| 34 |
+
> retained in `args.yaml`; the SFT runs in `phase_tree_models/sft/hyper_lora/`
|
| 35 |
+
> carry the complete training configurations for their respective fine-tuning
|
| 36 |
+
> stages.
|
| 37 |
+
|
| 38 |
+
## What is a hypermod?
|
| 39 |
+
|
| 40 |
+
A **hypermod** (hyper-modulator) is a hypernetwork that, conditioned on a
|
| 41 |
+
character profile embedding, emits a low-rank LoRA delta `ΞW = AB` for each
|
| 42 |
+
target layer of the base model on the fly. The base model weights themselves
|
| 43 |
+
are never updated; only the hypernet is trained. At inference time the
|
| 44 |
+
hypernet generates a personalised LoRA per character, giving one model that
|
| 45 |
+
covers an open-ended set of personas without needing to store per-character
|
| 46 |
+
adapters.
|
| 47 |
+
|
| 48 |
+
## Files
|
| 49 |
+
|
| 50 |
+
| File | Purpose |
|
| 51 |
+
|------|---------|
|
| 52 |
+
| `hypermod.pt` | The released pretrained hypermod (it_20000 of the original pretraining run). Use this as the entry point. |
|
| 53 |
+
| `args.yaml` | Architecture and loader metadata (no training schedule β this checkpoint is meant to be consumed, not resumed). |
|
| 54 |
+
| `adapter_config.json` | LoRA target-module stub (rank 8, alpha 16, `q_proj` + `v_proj`). |
|
| 55 |
+
|
| 56 |
+
## How to load
|
| 57 |
+
|
| 58 |
+
```python
|
| 59 |
+
from huggingface_hub import snapshot_download
|
| 60 |
+
from hyper_llm_modulator.hyper_modulator import load_hypermod_checkpoint
|
| 61 |
+
|
| 62 |
+
ckpt_dir = snapshot_download("<your-hf-username>/PHASE-Tree-pretrained-hypermod")
|
| 63 |
+
|
| 64 |
+
(
|
| 65 |
+
args, hypermod, base_model, tokenizer,
|
| 66 |
+
emb_model, emb_tokenizer, task_desc_format_fn, pooling_fn,
|
| 67 |
+
) = load_hypermod_checkpoint(f"{ckpt_dir}/hypermod.pt", device="cuda")
|
| 68 |
+
```
|
| 69 |
+
|
| 70 |
+
The loader reads `args.yaml` and `adapter_config.json` from the same directory
|
| 71 |
+
as `hypermod.pt` automatically; you do not need to pass them explicitly. The
|
| 72 |
+
full inference pipeline (profile β embedding β per-layer LoRA β generation)
|
| 73 |
+
lives in the PHASE-Tree codebase.
|
| 74 |
+
|
| 75 |
+
## Architecture
|
| 76 |
+
|
| 77 |
+
| Component | Value |
|
| 78 |
+
|-----------|-------|
|
| 79 |
+
| Base model | `Qwen/Qwen2.5-7B-Instruct` |
|
| 80 |
+
| Task encoder | `Qwen/Qwen3-Embedding-4B` |
|
| 81 |
+
| Target modules | `q_proj`, `v_proj` |
|
| 82 |
+
| LoRA rank `r` | 8 |
|
| 83 |
+
| LoRA alpha | 16 |
|
| 84 |
+
| LoRA dropout | 0.05 |
|
| 85 |
+
| Hypernet latent size | 1024 |
|
| 86 |
+
| Hypernet head input size | 2048 |
|
| 87 |
+
| `delta_w` scaling | 100 |
|
| 88 |
+
|
| 89 |
+
## Use as warm-start
|
| 90 |
+
|
| 91 |
+
SFT runs whose `args.yaml` sets
|
| 92 |
+
|
| 93 |
+
```yaml
|
| 94 |
+
init_hypermod_from: phase_tree_models/phase_tree_pretrained/hypermod.pt
|
| 95 |
+
```
|
| 96 |
+
|
| 97 |
+
consume this checkpoint as the initial hypernet weights. This is the
|
| 98 |
+
warm-start used by the released anchor SFT run under
|
| 99 |
+
`phase_tree_models/sft/hyper_lora/`.
|
| 100 |
+
|
| 101 |
+
## Limitations
|
| 102 |
+
|
| 103 |
+
- This is a **pretraining** checkpoint; downstream SFT is required for
|
| 104 |
+
competitive character-fidelity scores.
|
| 105 |
+
- Persona conditioning is mediated entirely by the profile embedding fed into
|
| 106 |
+
the task encoder; the model has no other persona-control mechanism.
|
| 107 |
+
- Generations may reproduce stylistic biases of the source corpora and are
|
| 108 |
+
intended for research evaluation only.
|
phase_tree_pretrained/adapter_config.json
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alpha_pattern": {},
|
| 3 |
+
"auto_mapping": null,
|
| 4 |
+
"base_model_name_or_path": "Qwen/Qwen2.5-7B-Instruct",
|
| 5 |
+
"bias": "none",
|
| 6 |
+
"corda_config": null,
|
| 7 |
+
"eva_config": null,
|
| 8 |
+
"exclude_modules": null,
|
| 9 |
+
"fan_in_fan_out": false,
|
| 10 |
+
"inference_mode": false,
|
| 11 |
+
"init_lora_weights": true,
|
| 12 |
+
"layer_replication": null,
|
| 13 |
+
"layers_pattern": null,
|
| 14 |
+
"layers_to_transform": null,
|
| 15 |
+
"loftq_config": {},
|
| 16 |
+
"lora_alpha": 16,
|
| 17 |
+
"lora_bias": false,
|
| 18 |
+
"lora_dropout": 0.05,
|
| 19 |
+
"megatron_config": null,
|
| 20 |
+
"megatron_core": "megatron.core",
|
| 21 |
+
"modules_to_save": null,
|
| 22 |
+
"peft_type": "LORA",
|
| 23 |
+
"qalora_group_size": 16,
|
| 24 |
+
"r": 8,
|
| 25 |
+
"rank_pattern": {},
|
| 26 |
+
"revision": null,
|
| 27 |
+
"target_modules": [
|
| 28 |
+
"q_proj",
|
| 29 |
+
"v_proj"
|
| 30 |
+
],
|
| 31 |
+
"task_type": "CAUSAL_LM",
|
| 32 |
+
"trainable_token_indices": null,
|
| 33 |
+
"use_dora": false,
|
| 34 |
+
"use_qalora": false,
|
| 35 |
+
"use_rslora": true
|
| 36 |
+
}
|
phase_tree_pretrained/args.yaml
ADDED
|
@@ -0,0 +1,63 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# ==========================================================================
|
| 2 |
+
# Architecture / loading metadata for the PHASE-Tree pretrained hypermod.
|
| 3 |
+
#
|
| 4 |
+
# The released warm-start checkpoint is
|
| 5 |
+
# phase_tree_models/phase_tree_pretrained/hypermod.pt
|
| 6 |
+
# (= the it_20000 snapshot of the original pretraining run).
|
| 7 |
+
#
|
| 8 |
+
# Only fields read by `load_hypermod_checkpoint` (path resolution +
|
| 9 |
+
# hypermod architecture) are kept here; the dataset lists and the
|
| 10 |
+
# original training schedule are intentionally omitted, because the
|
| 11 |
+
# PHASE-Tree SFT runs warm-start from these weights and override every
|
| 12 |
+
# training hyperparameter from `train_phase_tree_qwen_7b.sh`.
|
| 13 |
+
# ==========================================================================
|
| 14 |
+
|
| 15 |
+
# ββ Paths ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 16 |
+
model_dir: Qwen/Qwen2.5-7B-Instruct
|
| 17 |
+
emb_model: Qwen/Qwen3-Embedding-4B
|
| 18 |
+
mt_lora_path: null
|
| 19 |
+
|
| 20 |
+
# ββ Task setup βββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 21 |
+
training_task: sft
|
| 22 |
+
exp_setup: hyper_lora
|
| 23 |
+
sft_mode: completion
|
| 24 |
+
encoder_type: linear
|
| 25 |
+
|
| 26 |
+
# ββ Task-embedding mode βββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 27 |
+
use_hypernet: true
|
| 28 |
+
use_per_task_emb: true
|
| 29 |
+
use_one_hot_task_emb: false
|
| 30 |
+
use_inp_as_desc: false
|
| 31 |
+
use_per_sample_desc: false
|
| 32 |
+
use_default_desc: false
|
| 33 |
+
|
| 34 |
+
# ββ Hypermod architecture ββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 35 |
+
head_in_size: 2048
|
| 36 |
+
head_use_bias: false
|
| 37 |
+
hypernet_latent_size: 1024
|
| 38 |
+
delta_w_scaling: 100
|
| 39 |
+
pred_z_score: true
|
| 40 |
+
factorized: false
|
| 41 |
+
shared_AB_head: false
|
| 42 |
+
autoreg_gen: false
|
| 43 |
+
learnable_pos_emb: false
|
| 44 |
+
learnable_AB_offset: false
|
| 45 |
+
|
| 46 |
+
# ββ Fusion (disabled; kept for loader compatibility) ββββββββββββββββββββ
|
| 47 |
+
use_conv_fusion: false
|
| 48 |
+
conv_fusion_type: 1d
|
| 49 |
+
conv_fusion_kernel_size: 3
|
| 50 |
+
conv_fusion_num_layers: 2
|
| 51 |
+
conv_fusion_channels: 64
|
| 52 |
+
conv_fusion_dropout: 0.1
|
| 53 |
+
use_attention_fusion: false
|
| 54 |
+
attention_fusion_type: self
|
| 55 |
+
attention_num_heads: 8
|
| 56 |
+
attention_num_layers: 2
|
| 57 |
+
attention_dropout: 0.1
|
| 58 |
+
|
| 59 |
+
# ββ Target LoRA modules and context window ββββββββββββββββββββββββββββββ
|
| 60 |
+
target_modules:
|
| 61 |
+
- q_proj
|
| 62 |
+
- v_proj
|
| 63 |
+
inp_max_len: 1024
|
phase_tree_pretrained/hypermod.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:f807533480a5353bd8637aab87fc50ae7039f708d73425dd467bad0f0811413e
|
| 3 |
+
size 925242042
|
sft/hyper_lora/README.md
ADDED
|
@@ -0,0 +1,137 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc-by-nc-4.0
|
| 3 |
+
base_model: Qwen/Qwen2.5-7B-Instruct
|
| 4 |
+
pipeline_tag: text-generation
|
| 5 |
+
library_name: peft
|
| 6 |
+
language:
|
| 7 |
+
- en
|
| 8 |
+
- zh
|
| 9 |
+
tags:
|
| 10 |
+
- hypernetwork
|
| 11 |
+
- hyper-lora
|
| 12 |
+
- lora
|
| 13 |
+
- role-play
|
| 14 |
+
- character-impersonation
|
| 15 |
+
- sft
|
| 16 |
+
- phase-tree
|
| 17 |
+
datasets:
|
| 18 |
+
- IAAR-Shanghai/phase_tree_data
|
| 19 |
+
---
|
| 20 |
+
|
| 21 |
+
# PHASE-Tree Hyper-LoRA SFT (anchor run)
|
| 22 |
+
|
| 23 |
+
**Variant:** Warm-start, lr=5e-6 (anchor run)
|
| 24 |
+
|
| 25 |
+
The **anchor** SFT run: hypernet warm-started from the PHASE-Tree pretrained
|
| 26 |
+
checkpoint and fine-tuned at a conservative learning rate of 5e-6 with label
|
| 27 |
+
smoothing 0.1 and NEFTune noise 5.0. This is the checkpoint reported in the
|
| 28 |
+
PHASE-Tree paper.
|
| 29 |
+
|
| 30 |
+
During development, six hyper-LoRA SFT cells were trained β an ablation grid
|
| 31 |
+
over initialisation (warm-start vs cold-start), learning rate (5e-6 vs 1e-5),
|
| 32 |
+
and trainable vs frozen hypernet output heads. Only this anchor cell is
|
| 33 |
+
bundled here; the other five are kept locally for reproducibility.
|
| 34 |
+
|
| 35 |
+
## What is a hypermod?
|
| 36 |
+
|
| 37 |
+
A **hypermod** (hyper-modulator) is a hypernetwork that, conditioned on a
|
| 38 |
+
character profile embedding, emits a low-rank LoRA delta `ΞW = AB` for each
|
| 39 |
+
target layer of the base model at inference time. The base model weights are
|
| 40 |
+
never updated; only the hypernet is trained. A single hypermod therefore
|
| 41 |
+
generalises across an open-ended set of personas without needing to store a
|
| 42 |
+
separate adapter per character.
|
| 43 |
+
|
| 44 |
+
## Files
|
| 45 |
+
|
| 46 |
+
| File | Purpose |
|
| 47 |
+
|------|---------|
|
| 48 |
+
| `hypermod.pt` | **Recommended checkpoint.** The anchor SFT step selected from per-step LLM-as-judge ratings (`character`, `semantic`) and Qwen3-Embedding-4B response-vs-reference cosine similarity. |
|
| 49 |
+
| `args.yaml` | Full training configuration; consumed by the loader to instantiate the hypernet architecture. |
|
| 50 |
+
| `adapter_config.json` | LoRA target-module stub (rank 8, alpha 16, `q_proj` + `v_proj`). |
|
| 51 |
+
| `timing_stats.json` | Wall-clock breakdown of the training run (training / validation / other overhead, in seconds). |
|
| 52 |
+
|
| 53 |
+
> Per-step snapshots (`checkpoints/it_5000` β¦ `it_40000`) and the post-hoc
|
| 54 |
+
> evaluation artefacts (`eval_ckpt_judge_scores/`, `eval_ckpt_val_loss/`)
|
| 55 |
+
> generated during training are **not bundled** with this release. They can
|
| 56 |
+
> be regenerated by re-running `src/scripts/train_phase_tree_qwen_7b.sh`
|
| 57 |
+
> followed by the evaluation scripts under `src/scripts/`.
|
| 58 |
+
|
| 59 |
+
## How to load
|
| 60 |
+
|
| 61 |
+
```python
|
| 62 |
+
from huggingface_hub import snapshot_download
|
| 63 |
+
from hyper_llm_modulator.hyper_modulator import load_hypermod_checkpoint
|
| 64 |
+
|
| 65 |
+
ckpt_dir = snapshot_download("<your-hf-username>/PHASE-Tree-hyper-lora-anchor")
|
| 66 |
+
|
| 67 |
+
(
|
| 68 |
+
args, hypermod, base_model, tokenizer,
|
| 69 |
+
emb_model, emb_tokenizer, task_desc_format_fn, pooling_fn,
|
| 70 |
+
) = load_hypermod_checkpoint(f"{ckpt_dir}/hypermod.pt", device="cuda")
|
| 71 |
+
```
|
| 72 |
+
|
| 73 |
+
The loader reads `args.yaml` and `adapter_config.json` from the same directory
|
| 74 |
+
as `hypermod.pt` automatically. The full inference pipeline (profile β
|
| 75 |
+
embedding β per-layer LoRA β generation) lives in the PHASE-Tree codebase.
|
| 76 |
+
|
| 77 |
+
## Training configuration
|
| 78 |
+
|
| 79 |
+
| Hyperparameter | Value |
|
| 80 |
+
|----------------|-------|
|
| 81 |
+
| Base model | `Qwen/Qwen2.5-7B-Instruct` |
|
| 82 |
+
| Task encoder | `Qwen/Qwen3-Embedding-4B` |
|
| 83 |
+
| Initialisation | Warm-start from `phase_tree_models/phase_tree_pretrained/hypermod.pt` |
|
| 84 |
+
| Target modules | `q_proj`, `v_proj` |
|
| 85 |
+
| LoRA rank `r` | 8 |
|
| 86 |
+
| LoRA alpha | 16 |
|
| 87 |
+
| LoRA dropout | 0.05 |
|
| 88 |
+
| Hypernet latent size | 1024 |
|
| 89 |
+
| Hypernet head input size | 2048 |
|
| 90 |
+
| Freeze hypernet heads | `false` |
|
| 91 |
+
| Optimizer steps | 40000 |
|
| 92 |
+
| Effective batch size | 8 (per-device 4 Γ grad-accum 2) |
|
| 93 |
+
| Learning rate | 5e-6 |
|
| 94 |
+
| Warmup fraction | 0.05 |
|
| 95 |
+
| Weight decay | 0.01 |
|
| 96 |
+
| Label smoothing | 0.1 |
|
| 97 |
+
| NEFTune noise Ξ± | 5.0 |
|
| 98 |
+
| Checkpoint cadence | every 5000 steps |
|
| 99 |
+
| Random seed | 42 |
|
| 100 |
+
|
| 101 |
+
The complete configuration (including dataset lists, sampler settings, and
|
| 102 |
+
fusion-module placeholders kept for loader compatibility) lives in `args.yaml`.
|
| 103 |
+
|
| 104 |
+
## Training data
|
| 105 |
+
|
| 106 |
+
The hypermod is jointly fine-tuned on the *train* splits of the eight
|
| 107 |
+
PHASE-Tree character-dialogue datasets (RAIDEN, CharacterEval, HPD, SimsConv,
|
| 108 |
+
ChatHaruhi, Friends, StarTrek_TNG, TheOffice), `m6_phase_tree` profile variant.
|
| 109 |
+
Sampling follows the hierarchical `sqrt_size` strategy with 6 tasks Γ 2 points
|
| 110 |
+
per batch.
|
| 111 |
+
|
| 112 |
+
## Evaluation
|
| 113 |
+
|
| 114 |
+
The released `hypermod.pt` was selected from per-step snapshots of the
|
| 115 |
+
training run by scoring predictions on a held-out evaluation set along
|
| 116 |
+
three axes:
|
| 117 |
+
|
| 118 |
+
- **`character` (1β5)** β profile-consistency rating by an LLM judge (see
|
| 119 |
+
`evaluation/persona_rubric.md` in the PHASE-Tree codebase for the rubric).
|
| 120 |
+
- **`semantic` (1β5)** β contextual-coherence rating by the same judge.
|
| 121 |
+
- **`embedding`** β cosine similarity of the predicted and reference response
|
| 122 |
+
embeddings computed with Qwen3-Embedding-4B.
|
| 123 |
+
|
| 124 |
+
The per-step intermediate snapshots and full evaluation artefacts produced
|
| 125 |
+
during model selection are not bundled (see the note above the loading
|
| 126 |
+
section); they can be regenerated from a re-training run via the scripts
|
| 127 |
+
under `src/scripts/`.
|
| 128 |
+
|
| 129 |
+
## Limitations
|
| 130 |
+
|
| 131 |
+
- Persona conditioning is mediated entirely by the profile embedding fed into
|
| 132 |
+
the task encoder; the model has no other persona-control surface.
|
| 133 |
+
- Generations may reproduce stylistic biases of the source corpora; intended
|
| 134 |
+
for research evaluation only.
|
| 135 |
+
- The checkpoint depends on the PHASE-Tree codebase for inference and is not a
|
| 136 |
+
drop-in `peft.PeftModel`: `adapter_config.json` describes only which layers
|
| 137 |
+
receive a generated LoRA, not directly loadable weights.
|
sft/hyper_lora/adapter_config.json
ADDED
|
@@ -0,0 +1,36 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"alpha_pattern": {},
|
| 3 |
+
"auto_mapping": null,
|
| 4 |
+
"base_model_name_or_path": "Qwen/Qwen2.5-7B-Instruct",
|
| 5 |
+
"bias": "none",
|
| 6 |
+
"corda_config": null,
|
| 7 |
+
"eva_config": null,
|
| 8 |
+
"exclude_modules": null,
|
| 9 |
+
"fan_in_fan_out": false,
|
| 10 |
+
"inference_mode": false,
|
| 11 |
+
"init_lora_weights": true,
|
| 12 |
+
"layer_replication": null,
|
| 13 |
+
"layers_pattern": null,
|
| 14 |
+
"layers_to_transform": null,
|
| 15 |
+
"loftq_config": {},
|
| 16 |
+
"lora_alpha": 16,
|
| 17 |
+
"lora_bias": false,
|
| 18 |
+
"lora_dropout": 0.05,
|
| 19 |
+
"megatron_config": null,
|
| 20 |
+
"megatron_core": "megatron.core",
|
| 21 |
+
"modules_to_save": null,
|
| 22 |
+
"peft_type": "LORA",
|
| 23 |
+
"qalora_group_size": 16,
|
| 24 |
+
"r": 8,
|
| 25 |
+
"rank_pattern": {},
|
| 26 |
+
"revision": null,
|
| 27 |
+
"target_modules": [
|
| 28 |
+
"v_proj",
|
| 29 |
+
"q_proj"
|
| 30 |
+
],
|
| 31 |
+
"task_type": "CAUSAL_LM",
|
| 32 |
+
"trainable_token_indices": null,
|
| 33 |
+
"use_dora": false,
|
| 34 |
+
"use_qalora": false,
|
| 35 |
+
"use_rslora": true
|
| 36 |
+
}
|
sft/hyper_lora/args.yaml
ADDED
|
@@ -0,0 +1,115 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
# ==========================================================================
|
| 2 |
+
# Training args for the released anchor SFT run.
|
| 3 |
+
# PHASE-Tree hyper-LoRA SFT on Qwen2.5-7B-Instruct.
|
| 4 |
+
# Variant: warm-start, lr=5e-6 (anchor SFT run).
|
| 5 |
+
# Effective batch size = batch_size * grad_accum_steps = 4 * 2 = 8.
|
| 6 |
+
# 40000 optimizer steps; a checkpoint is saved every val_freq=5000 steps.
|
| 7 |
+
# ==========================================================================
|
| 8 |
+
run_name: anchor
|
| 9 |
+
save_dir: phase_tree_models/sft/hyper_lora
|
| 10 |
+
|
| 11 |
+
# ββ Model ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 12 |
+
model_dir: Qwen/Qwen2.5-7B-Instruct
|
| 13 |
+
emb_model: Qwen/Qwen3-Embedding-4B
|
| 14 |
+
init_hypermod_from: phase_tree_models/phase_tree_pretrained/hypermod.pt
|
| 15 |
+
init_hypermod_strict: false
|
| 16 |
+
|
| 17 |
+
# ββ Task / data ββββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 18 |
+
training_task: sft
|
| 19 |
+
exp_setup: hyper_lora
|
| 20 |
+
sft_mode: completion
|
| 21 |
+
encoder_type: linear
|
| 22 |
+
train_ds_names:
|
| 23 |
+
- RAIDEN_train
|
| 24 |
+
- CharacterEval_train
|
| 25 |
+
- HPD_train
|
| 26 |
+
- SimsConv_train
|
| 27 |
+
- ChatHaruhi_train
|
| 28 |
+
- Friends_train
|
| 29 |
+
- StarTrek_TNG_train
|
| 30 |
+
- TheOffice_train
|
| 31 |
+
eval_ds_info:
|
| 32 |
+
- RAIDEN_random_test
|
| 33 |
+
- RAIDEN_ood_test
|
| 34 |
+
- CharacterEval_random_test
|
| 35 |
+
- CharacterEval_ood_test
|
| 36 |
+
- HPD_random_test
|
| 37 |
+
- HPD_ood_test
|
| 38 |
+
- SimsConv_random_test
|
| 39 |
+
- SimsConv_ood_test
|
| 40 |
+
- ChatHaruhi_random_test
|
| 41 |
+
- ChatHaruhi_ood_test
|
| 42 |
+
- Friends_random_test
|
| 43 |
+
- Friends_ood_test
|
| 44 |
+
- StarTrek_TNG_random_test
|
| 45 |
+
- StarTrek_TNG_ood_test
|
| 46 |
+
- TheOffice_random_test
|
| 47 |
+
- TheOffice_ood_test
|
| 48 |
+
use_per_task_emb: true
|
| 49 |
+
use_one_hot_task_emb: false
|
| 50 |
+
use_inp_as_desc: false
|
| 51 |
+
use_per_sample_desc: false
|
| 52 |
+
use_default_desc: false
|
| 53 |
+
use_hierarchical_sampler: true
|
| 54 |
+
dataset_sampling_strategy: sqrt_size
|
| 55 |
+
equally_weight_sample: true
|
| 56 |
+
n_tasks_per_batch: 6
|
| 57 |
+
n_points_per_task: 2
|
| 58 |
+
inp_max_len: 1024
|
| 59 |
+
target_modules:
|
| 60 |
+
- q_proj
|
| 61 |
+
- v_proj
|
| 62 |
+
|
| 63 |
+
# ββ Hypermod architecture ββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 64 |
+
use_hypernet: true
|
| 65 |
+
head_in_size: 2048
|
| 66 |
+
head_use_bias: false
|
| 67 |
+
hypernet_latent_size: 1024
|
| 68 |
+
delta_w_scaling: 100
|
| 69 |
+
pred_z_score: true
|
| 70 |
+
factorized: false
|
| 71 |
+
shared_AB_head: false
|
| 72 |
+
autoreg_gen: false
|
| 73 |
+
learnable_pos_emb: false
|
| 74 |
+
learnable_AB_offset: false
|
| 75 |
+
freeze_heads: false
|
| 76 |
+
|
| 77 |
+
# ββ Fusion (disabled for this run, kept for loader compatibility) ββββββββ
|
| 78 |
+
use_conv_fusion: false
|
| 79 |
+
conv_fusion_type: 1d
|
| 80 |
+
conv_fusion_kernel_size: 3
|
| 81 |
+
conv_fusion_num_layers: 2
|
| 82 |
+
conv_fusion_channels: 64
|
| 83 |
+
conv_fusion_dropout: 0.1
|
| 84 |
+
use_attention_fusion: false
|
| 85 |
+
attention_fusion_type: self
|
| 86 |
+
attention_num_heads: 8
|
| 87 |
+
attention_num_layers: 2
|
| 88 |
+
attention_dropout: 0.1
|
| 89 |
+
|
| 90 |
+
# ββ Optimisation βββββββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 91 |
+
lr: 5.0e-06
|
| 92 |
+
weight_decay: 0.01
|
| 93 |
+
warmup_frac: 0.05
|
| 94 |
+
max_grad_norm: 1.0
|
| 95 |
+
label_smoothing: 0.1
|
| 96 |
+
l2_reg_generated_w: 0.001
|
| 97 |
+
neftune_noise_alpha: 5.0
|
| 98 |
+
gradient_checkpointing: true
|
| 99 |
+
|
| 100 |
+
# ββ Schedule / batching ββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 101 |
+
epochs: 40000
|
| 102 |
+
batch_size: 4
|
| 103 |
+
grad_accum_steps: 2
|
| 104 |
+
val_batch_size: 16
|
| 105 |
+
seed: 42
|
| 106 |
+
|
| 107 |
+
# ββ Logging / saving βββββββββββββββββββββββββββββββββββββββββββββββββββββ
|
| 108 |
+
logging_freq: 50
|
| 109 |
+
val_freq: 5000 # validation + checkpoint cadence (steps)
|
| 110 |
+
top_k_checkpoints: 999 # keep every checkpoint
|
| 111 |
+
skip_val: true # validation done post-hoc via eval scripts
|
| 112 |
+
skip_eval: false
|
| 113 |
+
use_early_stopping: true
|
| 114 |
+
early_stopping_patience: 5
|
| 115 |
+
early_stopping_min_delta: 0.0
|
sft/hyper_lora/hypermod.pt
ADDED
|
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
version https://git-lfs.github.com/spec/v1
|
| 2 |
+
oid sha256:a5b00e1e07d793b199375a0def2abe387ca49b37a193981d58d93532deb8ffb0
|
| 3 |
+
size 925242042
|
sft/hyper_lora/timing_stats.json
ADDED
|
@@ -0,0 +1,6 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"total_runtime_seconds": 80705.80009109527,
|
| 3 |
+
"training_time_seconds": 78332.75524883717,
|
| 4 |
+
"validation_time_seconds": 0.025737229734659195,
|
| 5 |
+
"other_overhead_seconds": 2373.019105028361
|
| 6 |
+
}
|