FRIDA-Decisions: bf16 weights, head, int8 ONNX, model card

.gitattributes

@@ -0,0 +1,35 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,144 @@
+---
+license: mit
+language:
+- ru
+- en
+base_model: ai-forever/FRIDA
+base_model_relation: finetune
+pipeline_tag: text-classification
+library_name: frida-decisions
+tags:
+- decisions
+- routing
+- intent-classification
+- guardrails
+- zero-shot-classification
+- reranking
+- encoder
+- russian
+---
+
+# FRIDA-Decisions
+
+**FRIDA-Decisions** makes structured decisions over Russian text in a single encoder pass: pick one of K options, place a text on an ordinal scale, answer yes / no, or rank candidates. The options are written as text inside the request, so a new label set is a new JSON, not a new training run. No generation, no output tokens, no parsing: every answer is one of the declared options, with its confidence.
+
+It is built on [ai-forever/FRIDA](https://huggingface.co/ai-forever/FRIDA) (T5 encoder, 823M parameters) and runs on a consumer GPU.
+
+* **razvilka.** 0.890 on [razvilka](https://huggingface.co/datasets/artemsnegirev/razvilka) (735 items); TypeSafe Jev, a commercial API, scores 0.897 on the same items (paired McNemar p = 0.68). The highest among the open models we ran on razvilka.
+* **Fast.** About 30–45 ms per request on an RTX 5060 Ti, in process; exact conditions in the latency table below.
+* **Cheap to run.** 4.3 GiB of GPU memory at peak; an int8 ONNX build runs on CPU.
+* **Packing.** All options of all questions share one sequence and the text is encoded once; with the state cache a follow-up question about the same text costs only its own tokens. A catalog of 243 intents is answered in 0.44 s, against 4.65 s for one sequence per option.
+
+## Quickstart
+
+```bash
+pip install "frida-decisions[torch] @ git+https://github.com/ai-forever/frida-decisions"
+```
+
+```python
+from frida_decisions import Judge
+
+judge = Judge.from_pretrained("ai-forever/FRIDA-Decisions", device="cuda")
+
+request = {
+    "state": "Здравствуйте, у меня не приходит код подтверждения уже час.",
+    "questions": {
+        "topic": {
+            "type": "choice",
+            "instructions": "К какой теме относится обращение?",
+            "criteria": {
+                "login": "вход в аккаунт, коды подтверждения, пароли",
+                "payment": "оплата, списания, возвраты",
+                "delivery": "доставка заказа",
+            },
+        },
+        "urgency": {
+            "type": "score",
+            "instructions": "Оцени срочность обращения.",
+            "criteria": ["низкая", "средняя", "высокая"],
+        },
+        "spam": {
+            "type": "noul",
+            "instructions": "Является ли сообщение спамом?",
+            "criteria": {"true": "реклама, чужие ссылки, просьба перевести деньги",
+                         "false": "вопрос или жалоба по нашему сервису"},
+        },
+    },
+}
+print(judge.judge(request)["answers"])
+```
+
+CPU without PyTorch: `pip install "frida-decisions[onnx] @ git+..."` and `OnnxJudge.from_pretrained("ai-forever/FRIDA-Decisions")` — int8 weights and per-token int8 activations; its decisions agree with the fp32 model on 120 of 122 test decisions, and a 384-token request with 3 questions takes 0.88 s on 6 CPU threads (2.6x faster than fp32).
+
+## Question types
+
+| type | asks | returns |
+|---|---|---|
+| `choice` | which of K options is right | the option key and a distribution over all options |
+| `score` | where the text sits on an ordinal scale | a level 0..K−1 and its distribution |
+| `noul` | is a statement true | `p(true)` |
+| `ranking` | order candidates for a query | the order and a margin per candidate |
+
+Texts up to 512 tokens are the recommended range.
+
+## Benchmarks
+
+**razvilka** — 735 Russian items, 15 tasks (routing, intents, topic and sentiment classification, moderation, relevance ranking), all four question types, gold from published datasets. All models are run on the same requests and scored with the same script ([razvilka_eval.py](https://github.com/ai-forever/frida-decisions/tree/main/benchmarks/razvilka)).
+
+| model | parameters | accuracy |
+|---|--:|--:|
+| TypeSafe Jev (commercial API) | — | 0.897 |
+| **FRIDA-Decisions** | 823M | **0.890** |
+| smolnikov/migom-2b | 1.9B | 0.853 |
+| Mapika/decider-2b | 1.9B | 0.833 |
+| smolnikov/kivok-0.3b | 0.3B | 0.619 |
+| fastino/GLiNER2.5-multi-Decide | 287M | 0.576 |
+| convaiinnovations/laya (multilingual) | — | 0.559 |
+| KaLM-Reranker-V1-Nano-R2 | 786M | 0.521 |
+| open-jev (DeBERTa-v3-large) | 437M | 0.490¹ |
+| lexical baseline | — | 0.333 |
+| chance | — | 0.257 |
+
+¹ 140 of 735 texts exceed open-jev's input window and count as random answers; on the other 595 it scores 0.565.
+
+**Latency**, one request, single stream, in process:
+
+| hardware | request | time |
+|---|---|--:|
+| RTX 5060 Ti, bf16 | 384-token state, 1 question | 31.9 ms² |
+| RTX 5060 Ti, bf16 | 384-token state, 3 questions | 44.5 ms² |
+| RTX 5060 Ti, bf16 | 317-token state, 3 questions (8 options) | 28.3 ms |
+| CPU, 6 threads, PyTorch fp32 | 384-token state, 3 questions (8 options) | 2.28 s |
+| CPU, 6 threads, ONNX int8 | 384-token state, 3 questions (8 options) | 0.88 s (x2.6) |
+
+² An earlier measurement on the same architecture; the other rows were measured on the released weights.
+
+**Packing**, RTX 5060 Ti, bf16, same model, one sequence per option vs packed with the state cache:
+
+| scenario | candidates | tokens, naive / packed | time, naive / packed |
+|---|--:|--:|--:|
+| intent from a 243-intent support-bot catalog | 243 | 97,685 / 4,871 | 4.65 s / 0.44 s |
+| full triage of a support email | 16 questions, 65 options | 23,487 / 1,853 | 1.10 s / 0.18 s |
+| follow-up question about a cached document | 13 | 5,081 / 277 | 0.24 s / 25 ms |
+
+## Training
+
+* 1.42M examples (1.5M questions) over 151 question types in eight domains: relevance and RAG grounding, NLI and fact checking, moderation and safety, agents and tool choice, topic classification, sentiment and emotion, LLM request routing, intents and customer support.
+* 71% Russian, 29% English; mostly human or naturally labelled data, about a quarter with LLM-generated text or model labels; instruction wordings augmented with paraphrases.
+* LoRA rank 16 on the attention projections (q, k, v, o) of all 24 layers plus a scalar head, 4.7M trainable parameters; merged into the weights in this release. Listwise softmax for `choice` / `ranking`, pairwise BCE for `noul`.
+* One epoch, with a compute budget comparable to about 50 hours of a single consumer GPU (RTX 5060 Ti class).
+
+## Files
+
+* `model.safetensors` — encoder weights, bf16
+* `head.safetensors` — readout head, fp32
+* `decisions_config.json` — packing limits and instruction suffixes
+* `onnx/model_int8_pertoken.onnx` — CPU build
+
+## License
+
+MIT. Based on [ai-forever/FRIDA](https://huggingface.co/ai-forever/FRIDA) (MIT).
+
+## Citation
+
+TODO

config.json

@@ -0,0 +1,35 @@
+{
+  "architectures": [
+    "T5EncoderModel"
+  ],
+  "bos_token_id": 1,
+  "classifier_dropout": 0.0,
+  "d_ff": 4096,
+  "d_kv": 64,
+  "d_model": 1536,
+  "decoder_start_token_id": 0,
+  "dense_act_fn": "gelu_new",
+  "dropout_rate": 0.0,
+  "dtype": "bfloat16",
+  "eos_token_id": 2,
+  "feed_forward_proj": "gated-gelu",
+  "gradient_checkpointing": false,
+  "initializer_factor": 1.0,
+  "is_decoder": false,
+  "is_encoder_decoder": false,
+  "is_gated_act": true,
+  "layer_norm_epsilon": 1e-06,
+  "model_type": "t5",
+  "num_decoder_layers": 24,
+  "num_heads": 24,
+  "num_layers": 24,
+  "output_past": true,
+  "pad_token_id": 0,
+  "relative_attention_max_distance": 128,
+  "relative_attention_num_buckets": 32,
+  "scale_decoder_outputs": false,
+  "tie_word_embeddings": true,
+  "transformers_version": "5.17.0",
+  "use_cache": false,
+  "vocab_size": 93651
+}

decisions_config.json

@@ -0,0 +1,24 @@
+{
+  "state_max_tokens": 512,
+  "instruction_max_tokens": 96,
+  "option_max_tokens": 256,
+  "max_options_per_row": 16,
+  "max_row_tokens": 1024,
+  "align": 8,
+  "eos_token_id": 2,
+  "pad_token_id": 0,
+  "tokenizer_max_length": 1024,
+  "num_buckets": 32,
+  "max_distance": 128,
+  "instruction_suffixes": {
+    "choice": "Оцени, является ли вариант, описанный в кандидате, подходящим ответом на вопрос выше, с учётом сведений в запросе. Суди о варианте по его описанию, а не по одной лишь тематической близости.",
+    "score": "Оцени, точно ли уровень, описанный в кандидате, характеризует запрос в рамках оценки выше. Суди об уровне по его описанию, а не по одной лишь тематической близости.",
+    "noul": "Оцени, верно ли кандидат описывает ответ на вопрос выше, с учётом сведений в запросе. Отвечай утвердительно, если этот вариант выполняется, иначе отрицательно.",
+    "ranking": "Оцени, насколько кандидат подходит под вопрос выше с учётом сведений в запросе. Кандидат — это сам материал, а не описание варианта: суди по его содержанию, а не по одной лишь тематической близости."
+  },
+  "yes_no_default_criteria": {
+    "true": "The answer to the question is yes.",
+    "false": "The answer to the question is no."
+  },
+  "format_version": 1
+}

head.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e822115e760c243d9b36010573efadc2e29c315323cccdbd5597a4f6dbd74f91
+size 6284

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:70a8cb915323c072f549f46b0f89c57a83b54128458e3cc7178a7bc3c013de5f
+size 1646829336

onnx/model_int8_pertoken.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:09e17f504d100551108d2af96f20d0e8a5b4ee656bf39e18dd092d0115694124
+size 1257200228

tokenizer_config.json

@@ -0,0 +1,16 @@
+{
+  "add_prefix_space": false,
+  "backend": "tokenizers",
+  "bos_token": "<s>",
+  "clean_up_tokenization_spaces": true,
+  "cls_token": "<s>",
+  "eos_token": "</s>",
+  "errors": "replace",
+  "mask_token": "<mask>",
+  "model_max_length": 1024,
+  "pad_token": "<pad>",
+  "sep_token": "</s>",
+  "tokenizer_class": "RobertaTokenizer",
+  "trim_offsets": true,
+  "unk_token": "<unk>"
+}