ksanjiv05/laya-for-rn-executorch

Laya — ExecuTorch (.pte) for on-device / React Native

ExecuTorch exports of convaiinnovations/laya (the typed-decisions checkpoint: ModernBERT-large encoder + Laya's from-scratch decision head), ready to run on-device via the ExecuTorch XNNPACK and Core ML backends — e.g. from react-native-executorch.

Laya is a System-1 decision model: you give it a state (JSON/text) plus a typed question (choice / score / noul) and it returns a calibrated answer in one forward pass — it does not generate text.

Files

File	Size	What
laya_xnnpack_int8wo.pte	603 MB	int8 weight-only quantized, XNNPACK — recommended for mobile (iOS + Android)
laya_xnnpack.pte	1.69 GB	fp32, XNNPACK — reference / desktop
laya_coreml.pte	845 MB	Core ML (Apple Neural Engine / GPU) — recommended for iOS, device-verified on iPhone 15
laya_vulkan_int8.pte	603 MB	int8, Vulkan (Android GPU) — experimental, see BACKENDS.md
laya_vulkan.pte	1.69 GB	fp32, Vulkan (Android GPU) — experimental
*.meta.json	—	runtime contract (I/O shapes, temperatures, qtype map)
laya_testcases.json	—	4 tokenized test cases + reference answers for verification

See BACKENDS.md for the per-platform backend matrix (XNNPACK / Vulkan / Core ML) and how to build the iOS Core ML variant.

Runtime contract (static shapes)

The graph is exported with fixed shapes for mobile:

Inputs (in order):

• input_ids [1, 192] int64
• attention_mask [1, 192] int64
• marker_pos [1, 12] int64 — token index of each option's [MASK] marker
• marker_mask [1, 12] bool — which of the 12 option slots are real
• qtype [1] int64 — 0=choice, 1=score, 2=noul

Outputs:

• logits [1, 12] float32 — per-option scores (take the first k = #real options)
• act_logits [1, 2] float32 — auxiliary escalate/answer head

Post-processing (see rl_agent_api.py in the base repo): divide the k logits by the temperature for the (qtype, k) bucket (in the .meta.json), softmax, then: choice → argmax option; score → Σ i·pᵢ; noul → p[1] = P(true).

Input construction mirrors Laya's build_sequence: [CLS] "<type> question: <instructions>" [SEP] [MASK] opt0 [MASK] opt1 … [SEP] <state> [SEP] (tokenizer: the base repo's ModernBERT tokenizer; CLS=50281 SEP=50282 MASK=50284 PAD=50283).

Benchmark — iPhone 15 (Core ML)

laya_coreml.pte running fully offline in the React Native demo app (react-native-executorch 0.10.2, New Architecture) on an iPhone 15. Batch 1, seq len 192.

Case	Type	Decision	On-device	Reference (desktop)	Latency
Sentiment	choice 3-way	Positive	67 / 25 / 7 %	66.2 / 25.7 / 8.1 %	2829 ms ¹
Intent routing	choice 4-way	Billing	76 / 8 / 7 / 9 %	73.7 / 8.8 / 7.5 / 10.0 %	47 ms
Urgency	score 0–4	Critical · 3.5 / 4	1 / 2 / 8 / 30 / 60 %	score 3.446 (0.9 / 2.0 / 8.1 / 29.9 / 59.2 %)	45 ms
Policy check	noul yes/no	Yes (phishing)	P(yes) = 54 %	P(yes) = 53.5 %	42 ms

¹ First inference after load includes Core ML's one-time model preparation / warm-up. Steady-state latency is ~42–47 ms per decision (median 45 ms over the three warm runs).

4/4 decisions match the reference, with probabilities within ~2.5 points (Core ML computes in fp16).

Compared with other targets

Target	Model	Warm latency / decision
iPhone 15 — Core ML (ANE/GPU)	laya_coreml.pte	~45 ms
Android phone — XNNPACK CPU	laya_xnnpack_int8wo.pte	~7.7 s avg
Desktop CPU — XNNPACK (reference)	laya_xnnpack_int8wo.pte	~1.6 s

On iPhone, Core ML is roughly 35× faster than desktop CPU and ~170× faster than the Android CPU build; ship laya_coreml.pte for iOS.

Verification

Device-verified on a physical Android phone (ModernBERT-large int8, XNNPACK CPU): all 4 laya_testcases.json cases reproduced the desktop int8 reference exactly. int8 vs fp32: argmax and probabilities agree to ~1e-3. Use laya_testcases.json (tokenized inputs + reference answers) to verify your own integration.

Device-verified on iPhone 15 (Core ML): all 4 cases give the same decisions as the reference — see the benchmark above.

Reproduce the export

See export/export_laya_pte.py in the laya-for-react-native project: rebuild DecisionModel, load the typed-decisions weights, torch.export → XNNPACK lower → .pte, with torchao Int8WeightOnlyConfig for the int8 build. (torch>=2.11, executorch>=1.5.)

License

Apache-2.0, inherited from the base model convaiinnovations/laya.