SAFE-LSTM (GR00T-N1.7 / LIBERO-10 + Occ, paper split seed 0)
SAFE LSTM: causal LSTM(256) + Linear + sigmoid per step, BCE on the per-step probabilities with the rollout outcome as label, the baselines' recipe (Adam 3e-4, batches of 32 rollouts, patience 25, model selection by val AUROC). The alarm uses SAFE's functional conformal band.
Input
Per-modality pooled GR00T-N1.7 backbone features: [mean of the 64 base-image tokens | mean of the 64 wrist-image tokens | masked mean of the real language tokens], each 2048-d, i.e. one 6144-d vector per policy step.
Standardise with normalization.npz (train-split mean/std), run the model causally over the policy steps, and raise an alarm at the first step whose
score reaches the band in thresholds.json (SAFE functional conformal band,
target false-alarm rate alpha). One step = 8 control steps = 0.4 s.
pip install torch safetensors numpy
python probe.py features.npy --alpha 0.10
Files
model.safetensors, config.json, normalization.npz, thresholds.json, eval_task_split.json, EVAL.md, models.py, probe.py.
Results on the LIBERO-10 paper split (seed 0; alpha-averaged over 0.01..0.30, alpha = 0.10 in brackets)
| TPR | TPR early | on-time TPR | LDR | FPR | |
|---|---|---|---|---|---|
| held-out seen tasks | 79.0 (73.3) | 78.1 (73.3) | 6.7 (6.7) | 1.0 (0.0) | 12.2 (0.0) |
| unseen tasks | 61.4 (56.7) | 56.2 (50.0) | 14.8 (13.3) | 11.5 (15.4) | 21.2 (19.2) |
Full per-alpha table: EVAL.md.
- Downloads last month
- 13