nielsr HF Staff commited on
Commit
006a8e8
·
verified ·
1 Parent(s): e20e99a

Update pipeline_tag, add library_name, update paper link and project page link

Browse files

This PR improves the model card for the RLinf-VLA model by:
* Updating the `pipeline_tag` to `robotics` for better discoverability.
* Adding `library_name: transformers` as the model is compatible with the Hugging Face Transformers library.
* Adding direct links to the Hugging Face paper, and the project page.

Please review these changes.

Files changed (1) hide show
  1. README.md +22 -26
README.md CHANGED
@@ -1,56 +1,53 @@
1
  ---
2
- license: mit
3
- tags:
4
- - RLinf
5
  language:
6
  - en
 
7
  metrics:
8
  - accuracy
9
- base_model:
10
- - Haozhan72/Openvla-oft-SFT-libero-object-traj1
11
- pipeline_tag: reinforcement-learning
 
12
  model-index:
13
  - name: RLinf-OpenVLAOFT-GRPO-LIBERO-object
14
  results:
15
  - task:
16
- type: VLA # Required. Example: automatic-speech-recognition
17
  dataset:
18
- type: libero_object # Required. Example: common_voice. Use dataset id from https://hf.co/datasets
19
- name: libero_object # Required. A pretty name for the dataset. Example: Common Voice (French)
20
  metrics:
21
- - type: accuracy # Required. Example: wer. Use metric id from https://hf.co/metrics
22
- value: 98.99 # Required. Example: 20.90
23
  ---
24
 
25
  <div align="center">
26
  <img src="logo.svg" alt="RLinf-logo" width="500"/>
27
  </div>
28
 
29
-
30
  <div align="center">
31
- <!-- <a href="TODO"><img src="https://img.shields.io/badge/arXiv-Paper-red?logo=arxiv"></a> -->
32
- <!-- <a href="TODO"><img src="https://img.shields.io/badge/HuggingFace-yellow?logo=huggingface&logoColor=white" alt="Hugging Face"></a> -->
33
  <a href="https://github.com/RLinf/RLinf"><img src="https://img.shields.io/badge/Github-blue"></a>
34
- <a href="https://rlinf.readthedocs.io/en/latest/"><img src="https://img.shields.io/badge/Documentation-Purple?color=8A2BE2&logo=readthedocs"></a>
35
- <!-- <a href="TODO"><img src="https://devin.ai/assets/deepwiki-badge.png" alt="Ask DeepWiki.com" style="height:20px;"></a>
36
- <a href="TODO"><img src="https://img.shields.io/badge/微信-green?logo=wechat&amp"></a> -->
37
  </div>
38
 
39
  <h1 align="center">RLinf: Reinforcement Learning Infrastructure for Agentic AI</h1>
40
 
41
  [RLinf](https://github.com/RLinf/RLinf) is a flexible and scalable open-source infrastructure designed for post-training foundation models (LLMs, VLMs, VLAs) via reinforcement learning. The 'inf' in RLinf stands for Infrastructure, highlighting its role as a robust backbone for next-generation training. It also stands for Infinite, symbolizing the system’s support for open-ended learning, continuous generalization, and limitless possibilities in intelligence development.
42
 
43
-
44
  <div align="center">
45
  <img src="overview.png" alt="RLinf-overview" width="600"/>
46
  </div>
47
 
48
  ## Model Description
 
 
49
  The RLinf-openvlaoft-libero series is trained on RLinf/RLinf-OpenVLAOFT-LIBERO-xxx-Base-Lora (including libero90 and libero130) and Haozhan72/Openvla-oft-SFT-libero-xxx-traj1 (including libero10, libero-object, libero-goal and libero-spatial), using the same base models and training datasets as verl. Training with RLinf yields SOTA performance.
50
 
51
  We use a mask to focus on valid action tokens, and compute token-level loss based on the Group Relative Policy Optimization (GRPO) advantage function, in order to enhance the model’s performance on spatial reasoning, object generalization, instruction generalization, and long-horizon tasks.
52
 
53
-
54
  ## Evaluation and Results
55
  We trained and evaluated four models using RLinf:
56
 
@@ -72,23 +69,22 @@ We trained and evaluated four models using RLinf:
72
  - RLinf-openvlaoft-libero10 Model (based on [Haozhan72/Openvla-oft-SFT-libero10-traj1]((https://huggingface.co/Haozhan72/Openvla-oft-SFT-libero10-traj1)))
73
  - Recommended sampling settings: `temperature = 1.6`, `top_p = 1.0`
74
 
75
-
76
  ### Benchmark Results
77
 
78
- Sft models for LIBERO-90 and LIBERO-130 are trained by ourself following training reciepe from [OpenVLA-OFT](https://github.com/moojink/openvla-oft/blob/main/vla-scripts/finetune.py). And other sft models are from [SimpleVLA-RL](https://huggingface.co/collections/Haozhan72/simplevla-rl-6833311430cd9df52aeb1f86).
79
  - Recommended sampleing setting for evaluation: `libero seed=0`; `episode number=500`; `do_sample=False`
80
 
81
  | Model | Object | Spatial | Goal | Long | 90 | Average |
82
  | ------------------ | ------ | ------- | ----- | ----- | ------- |------- |
83
- | sft models | 25.60 | 56.45 | 45.59 | 9.68 | 78.63 | 43.19 |
84
- | trained with RLinf | 98.99 | 98.99 | 98.99 | 94.35 | 96.77 | 97.62 |
85
 
86
  Besides, we train one model (we named it libero-130 model) for all tasks in libero.
87
 
88
  | libero-130 model | Object | Spatial | Goal | Long | 90 | 130(all) |
89
  | ------------------ | ------ | ------- | ----- | ----- | ------- |------- |
90
- | sft models | 71.48 | 72.18 | 64.06 | 48.44 | 70.97 | 70.78 |
91
- | trained with RLinf | 99.80 | 99.40 | 98.79 | 93.95 | 98.32 | 98.09 |
92
 
93
  <div align="center">
94
  <img src="tensorboard-success_once.png" alt="RLinf-libero-result" width="600"/>
@@ -102,4 +98,4 @@ Please integrate the provided model with the [RLinf](https://github.com/RLinf/RL
102
  Note: If you intend to evaluate the model directly, make sure to set ``actor.model.is_lora`` to ``false``.
103
 
104
  ## License
105
- This code repository and the model weights are licensed under the MIT License.
 
1
  ---
2
+ base_model:
3
+ - Haozhan72/Openvla-oft-SFT-libero-object-traj1
 
4
  language:
5
  - en
6
+ license: mit
7
  metrics:
8
  - accuracy
9
+ pipeline_tag: robotics
10
+ tags:
11
+ - RLinf
12
+ library_name: transformers
13
  model-index:
14
  - name: RLinf-OpenVLAOFT-GRPO-LIBERO-object
15
  results:
16
  - task:
17
+ type: VLA
18
  dataset:
19
+ name: libero_object
20
+ type: libero_object
21
  metrics:
22
+ - type: accuracy
23
+ value: 98.99
24
  ---
25
 
26
  <div align="center">
27
  <img src="logo.svg" alt="RLinf-logo" width="500"/>
28
  </div>
29
 
 
30
  <div align="center">
31
+ <a href="https://huggingface.co/papers/2510.06710"><img src="https://img.shields.io/badge/Paper-RLinf--VLA-red?logo=arxiv"></a>
32
+ <a href="https://rlinf.readthedocs.io/en/latest/"><img src="https://img.shields.io/badge/Project_Page-Purple?color=8A2BE2&logo=readthedocs"></a>
33
  <a href="https://github.com/RLinf/RLinf"><img src="https://img.shields.io/badge/Github-blue"></a>
 
 
 
34
  </div>
35
 
36
  <h1 align="center">RLinf: Reinforcement Learning Infrastructure for Agentic AI</h1>
37
 
38
  [RLinf](https://github.com/RLinf/RLinf) is a flexible and scalable open-source infrastructure designed for post-training foundation models (LLMs, VLMs, VLAs) via reinforcement learning. The 'inf' in RLinf stands for Infrastructure, highlighting its role as a robust backbone for next-generation training. It also stands for Infinite, symbolizing the system’s support for open-ended learning, continuous generalization, and limitless possibilities in intelligence development.
39
 
 
40
  <div align="center">
41
  <img src="overview.png" alt="RLinf-overview" width="600"/>
42
  </div>
43
 
44
  ## Model Description
45
+ This repository hosts the **RLinf-VLA** model, a unified and efficient framework for VLA+RL Training, as presented in the paper [RLinf-VLA: A Unified and Efficient Framework for VLA+RL Training](https://huggingface.co/papers/2510.06710).
46
+
47
  The RLinf-openvlaoft-libero series is trained on RLinf/RLinf-OpenVLAOFT-LIBERO-xxx-Base-Lora (including libero90 and libero130) and Haozhan72/Openvla-oft-SFT-libero-xxx-traj1 (including libero10, libero-object, libero-goal and libero-spatial), using the same base models and training datasets as verl. Training with RLinf yields SOTA performance.
48
 
49
  We use a mask to focus on valid action tokens, and compute token-level loss based on the Group Relative Policy Optimization (GRPO) advantage function, in order to enhance the model’s performance on spatial reasoning, object generalization, instruction generalization, and long-horizon tasks.
50
 
 
51
  ## Evaluation and Results
52
  We trained and evaluated four models using RLinf:
53
 
 
69
  - RLinf-openvlaoft-libero10 Model (based on [Haozhan72/Openvla-oft-SFT-libero10-traj1]((https://huggingface.co/Haozhan72/Openvla-oft-SFT-libero10-traj1)))
70
  - Recommended sampling settings: `temperature = 1.6`, `top_p = 1.0`
71
 
 
72
  ### Benchmark Results
73
 
74
+ Sft models for LIBERO-90 and LIBERO-130 are trained by ourself following training reciepe from [OpenVLA-OFT](https://github.com/moojink/openvla-oft/blob/main/vla-scripts/finetune.py). And other sft models are from [SimpleVLA-RL](https://huggingface.co/collections/Haozhan72/simplevla-rl-6833311430cd9df52aeb1f86).
75
  - Recommended sampleing setting for evaluation: `libero seed=0`; `episode number=500`; `do_sample=False`
76
 
77
  | Model | Object | Spatial | Goal | Long | 90 | Average |
78
  | ------------------ | ------ | ------- | ----- | ----- | ------- |------- |
79
+ | sft models | 25.60 | 56.45 | 45.59 | 9.68 | 78.63 | 43.19 |
80
+ | trained with RLinf | 98.99 | 98.99 | 98.99 | 94.35 | 96.77 | 97.62 |
81
 
82
  Besides, we train one model (we named it libero-130 model) for all tasks in libero.
83
 
84
  | libero-130 model | Object | Spatial | Goal | Long | 90 | 130(all) |
85
  | ------------------ | ------ | ------- | ----- | ----- | ------- |------- |
86
+ | sft models | 71.48 | 72.18 | 64.06 | 48.44 | 70.97 | 70.78 |
87
+ | trained with RLinf | 99.80 | 99.40 | 98.79 | 93.95 | 98.32 | 98.09 |
88
 
89
  <div align="center">
90
  <img src="tensorboard-success_once.png" alt="RLinf-libero-result" width="600"/>
 
98
  Note: If you intend to evaluate the model directly, make sure to set ``actor.model.is_lora`` to ``false``.
99
 
100
  ## License
101
+ This code repository and the model weights are licensed under the MIT License.