Update pipeline_tag, add library_name, update paper link and project page link
Browse filesThis PR improves the model card for the RLinf-VLA model by:
* Updating the `pipeline_tag` to `robotics` for better discoverability.
* Adding `library_name: transformers` as the model is compatible with the Hugging Face Transformers library.
* Adding direct links to the Hugging Face paper, and the project page.
Please review these changes.
README.md
CHANGED
|
@@ -1,56 +1,53 @@
|
|
| 1 |
---
|
| 2 |
-
|
| 3 |
-
|
| 4 |
-
- RLinf
|
| 5 |
language:
|
| 6 |
- en
|
|
|
|
| 7 |
metrics:
|
| 8 |
- accuracy
|
| 9 |
-
|
| 10 |
-
|
| 11 |
-
|
|
|
|
| 12 |
model-index:
|
| 13 |
- name: RLinf-OpenVLAOFT-GRPO-LIBERO-object
|
| 14 |
results:
|
| 15 |
- task:
|
| 16 |
-
type: VLA
|
| 17 |
dataset:
|
| 18 |
-
|
| 19 |
-
|
| 20 |
metrics:
|
| 21 |
-
|
| 22 |
-
|
| 23 |
---
|
| 24 |
|
| 25 |
<div align="center">
|
| 26 |
<img src="logo.svg" alt="RLinf-logo" width="500"/>
|
| 27 |
</div>
|
| 28 |
|
| 29 |
-
|
| 30 |
<div align="center">
|
| 31 |
-
<
|
| 32 |
-
<
|
| 33 |
<a href="https://github.com/RLinf/RLinf"><img src="https://img.shields.io/badge/Github-blue"></a>
|
| 34 |
-
<a href="https://rlinf.readthedocs.io/en/latest/"><img src="https://img.shields.io/badge/Documentation-Purple?color=8A2BE2&logo=readthedocs"></a>
|
| 35 |
-
<!-- <a href="TODO"><img src="https://devin.ai/assets/deepwiki-badge.png" alt="Ask DeepWiki.com" style="height:20px;"></a>
|
| 36 |
-
<a href="TODO"><img src="https://img.shields.io/badge/微信-green?logo=wechat&"></a> -->
|
| 37 |
</div>
|
| 38 |
|
| 39 |
<h1 align="center">RLinf: Reinforcement Learning Infrastructure for Agentic AI</h1>
|
| 40 |
|
| 41 |
[RLinf](https://github.com/RLinf/RLinf) is a flexible and scalable open-source infrastructure designed for post-training foundation models (LLMs, VLMs, VLAs) via reinforcement learning. The 'inf' in RLinf stands for Infrastructure, highlighting its role as a robust backbone for next-generation training. It also stands for Infinite, symbolizing the system’s support for open-ended learning, continuous generalization, and limitless possibilities in intelligence development.
|
| 42 |
|
| 43 |
-
|
| 44 |
<div align="center">
|
| 45 |
<img src="overview.png" alt="RLinf-overview" width="600"/>
|
| 46 |
</div>
|
| 47 |
|
| 48 |
## Model Description
|
|
|
|
|
|
|
| 49 |
The RLinf-openvlaoft-libero series is trained on RLinf/RLinf-OpenVLAOFT-LIBERO-xxx-Base-Lora (including libero90 and libero130) and Haozhan72/Openvla-oft-SFT-libero-xxx-traj1 (including libero10, libero-object, libero-goal and libero-spatial), using the same base models and training datasets as verl. Training with RLinf yields SOTA performance.
|
| 50 |
|
| 51 |
We use a mask to focus on valid action tokens, and compute token-level loss based on the Group Relative Policy Optimization (GRPO) advantage function, in order to enhance the model’s performance on spatial reasoning, object generalization, instruction generalization, and long-horizon tasks.
|
| 52 |
|
| 53 |
-
|
| 54 |
## Evaluation and Results
|
| 55 |
We trained and evaluated four models using RLinf:
|
| 56 |
|
|
@@ -72,23 +69,22 @@ We trained and evaluated four models using RLinf:
|
|
| 72 |
- RLinf-openvlaoft-libero10 Model (based on [Haozhan72/Openvla-oft-SFT-libero10-traj1]((https://huggingface.co/Haozhan72/Openvla-oft-SFT-libero10-traj1)))
|
| 73 |
- Recommended sampling settings: `temperature = 1.6`, `top_p = 1.0`
|
| 74 |
|
| 75 |
-
|
| 76 |
### Benchmark Results
|
| 77 |
|
| 78 |
-
Sft models for LIBERO-90 and LIBERO-130 are trained by ourself following training reciepe from [OpenVLA-OFT](https://github.com/moojink/openvla-oft/blob/main/vla-scripts/finetune.py). And other sft models are from [SimpleVLA-RL](https://huggingface.co/collections/Haozhan72/simplevla-rl-6833311430cd9df52aeb1f86).
|
| 79 |
- Recommended sampleing setting for evaluation: `libero seed=0`; `episode number=500`; `do_sample=False`
|
| 80 |
|
| 81 |
| Model | Object | Spatial | Goal | Long | 90 | Average |
|
| 82 |
| ------------------ | ------ | ------- | ----- | ----- | ------- |------- |
|
| 83 |
-
| sft models | 25.60 | 56.45 | 45.59 | 9.68 | 78.63 | 43.19 |
|
| 84 |
-
| trained with RLinf | 98.99 | 98.99 | 98.99 | 94.35 | 96.77 | 97.62 |
|
| 85 |
|
| 86 |
Besides, we train one model (we named it libero-130 model) for all tasks in libero.
|
| 87 |
|
| 88 |
| libero-130 model | Object | Spatial | Goal | Long | 90 | 130(all) |
|
| 89 |
| ------------------ | ------ | ------- | ----- | ----- | ------- |------- |
|
| 90 |
-
| sft models | 71.48 | 72.18 | 64.06 | 48.44 | 70.97 | 70.78 |
|
| 91 |
-
| trained with RLinf | 99.80 | 99.40 | 98.79 | 93.95 | 98.32 | 98.09 |
|
| 92 |
|
| 93 |
<div align="center">
|
| 94 |
<img src="tensorboard-success_once.png" alt="RLinf-libero-result" width="600"/>
|
|
@@ -102,4 +98,4 @@ Please integrate the provided model with the [RLinf](https://github.com/RLinf/RL
|
|
| 102 |
Note: If you intend to evaluate the model directly, make sure to set ``actor.model.is_lora`` to ``false``.
|
| 103 |
|
| 104 |
## License
|
| 105 |
-
This code repository and the model weights are licensed under the MIT License.
|
|
|
|
| 1 |
---
|
| 2 |
+
base_model:
|
| 3 |
+
- Haozhan72/Openvla-oft-SFT-libero-object-traj1
|
|
|
|
| 4 |
language:
|
| 5 |
- en
|
| 6 |
+
license: mit
|
| 7 |
metrics:
|
| 8 |
- accuracy
|
| 9 |
+
pipeline_tag: robotics
|
| 10 |
+
tags:
|
| 11 |
+
- RLinf
|
| 12 |
+
library_name: transformers
|
| 13 |
model-index:
|
| 14 |
- name: RLinf-OpenVLAOFT-GRPO-LIBERO-object
|
| 15 |
results:
|
| 16 |
- task:
|
| 17 |
+
type: VLA
|
| 18 |
dataset:
|
| 19 |
+
name: libero_object
|
| 20 |
+
type: libero_object
|
| 21 |
metrics:
|
| 22 |
+
- type: accuracy
|
| 23 |
+
value: 98.99
|
| 24 |
---
|
| 25 |
|
| 26 |
<div align="center">
|
| 27 |
<img src="logo.svg" alt="RLinf-logo" width="500"/>
|
| 28 |
</div>
|
| 29 |
|
|
|
|
| 30 |
<div align="center">
|
| 31 |
+
<a href="https://huggingface.co/papers/2510.06710"><img src="https://img.shields.io/badge/Paper-RLinf--VLA-red?logo=arxiv"></a>
|
| 32 |
+
<a href="https://rlinf.readthedocs.io/en/latest/"><img src="https://img.shields.io/badge/Project_Page-Purple?color=8A2BE2&logo=readthedocs"></a>
|
| 33 |
<a href="https://github.com/RLinf/RLinf"><img src="https://img.shields.io/badge/Github-blue"></a>
|
|
|
|
|
|
|
|
|
|
| 34 |
</div>
|
| 35 |
|
| 36 |
<h1 align="center">RLinf: Reinforcement Learning Infrastructure for Agentic AI</h1>
|
| 37 |
|
| 38 |
[RLinf](https://github.com/RLinf/RLinf) is a flexible and scalable open-source infrastructure designed for post-training foundation models (LLMs, VLMs, VLAs) via reinforcement learning. The 'inf' in RLinf stands for Infrastructure, highlighting its role as a robust backbone for next-generation training. It also stands for Infinite, symbolizing the system’s support for open-ended learning, continuous generalization, and limitless possibilities in intelligence development.
|
| 39 |
|
|
|
|
| 40 |
<div align="center">
|
| 41 |
<img src="overview.png" alt="RLinf-overview" width="600"/>
|
| 42 |
</div>
|
| 43 |
|
| 44 |
## Model Description
|
| 45 |
+
This repository hosts the **RLinf-VLA** model, a unified and efficient framework for VLA+RL Training, as presented in the paper [RLinf-VLA: A Unified and Efficient Framework for VLA+RL Training](https://huggingface.co/papers/2510.06710).
|
| 46 |
+
|
| 47 |
The RLinf-openvlaoft-libero series is trained on RLinf/RLinf-OpenVLAOFT-LIBERO-xxx-Base-Lora (including libero90 and libero130) and Haozhan72/Openvla-oft-SFT-libero-xxx-traj1 (including libero10, libero-object, libero-goal and libero-spatial), using the same base models and training datasets as verl. Training with RLinf yields SOTA performance.
|
| 48 |
|
| 49 |
We use a mask to focus on valid action tokens, and compute token-level loss based on the Group Relative Policy Optimization (GRPO) advantage function, in order to enhance the model’s performance on spatial reasoning, object generalization, instruction generalization, and long-horizon tasks.
|
| 50 |
|
|
|
|
| 51 |
## Evaluation and Results
|
| 52 |
We trained and evaluated four models using RLinf:
|
| 53 |
|
|
|
|
| 69 |
- RLinf-openvlaoft-libero10 Model (based on [Haozhan72/Openvla-oft-SFT-libero10-traj1]((https://huggingface.co/Haozhan72/Openvla-oft-SFT-libero10-traj1)))
|
| 70 |
- Recommended sampling settings: `temperature = 1.6`, `top_p = 1.0`
|
| 71 |
|
|
|
|
| 72 |
### Benchmark Results
|
| 73 |
|
| 74 |
+
Sft models for LIBERO-90 and LIBERO-130 are trained by ourself following training reciepe from [OpenVLA-OFT](https://github.com/moojink/openvla-oft/blob/main/vla-scripts/finetune.py). And other sft models are from [SimpleVLA-RL](https://huggingface.co/collections/Haozhan72/simplevla-rl-6833311430cd9df52aeb1f86).
|
| 75 |
- Recommended sampleing setting for evaluation: `libero seed=0`; `episode number=500`; `do_sample=False`
|
| 76 |
|
| 77 |
| Model | Object | Spatial | Goal | Long | 90 | Average |
|
| 78 |
| ------------------ | ------ | ------- | ----- | ----- | ------- |------- |
|
| 79 |
+
| sft models | 25.60 | 56.45 | 45.59 | 9.68 | 78.63 | 43.19 |
|
| 80 |
+
| trained with RLinf | 98.99 | 98.99 | 98.99 | 94.35 | 96.77 | 97.62 |
|
| 81 |
|
| 82 |
Besides, we train one model (we named it libero-130 model) for all tasks in libero.
|
| 83 |
|
| 84 |
| libero-130 model | Object | Spatial | Goal | Long | 90 | 130(all) |
|
| 85 |
| ------------------ | ------ | ------- | ----- | ----- | ------- |------- |
|
| 86 |
+
| sft models | 71.48 | 72.18 | 64.06 | 48.44 | 70.97 | 70.78 |
|
| 87 |
+
| trained with RLinf | 99.80 | 99.40 | 98.79 | 93.95 | 98.32 | 98.09 |
|
| 88 |
|
| 89 |
<div align="center">
|
| 90 |
<img src="tensorboard-success_once.png" alt="RLinf-libero-result" width="600"/>
|
|
|
|
| 98 |
Note: If you intend to evaluate the model directly, make sure to set ``actor.model.is_lora`` to ``false``.
|
| 99 |
|
| 100 |
## License
|
| 101 |
+
This code repository and the model weights are licensed under the MIT License.
|