strands-isaaclab-ant-131k-policy

rsl_rl PPO policy for the Isaac Lab Ant (Isaac-Ant), trained at 131,072 parallel envs, trained with strands-robots' isaaclab train_policy provider (PR #4227). Rollouts recorded through strands: cagataydev/strands-isaaclab-ant-131k.

Results

131,072 envs × 100 PPO iterations (419 M env steps, 11 min 14 s, Newton/MJWarp, 1× L40S, median 624k env-steps/s): mean reward -0.09 → 62.6 (still rising). Scaling curve: see the dataset card. Recorded 12 rollouts: mean return 19.6.

PPO iteration	mean reward	mean episode length (of 1000)	env-steps/s
0	-0.09	32	521,432
10	1.04	352	602,889
25	9.48	832	611,975
50	36.93	960	628,789
75	54.29	944	635,691
99	62.62	957	636,633

Files

file	what
model_99.pt	rsl_rl checkpoint (actor-critic + optimizer), final iteration
exported/policy.pt	TorchScript actor incl. observation normalizer; input (N, 60) → (N, 8) (joint efforts)
exported/policy.onnx (+ .data)	ONNX export of the same actor
params/agent.yaml, params/env.yaml	exact Isaac Lab agent / env configs of this run
train_curve.json	per-iteration training log parsed by the strands provider
record.json, verify.json	recording + verification reports
playback.gif, playback.mp4, frame.png	rendered rollouts
examples/	recording script used to make the dataset + scale.py (scaling sweep)

How it was made with strands-robots

# Isaac Lab in its OWN venv (its pins clash with strands; strands never imports it)
uv venv --python 3.12 ~/il && uv pip install --python ~/il/bin/python --prerelease=allow \
  --index https://pypi.nvidia.com --index-strategy unsafe-best-match "isaaclab[rsl-rl,isaacsim]==3.0.0rc1"
export ISAACLAB_PYTHON=~/il/bin/python
export OMNI_KIT_ACCEPT_EULA=YES          # you accept the NVIDIA Omniverse / Isaac Sim EULA yourself
pip install "git+https://github.com/cagataycali/robots@feat/isaaclab-trainer"   # strands-robots with PR #4227

As an agent tool call (the train_policy tool is a Strands @tool):

from strands import Agent
from strands_robots.tools.train_policy import train_policy

agent = Agent(tools=[train_policy])
agent("Train Isaac-Ant with the isaaclab provider at the largest scale this GPU fits: 131072 envs, 100 iterations, seed 1.")
# -> train_policy(action="train", provider="isaaclab", steps=100, seed=1, output_dir="runs/c5_ant_131k",
#                 extra={"task": "Isaac-Ant", "num_envs": 131072, "timeout_s": 3600})

As plain Python (exactly what produced this run):

from strands_robots.tools.train_policy import train_policy

job = train_policy(action="train", provider="isaaclab", steps=100, seed=1, output_dir="runs/c5_ant_131k",
                   extra={"task": "Isaac-Ant", "num_envs": 131072, "timeout_s": 3600})   # Newton/MJWarp (task default)
train_policy(action="status", provider="isaaclab", job_id="<job_id from the result>")

Under the hood: python -m isaaclab train --rl_library rsl_rl --task Isaac-Ant --max_iterations 100 --num_envs 131072 --seed 1 in $ISAACLAB_PYTHON. Job id of this run: isaaclab-20260929-095103-667764acb8c8. Docs: docs/learn/training/isaaclab.md · PR: strands-labs/robots#4227.

Use it

Play it in Isaac Lab:

huggingface-cli download cagataydev/strands-isaaclab-ant-131k-policy --local-dir ant_policy
OMNI_KIT_ACCEPT_EULA=YES $ISAACLAB_PYTHON -m isaaclab play --rl_library rsl_rl --task Isaac-Ant \
  --num_envs 16 --checkpoint ant_policy/model_99.pt     # add --video --video_length 300

Or use the exported actor anywhere (TorchScript):

import torch
pi = torch.jit.load("ant_policy/exported/policy.pt").eval()
actions = pi(obs)          # obs: (N, 60) Isaac Lab 'policy' observation group -> (N, 8)

As a strands Policy (what the recording used):

import json, numpy as np, torch
from strands_robots.policies.base import Policy

class RslRlJitPolicy(Policy):
    def __init__(self, jit_path, action_names, device="cpu"):
        self.net = torch.jit.load(jit_path, map_location=device).eval()
        self.device, self.action_names, self.robot_state_keys = device, list(action_names), []
    @property
    def provider_name(self): return "isaaclab_rsl_rl_jit"
    def set_robot_state_keys(self, keys): self.robot_state_keys = list(keys)
    def reset(self, seed=None): self.net.reset()
    async def get_actions(self, observation_dict, instruction, **kw):
        x = torch.as_tensor(np.asarray(observation_dict["policy_obs"], np.float32), device=self.device)
        with torch.inference_mode():
            y = self.net(x.unsqueeze(0))[0].cpu().numpy()
        return [dict(zip(self.action_names, map(float, y)))]

names = json.load(open("ant_policy/record.json"))["action_names"]
policy = RslRlJitPolicy("ant_policy/exported/policy.pt", names)
chunk = policy.get_actions_sync({"policy_obs": obs_60}, "run forward")   # [{"joint_effort.front_left_leg": ..., ...}] (8 joint efforts)

Record your own dataset with strands:

The final checkpoint was rolled out and recorded with examples/record_trained_policy.py (included): rebuild the task env in play mode with an RTX camera → load model_99.pt with rsl_rl and export TorchScript/ONNX → wrap the actor as a strands Policy (RslRlJitPolicy, max |Δa| vs rsl_rl = 6.0e-08) → step with policy.get_actions_sync(...) → write every frame through strands DatasetRecorder (LeRobot v3) → verify with strands verify_dataset.

OMNI_KIT_ACCEPT_EULA=YES PYTHONPATH=/path/to/strands-robots $ISAACLAB_PYTHON examples/record_trained_policy.py \
  --task Isaac-Ant --checkpoint model_99.pt --episodes 12 --frames 300 \
  --cam chase --attach Robot/torso --eye 2.5,-3.0,1.8 \
  --task_str "run forward as fast as possible" --robot_type ant --root out/ds --repo_id cagataydev/strands-isaaclab-ant-131k
$ISAACLAB_PYTHON examples/record_trained_policy.py --verify out/ds --repo_id cagataydev/strands-isaaclab-ant-131k

Provenance

• strands-robots: feat/isaaclab-trainer @ fa66fc68 — strands-labs/robots#4227 (isaaclab train_policy provider; DatasetRecorder; verify_dataset)
• Isaac Lab 3.0.0rc1 · Isaac Sim 6.1.0.0 · Newton / MJWarp (task default physics) · rsl-rl-lib 5.4.1 (PPO) · lerobot 0.6.1
• GPU: 1× NVIDIA L40S (46 GB), shared with the Allegro reorientation training (~8.4 GB)
• Seeds: training seed 1 (params/agent.yaml, params/env.yaml); recording seed 7
• Training job: isaaclab-20260929-095103-667764acb8c8, 2026-09-29

Limitations

• Simulation only; the Ant is a classic simulated benchmark robot.
• Release candidates: Isaac Lab 3.0.0rc1 / Isaac Sim 6.1.0.0. Trained on the task's default Newton/MJWarp physics; the checkpoint does not remember the preset (IL-X-011) — replay on the same physics.
• This is a throughput demo, not a converged policy: 100 iterations at 131,072 envs (419 M env steps in ~11 min). The reward was still rising at the end; the Ant runs forward but is not tuned. 262,144 envs ran out of CUDA memory on the shared 46 GB L40S, and the strands status message did not surface the OOM (finding IL-X-007).
• Scaling numbers were measured with another training (Allegro, ~8.4 GB) on the same GPU; a solo GPU would be somewhat faster.
• Recording: 12 parallel envs, 300 frames (5 s @ 60 fps) each from one fixed camera; all 12 episodes ran the full 300 frames (5 s @ 60 fps). observation.state = 8 joint pos + 60-D policy observation + root pos/quat (75-D), not a standard LeRobot robot state.
• create_policy("rl") in strands cannot load rsl_rl checkpoints yet (IL-X-006); use the exported TorchScript + the wrapper above.

License

Card choice: license: other — our generated data / weights under CC-BY-4.0, plus NVIDIA notices. Why:

• Trajectories, rendered camera video, playback clips and trained weights are user-generated content made with NVIDIA Isaac Sim / Isaac Lab; the NVIDIA Omniverse License Agreement (governs Isaac Sim 6.1, isaacsim/LICENSE.txt) §2.1 allows distributing "user generated content that you develop using Omniverse, such as video, audio, stills, models, 3D assets and screen captures". We release it under CC-BY-4.0.
• No NVIDIA Content is redistributed: the Ant USD (Robots/IsaacSim/Ant/ant_instanceable.usd) comes from the Isaac Lab / Isaac Sim asset packs and are not in this repo (params/env.yaml only references their paths). No endorsement by NVIDIA is implied.
• params/*.yaml are Isaac Lab configurations (BSD-3-Clause); examples/* are Apache-2.0 like strands-robots. Running Isaac Sim requires your own acceptance of the NVIDIA Isaac Sim / Omniverse EULA. Full text: LICENSE.md.