Vision-Language-Action · Robotics · Reinforcement Learning
PhysBrain vs RATs vs RLDX-1: Where a Robot's Skills Come From
PhysBrain mines physics QA from human video (Franka grasping 47.1% to 63.3%), RATs banks play-generated code skills (LIBERO-PRO 23.2% to 43.8%), RLDX-1 adds motion, memory and tactile streams (catching 87.5% vs 29.2%).
All three papers in this comparison ask the same question, just at different points in the pipeline: when a new task arrives, where should the robot’s competence already be? Classical robot learning starts the clock at task time, collecting demonstrations or rewards for that instruction and training from there. PhysBrain 1.0, RATs and RLDX-1 each move that learning earlier, and they move it to three very different places. One mines the physical world understanding already recorded in human egocentric video. One lets the robot entertain itself and banks the wins as reusable code. One rebuilds the policy’s senses so that less task-time learning is needed at all. None of the three cites the other two, they appeared within weeks of each other in mid-2026, and they differ most in what they assume you already have: data, compute, or hardware.
PhysBrain 1.0: competence from video humans already recorded
PhysBrain 1.0 attacks the data bottleneck from the cheapest possible angle. Teleoperated robot demonstrations are slow and embodiment-specific, but the internet holds enormous quantities of first-person human video (Ego4D, EgoDex, EPIC) that has no robot action labels at all. The paper’s claim is that the action labels were never the valuable part. Its pipeline treats raw video as source code and compiles it, in three layers, into structured scene metadata, depth-aware spatial relations grounded with Depth Anything v3, and finally physics question-answer pairs. Those QA pairs pretrain a VLM; the VLM is then adapted into a robot policy, with preserving general vision-language ability stated as an explicit adaptation goal rather than an afterthought.
The real-robot evidence is a single Franka platform, 50 trials per setting, against a pi0.5 baseline: single-object vegetable grasping rises from 47.1% to 63.3%, and long-horizon semantic instructions from 31.0% to 45.0%. The 14-point long-horizon lift is the more telling number, because long-horizon instruction following is where most policies collapse first. Read the honest caveats alongside: this is a technical report, the broad VLM and simulation wins (ERQA, PhysBench, LIBERO, RoboCasa) are asserted without per-benchmark tables to audit, and how much VLM capability survives adaptation is not quantified.
RATs: competence the robot writes for itself
RATs (Robotics Agent Teams) spends no training budget at all. Before any instruction arrives, teams of LLM-driven agents attached to a simulator propose their own practice goals, attempt them as Python code-as-policy, verify and diagnose the outcomes, and distill the successes into a reusable skill library. Every agent queries the same frozen gemini-3.1pro-preview; nothing is finetuned. Goal selection is intrinsic, novelty multiplied by a competence-frontier term 4r(1-r) that peaks near 50% success, so the robot practices tasks that are new but still learnable. Skills carry a reliability lifecycle (Experimental, then Verified after at least 3 uses above 50% success, Deprecated after 10 or more uses at 20% or below), and failures are compressed into lessons rather than discarded.
The headline results, on the paper’s own brutal suites: LIBERO-PRO average success rises from 23.2% (a no-play code-as-policy agent) to 43.8%, across 600 trials on 60 held-out tasks, with the vision-language-action models pi0.5 at 12.8% and OpenVLA at 0.0% on the same suite for scale. MolmoSpaces goes 21.0% to 38.0% over 400 trials. The ablation is the part worth remembering: curious play alone lifts the baseline executor from 24.7% to 32.3%, the multi-agent executor alone with no play reaches 36.3%, and the two together reach 44.3%. About 20 points of the headline gain belongs to the harness, not the play. Transfer is measured conservatively, by dropping the frozen library into a different agent on a physical arm: +8.8 points (30.0% to 38.8%). In 391 of 400 evaluation trials at least one learned skill was invoked, 5,169 calls in total, roughly 12.9 per trial. And 43.8% still means most trials fail.
RLDX-1: competence engineered into the senses
RLDX-1 argues that the missing competence was never in the data, it was in the senses. A standard VLA maps one image plus one instruction to an action, which fails on tasks that are fast, contact-rich, or sequential: a still frame cannot show motion, a stateless policy forgets what it saw two seconds ago, and a vision-only model is blind to force. RLDX-1 bolts three modules onto a Qwen3-VL 8B backbone: a motion module computing space-time self-similarity at layer 9 of the vision encoder, a memory queue holding the three most recent compressed scene features, and a Multi-Stream Action Transformer whose physics stream predicts future tactile and torque signals, forcing the model to anticipate contact.
The price is real: pre-training on 1.5M episodes across embodiments (100K steps on 64 H200 GPUs), plus mid-training and post-training stages, and the weights are not released. The payoff concentrates exactly where the thesis predicts. Catching fast-moving objects with the ALLEX humanoid hand: 87.5% versus 29.2% for pi0.5. Object-in-box selection: 91.7% versus roughly 30% for both pi0.5 and GR00T N1.6. On LIBERO-Plus, 86.7% versus 72.6% for GR00T N1.6. On saturated LIBERO, 97.8% versus 96.7%, a rounding error. Inference lands at 43.7 ms per step on an RTX 5090 after a 1.63x static-graph speedup.
Key numbers
| Dimension | PhysBrain 1.0 | RATs | RLDX-1 |
|---|---|---|---|
| What the robot banks before the task | Physical understanding compiled from human egocentric video into physics QA | A library of verified code-as-policy skills from 50 self-proposed play iterations | Motion, memory, and force-prediction modules on a Qwen3-VL 8B backbone |
| Headline number | Franka grasping 63.3% vs pi0.5 47.1% (50 trials, same arm) | LIBERO-PRO 43.8% vs 23.2% no-play baseline (600 trials, 60 tasks) | Catching fast objects 87.5% vs pi0.5 29.2% (ALLEX hand) |
| Training cost | VLM pretraining plus adaptation; data is free human video | Zero gradient training; frozen gemini-3.1pro-preview plus inference-time agent loop | 100K steps on 64 H200 GPUs over 1.5M episodes; weights not released |
| Best baseline gap | +16 points grasping, +14 long-horizon, both vs pi0.5 | +20.6 over no-play agent; pi0.5 manages 12.8 and OpenVLA 0.0 on the same suite | +58.3 catching; +14.1 LIBERO-Plus vs GR00T N1.6; LIBERO nearly tied |
| Transfer evidence | Real Franka only; cross-embodiment unproven in the real world | Skills moved into a different agent on a real arm: +8.8 (30.0% to 38.8%) | Real OpenArm unseen object and unseen task both 54.2%, vs pi0.5 at 37.5% and 45.8% |
| Honest probe | 50-trial samples; forgetting after adaptation not quantified | Executor alone reaches 36.3%, so the harness earns most of the gain | Gaps vanish on saturated benchmarks (LIBERO 97.8 vs 96.7) |
| Same harness as the others? | No: own Franka protocol | No: own sim suites and agent harness | No: own ALLEX and OpenArm setups, authors’ evaluation |
Read the columns as three currencies. PhysBrain reports real-arm success rates, RATs reports simulation success with strict cross-agent real-robot transfer, and RLDX-1 reports its own hardware suite against public baselines. Every cell in the middle rows comes from the paper that owns the column, on that paper’s own setup; there is no row in this table where two of the three systems were measured on the same task.
When to use which
- You have no robot data, no training budget, but access to a simulator and an LLM API: start with the RATs pattern. Play plus a verified code-as-policy library took LIBERO-PRO from 23.2% to 43.8% with a frozen model, and the +8.8 real-arm transfer number comes from dropping the library into a different agent, which is the stricter test.
- You have abundant human video and a VLM pipeline, and your tasks are tabletop manipulation: the PhysBrain pattern is the cheapest data flywheel to try. Do not expect more than the measured effect: 50 trials on one arm is evidence of direction, not of scale.
- Your task is dexterous, fast, or contact-rich, and you control the hardware stack: RLDX-1’s thesis, that motion, memory, and predicted force are the missing senses, is supported most cleanly by the 87.5% versus 29.2% catching gap. Budget for its prerequisites: tactile or torque signals, 64 H200-class training, and no released weights to copy.
- You can stack: the three axes are mostly orthogonal. A RATs-style harness could wrap an RLDX-1 policy or a PhysBrain-initialized VLM, and the physics stream of RLDX-1 predicts exactly the contact signal that PhysBrain’s video pipeline never extracts and that RATs’ verifier only sees as binary success. Nobody has published that combination yet.
Limits and open questions
The three systems never meet on a benchmark. PhysBrain evaluates on a Franka arm, RATs on LIBERO-PRO and MolmoSpaces, RLDX-1 on LIBERO-family suites plus its own ALLEX hand and OpenArm. The single shared reference point is pi0.5, which appears as a baseline in two of the three papers, at 12.8% on RATs’ LIBERO-PRO suite and at 29.2% on RLDX-1’s catching task; those numbers describe different tasks and harnesses and cannot be lined up. Any cross-paper sentence above is a design-level comparison, not a measurement.
Each paper also carries its own self-report caveat. PhysBrain’s headline rests on 50 trials per setting on one embodiment, and its broad benchmark wins are asserted without auditable tables. RLDX-1’s standout results come from proprietary hardware evaluated by the authors, with no released weights, so the pi0.5 and GR00T N1.6 comparisons rest on their harness. RATs’ entire effect rides on one frozen backbone (gemini-3.1pro-preview), so it is unknown whether play helps a weaker model or the harness is just surfacing the backbone’s code ability; verifier error is also unbounded, since a goal verifier that mislabels failure as success banks a bad skill, and the reliability tiers catch that only after several logged uses.
The missing experiment sits at the intersection: take RATs’ play loop, wrap it around RLDX-1’s multi-stream policy, and measure whether a self-generated skill library compounds with engineered senses on contact-rich tasks. That run would answer the question this page cannot.
FAQ
Which of PhysBrain 1.0, RATs and RLDX-1 can I try with no robot data at all?
RATs needs no robot data, only a simulator and an LLM API, because skills are code written and verified during play rather than network weights. PhysBrain also starts from data with no robot actions (human egocentric video), but it assumes you can run a VLM pretraining and adaptation pipeline. RLDX-1 is the opposite end: 1.5M robot episodes across embodiments and 64 H200 GPUs of pretraining.
Why does RLDX-1 beat pi0.5 by 58 points on catching but only 1 point on LIBERO?
Because its three added senses target exactly what static benchmarks do not test. The motion module, memory queue, and tactile-torque prediction stream matter when objects move fast, when contact force decides success, and when the policy must reason over a short sequence. LIBERO is close to saturated, so 97.8% versus 96.7% says almost nothing; the 87.5% versus 29.2% catching result is where the architecture’s thesis is actually tested.
Is RATs’ 43.8% on LIBERO-PRO better than RLDX-1’s 86.7% on LIBERO-Plus?
They cannot be ranked. LIBERO-PRO and LIBERO-Plus are different suites with different task distributions, and no paper in this comparison ran both. The honest reading is relative to baselines inside each suite: RATs beats a no-play agent by 20.6 points while pi0.5 manages 12.8% and OpenVLA 0.0% on LIBERO-PRO, and RLDX-1 beats GR00T N1.6 by 14.1 points on LIBERO-Plus.
Can PhysBrain 1.0 priors, RATs play, and RLDX-1 senses be combined?
In principle yes, and the axes barely overlap. PhysBrain supplies pre-task knowledge from human video, RATs supplies an inference-time harness and skill library that can wrap any policy, and RLDX-1 supplies architecture-level senses. RATs’ library was already shown to transfer into a different agent on a real arm (+8.8 points), which is the closest published evidence that such stacking can work. Nobody has measured the full combination.
Which system has the strongest real-robot evidence?
Each has some, and each is weakest where the others are strongest. PhysBrain’s Franka numbers (63.3% grasping, 45.0% long-horizon, 50 trials each) are direct but small-sample. RATs’ +8.8 point transfer is indirect but methodologically strict, since the skills moved into a different agent. RLDX-1’s ALLEX catching and OpenArm numbers are the most dramatic but come from proprietary hardware in the authors’ own harness.