Playful Agentic Robot Learning: RATs Learn Skills Before the Task
RATs lets a robot play before it is told what to do: agent teams propose their own goals, write code-as-policy, and bank wins as reusable skills. The library lifts LIBERO-PRO success from 23.2% to 43.8%.
Quick answer
RATs (Robotics Agent Teams) is a system that lets a robot practice without a task. Before any instruction arrives, teams of LLM-driven agents propose their own exploratory goals, write Python code-as-policy to attempt them, verify and diagnose what happened, and distill the successful programs into a reusable skill library. At test time the robot retrieves those skills and composes them, with no model finetuning. On LIBERO-PRO the play-built library raises average success from 23.2% (a code-as-policy agent with no play) to 43.8%, a +20.6 point jump; on MolmoSpaces it goes 21.0% to 38.0%, +17.0 points. The skills also move: dropped into a separate code-as-policy agent on a real arm, they add +8.8 points.
The gain belongs to the harness and the learned skill library, not to a smarter base model. Every agent here is querying the same frozen gemini-3.1pro-preview. What changes between 23.2% and 43.8% is the library of verified code the system wrote for itself during play, plus the multi-agent execution loop.
The problem: robots learn the task, then forget how to do anything else
Most robot learning starts the clock when the task arrives. You get an instruction, you collect demonstrations or rewards for that instruction, you train. Skills do not survive the move to the next task. RATs borrows the developmental-psychology idea that play comes first: a child stacks and topples blocks with no external reward, and the motor skills that fall out are reused later for real goals. The bet is that an agent which spends a play budget acquiring general manipulation code will start any downstream task already holding the pieces.
What makes this tractable now is code-as-policy. Skills are not opaque network weights; they are Python functions with docstrings that compose perception, planning, and control APIs. That representation is what lets the system store a behavior, name it, score its reliability, and retrieve it later by description. Read the original paper on arXiv for the full API surface.
How the play loop actually runs
Three agent teams share two persistent stores, a skill library and a failure memory.
A task proposer team picks what to practice using intrinsic motivation, not reward. It balances two scores: novelty, which rewards rare object-skill pairings the robot has barely touched, and a competence frontier 4r(1-r) that peaks when a task’s success rate sits near 50%. The product picks tasks that are new but learnable, the Goldilocks zone. Too easy is boring; too hard is noise.
An execution team runs a write-execute-verify-diagnose loop. A planner sequences actions and tags which library skills to use, a policy writer turns the plan into code, a quality checker screens for unsafe or broken code, and a stack of verifiers grades scene grounding, each substep, and final success. A failure diagnoser categorizes what broke and proposes a fix, and a SubAgent drills a stuck sub-action in isolation.
A memory-management team decides what to keep. Successful runs get distilled into self-contained functions. Failures get compressed into lessons: a missing precondition, a suggested correction. Skills carry a reliability lifecycle that matters more than it looks: a skill starts Experimental, becomes Verified after at least 3 uses at 50%-plus success, and is Deprecated if it logs 10-plus uses at 20% or below. Every five iterations a curator merges duplicate skills.
What the ablation actually credits
The headline number hides a two-part story, and the ablation separates it cleanly. On LIBERO-PRO with 50 play iterations, swapping random play for curious play lifts the plain code-as-policy agent from 24.7% to 32.3%, so curiosity is doing real work, not just burning a budget. Separately, the RATs execution team with no play at all already reaches 36.3%, so the multi-agent harness is a large independent lever. Combine curious play with the RATs executor and you get 44.3%, above either piece alone. The play and the harness are complementary, and crediting the whole 20-point gain to “play” would overstate it.
Key results
- LIBERO-PRO (600 trials, 60 held-out tasks): RATs 43.8% vs CaP-Agent0 23.2%, +20.6 points. For scale, the vision-language-action models π₀.₅ scored 12.8% and OpenVLA 0.0% on the same suite.
- MolmoSpaces (400 trials): RATs 38.0% vs 21.0%. Biggest category gain is Close, 73.0% vs 36.0%; Pick-and-Place doubles, 22.0% vs 11.0%.
- Cross-simulator transfer (RoboSuite): LIBERO-PRO skills plugged into the baseline agent add +8.9 points on average (40.3% to 49.1%), including +24.0 on two-arm lifting, a single-arm-to-dual-arm jump the library was never trained for.
- Real-world transfer: LIBERO-PRO skills add +8.8 points on a physical arm (30.0% to 38.8%); MolmoSpaces skills add +21.7 points on a second task set (3.3% to 25.0%), with one task, Swap Cubes, going from 0.0% to 23.3%.
- Skill reuse is real, not decorative: 391 of 400 evaluation trials invoked at least one learned skill, 5,169 learned-skill calls total, about 12.9 per trial. The most-called helper,
localize_and_verify_object_point_cloud, fired 1,873 times. - Library growth: over a 50-iteration MolmoSpaces run the library grew from 6 to 27 learned helpers while distilled failure lessons grew from 8 to 121.
Read the success rates before you read the percentages
The absolute numbers are easy to misread. RATs wins by a wide margin, and it still fails the majority of LIBERO-PRO tasks: 43.8% means most trials do not complete. The right comparison is the baseline on the same brutal benchmark, where a fresh code-as-policy agent manages 23.2% and a real VLA like OpenVLA scores zero. Inside that frame the result is strong. Read as an absolute “the robot can do this,” it is not.
Two other guardrails. The “teacher” here is intrinsic curiosity, not a privileged oracle feeding answers, so the skills are earned through trial and verification. And the transfer claim is conservative: the real-world gain comes from dropping the frozen library into a different agent, which is a harder test than reusing skills in the same system that grew them. If you want the broader argument for code as the unit of agent skill, the sibling page on code-as-agent harnesses and the work on reusable learned skills cover adjacent ground.
Limits and open questions
The honest ceiling is the success rate: a method that tops out at 43.8% on its own benchmark is a research result, not a deployable manipulator. Everything rides on a single frozen backbone, gemini-3.1pro-preview, so the paper does not tell you whether play helps a weaker model or whether the whole effect is the backbone’s code-writing ability surfaced through a good harness. The play budget is fixed at 50 iterations with no scaling curve, so we do not know if the library keeps paying off or saturates. There is no wall-clock or token-cost accounting for a loop that fans out across a proposer, an executor with multiple verifiers, and a curator, and verifier error is unbounded: a goal verifier that mislabels a failure as success will bank a bad skill, and the reliability tiers only catch that after several logged uses.
FAQ
Does RATs make the base robot model smarter, or just give it a toolbox?
A toolbox. Every agent in RATs queries the same frozen gemini-3.1pro-preview; nothing is finetuned. The 23.2% to 43.8% jump on LIBERO-PRO comes from two things the harness adds around that fixed model: a library of verified code-as-policy skills written during play, and a multi-agent execution loop that plans, verifies, and diagnoses. The ablation shows the harness alone (no play) already reaches 36.3%, so the model is the constant, not the variable.
What is the “Goldilocks” task selection in Playful Agentic Robot Learning?
It is how the proposer decides what to practice without any reward signal. Each candidate task gets a novelty score that favors rare object-skill combinations and a competence-frontier score 4r(1-r) that peaks when the robot’s current success on that task is about 50%. Multiplying them selects tasks that are new but still learnable, avoiding both trivial and impossible goals so play time produces skills that actually generalize.
Does the RATs skill library transfer to a real robot, or only in simulation?
Both, and the real-robot test is the stricter one. Skills grown in LIBERO-PRO, dropped into a separate code-as-policy agent on a physical arm, add +8.8 points (30.0% to 38.8%); a second set of MolmoSpaces skills adds +21.7 points, with one task going from 0.0% to 23.3%. Because the library is plugged into a different agent than the one that grew it, the transfer is not just memorization of the training system.
Why does RATs need both the play phase and the RATs execution team?
Because they fix different bottlenecks and the ablation proves they stack. On LIBERO-PRO, curious play with the baseline executor reaches 32.3%, the RATs executor with no play reaches 36.3%, and the two together reach 44.3%, above either alone. Play supplies reusable skills; the execution loop supplies planning, verification, and failure diagnosis at test time. Drop either and you lose several points.
One line: RATs spends a play budget letting a frozen model write and verify its own robot skills, then reuses that code library to beat a no-play baseline by 20 points and carry +8.8 to a real arm. Read the original paper on arXiv.