Vision-Language-Action · Robotics · Reinforcement Learning
ACE-Ego-0 vs Humanoid-GPT vs DragMesh-2 on the Robot Data Bottleneck
ACE-Ego-0 imports human egocentric video into VLA pretraining, Humanoid-GPT makes 2 billion motion frames and distills 384 RL experts into one model, and DragMesh-2 strips actuation shortcuts.
The shared bottleneck
All three papers start from the same observation, stated as plainly as possible by the position paper Robots Need More than VLA and World Models: robotics is compute-rich and supervision-poor. Teleoperated demonstrations are slow, expensive, and embodiment-specific, so the default recipe of “collect more demos, train a bigger VLA” cannot 10x the way language modeling did. The position paper names four missing interfaces (data, embodiment, world-model, reward) whose job is to convert the world’s abundant action-less behaviour footage into supervision a policy can train on.
What the three systems in this comparison do is take that diagnosis seriously and pick a different interface to build:
- ACE-Ego-0 attacks the data interface. It imports 1,479 hours of human egocentric video, converts it into robot-shaped pseudo-actions, and pours it into the same pretraining pool as real robot demonstrations.
- Humanoid-GPT attacks the data interface from the supply side. It manufactures 2 billion frames of retargeted motion, trains roughly 384 specialized RL experts on it, and distills them into a single causal Transformer.
- DragMesh-2 attacks the reward and honesty side. It removes the actuation shortcut from articulated manipulation (the object joint gets no action channel, so the hand must drag the door open through sustained contact), which turns the benchmark itself into a probe of whether the training signal survives honest physics.
None of the three cites the other two. They appeared within weeks of each other in mid-2026, and they answer the same question with incompatible strategies, which is exactly why they belong on one page.
What ACE-Ego-0 buys: 1,479 hours of video for +4.5 points
ACE-Ego-0’s pretraining pool is 6,013 hours: 4,534 hours of robot demonstrations plus 1,479 hours of human egocentric video that has no recorded actions at all. A five-stage pipeline (curate clips, filter to manipulation segments, reconstruct 3D hands, parameterize MANO poses into camera-space pseudo-actions, drop corrupted clips) turns the human footage into a 22-D bimanual action stream aligned with the robot data by morphology tokens and by 2-second physical-duration chunks instead of step counts.
The load-bearing detail is that the model is told how much to trust each channel. Position channels get weight 1.0; rotation and gripper channels get 0.001. Human video supervises where the hand goes and is effectively muted on orientation and grasp, because reconstructed hand poses are good for position and weak for everything else. The ablation rewards this bluntness: removing the reliability-aware human loss drops RoboCasa from 72.8% to 69.2%, a 3.6-point fall, larger than removing morphology tokens (1.9) or time-aligned chunking (1.1).
The headline numbers: 78.3% average success on six real bimanual ARX tasks, ahead of pi-0.5 at 71.7% and GR00T-N1.7 at 35.6%; 72.8% on RoboCasa GR1 TableTop against DIAL’s 70.2%; 91.12% / 90.62% on the easy / hard splits of RoboTwin 2.0. The cleanest evidence that the human video is doing real work is the data ablation: robot-only training reaches 68.3% on RoboCasa, adding human video lifts it to 72.8%, a +4.5 point gain on a fixed architecture. In the data-scarce regime, 34 robot demos (covering 0.062 m² of workspace) plus 419 human episodes (covering 0.296 m², 4.8x broader) lift Sweep Cubes success from 10% to 40%. The human footage fills workspace regions the teleoperation never reached.
What Humanoid-GPT buys: 200x data and one generalist tracker
Humanoid-GPT’s bet is that motion tracking failed to scale for the same reason early NLP did: too little data and too small a model. It unifies AMASS, LAFAN1, MotionMillion, PHUMA, and in-house recordings into a 2-billion-frame retargeted corpus, described as 200x larger than prior tracking datasets. Because a single policy trained broadly used to go soft on agile motions, the pipeline first trains roughly 384 PPO experts, each specialized on a motion cluster discovered by a Harmonic Motion Embedding, and then distills all of them via DAgger into one GPT-style causal Transformer of 22.1M to 80.4M parameters that outputs per-joint PD targets.
The results read like a scaling-law paper: 92.58% tracking success in simulation with 40.99mm mean per-keypoint error, clean gains with both model size and data size, and under 1.5ms per step on a single RTX 4090. It runs on real hardware too: four dance motions on a physical humanoid with MPJPE between 0.0856 and 0.1180 rad, visibly worse than the sim figure of 0.0735 rad, which is the honest sim-to-real confession.
Two caveats matter for anyone borrowing the recipe. First, the gains from 200M to 2B frames are already marginal, which the authors read as the onset of a data-limited regime at fixed model capacity; 2B frames did not buy a second scaling curve. Second, the system is a tracker, not a planner: it follows reference trajectories, and where those come from is out of scope.
What DragMesh-2 buys: a metric that catches nominal-success collapse
DragMesh-2 asks a question the other two never touch: what does your training signal look like when you stop helping the policy? Most manipulation benchmarks quietly give the object’s joint its own controller, so the hand rides along a scripted door. DragMesh-2 removes the crutch. A 51-DoF SMPL-X hand controls only the hand; the dishwasher door or drawer moves only while sustained contact drags it, and the policy sees no RGB, depth, force, or tactile input, only kinematics.
PICA (Physically Informed Contact-Aware training) reconstructs contact from motion history: a gated linear attention encoder over recent interaction blocks plus an auxiliary head predicting four contact-response signals (object-joint response, palm-handle distance, detachment risk, tracking stress), with explicit contact-maintenance and detachment-penalty reward terms and damping randomization over [1.0, 2.0].
The headline is a split number, and the split is the point: 0.89 success at nominal damping, 0.56 at 4x damping, on seven GAPartNet objects. State-only PPO manages 0.58 / 0.27, and a Transformer-PPO variant collapses to 0.35 / 0.09. Neither ablation half reproduces the whole: the GLA encoder alone gets 0.65 / 0.56 / 0.36, PICA signals alone get 0.75 / 0.71 / 0.43, the full system 0.89 / 0.80 / 0.56.
The paper’s most transferable result is a warning about metrics. Extending training from 150 to 500 epochs raised nominal success from 0.90 to a perfect 1.00 while 4x-damping success fell from 0.55 to 0.10, with action saturation climbing to 0.99. Selecting the checkpoint by the nominal number would have picked the least contact-capable policy. A scripted trajectory-tracking baseline beats PICA in-distribution (1.00 at 1x) and only ties it under load (0.71 at 4x); a parallel-jaw primitive flatlines at 0.14 across all damping settings. The dexterous formulation earns its keep precisely where the nominal number says nothing.
Key numbers
| Dimension | ACE-Ego-0 | Humanoid-GPT | DragMesh-2 |
|---|---|---|---|
| Supervision source | 4,534 h robot demos + 1,479 h human egocentric video as pseudo-actions | 2 billion retargeted frames distilled from ~384 PPO experts | No new data; contact inferred from kinematics on 7 GAPartNet objects |
| Headline result | 78.3% avg on 6 real ARX bimanual tasks (pi-0.5: 71.7%, GR00T-N1.7: 35.6%) | 92.58% sim tracking success, 40.99mm MPKPE | 0.89 success at 1x damping, 0.56 at 4x (state-only PPO: 0.58 / 0.27) |
| Honesty probe | Human video worth +4.5 on RoboCasa; reliability loss removal costs 3.6 | 200M to 2B frame gains marginal; real MPJPE 0.086 to 0.118 rad vs sim 0.0735 | 150 to 500 epochs: 1x success 0.90 to 1.00 while 4x falls 0.55 to 0.10 |
| Real hardware | Yes: ARX bimanual, 30 trials per task | Yes: 4 dance motions, under 1.5ms per step on an RTX 4090 | No: simulation only, privileged state |
| Ablation signal | Reliability weighting is the single largest component (3.6 of 4.5 human-video gain) | Scaling holds with both model size (22.1M to 80.4M) and data size | GLA alone 0.65 / 0.56 / 0.36; PICA signals alone 0.75 / 0.71 / 0.43 |
| Where it stops | Tabletop only; no dexterous hand, force, or torque data in the pool | A tracker, not a planner; ~80M parameters is the ceiling tested | 7 objects, sim only, no vision, force, or tactile input |
Read the columns as three different currencies. ACE-Ego-0’s points come from real-robot success rates on broad task suites, Humanoid-GPT’s from sim tracking precision on an enormous motion corpus, and DragMesh-2’s from success under shifted dynamics on a handful of contact-heavy objects. They are not exchangeable, and no row in this table is a head-to-head result.
When to use which
- Your bottleneck is robot demonstration hours and your tasks are tabletop manipulation: start with the ACE-Ego-0 pattern. Human egocentric video is nearly free compared to teleoperation, the +4.5 RoboCasa gain is measured on a fixed architecture, and the reliability-weighting trick (trust position, mute rotation and gripper) is cheap to graft onto an existing VLA pipeline.
- You control a simulator and need whole-body or highly dynamic skills: the Humanoid-GPT recipe of “many RL experts, then distill into one causal Transformer” is the one with demonstrated scaling behavior and real-time inference. Budget for its ceiling: the authors’ own data shows diminishing returns past 200M frames at these model sizes.
- Your task is contact-dense (doors, drawers, latches, insertion under load): read DragMesh-2 before you trust any nominal success number. Adopt the two diagnostics that caught the collapse (detachment proxy and action-saturation tracking) even if you keep your current architecture.
- You are deciding where the field’s effort should go: the position paper’s four interfaces are the map. ACE-Ego-0 and Humanoid-GPT both build data interfaces; a contact-capable reward interface of the kind DragMesh-2 implicitly demands is still missing from all three.
Limits and open questions
The three papers never meet on a single task, so every cross-paper sentence above is a design-level comparison, not a measurement. ACE-Ego-0 evaluates on tabletop manipulation, Humanoid-GPT on motion tracking, DragMesh-2 on seven articulated objects in simulation; a policy that wins all three benchmarks does not exist yet, and none of these papers trains one.
The missing experiment sits exactly at the intersection. Take Humanoid-GPT’s expert-distillation pipeline and train it on DragMesh-2’s damping-randomized tasks, then select checkpoints by 4x-damping success instead of nominal success. That run would answer the question this comparison cannot: does manufactured data buy contact capability, or only breadth? Pointing the other way, ACE-Ego-0’s own ablation says its human-video pipeline mutes rotation and gripper channels, and its pretraining pool contains no force or torque data, which is precisely the signal DragMesh-2 shows is missing when damping shifts. The data interface and the contact interface are complementary on paper and untested in combination.
Smaller caveats: ACE-Ego-0’s 10% to 40% data-scarce result is one task with 10 trials, so 1 success in 10 versus 4 in 10; Humanoid-GPT’s real-world evidence is four dances; DragMesh-2’s is zero real-world quantitative evidence. All three gaps are disclosed by the authors, and all three matter when you budget a replication.
FAQ
How much training data does each method need?
ACE-Ego-0 pretrains on 6,013 hours (4,534 robot + 1,479 human video). Humanoid-GPT trains on 2 billion retargeted frames, which its authors call 200x prior tracking datasets, and reports that going from 200M to 2B frames adds little. DragMesh-2 adds no data at all: seven GAPartNet objects with damping randomization over [1.0, 2.0].
Does ACE-Ego-0 really beat GR00T-N1.7 by 42.7 points?
On six ARX bimanual tasks, yes: 78.3% against 35.6%, with 30 trials per task. Read it with the caveat that GR00T is a generalist not tuned for this exact hardware setup, and the tasks are a narrow slice. The fairer comparison is +6.6 over pi-0.5, a strong same-class baseline, plus the controlled +4.5 from human video on RoboCasa.
Does Humanoid-GPT run on real hardware?
Yes, but the evidence is narrow: four dance motions on a physical humanoid, with MPJPE between 0.0856 and 0.1180 rad where simulation reports 0.0735 rad. Inference runs under 1.5ms per step on a single RTX 4090. Everything else in the paper is simulation.
Why does DragMesh-2 call 0.89 a misleading headline?
Because 0.89 is the nominal-damping number. At 4x damping the same policy scores 0.56, and the checkpoint that reaches 1.00 at nominal damping (trained for 500 instead of 150 epochs) drops to 0.10 at 4x with action saturation at 0.99. Selecting on the nominal metric picks the worst policy under shifted dynamics.
Which method should a robot team try first?
Match the method to the bottleneck. Scarce demonstration data on tabletop tasks points to ACE-Ego-0’s import strategy. A simulator and a need for dynamic whole-body skills points to Humanoid-GPT’s manufacture-and-distill recipe. Contact-heavy manipulation points to DragMesh-2’s diagnostics, whatever architecture you keep.