Research Papers
EN ZH

Reinforcement Learning · Vision-Language-Action · Efficient AI · LLM Reasoning

Rethinking Muon Explained: Where Muon Collapses and Pion's High-Pass Fix

Muon's uniform whitening wins pretraining but collapses under RL with verifiable rewards and lags on robot policies; Pion keeps dominant directions, suppresses the noisy tail, and beats both Muon and AdamW.

Michigan State University, Cisco Research, University of Minnesota · Paper published 2026-05-19 · 5 min read

Rethinking Muon Explained: Where Muon Collapses and Pion's High-Pass Fix

Quick answer

Muon orthogonalizes the momentum matrix with Newton-Schulz iterations so all singular values move toward 1, which is a uniform spectral whitening. This May 2026 paper from Michigan State, Cisco Research and the University of Minnesota shows that whitening, the reason Muon beats AdamW in pretraining, is the wrong inductive bias in two post-pretraining regimes. In vision-language-action (VLA) training the action module’s gradients are nearly low-rank, so whitening amplifies noise in the tail directions. In reinforcement learning with verifiable rewards (RLVR) the gradients have low signal-to-noise ratio, and Muon collapses: across eight GRPO and GMPO settings on Qwen3-1.7B and 4B, accuracy stays near zero and often drops below the starting checkpoint. The proposed replacement, Pion, keeps Muon’s cost and replaces whitening with a two-stage Promotion plus Suppression polynomial that anchors dominant singular values at 1 and pushes the tail toward 0. On LIBERO with VLA-Adapter it reaches 100% success on the Object suite after 1,500 steps against 97.0% for Muon and 32.2% for AdamW; on a real Franka robot with a pi-0.5 backbone it lifts average success from 31.1% (AdamW) and 38.9% (Muon) to 85.6%.

Why whitening breaks after pretraining

The paper’s diagnosis rests on two measurements. First, the effective rank of gradients per module during VLA training: vision and language modules keep high-rank gradients, but the action head’s gradients are close to low-rank. Newton-Schulz treats every singular direction the same, so the small, noisy singular values of a low-rank gradient get lifted to the same magnitude as the informative ones. Second, gradient signal-to-noise ratio under GRPO versus supervised fine-tuning on the same model and data: GRPO’s is far lower throughout training, because its supervision is a trajectory-level reward rather than per-token targets, and importance sampling, clipping and group normalization discard or reweight parts of the gradient. Under that regime, Figure 2 shows GRPO with AdamW improving steadily on MATH500 while GRPO with Muon collapses from the initial checkpoint.

A third observation drives the per-head design. A pretrained model’s attention projections have heterogeneous per-head weight norms, and those norms govern attention sharpness and gradient scale. Whole-matrix orthogonalization treats all heads as one block and erases that structure.

What Pion changes

Pion replaces the Newton-Schulz polynomial with two: a Promotion step that drives large singular values to 1 and a Suppression step that drives small ones toward 0, with a tunable threshold that sets where “large” starts. The composition is a spectral high-pass filter. Per-step cost is identical to Muon because it is still a few matrix polynomial iterations on the momentum. A per-head mode reshapes the momentum of the Q, K and V projections so each head’s block is filtered separately, at no extra cost. In the VLA experiments Pion is applied to the action module’s matrices while vision and language matrices keep Muon; in RLVR the per-head mode is used throughout.

Two ablations pin the mechanism. Flipping the filter direction into a low-pass Muon (LPMuon), which contracts large singular values and amplifies small ones, fails to train at all on RLVR, so the direction of spectral shaping is the active ingredient. And default-mode Pion underperforms AdamW on RLVR while per-head Pion wins, whereas per-head Muon still collapses, so per-head structure helps only once the noise is filtered.

Key results

  • VLA-Adapter on LIBERO Object (Figure 5): Pion 95.4% success at 500 steps and 100% by 1,500; Muon 97.0% and AdamW 32.2% at the 1,500-step budget. Muon beats AdamW on all four LIBERO suites; Pion beats Muon on all four.
  • VLANeXt, flow-matching VLA (Table 1): LIBERO average AdamW 79.45, Muon 93.65, Pion 96.35. LIBERO-Plus average 64.57, 72.34, 75.93. Under the Language perturbation: 54.50, 77.53, 86.93. Under the Robot perturbation: 47.04, 57.36, 63.18.
  • Real robot, pi-0.5 under DROID, 20,000 steps, 30 trials per task (Table 3): average success AdamW 31.1%, Muon 38.9%, Pion 85.6%. Cucumber to plate 40.0 / 56.7 / 93.3; cube to plate 33.3 / 33.3 / 83.3; cube to bowl 20.0 / 26.7 / 80.0.
  • RLVR (Figure 6): across GRPO and GMPO, Qwen3-1.7B and 4B, MATH levels 3 to 5 and GSM8K, Muon’s validation accuracy stays near zero; Pion converges faster than AdamW in every setting and shows higher gradient SNR throughout (Figure 7).
  • Reverse ablation: low-pass Muon stays at the initial checkpoint’s accuracy on GSM8K.

Limits and open questions

The VLA evidence is LIBERO simulation plus one real-robot setup with three grasp-and-place tasks and 30 trials each; the RLVR evidence is two small Qwen3 models on math. Nothing here tests Pion in pretraining, where Muon’s whitening is the point, or in large-scale RLVR on frontier models. Pion adds a filter-strength threshold, and the paper’s per-modality optimizer assignment (Pion for action, Muon for vision and language, AdamW elsewhere) is itself a tuned choice. The SNR theory in Appendix C explains Muon’s collapse under GRPO but does not predict how much filtering is enough for a new task. The broader picture of when to prefer each optimizer is in Muon vs AdamW.

FAQ

Why does Muon collapse in RL with verifiable rewards?

GRPO and GMPO gradients have low signal-to-noise ratio because rewards are trajectory-level and clipping, importance sampling and group normalization discard parts of the signal. Muon’s Newton-Schulz step lifts every singular direction to the same magnitude, so noise directions get as much weight as informative ones and the policy collapses; in the paper accuracy stays near zero across eight settings.

How does Pion differ from Muon?

Pion replaces uniform whitening with a high-pass filter: a Promotion polynomial anchors dominant singular values at 1 and a Suppression polynomial pushes small ones toward 0, with a tunable threshold. It also supports a per-head mode for attention projections. Per-step cost is the same as Muon.

What are the Pion vs Muon vs AdamW results on LIBERO and the real robot?

With VLANeXt, LIBERO averages are 96.35 for Pion, 93.65 for Muon and 79.45 for AdamW. On a real Franka robot with a pi-0.5 backbone, average success over three grasp-and-place tasks is 85.6% for Pion against 38.9% for Muon and 31.1% for AdamW after 20,000 steps.

Does per-head Muon fix the RLVR problem?

No. Per-head Muon still collapses, and default-mode Pion underperforms AdamW; only per-head Pion wins. The paper reads this as noise filtering being the primary requirement, with per-head structure helping once filtering is in place.