Research Papers Compare
Head-to-head comparisons of methods and models, with numbers pulled from the source papers.
Reinforcement Learning · GRPO vs PPO
GRPO swaps PPO's learned value model for the mean reward of a sampled group. That frees one policy-sized model and made open reasoning RL affordable, at the cost of per-token credit assignment.
Diffusion Language Models · Diffusion language models vs Autoregressive language models
An 8B masked-diffusion LLM matches an 8B autoregressive one on MMLU, beats it on GSM8K with a sixth of the tokens, and fixes the reversal curse. Decoding speed and serving maturity still favor autoregression.
Fine-Tuning & Adaptation · LoRA vs Full fine-tuning
LoRA matches full fine-tuning within a point on standard tasks at 10,000x fewer trainable parameters, but it gets there through intruder dimensions that carry its forgetting and hurt under continual training.
Sequence Modeling · Mamba vs Transformer
Mamba swaps attention's quadratic compute and growing KV cache for a fixed-size selective state: 5x decode throughput and parity with 2x-larger Transformers at 3B, but exact recall stays with attention.
Language Models · Muon vs AdamW
Muon reaches AdamW's loss with about half the pretraining FLOPs and now trains DeepSeek-V4, Kimi K3 and Qwen3.8-Next; used for SFT of an AdamW-pretrained model it loses points, and in RL it collapses.
Long Context · Sparse attention vs Full attention
Sparse attention buys 7x to 28x less attention compute at 1M tokens for roughly half a point on long-context suites, but the wall-clock gain is far smaller than the FLOPs gain and most of it is invisible at 32K.
World Models · World models vs LLMs
An LLM reasons over rules and goals in language; a world model predicts what a scene does next. The 2026 evidence says they are complementary, and the hard part is knowing when a generated future is credible.