Research Papers
EN ZH

Text-to-Image · Diffusion Models · Efficient AI

Moebius: a 0.22B Inpainting Model That Matches FLUX.1-Fill

Moebius is a 0.22B diffusion inpainter that rivals the 11.9B FLUX.1-Fill-Dev. On CelebA-HQ it scores FID 5.39 vs FLUX 10.13, using under 2% of the parameters and roughly 15x less total inference time.

Huazhong University of Science and Technology, vivo · Paper published 2026-06-17 · 7 min read

Moebius: a 0.22B Inpainting Model That Matches FLUX.1-Fill

Quick answer

Moebius is a 0.22B-parameter diffusion inpainting model that matches or beats the 11.9B FLUX.1-Fill-Dev on standard benchmarks while running far cheaper. On the CelebA-HQ (512) portrait benchmark it reaches FID 5.39 against FLUX’s 10.13, and on FFHQ (256) it scores 8.15 against 11.19. On natural scenes (Places2) the picture is more even: FLUX wins the harder Test split (8.02 vs 9.48 FID) while Moebius edges it on the Small-hole split (0.92 vs 0.94). It does all of this with under 2% of FLUX’s parameters (0.22B vs 11.9B), 26.01 ms/step latency against FLUX’s 161.01, and 20 sampling steps against FLUX’s 50, which compounds to roughly a 15x cut in total inference time. The win comes from two pieces working together: a redesigned backbone block and a latent-space distillation recipe. Neither alone is enough.

The problem: compress the backbone and the representation collapses

Industrial inpainters in the 10B range produce excellent fills but cost too much to deploy at scale. The obvious fix is to train a small task-specific model instead of a giant generalist. The trap is that aggressive parameter cuts choke the model’s ability to represent context, and quality falls off a cliff. The authors call this the representation bottleneck, and their Table 2 makes it concrete: a compact backbone trained without distillation lands at FID 33.21, while the same architecture with their distillation recipe drops to 24.73 on the same Places2 checkpoint. So the small architecture is necessary but not sufficient. You have to also teach it well.

How the Local-lambda Mix Interaction block works

The backbone is a latent diffusion U-Net rebuilt around a single repeating unit, the Local-lambda Mix Interaction block. It has three parts. Local-lambda summarizes nearby spatial context into a fixed-size linear matrix (a lambda), so the cost of mixing context does not grow with the number of tokens the way attention does. Interactive-lambda does the same trick for the global semantic prior, fusing the Latent Categories Guidance signal into another compact lambda. Mix-FFN is the feed-forward stage. The point of folding context and global prior into fixed-size matrices is that you keep the latent interactions a heavy attention block would capture, but pay a flat cost instead of a quadratic one, which is where most of the parameter and FLOP savings come from.

Swapping standard attention for the lambda modules is what shrinks the model. The standard global-attention backbone (GLA-CA-FFN) uses 526M parameters; the full Local-lambda/Interactive-lambda design matches it at 485M (FID 33.21 vs 32.75 without distillation, essentially even), and adding depthwise convolutions plus Mix-FFN compresses it to the shipped 226M and 154 GFLOPs. With distillation the 485M variant reaches FID 24.73 and the 226M shipped backbone lands at 26.43, so the final 53% parameter cut costs about 1.7 FID, a deliberate trade.

Where the quality actually comes from

The lift that makes 0.22B competitive is the distillation, not the block design. PixelHacker (0.86B) is the teacher, and Moebius is trained to match its internal representations entirely in latent space, skipping the expensive pixel decode that pixel-space distillation needs. The recipe stacks four losses, and the ablation in Table 5 shows each one earning its place. With only the coarse KD loss, FID sits at a useless 74.20. Adding a fine-grained KD term drops it to 36.17. A task loss takes it to 32.59. A perceptual term lands the final 26.43. The headline detail is the gap between the architecture alone and the architecture plus distillation: 33.21 without, 24.73 with, on the matched setup. The block makes the model small; the distillation is what fills the resulting hole.

This is the same efficiency lever as classic model compression like DistilBERT, applied to a diffusion backbone instead of a transformer encoder, with the twist that alignment happens across multiple spatial granularities in latent space.

A misread to avoid

The title says “10B-level performance,” and the clean reading is that Moebius beats FLUX everywhere. It does not. On the natural-scene Places2 Test split, FLUX.1-Fill-Dev still leads on FID (8.02 vs 9.48) and on the 256 split (10.44 vs 14.13). Moebius’s clear wins are on portraits (CelebA-HQ and FFHQ) and on small-hole and large-hole natural splits, plus it tracks its own teacher closely. The honest claim is “rivals or surpasses depending on the split, at a fraction of the cost,” which the paper’s user study supports: human raters preferred Moebius over the 10B generalists, especially on faces, while it merely matched its teacher. Read it as a specialist that closes the gap, not a universal winner.

Worth noting on the speedup too: the roughly 15x total-time advantage is not purely architectural. Part comes from lower per-step latency (26.01 vs 161.01 ms) and part from running 20 sampling steps instead of FLUX’s default 50. Both are legitimate, but if you held step count equal the multiplier would shrink.

Key results

  • CelebA-HQ (512) portraits, FID: Moebius 5.39 vs FLUX.1-Fill-Dev 10.13 and SD3.5 Large-Inpaint 11.80, so the 0.22B model nearly halves FLUX’s FID on faces. Beats every academic and industrial baseline in the table except its own teacher PixelHacker (4.75).
  • FFHQ (256) portraits, FID: Moebius 8.15 vs FLUX 11.19 and SD3.5 109.42, a clear portrait win over both industrial generalists.
  • Places2 Test (natural), FID: FLUX 8.02 leads Moebius 9.48, the split where the 10B generalist still wins, so the parity claim is split-dependent.
  • Cost: 0.22B params vs FLUX’s 11.9B (under 2%), 154 GFLOPs, 26.01 ms/step vs 161.01, 20 steps vs 50, compounding to about 15x less total inference time.
  • Distillation ablation (Places2, 18K-step checkpoint): coarse-KD only FID 74.20, +fine-KD 36.17, +task loss 32.59, +perceptual 26.43, so all four losses contribute and the coarse term alone is unusable.
  • Architecture-vs-distillation (matched setup): same compact backbone FID 33.21 without distillation vs 24.73 with, isolating the lift as coming from the teacher signal rather than the block.
  • Out-of-distribution generalization: on LVIS (natural) and DeepFakeFace (portrait) zero-shot, Moebius posts FID 17.81 / 15.32 against SD3.5’s 114.21 / 81.91 and stays close to FLUX (14.52 / 12.24), so the specialist does not overfit its training distribution.

Limits and open questions

Moebius is a specialist, not a generalist. It is trained for mask-based inpainting on natural and portrait scenes, not text-prompted editing, so the comparison to FLUX is fair only on the inpainting task both can do. Its ceiling is its teacher: the failure-case figure shows it can lose fine detail or produce less plausible texture in extremely tiny background regions where context is thin, exactly where a 0.22B capacity budget bites. The natural-scene Places2 Test result confirms it has not fully closed the gap on hard general scenes. And the 15x speed number mixes a per-step latency win with a lower step count, so a like-for-like step comparison would report a smaller (still large) multiplier. There is no public training-cost figure for the distillation stage, and results are confined to two domains at fixed resolutions.

FAQ

Does Moebius actually beat FLUX.1-Fill-Dev everywhere?

No. Moebius wins decisively on portraits (CelebA-HQ FID 5.39 vs 10.13, FFHQ 8.15 vs 11.19) and on small and large hole natural splits, but FLUX.1-Fill-Dev still leads on the harder Places2 Test split (8.02 vs 9.48 FID) and the 256 split. The accurate summary is that Moebius rivals or surpasses FLUX depending on the benchmark, at under 2% of the parameters, rather than beating it across the board.

Where does the Moebius quality come from, the new block or the distillation?

Mostly the distillation. The Local-lambda Mix Interaction block is what makes the model small and cheap, but in the matched ablation the same compact backbone scores FID 33.21 without distillation and 24.73 with it. The four-loss latent-space distillation from the PixelHacker teacher is what fills the representation bottleneck that compression creates.

Is the Moebius 15x speedup from a faster architecture or fewer sampling steps?

Both. Moebius runs at 26.01 ms/step against FLUX’s 161.01 (a per-step architectural win) and uses 20 sampling steps against FLUX’s default 50. Multiplying step latency by step count gives the roughly 15x total-time advantage. Hold the step count equal and the multiplier drops, though Moebius is still far cheaper per step.

What is the Local-lambda Mix Interaction block in Moebius?

It is the repeating backbone unit that replaces standard attention. Local-lambda compresses local spatial context into a fixed-size linear matrix, Interactive-lambda compresses the global semantic prior the same way, and Mix-FFN handles the feed-forward stage. Folding context into fixed-size matrices keeps rich latent interactions while paying a flat rather than quadratic cost, which is the main source of the parameter and FLOP savings.

One line: Moebius pairs a lambda-based compact backbone with four-loss latent-space distillation from a 0.86B teacher to reach FLUX-level inpainting at 0.22B and roughly 15x less inference time, winning on portraits and trailing slightly on hard natural scenes. For more lightweight-model work see the efficient AI topic and the inpainting-adjacent ControlNet. Read the original paper on arXiv.