Research Papers
EN ZH

LLM Reasoning · Multimodal Models · Vision Foundation Models

ReRe: Cross-view Revisiting Lifts MLLM Spatial Reasoning

ReRe lets an MLLM answer a spatial question, then re-watch a synthesized novel-view video and revise. Training-free, it pushes Qwen3-VL-2B from 22.5 to 31.0 on VSI-Bench (+8.5).

Shanghai Jiao Tong University, Tongji University, Jilin University · Paper published 2026-06-10 · 7 min read

ReRe: Cross-view Revisiting Lifts MLLM Spatial Reasoning

Quick answer

ReRe (Reason, then Re-reason) is a training-free, inference-time wrapper that lets a multimodal LLM revise its spatial answer after watching a second video it never saw. The model first forms a hypothesis from the original egocentric clip (Reason Phase), then a Geometry-to-Video pipeline reconstructs the scene in 3D and renders an elevated, oblique fly-through, and the same model re-watches that synthesized view to verify or correct itself (Re-reason Phase). On VSI-Bench it moves Qwen3-VL-2B from 22.5 to 31.0 (+8.5), Qwen3-VL-4B from 30.7 to 36.5 (+5.8), and Qwen2.5-VL-7B from 24.8 to 29.5 (+4.7). No weights change. The gain comes from showing the model a viewpoint the camera never captured, not from letting it think twice.

Why one camera path is the bottleneck

Spatial questions from egocentric video ask things like how far two objects sit apart or what the room layout is, but the camera only walked one path. Whatever it failed to frame, the model never gets to see. A single-turn model is then forced to fill the geometric gaps with semantic priors, guessing the typical distance between a sofa and a TV instead of reading it off the scene. ReRe’s premise is that a spatial conclusion drawn under partial evidence should stay revisable once a better vantage point exists. So the system manufactures that vantage point.

How the Geometry-to-Video pipeline builds the second view

The pipeline runs VGGT (a feed-forward 3D geometry transformer) on the original frames to predict per-pixel 3D points, filters out low-confidence points below 0.5, applies median filtering, then rasterizes a point cloud into a new video along a designed camera path. That path, the “Oblique Sweep,” follows p(t) = c + r·(1-2t)·d with the direction vector at a 45-degree elevation, giving an elevated angle with scene-spanning coverage. Crucially the output is a video, so the MLLM consumes it through the same interface as the original clip. No projector, adapter, or retraining is added.

Which part actually causes the gain

The interesting result is that re-watching alone does almost nothing; the new viewpoint does the work. On Qwen2.5-VL-7B (baseline 24.8), naively concatenating the original video with itself reaches 25.4 and interleaving frames reaches 25.6, both inside noise. Structured re-reason with the synthesized allocentric view reaches 29.5. The view choice is load-bearing too: the proposed Oblique Sweep scores 29.5, a bird’s-eye orbit 27.4, and a mid-level traverse only 25.6. The sharpest evidence is the failure case. When the paper pairs the original video with the allocentric view in the wrong protocol, the score drops to 23.5, below the 24.8 baseline. A second view helps only when it shows complementary geometry and the model is told to treat it as a revision, not extra footage. That is a stronger claim than the authors foreground: the method can hurt if you feed it the wrong angle.

Key results

  • VSI-Bench, open-source backbones: Qwen3-VL-2B 22.5 to 31.0 (+8.5), Qwen3-VL-4B 30.7 to 36.5 (+5.8), Qwen2.5-VL-7B 24.8 to 29.5 (+4.7); InternVL gains are smaller (InternVL2.5-8B 35.5 to 36.7, +1.2). For scale, GPT-4o sits at 34.0 and Gemini-1.5 Pro at 45.4, so a 2B model with ReRe (31.0) closes most of the gap to GPT-4o.
  • STI-Bench static subset: Qwen3-VL-2B 22.2 to 30.2 (+8.0), Qwen3-VL-4B 29.7 to 34.4 (+4.7); proprietary comparators are GPT-4o 31.0, Claude-3.7-Sonnet 37.0, Gemini-2.5-Pro 37.1.
  • Revisiting protocol ablation (Qwen2.5-VL-7B): single-turn 24.8, concat 25.4, interleaved 25.6, ReRe 29.5; naive re-watching is worthless, the structured revise step is what lands the points.
  • View-source ablation: original-only 24.8, re-reason with allocentric view 27.3, wrong original-plus-allocentric pairing 23.5 (below baseline), full ReRe 29.5; a bad second view actively degrades the answer.
  • Trajectory ablation: Oblique Sweep 29.5 beats bird’s-eye orbit 27.4 and mid-level traverse 25.6; the elevated oblique angle, not just any novel view, drives the result.
  • Flip analysis: among samples whose answer changed, 71.6 percent flipped wrong-to-correct and 28.4 percent correct-to-wrong; net effect is 11.21 percent corrected against 4.44 percent broken across all samples.

How to read the headline

The gains are largest on small models (the 2B backbones jump 8 points) and shrink on stronger InternVL backbones (about 1 point). ReRe is a crutch for weak spatial perception, so the better a model already reads geometry, the less a synthesized view adds. The proprietary comparison is also worth reading carefully: a 2B model at 31.0 beats GPT-4o’s 34.0 only after you add VGGT reconstruction plus rendering plus a second inference pass, so “rivals proprietary SOTA” describes the augmented pipeline, not the base model, and still trails Gemini-1.5 Pro at 45.4. This sits in the same line of work as SpatialClaw and SpatialWorld on agentic spatial reasoning, but ReRe keeps the model frozen and moves all the work to view synthesis.

Limits and open questions

The whole method rests on monocular 3D reconstruction, which is ill-posed, and the paper filters points below 0.5 confidence precisely because the geometry is noisy. When VGGT misreads the scene, the rendered view encodes that error and the 28.4 percent correct-to-wrong flip rate is the visible cost. The STI-Bench results are reported on the static subset only, so dynamic-scene spatial reasoning is untested here. Latency is real: about 11 seconds per sample on an A100 with Qwen3-VL-8B (VGGT on 100 frames is roughly 9 of those seconds), versus about 2 seconds for the two MLLM passes alone; a 20-frame variant cuts it to about 4 seconds but only delivers +2.8 instead of +5.2. There is no result with a different 3D backbone, so how much of the ceiling is VGGT’s reconstruction quality versus the revisiting idea itself is unmeasured. And the method is evaluated on benchmark QA, not open-ended embodied tasks where a wrong synthesized view could mislead an agent acting in the world.

FAQ

Does ReRe retrain or modify the MLLM?

No. ReRe is training-free and inference-time. It freezes the base MLLM and adds no architecture, adapter, or projector. The second view is rendered into a normal video so the model reads it through its native video interface. The VSI-Bench gain, for example Qwen3-VL-2B from 22.5 to 31.0, comes entirely from the synthesized novel-view pass and the revise prompt, using the same weights.

In ReRe, does the gain come from the novel view or from re-reasoning twice?

From the novel view. The ablation on Qwen2.5-VL-7B shows naive re-watching barely moves the score (concat 25.4, interleaved 25.6 against a 24.8 baseline), while the synthesized allocentric view reaches 29.5. The trajectory ablation confirms it: an Oblique Sweep scores 29.5 but a mid-level traverse only 25.6. Re-reading the same footage is nearly free of benefit; a complementary viewpoint is what corrects the geometry.

Can a wrong synthesized view hurt ReRe’s accuracy?

Yes. When the paper pairs the original video with the allocentric view under the wrong protocol, the score falls to 23.5, below the 24.8 single-turn baseline. A second view only helps when it carries complementary geometry and the model treats it as a revision step. This is the main caution for builders: a bad reconstruction can make the answer worse than not revisiting at all.

What is ReRe’s accuracy and latency tradeoff on VSI-Bench?

On an A100 with Qwen3-VL-8B, the full pipeline adds about 11 seconds per sample (VGGT on 100 frames is roughly 9 seconds) for a +5.2 point gain, while the two MLLM passes alone take about 2 seconds. A reduced 20-frame variant runs in about 4 seconds but yields +2.8 instead of +5.2, so reconstruction frame count is the main accuracy-versus-latency knob.

How does ReRe compare to proprietary models on VSI-Bench and STI-Bench?

On VSI-Bench, ReRe lifts Qwen3-VL-2B to 31.0, between GPT-4o’s 34.0 and below Gemini-1.5 Pro’s 45.4. On the STI-Bench static subset, ReRe reaches 30.2 to 34.4 across small Qwen3-VL backbones, against GPT-4o 31.0, Claude-3.7-Sonnet 37.0, and Gemini-2.5-Pro 37.1. So ReRe closes much of the gap to mid-tier proprietary models but does not overtake the strongest Gemini results.

One line: ReRe shows egocentric spatial reasoning is limited by the camera path more than the model, and rendering a complementary novel view to re-watch is enough to lift a frozen 2B MLLM from 22.5 to 31.0 on VSI-Bench. Read the original paper on arXiv.