Multimodal Models · Vision Foundation Models · Robotics
OVO-S-Bench vs TVRBench: How to Benchmark Spatial Intelligence in MLLMs
OVO-S-Bench scores 38 MLLMs on streaming spatial questions (1,680 over 348 videos; Gemini-3.1-Pro 59.2 vs human 92.2); TVRBench tests active viewpoint reproduction (12.0% best zero-shot, 51.4% after GRPO on a 9B model).
The question both papers are circling
A multimodal model can caption a photo convincingly, but can it keep track of the space around it? That question has two very different operational meanings, and two 2026 benchmarks answer it with numbers instead of demo clips. OVO-S-Bench gives the passive version: watch an egocentric video stream and answer a spatial question at a specific timestamp, seeing only the footage that came before it. TVRBench gives the active version: place the model inside a 3D environment and let it move its head and body until what it sees matches a target image.
The two papers agree on the headline before any number is read: spatial intelligence in foundation models is not where language intelligence is. They disagree, usefully, on where the gap comes from and on whether training can close it. Reading them together tells you what to measure, what to train, and what not to claim.
OVO-S-Bench: spatial understanding under a streaming protocol
The design choice that defines OVO-S-Bench is prefix-only access. Every one of the 1,680 questions carries a query timestamp and an evidence interval, and at evaluation time the model sees only the stream prefix that precedes the question. An answer that required peeking at future frames is simply unavailable. Questions climb four levels of abstraction: instantaneous egocentric perception, spatiotemporal context tracking, generative spatial reasoning, and allocentric spatial mapping, which means building a map of the space beyond anything currently on screen.
The annotation investment signals how hard the labeling problem is: 12 trained annotators, each also serving as a blind cross-reviewer, spent roughly 804 person-hours on multi-round quality assurance across 348 source videos. The benchmark is fully human-annotated, which is why it can afford checkable answers rather than model-graded ones.
Across 38 proprietary and open-source MLLMs, the best model, Gemini-3.1-Pro, scores 59.2, while human experts judged under the same prefix-access protocol reach 92.2. That 33-point gap is the paper’s headline, and its distribution matters more than its size: allocentric spatial mapping is the dominant bottleneck, so the deficit lives precisely at the level that robotics, AR, and autonomous-driving assistants actually need.
Two secondary findings are the ones builders remember. First, MLLMs fine-tuned for streaming or spatial tasks underperformed their own backbones, so targeted tuning on related data did not buy anything here. Second, chain-of-thought reasoning amplifies spatial errors when the reasoning is not grounded in the stream; longer thinking made answers worse, not better.
TVRBench: spatial understanding as a closed-loop control task
TVRBench replaces the question with a task. Target Viewpoint Reproduction (TVR) gives the agent a target image and asks it to adjust its viewpoint in a 3D indoor environment until its live observation matches that image. Success is binary per episode, and the benchmark varies scene scale and target-view visual richness so that a single number cannot hide behind one room type.
Zero-shot, the task is far from solved: on the evaluation split, the strongest open-source model reaches 7.8% success and the strongest closed-source model reaches 12.0%. The fine-grained analysis names two consistent bottlenecks. Off-the-shelf models struggle to use multi-turn visual history, and performance drops sharply whenever reproduction requires body translation rather than in-place rotation, which exposes a specific failure to map spatial discrepancies onto embodied movement.
The paper’s second half is a controlled experiment on whether training helps. The authors build a unified post-training framework with four arms: expert-trajectory SFT, rationale-supervised CoT-SFT, offline single-turn GRPO, and on-policy multi-turn GRPO from live simulator rollouts. The result ordering is the scientifically interesting part. Visual-action SFT supplies the main gain, lifting a 9B open-source model from single-digit success to 50.8%. Multi-turn GRPO adds targeted multi-room refinement and reaches 51.4% overall. Both CoT supervision and single-turn GRPO degrade closed-loop performance. In other words, the recipe that wins on static benchmarks, more verbal reasoning, actively hurts when the model must act on what it sees. Code, data, and models are released, so the framework doubles as a training recipe, not only a scoreboard.
Where the two probes agree
Three conclusions survive the protocol differences. First, the binding constraint is time-extended state, not single-frame perception: OVO-S-Bench forces models to reason from a prefix without future context, and TVRBench forces them to accumulate visual history across moves. Both find that this is where current models break.
Second, naive scaling of familiar recipes backfires in both settings. OVO-S-Bench reports that streaming and spatial fine-tuning left models below their own backbones and that ungrounded chain-of-thought amplified spatial errors; TVRBench reports that CoT-SFT and single-turn GRPO degraded closed-loop success. Spatial tasks punish reasoning that is not anchored to observations.
Third, the gap is trainable in the active setting but not obviously in the passive one. TVRBench moves a 9B model from 7.8% to 51.4% with visual-action SFT plus on-policy multi-turn GRPO, a sevenfold jump. OVO-S-Bench shows no equivalent shortcut: the best zero-shot model still trails humans by 33 points, and the interventions the authors tested did not close the distance. Capability at scale did not transfer into spatial streaming competence on its own.
Keep the percentages apart: the two benchmarks measure different tasks under different protocols, and only within each panel do the numbers share a ruler. The on-site paper pages carry the same caveat, and the table below states it per row where relevant.
Key numbers
| Dimension | OVO-S-Bench | TVRBench |
|---|---|---|
| Task | Answer a spatial question at a timestamp from the preceding stream prefix only | Move in 3D until the live view matches a target image |
| Scale | 1,680 questions over 348 egocentric videos | Indoor simulation episodes varying scene scale and target richness |
| Annotation | 12 annotators, about 804 person-hours, blind cross-review | Expert trajectories for SFT, live simulator rollouts for GRPO |
| Models tested | 38 proprietary and open-source MLLMs | Open and closed foundation models, plus a 9B open model post-trained by the authors |
| Best zero-shot | Gemini-3.1-Pro at 59.2 | 12.0% closed-source, 7.8% open-source success |
| Human reference | Experts at 92.2 under the same prefix protocol | Not reported in the abstract |
| Effect of training | Streaming or spatial fine-tuning underperformed the backbones | Visual-action SFT lifts the 9B model to 50.8%, multi-turn GRPO to 51.4% |
| Effect of chain-of-thought | Ungrounded CoT amplifies spatial errors | CoT-SFT and single-turn GRPO degrade closed-loop success |
A related line of work worth cross-reading: CrossView revisits how spatial reasoning is trained in MLLMs, which pairs naturally with the OVO-S finding that fine-tuning did not transfer. Its numbers come from a different setup, so treat it as context, not as a third column in the table.
When to use which
Pick OVO-S-Bench numbers when the question is whether an MLLM can serve as the perception layer of a streaming system: a robotics assistant, an AR guide, an autonomous-driving copilot. Its four levels tell you whether the failure is instantaneous perception, context tracking, generative spatial reasoning, or map-level allocentric thinking, and that diagnosis is more actionable than a single capability score.
Pick TVRBench when the question is whether a model can act on space, not just describe it. Its success metric is binary, its post-training framework is released, and the 7.8% to 51.4% trajectory makes it the rarer kind of benchmark: one that doubles as a training recipe with a measured payoff per arm.
Do not use either as a procurement leaderboard. Both papers report results under their own protocols on their own data, the percentages are not comparable across panels, and a strong score on a benchmark with human experts at 92.2 still leaves a deployment question open.
Limits and open questions
The caveats are mostly about transfer. OVO-S-Bench is fully human-annotated and protocol-clean, but a prefix-only streaming setup is still a simplification of a live agent that can ask for more views, and the human ceiling of 92.2 rather than 100 suggests the protocol itself costs something even to experts. TVRBench is indoor simulation only; how success rates move into real embodied hardware is untested, and the abstract does not report a human baseline, so the size of the human-machine gap on TVR is unknown from the paper alone.
The training results deserve the same caution. The 50.8% and 51.4% figures come from the authors’ own post-training framework on their own benchmark, with a single 9B open model as the carrier. That is internally consistent evidence that the gap is trainable, not yet independent evidence that the recipe transfers across model families or environments.
One fair question the pair raises together: if the active gap closes with training while the passive gap resists it, part of the OVO-S deficit may be a data and tuning problem rather than a capability ceiling. Neither paper tests that directly, and it is the most useful follow-up the comparison suggests.
FAQ
Which benchmark should I cite for a spatial intelligence claim, OVO-S-Bench or TVRBench?
Match the claim to the task form. If the claim is about understanding streamed egocentric video, cite OVO-S-Bench and quote the 59.2 vs 92.2 human comparison under the same prefix protocol. If the claim is about acting to reach a desired viewpoint, cite TVRBench and quote the 7.8% and 12.0% zero-shot success rates plus the 50.8% and 51.4% post-training results.
Can TVRBench post-training really lift spatial success from 7.8% to 51.4%?
Within TVRBench, yes, with qualifications. Visual-action SFT on expert trajectories supplies the main gain, taking a 9B open-source model to 50.8% success, and on-policy multi-turn GRPO from live simulator rollouts adds multi-room refinement for 51.4% overall. The two other arms, CoT-SFT and single-turn GRPO, degraded closed-loop performance, so the lift depends on training the model to act on visual feedback rather than to talk about it.
Does chain-of-thought help on OVO-S-Bench?
No, and that is one of the paper’s more consequential findings. Chain-of-thought reasoning amplifies spatial errors when it is not grounded in the stream. Longer verbal reasoning made spatial answers worse, which lines up with TVRBench’s result that CoT supervision degrades closed-loop viewpoint control.
In OVO-S-Bench, what does the 33-point gap between Gemini-3.1-Pro and human experts mean for robotics MLLMs?
It localizes the deficit. The gap is largest at allocentric spatial mapping, the level where a system must represent space beyond the current view, which is exactly what robotics, AR, and driving assistants need. So the number is best read as a statement about which layer to invest in, not as a verdict that MLLMs are unusable for spatial tasks.