aiqbench analysis
ReRef-3D establishes a rigorous new benchmark for spatial reasoning in 3D environments, revealing significant gaps between current models' ability to satisfy relational constraints versus maintaining physical validity. The study highlights that complex relative positioning tasks remain the primary bottleneck for language-guided scene rearrangement systems using LLaVA-3D and similar architectures. This work advances machine learning by providing a standardized testbed specifically designed to evaluate multi-hop spatial understanding beyond simple object placement.
ReRef-3D: A Benchmark for Spatial Referring Expression-Guided 3D Scene Rearrangement
arXiv:2608.16011v1 Announce Type: new Abstract: We introduce ReRef-3D, a benchmark for language-guided placement in 3D scenes. It contains 33,826 instructions across 998 CLEVR-derived scenes, spanning 16 placement families and direct, one-hop, and two-hop references. Each instruction must be resolved into a valid new placement position. Given that an instruction defines a region of acceptable placements rather than one coordinate, our evaluation inserts a prediction into the scene, recomputes relations, and tests relation satisfaction and physical validity. Each instruction also includes a verified naturalized rewrite. After fine-tuning, LLaVA-3D, 3D-LLM, and PlaceIt3D produce valid placements for 68.3%, 31.6%, and 22.4% of instructions, respectively. Across models, relation satisfaction surpasses physical validity, relations such as nearest and between are the most difficult, and phrasing has minimal effect on performance.
Abstract:We introduce ReRef-3D, a benchmark for language-guided placement in 3D scenes. It contains 33,826 instructions across 998 CLEVR-derived scenes, spanning 16 placement families and direct, one-hop, and two-hop references. Each instruction must be resolved into a valid new placement position. Given that an instruction defines a region of acceptable placements rather than one coordinate, our evaluation inserts a prediction into the scene, recomputes relations, and tests relation satisfaction and physical validity. Each instruction also includes a verified naturalized rewrite. After fine-tuning, LLaVA-3D, 3D-LLM, and PlaceIt3D produce valid placements for 68.3%, 31.6%, and 22.4% of instructions, respectively. Across models, relation satisfaction surpasses physical validity, relations such as nearest and between are the most difficult, and phrasing has minimal effect on performance.
18 pages, 4 figures. Submitted to ACL Rolling Review (ARR)
Computation and Language (cs.CL); Computer Vision and Pattern Recognition (cs.CV)
arXiv:2608.16011 [cs.CL]
(or arXiv:2608.16011v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2608.16011 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 231 words
Related coverage
Apples on the Table? Evaluating Text-Guided 3D Scene Synthesis via Fine-Grained Constraint Verification
arXiv · Sep 2, 2026
From Scene-Centric to Observer-Centric: Modeling Observer-Aware Relations for 3D Scene Graph Generation
arXiv · Jul 21, 2026
PruneGround: Plug-and-play Spatial Pruning for 3D Visual Grounding
arXiv · Sep 9, 2026
Anvil: Scaling Embodied Spatial Intelligence through Knowledge Recirculation
arXiv · Oct 17, 2026