aiqbench analysis
This research introduces Prior-SG, an advanced framework for generating hierarchical 3D scene graphs in open-plan environments by integrating Large Language Model-derived topological priors with visual and geometric data. By casting spatial reasoning as a probabilistic alignment problem optimized via Markov Random Fields, the system achieves robust semantic segmentation without relying on strict physical walls or predefined room structures. The approach demonstrates significant gains in handling arbitrary layouts and offers zero-shot flexibility to restructure space based on high-level tasks, advancing autonomous mobile platform capabilities.
Prior-SG: Task and Prior Driven Region Segmentation for Scene Graphs in Arbitrarily-Structured Environments
arXiv:2608.06170v1 Announce Type: new Abstract: Hierarchical 3D scene graphs are a promising representation for high-level spatial reasoning in autonomous mobile platforms. However, existing extraction frameworks typically rely on purely local visual clustering or strict geometric heuristics, such as wall-separated rooms, which fail in open-plan or arbitrarily-structured environments. We propose Prior-SG, a task- and prior-driven framework that casts scene graph generation fundamentally as a probabilistic alignment problem. As the robot explores, it continuously aggregates an incoming RGB-D sensor stream into a physically grounded Instance Graph utilizing a multi-scale, open-vocabulary feature fusion strategy. The system then infers the high-level functional semantics of this map through a Maximum A Posteriori (MAP) estimate, guided by a Prior Graph-a logical expectation of the environment's structure and task-relevant vocabulary synthesized dynamically by a Large Language Model. By optimizing a Markov Random Field that fuses heterogeneous experts (visual, geometric, and discrete objects) with these topological priors, the system resolves local perceptual ambiguities. We validate this approach across diverse simulated residential datasets and large, open-plan real-world environments. Prior-SG achieves state-of-the-art semantic region segmentation accuracy compared to recent baselines, robustly delineates distant functional boundaries in the absence of physical walls, and uniquely provides zero-shot ontological flexibility, enabling the robot to entirely restructure its spatial partitioning based on a given high-level task.
Abstract:Hierarchical 3D scene graphs are a promising representation for high-level spatial reasoning in autonomous mobile platforms. However, existing extraction frameworks typically rely on purely local visual clustering or strict geometric heuristics, such as wall-separated rooms, which fail in open-plan or arbitrarily-structured environments. We propose Prior-SG, a task- and prior-driven framework that casts scene graph generation fundamentally as a probabilistic alignment problem. As the robot explores, it continuously aggregates an incoming RGB-D sensor stream into a physically grounded Instance Graph utilizing a multi-scale, open-vocabulary feature fusion strategy. The system then infers the high-level functional semantics of this map through a Maximum A Posteriori (MAP) estimate, guided by a Prior Graph-a logical expectation of the environment's structure and task-relevant vocabulary synthesized dynamically by a Large Language Model. By optimizing a Markov Random Field that fuses heterogeneous experts (visual, geometric, and discrete objects) with these topological priors, the system resolves local perceptual ambiguities. We validate this approach across diverse simulated residential datasets and large, open-plan real-world environments. Prior-SG achieves state-of-the-art semantic region segmentation accuracy compared to recent baselines, robustly delineates distant functional boundaries in the absence of physical walls, and uniquely provides zero-shot ontological flexibility, enabling the robot to entirely restructure its spatial partitioning based on a given high-level task.
Robotics (cs.RO); Computer Vision and Pattern Recognition (cs.CV)
arXiv:2608.06170 [cs.RO]
(or arXiv:2608.06170v1 [cs.RO] for this version)
https://doi.org/10.48550/arXiv.2608.06170 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 306 words
Related coverage
The Machines that Make the Machines
NVIDIA · Oct 7, 2026
Adaptive Parallel Reasoning: The Next Paradigm in Efficient Inference Scaling
BAIR Blog · May 8, 2026
T-FunS3D: Task-Driven Hierarchical Open-Vocabulary 3D Functionality Segmentation
arXiv · Jun 5, 2026
FOUND-IT: Foundation-model-first Task-driven 3D Scene Graphs with Granularity on Demand
arXiv · Jun 10, 2026