aiqbench analysis
This research introduces SG-Layout, a framework that enhances large language models by integrating structured spatial knowledge via graph encoders to improve geometric consistency in layout generation tasks. The approach utilizes a two-stage training paradigm involving feature alignment and instruction tuning with LoRA adapters, demonstrating superior performance on complex scenes without freezing the model backbone. By bridging natural language understanding with explicit relational geometry, this work advances controllable image synthesis for applications ranging from indoor scene creation to robotic manipulation.
SG-Layout: Structured Scene Graph-Guided Layout Generation with LLMs
arXiv:2608.01106v1 Announce Type: cross Abstract: Understanding and generating spatially coherent layouts from natural language remains a fundamental yet challenging task for large language models (LLMs). Existing LLMs often struggle to capture explicit geometric relationships and structural dependencies between objects. To address this issue, we propose SG-Layout, a graph-guided layout generation framework that explicitly incorporates structured spatial knowledge into LLMs. SG-Layout follows a two-stage training paradigm: (1) a graph-language feature alignment stage, where a relational graph encoder and a projector are trained to map scene-graph embeddings into the LLM's linguistic space; and (2) an instruction tuning stage, where LoRA-based adapters enable efficient fine-tuning for instruction-driven layout generation while keeping the backbone frozen. We evaluate SG-Layout on image layout generation, indoor scene synthesis and robotic object rearrangement tasks. Experimental results show that SG-Layout improves spatial reasoning accuracy and geometric consistency over the compact open-source backbone, with particularly clear advantages in relation-dense and compositionally complex scenes. These results highlight the effectiveness of graph-structured feature alignment for enhancing controllable layout generation.
Abstract:Understanding and generating spatially coherent layouts from natural language remains a fundamental yet challenging task for large language models (LLMs). Existing LLMs often struggle to capture explicit geometric relationships and structural dependencies between objects. To address this issue, we propose SG-Layout, a graph-guided layout generation framework that explicitly incorporates structured spatial knowledge into LLMs. SG-Layout follows a two-stage training paradigm: (1) a graph-language feature alignment stage, where a relational graph encoder and a projector are trained to map scene-graph embeddings into the LLM's linguistic space; and (2) an instruction tuning stage, where LoRA-based adapters enable efficient fine-tuning for instruction-driven layout generation while keeping the backbone frozen. We evaluate SG-Layout on image layout generation, indoor scene synthesis and robotic object rearrangement tasks. Experimental results show that SG-Layout improves spatial reasoning accuracy and geometric consistency over the compact open-source backbone, with particularly clear advantages in relation-dense and compositionally complex scenes. These results highlight the effectiveness of graph-structured feature alignment for enhancing controllable layout generation.
16 pages, 5 figures. Accepted at WAICA 2026
Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
arXiv:2608.01106 [cs.CV]
(or arXiv:2608.01106v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2608.01106 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 270 words
Related coverage
100 things we announced at I/O 2026
Google · May 20, 2026
10 Solved Generative AI Projects to Boost your Profile
Analytics Vidhya · Sep 25, 2026
Presentation: Engineering AI for Creativity and Curiosity on Mobile
InfoQ · Jul 21, 2026
3D Scene Graph Prediction: Generating Hierarchical Models from Partially Observed Environments
arXiv · Jul 21, 2026