analiza aiqbench
This research introduces PSP-DiT, a novel architecture that integrates panoptic scene programs directly into the diffusion process to solve complex compositional reasoning tasks like counting and attribute binding. By jointly denoising image and program latents, the model achieves significant performance gains on benchmarks such as GenEval 2 while maintaining high visual fidelity. The work represents a meaningful advancement in aligning generative models with structured logical constraints rather than relying solely on flat text prompts.
Panoptic Scene Program Diffusion Transformer
arXiv:2609.31780v1 Announce Type: cross Abstract: Modern text-to-image models produce high-fidelity images but still struggle with compositional prompts that require instance identity, attribute ownership, counting, spatial ordering, and role-sensitive relations. We introduce Panoptic Scene Program Diffusion Transformer (PSP-DiT), a diffusion-transformer architecture that treats a panoptic scene program as a first-class latent variable rather than an external control signal or post-hoc parse. PSP-DiT jointly denoises image latents and scene-program latents through coupled transformer streams, while panoptic grounding and cycle-consistency objectives tie object instances, attributes, relations, and counts to visual support in the generated image. Under matched training and inference settings, PSP-DiT improves over a strong flat-text baseline across GenEval 2, SANEval-Simple, PSG-Score, and DetailMaster, with the largest gains on counting, attribute binding, role-sensitive relations, and long structured prompts. The method preserves image quality, adds modest inference overhead, and remains robust to imperfect scene programs.
Abstract:Modern text-to-image models produce high-fidelity images but still struggle with compositional prompts that require instance identity, attribute ownership, counting, spatial ordering, and role-sensitive relations. We introduce Panoptic Scene Program Diffusion Transformer (PSP-DiT), a diffusion-transformer architecture that treats a panoptic scene program as a first-class latent variable rather than an external control signal or post-hoc parse. PSP-DiT jointly denoises image latents and scene-program latents through coupled transformer streams, while panoptic grounding and cycle-consistency objectives tie object instances, attributes, relations, and counts to visual support in the generated image. Under matched training and inference settings, PSP-DiT improves over a strong flat-text baseline across GenEval 2, SANEval-Simple, PSG-Score, and DetailMaster, with the largest gains on counting, attribute binding, role-sensitive relations, and long structured prompts. The method preserves image quality, adds modest inference overhead, and remains robust to imperfect scene programs.
Computer Vision and Pattern Recognition (cs.CV); Machine Learning (cs.LG)
arXiv:2609.31780 [cs.CV]
(or arXiv:2609.31780v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2609.31780 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 237 cuvinte
Subiecte conexe
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · 25 sept. 2026
Developing Healthcare Robotics with GPU-Native Medical Physics Simulation
NVIDIA · 28 iul. 2026
Fine-tune video and image models at scale with NVIDIA NeMo Automodel and 🤗 Diffusers
Hugging Face · 17 iul. 2026
Build a Physical AI model factory with NVIDIA Cosmos 3 on SageMaker HyperPod
AWS · 4 sept. 2026