analiza aiqbench
This research introduces a post-hoc trust scoring method that evaluates conditional generation samples using only the training distribution, effectively resolving the evaluation circularity in extrapolative regimes. By combining global realism with attribute-wise faithfulness, the approach enables reliable filtering and ranking of generated data without requiring an unavailable reference target distribution. The technique demonstrates practical utility in scientific domains like biological imaging, where it improves sample quality and downstream predictive performance on off-the-shelf models.
Assessing Sample Quality in Conditional Generation under Compositional Shift
arXiv:2606.09601v2 Announce Type: replace Abstract: Conditional generators provide a natural tool for controllable generation, including settings where the desired condition is a new composition of observed attributes or experimental factors. In many applications, especially in scientific domains, such models are attractive to explore conditions for which real samples are rare, expensive, or not yet observed. However, this creates a circularity for evaluation: standard conditional quality metrics require a reference target distribution, but in the extrapolative regime that distribution is unavailable by definition. We address this problem with a post-hoc, per-sample trust score for assessing conditional samples using only the training distribution. The score combines two estimable quantities: global realism, measuring compatibility with the real data manifold, and attribute-wise faithfulness, measuring whether a sample is closer to the requested attributes than to plausible alternatives. We show that the score can recover meaningful comparisons across extrapolated generations, under a mild coverage condition on the observed attributes. These comparisons enable effective filtering, ranking, and abstention of generations and can be used directly on off-the-shelf pretrained models. In biological imaging, selected samples preserve real morphological structure better and improve downstream predictive performance, while similar gains are observed on controlled vision benchmarks. Finally, we show how the score can be applied during generation, enabling abstention before full decoding. Code is available at https://github.com/berkerdemirel/faithful-cond-gen.
Abstract:Conditional generators provide a natural tool for controllable generation, including settings where the desired condition is a new composition of observed attributes or experimental factors. In many applications, especially in scientific domains, such models are attractive to explore conditions for which real samples are rare, expensive, or not yet observed. However, this creates a circularity for evaluation: standard conditional quality metrics require a reference target distribution, but in the extrapolative regime that distribution is unavailable by definition. We address this problem with a post-hoc, per-sample trust score for assessing conditional samples using only the training distribution. The score combines two estimable quantities: global realism, measuring compatibility with the real data manifold, and attribute-wise faithfulness, measuring whether a sample is closer to the requested attributes than to plausible alternatives. We show that the score can recover meaningful comparisons across extrapolated generations, under a mild coverage condition on the observed attributes. These comparisons enable effective filtering, ranking, and abstention of generations and can be used directly on off-the-shelf pretrained models. In biological imaging, selected samples preserve real morphological structure better and improve downstream predictive performance, while similar gains are observed on controlled vision benchmarks. Finally, we show how the score can be applied during generation, enabling abstention before full decoding. Code is available at this https URL.
Machine Learning (cs.LG)
arXiv:2606.09601 [cs.LG]
(or arXiv:2606.09601v2 [cs.LG] for this version)
https://doi.org/10.48550/arXiv.2606.09601 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 307 cuvinte
Subiecte conexe
SciFusion: Physics-Guided Compositional Diffusion for Multimodal 3D Scientific Image Generation
arXiv · 23 oct. 2026
Assessing Sample Quality in Conditional Generation under Compositional Shift
arXiv · 9 iun. 2026
A Theoretical Analysis of Provable Compositional Generalization in Neural Networks: A Necessary and Sufficient Condition
arXiv · 9 sept. 2026
Collective Collaboration of Recursive Autonomous Agents
arXiv · 27 oct. 2026