analiza aiqbench
This research introduces Verifiable QA Generation (VQS), a method that replaces unreliable majority voting or model judges with fixed programs to parse images into structured records for self-supervised training. By decomposing complex visual tasks into verifiable claims, the approach significantly reduces hallucination rates and improves performance on Qwen3-VL across multiple scales without requiring labeled data. The technique advances the frontier of autonomous model evolution by ensuring that generated training data is grounded in programmatically verified facts rather than probabilistic guesses.
Program-Verified Self-Evolution for Vision-Language Models
arXiv:2609.33855v1 Announce Type: cross Abstract: Self-evolving vision-language models train on questions they generate from unlabeled images. Since these questions have no gold answers, prior methods label them by majority vote over sampled answers or by a model judge. In a human evaluation, we find that 24\% of majority-vote labels and 18\% of model-judge labels produced during self-evolution are wrong. To address this problem, we present Verifiable QA Generation for Self-Evolving Models (VQS), which changes how the model judges answers. Instead of voting on an answer, the model parses each image into a structured record, such as a scene graph, a chart table, or a diagram graph. Fixed programs then write a question from the record and compute its answer. The model still acts as a visual checker, but it only confirms the individual facts the program reads, one short claim at a time. These claim-level checks select the parser's training targets, so the parser also improves without labels. Human raters find 94\% of VQS answers correct, against 76\% for majority voting. Across ten benchmarks, VQS improves Qwen3-VL by up to 3.18 points at the 2B, 4B, and 8B scales and outperforms the strongest self-evolving baseline at each. Gains keep growing over three training rounds, reaching 3.84 points at 2B. Code is released at https://github.com/ahmedheakl/VQS
Abstract:Self-evolving vision-language models train on questions they generate from unlabeled images. Since these questions have no gold answers, prior methods label them by majority vote over sampled answers or by a model judge. In a human evaluation, we find that 24\% of majority-vote labels and 18\% of model-judge labels produced during self-evolution are wrong. To address this problem, we present Verifiable QA Generation for Self-Evolving Models (VQS), which changes how the model judges answers. Instead of voting on an answer, the model parses each image into a structured record, such as a scene graph, a chart table, or a diagram graph. Fixed programs then write a question from the record and compute its answer. The model still acts as a visual checker, but it only confirms the individual facts the program reads, one short claim at a time. These claim-level checks select the parser's training targets, so the parser also improves without labels. Human raters find 94\% of VQS answers correct, against 76\% for majority voting. Across ten benchmarks, VQS improves Qwen3-VL by up to 3.18 points at the 2B, 4B, and 8B scales and outperforms the strongest self-evolving baseline at each. Gains keep growing over three training rounds, reaching 3.84 points at 2B. Code is released at this https URL
Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Machine Learning (cs.LG); Neural and Evolutionary Computing (cs.NE)
arXiv:2609.33855 [cs.CV]
(or arXiv:2609.33855v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2609.33855 arXiv-issued DOI via DataCite (pending registration)
Text integral sub licența deschisă a publicației · 247 cuvinte
Subiecte conexe
Models Can't Remember Their Training. Neither Can You.
LessWrong · 19 iul. 2026
Self-Evolving Coding Agents: From Digital Programs to Physical-World Intelligence
arXiv · 29 sept. 2026
Humanity’s Last Invention — Richard Socher of Recursive
Latent Space · 14 sept. 2026
AI Futures: Racing to Lose, A sermon
LessWrong · 28 sept. 2026