analiza aiqbench
HALDETECT addresses the critical challenge of visual hallucinations in large multimodal models by using QLoRA fine-tuning to improve contrastive grounding between images and culturally plausible statements. The study demonstrates that answer ordering and adapter adaptation significantly outperform prompting alone, though results remain sensitive to random seeds rather than just data scaling. This work provides essential insights for deploying reliable AI vision systems where precise visual interpretation is required.
HALDETECT at ImageEval 2026 Shared Tasks: Answer-First Contrastive Grounding with QLoRA
arXiv:2609.11236v1 Announce Type: cross Abstract: Large multimodal models tend to hallucinate visual detail fluently, which limits their deployment for fine-grained interpretation. We present HALDETECT, our system for the English hallucination-detection track (Task 1b) of ImageEval 2026, in which a system must identify, from an image and three culturally plausible statements, the single visually grounded one. We frame the item as one contrastive decision, emit the answer before its explanation, and structure reasoning around colour/texture, shape/form, and context. Our best submitted adapter fine-tunes Qwen2.5-VL-7B-Instruct with 4-bit QLoRA while freezing the vision encoder and reaches Contrastive Instability (CI) 0.035 on the 1,000-item test set; we placed third of eight teams. Development experiments show that answer order can matter more than model scale and that adaptation beats prompting alone. Retrospective paired analysis of the released gold labels confirms the QLoRA gain over the best prompt but not the small gap between the devtest-selected and best-test adapters, and reseeding all four training sizes shows that the apparent data-scaling curve does not survive a seed change. The 35 residual errors are culturally plausible function, material, and recognition distinctions; naive adapter voting does not help.
Abstract:Large multimodal models tend to hallucinate visual detail fluently, which limits their deployment for fine-grained interpretation. We present HALDETECT, our system for the English hallucination-detection track (Task 1b) of ImageEval 2026, in which a system must identify, from an image and three culturally plausible statements, the single visually grounded one. We frame the item as one contrastive decision, emit the answer before its explanation, and structure reasoning around colour/texture, shape/form, and context. Our best submitted adapter fine-tunes Qwen2.5-VL-7B-Instruct with 4-bit QLoRA while freezing the vision encoder and reaches Contrastive Instability (CI) 0.035 on the 1,000-item test set; we placed third of eight teams. Development experiments show that answer order can matter more than model scale and that adaptation beats prompting alone. Retrospective paired analysis of the released gold labels confirms the QLoRA gain over the best prompt but not the small gap between the devtest-selected and best-test adapters, and reseeding all four training sizes shows that the apparent data-scaling curve does not survive a seed change. The 35 residual errors are culturally plausible function, material, and recognition distinctions; naive adapter voting does not help.
10 pages, 3 figures, 10 tables (including appendices). System description paper for Task 1b (English) of ImageEval 2026 Shared Tasks (Fourth Arabic Natural Language Processing Conference), to appear in the Shared Tasks proceedings
Computer Vision and Pattern Recognition (cs.CV); Artificial Intelligence (cs.AI)
I.2.7; I.2.10; I.4.8
arXiv:2609.11236 [cs.CV]
(or arXiv:2609.11236v1 [cs.CV] for this version)
https://doi.org/10.48550/arXiv.2609.11236 arXiv-issued DOI via DataCite (pending registration)
Text integral sub licența deschisă a publicației · 244 cuvinte
Subiecte conexe
Concrete Evaluations to Investigate the OpenAI Model That Hacked Hugging Face
AI Alignment Forum · 3 aug. 2026
Links #5: 2026/07
LessWrong · 31 iul. 2026
[AINews] Gemini 4 Argon: GDM’s answer to Astra/Fable, with 1M output
Latent Space · 1 oct. 2026
Why do models task game?
AI Alignment Forum · 6 aug. 2026