analiza aiqbench
This arXiv paper introduces AD4Edu-Bench and AD4Edu-Eval, a benchmark and rule-based metric for generating audio descriptions of slide-based university lectures, and it uses DPO to post-train models. Its main finding is that outputs complying with accessibility standards don't always match what blind and low-vision raters prefer, especially on description length. For AI readers, the useful lesson is that automated metrics built from written guidelines can miss user preferences, so evaluation for accessibility tasks needs direct input from the people it serves.
AD4EDU: Benchmarking University Lecture Audio Description across Accessibility Standards, Metrics, and Learner Preferences
arXiv:2610.17042v1 Announce Type: new Abstract: Audio description (AD) aims to enhance the accessibility of visual media for blind and low-vision (BLV) persons. However, in higher education, where slide-based lectures are a default mode of teaching, lecture AD remains largely underexplored. The main reason is that existing production guidelines, evaluation metrics, and benchmark datasets are oriented toward entertainment content and do not transfer to lectures, where ADs must complement the lecturer's speech, resolve deictic references, and fit within narrow pauses. In this work, we propose AD4Edu, consisting of an AD benchmark, AD4Edu-Bench, and a rule-diagnosable evaluation metric, AD4Edu-Eval, for generating and evaluating ADs in higher education. Specifically, we consolidated established accessibility guidance, including WCAG 2.2, the DCMP Description Key, and NCAM's STEM description practices. We further collected silver reference ADs, judge-ordered preference pairs, and pairwise preferences from four BLV raters, and post-trained models with direct preference optimization (DPO) for detailed analysis. Our findings indicate that while the metric reliably detects violations of the standard, ADs that comply with it do not necessarily fulfill the subjective preferences of BLV persons, most notably on description length, underscoring the necessity of active participation of BLV persons in co-designing inclusive AI solutions.
Abstract:Audio description (AD) aims to enhance the accessibility of visual media for blind and low-vision (BLV) persons. However, in higher education, where slide-based lectures are a default mode of teaching, lecture AD remains largely underexplored. The main reason is that existing production guidelines, evaluation metrics, and benchmark datasets are oriented toward entertainment content and do not transfer to lectures, where ADs must complement the lecturer's speech, resolve deictic references, and fit within narrow pauses. In this work, we propose AD4Edu, consisting of an AD benchmark, AD4Edu-Bench, and a rule-diagnosable evaluation metric, AD4Edu-Eval, for generating and evaluating ADs in higher education. Specifically, we consolidated established accessibility guidance, including WCAG 2.2, the DCMP Description Key, and NCAM's STEM description practices. We further collected silver reference ADs, judge-ordered preference pairs, and pairwise preferences from four BLV raters, and post-trained models with direct preference optimization (DPO) for detailed analysis. Our findings indicate that while the metric reliably detects violations of the standard, ADs that comply with it do not necessarily fulfill the subjective preferences of BLV persons, most notably on description length, underscoring the necessity of active participation of BLV persons in co-designing inclusive AI solutions.
Computation and Language (cs.CL)
arXiv:2610.17042 [cs.CL]
(or arXiv:2610.17042v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2610.17042 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 287 cuvinte
Subiecte conexe
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · 25 sept. 2026
Gradium Launches Voice Design: Write a Prompt, Get a Brand New Synthetic Voice in Seconds
MarkTechPost · 9 sept. 2026
Links #3: 2026/06 Part 1
LessWrong · 15 iun. 2026
How Condé Nast built multimodal video discovery with Amazon Bedrock
AWS · 29 sept. 2026