analiza aiqbench
This research introduces ParA-LLM, a specialized framework designed to enhance Audio LLMs' ability to reason over paralinguistic traits and acoustic conditions beyond basic speech recognition. By training on a curated dataset of 1.2 million pairs and a new benchmark, the model achieves significant accuracy gains over frontier models like GPT-4o-Audio in complex multi-attribute reasoning tasks. The work highlights a critical gap in current audio intelligence systems and provides a foundational step toward more robust multimodal understanding.
ParA-LLM: A Unified Approach to Paralinguistic and Acoustic Speech Understanding
arXiv:2609.22771v1 Announce Type: cross Abstract: Recent advances in Audio LLMs have achieved human-level speech recognition, yet existing systems struggle to capture paralinguistic aspects such as speaker traits, expressive variations, and environmental acoustic conditions. To address this, we design a framework of 22 paralinguistic characteristics and create a dataset of over 1.2M Audio-QA pairs. We develop ParA-LLM, trained with a two-stage curriculum: first on single-attribute questions to build foundational knowledge, then on multi-attribute questions for joint reasoning over speaker and acoustic characteristics. We also release ParA-Bench, a benchmark of 6,000 multiple-choice questions across speaker-speech, acoustic, and mixed categories, where frontier models like GPT-4o-Audio achieve only 36% accuracy. ParA-LLM surpasses state-of-the-art Audio LLMs like GPT-4o-Audio by 7.5% on ParA-Bench, with additional gains of 1.13% on MMAU-Pro Speech and 7.49% on MMAR Speech.
Abstract:Recent advances in Audio LLMs have achieved human-level speech recognition, yet existing systems struggle to capture paralinguistic aspects such as speaker traits, expressive variations, and environmental acoustic conditions. To address this, we design a framework of 22 paralinguistic characteristics and create a dataset of over 1.2M Audio-QA pairs. We develop ParA-LLM, trained with a two-stage curriculum: first on single-attribute questions to build foundational knowledge, then on multi-attribute questions for joint reasoning over speaker and acoustic characteristics. We also release ParA-Bench, a benchmark of 6,000 multiple-choice questions across speaker-speech, acoustic, and mixed categories, where frontier models like GPT-4o-Audio achieve only 36% accuracy. ParA-LLM surpasses state-of-the-art Audio LLMs like GPT-4o-Audio by 7.5% on ParA-Bench, with additional gains of 1.13% on MMAU-Pro Speech and 7.49% on MMAR Speech.
Accepted to Interspeech 2026. Project Website: this https URL
Sound (cs.SD); Artificial Intelligence (cs.AI); Machine Learning (cs.LG); Audio and Speech Processing (eess.AS); Signal Processing (eess.SP)
arXiv:2609.22771 [cs.SD]
(or arXiv:2609.22771v1 [cs.SD] for this version)
https://doi.org/10.48550/arXiv.2609.22771 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 241 cuvinte
Subiecte conexe
EMODE: Dynamic Para-Semantic Experts for Emotion-Aware Speech Language Modeling
arXiv · 7 oct. 2026
FireRedAudio: A General-Purpose Audio Language Model with Decoupled Continuous Representations for Understanding and Generation
arXiv · 26 aug. 2026
SpeechSense: A Paralinguistic-Focused Dataset for Fine-Grained Speech Sentiment Analysis
arXiv · 19 aug. 2026
INSPIRE: A Benchmark for Instruction-Aware Speech Retrieval
arXiv · 18 aug. 2026