aiqbench analysis
This research introduces AudioRE, a high-quality dataset designed specifically for post-training large audio-language models by eliminating acoustic redundancy found in aggregated public sources. The study demonstrates that filtering duplicate patterns and structuring data into chain-of-thought reasoning formats yields superior performance compared to simply scaling up raw corpus size. This work underscores the critical importance of data quality and deduplication strategies over mere volume expansion for advancing multimodal AI capabilities.
AudioDER: A Deduplication-Enhanced Reasoning Dataset for Post-Training Large Audio-Language Models
arXiv:2606.14591v2 Announce Type: replace-cross Abstract: Recent advances in pretrained large audio-language models (LALMs) have demonstrated strong capabilities across speech, sound, and music. To adapt these models to downstream tasks without the cost of pretraining from scratch, post-training has become a widely adopted paradigm. However, the effectiveness of post-training depends critically on the quality of the training corpus. We observe that existing post-training corpora, often constructed by aggregating public audio datasets, suffer from substantial acoustic redundancy, as many of these datasets are sourced from overlapping media platforms. Such redundancy leads to repeated exposure to similar acoustic patterns, causing diminishing returns in performance despite increased data volume. address this issue, we propose a three-stage data construction pipeline that performs acoustic redundancy filtering, converts retained samples into a unified multiple-choice question-answering format with chain-of-thought generation, and finally applies quality verification and filtering. Using this pipeline, we construct AudioRE, a post-training dataset of approximately 286k instances spanning sound, speech, and music. Supervised fine-tuning on AudioRE consistently improves the performance of Qwen2-Audio-7B-Instruct across diverse audio understanding and reasoning benchmarks, outperforming models trained on the unfiltered raw corpus with substantially more instances. These results validate the effectiveness of our redundancy-aware data construction pipeline and the resulting AudioRE dataset, and further highlight the importance of minimizing acoustic redundancy in audio-language post-training. To facilitate future research, we will release both the AudioRE and the fine-tuned Qwen2-AudioRE checkpoint.
Abstract:Recent advances in pretrained large audio-language models (LALMs) have demonstrated strong capabilities across speech, sound, and music. To adapt these models to downstream tasks without the cost of pretraining from scratch, post-training has become a widely adopted paradigm. However, the effectiveness of post-training depends critically on the quality of the training corpus. We observe that existing post-training corpora, often constructed by aggregating public audio datasets, suffer from substantial acoustic redundancy, as many of these datasets are sourced from overlapping media platforms. Such redundancy leads to repeated exposure to similar acoustic patterns, causing diminishing returns in performance despite increased data volume. address this issue, we propose a three-stage data construction pipeline that performs acoustic redundancy filtering, converts retained samples into a unified multiple-choice question-answering format with chain-of-thought generation, and finally applies quality verification and filtering. Using this pipeline, we construct AudioRE, a post-training dataset of approximately 286k instances spanning sound, speech, and music. Supervised fine-tuning on AudioRE consistently improves the performance of Qwen2-Audio-7B-Instruct across diverse audio understanding and reasoning benchmarks, outperforming models trained on the unfiltered raw corpus with substantially more instances. These results validate the effectiveness of our redundancy-aware data construction pipeline and the resulting AudioRE dataset, and further highlight the importance of minimizing acoustic redundancy in audio-language post-training. To facilitate future research, we will release both the AudioRE and the fine-tuned Qwen2-AudioRE checkpoint.
Sound (cs.SD); Artificial Intelligence (cs.AI)
arXiv:2606.14591 [cs.SD]
(or arXiv:2606.14591v2 [cs.SD] for this version)
https://doi.org/10.48550/arXiv.2606.14591 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 318 words
Related coverage
AudioDER: A Deduplication-Enhanced Reasoning Dataset for Post-Training Large Audio-Language Models
arXiv · Jun 15, 2026
Introducing North Mini Code: Cohere’s First Model For Developers
Hugging Face · Jun 9, 2026
NVIDIA GTC Taipei at COMPUTEX: Live Updates on What’s Next in AI
NVIDIA · May 21, 2026
Introducing Instella-MoE: A State-of-the-Art Fully Open Mixture-of-Experts Language Model
AMD · Jul 24, 2026