analiza aiqbench
This research presents D2, a novel framework that upgrades half-duplex language models into high-capability full-duplex systems capable of simultaneous listening and speaking. By leveraging decoder-free representation distillation and supervised fine-tuning, the approach achieves state-of-the-art performance on metrics like turn-taking success and interruption handling across Qwen3-Omni and LLaMA-Omni 2 architectures. The study provides critical insights into optimizing chunk sizes for inference computation without sacrificing general capabilities in real-time audio interactions.
D2: High-Capability Full-Duplex Spoken Language Models from Half-Duplex Backbones
arXiv:2610.17124v1 Announce Type: cross Abstract: Full-duplex spoken language models (FD-SLMs) support simultaneous listening and speaking, but often lag behind half-duplex spoken language models (HD-SLMs) in general capabilities. We propose Dialogue-to-Duplex (D2), a framework that converts pretrained HD-SLMs into high-capability FD-SLMs at fine temporal granularity. D2 adapts the original audio encoder to streaming through decoder-free representation distillation and learns full-duplex interaction through supervised fine-tuning. We systematically study the effects of chunk size on full-duplex interaction, general capabilities, and inference computation. We apply D2 to two distinct HD-SLM architectures, Qwen3-Omni and LLaMA-Omni 2. At a 12.5Hz update rate, Qwen3-Omni-D2 achieves 99.2% turn-taking success with 521-ms latency, 99.5% interruption success, and a state-of-the-art VoiceBench score of 70.1.
Abstract:Full-duplex spoken language models (FD-SLMs) support simultaneous listening and speaking, but often lag behind half-duplex spoken language models (HD-SLMs) in general capabilities. We propose Dialogue-to-Duplex (D2), a framework that converts pretrained HD-SLMs into high-capability FD-SLMs at fine temporal granularity. D2 adapts the original audio encoder to streaming through decoder-free representation distillation and learns full-duplex interaction through supervised fine-tuning. We systematically study the effects of chunk size on full-duplex interaction, general capabilities, and inference computation. We apply D2 to two distinct HD-SLM architectures, Qwen3-Omni and LLaMA-Omni 2. At a 12.5Hz update rate, Qwen3-Omni-D2 achieves 99.2% turn-taking success with 521-ms latency, 99.5% interruption success, and a state-of-the-art VoiceBench score of 70.1.
17 pages, including references and appendices. Demo: this https URL
Computation and Language (cs.CL); Artificial Intelligence (cs.AI); Audio and Speech Processing (eess.AS)
arXiv:2610.17124 [cs.CL]
(or arXiv:2610.17124v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2610.17124 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 222 cuvinte
Subiecte conexe
Context Spanning: A Communication Framework for Full-Duplex Speech Models and External LLM Backends
arXiv · 30 sept. 2026
[AINews] not much happened today
Latent Space · 10 sept. 2026
[AINews] DeepSeek v4.1-Flash: 763B-P8B-D16B novel causal Encoder–Decoder architecture with vision marks the Return of the Whale
Latent Space · 12 sept. 2026
Links #5: 2026/07
LessWrong · 31 iul. 2026