analiza aiqbench
This research introduces Instruct-FD, a new benchmark that exposes significant limitations in current full-duplex speech systems regarding their ability to follow explicit turn-taking instructions. The study reveals that even state-of-the-art models struggle with proactive behaviors like backchanneling and interruption, achieving only 64.4% adherence rates across varied scenarios. These findings highlight the critical need for improved instruction-following capabilities before such systems can be reliably deployed in diverse real-world applications.
Instruct-FD: Can Your Full-Duplex Speech System Follow Turn-Taking Instructions?
arXiv:2607.20460v1 Announce Type: new Abstract: Current full-duplex (FD) spoken dialogue systems can produce fluid interactions, yet it remains unclear whether they can adapt their turn-taking behavior when explicitly instructed. This is critical for real-world deployment, where conversational policies vary across applications (e.g., proactive tutoring vs. passive counseling). We introduce Instruct-FD, an instruction-conditioned benchmark for evaluating controllable turn management in FD systems. To enable this, we develop a human-validated, scalable synthetic pipeline that generates instruction-conditioned conversations, along with a deployment-agnostic multi-turn evaluation protocol and an LLM-based judge. Benchmarking six state-of-the-art full-duplex systems reveals a substantial gap in instruction-following turn management: the best model achieves only 64.4% adherence. Performance is highly uneven across behaviors and scenarios, with proactive behaviors such as model backchanneling and interruption remaining particularly challenging. These findings establish instruction-following turn management as a crucial direction for building adaptable and deployable full-duplex dialogue systems.
Abstract:Current full-duplex (FD) spoken dialogue systems can produce fluid interactions, yet it remains unclear whether they can adapt their turn-taking behavior when explicitly instructed. This is critical for real-world deployment, where conversational policies vary across applications (e.g., proactive tutoring vs. passive counseling). We introduce Instruct-FD, an instruction-conditioned benchmark for evaluating controllable turn management in FD systems. To enable this, we develop a human-validated, scalable synthetic pipeline that generates instruction-conditioned conversations, along with a deployment-agnostic multi-turn evaluation protocol and an LLM-based judge. Benchmarking six state-of-the-art full-duplex systems reveals a substantial gap in instruction-following turn management: the best model achieves only 64.4% adherence. Performance is highly uneven across behaviors and scenarios, with proactive behaviors such as model backchanneling and interruption remaining particularly challenging. These findings establish instruction-following turn management as a crucial direction for building adaptable and deployable full-duplex dialogue systems.
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2607.20460 [cs.CL]
(or arXiv:2607.20460v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2607.20460 arXiv-issued DOI via DataCite (pending registration)
Text integral sub licența deschisă a publicației · 161 cuvinte
Subiecte conexe
DuplexSpeechBench-IFEval: Evaluating Implicit Instruction Following in Full-Duplex Voice Agents
arXiv · 5 sept. 2026
Evaluate your Amazon Nova Sonic voice agent at scale, no microphone required
AWS · 8 iun. 2026
[AINews] not much happened today
Latent Space · 3 oct. 2026
Links #5: 2026/07
LessWrong · 31 iul. 2026