analiza aiqbench
This research introduces FASTDIAR, a novel frame-level speaker encoder designed specifically for efficient, real-time streaming diarization on CPU hardware. By replacing computationally wasteful utterance-level processing with a causal architecture that updates embeddings every 80ms, the system achieves sub-second latency while maintaining high accuracy even as speaker counts increase. The approach relies on distillation from a teacher model and online clustering, offering a significant speedup over traditional cache-based methods without sacrificing performance on low-overlap benchmarks.
FASTDIAR: Frame-level speaker encoder for Streaming Diarization
arXiv:2610.02941v1 Announce Type: cross Abstract: Real-time conversational agents require speaker diarization that streams and runs on a CPU. Most systems apply an utterance-level speaker encoder to short, heavily overlapping chunks, which wastes computation and leaves the model optimized for the wrong task. We instead turn a state-of-the-art speaker recognition architecture into a causal frame-level encoder that reads the stream once and emits one embedding every 80~ms from a bounded two-second window of past audio, and pair it with online clustering that gates every update on the self-similarity of the stream. Trained only by distillation from an utterance-level teacher on simulated and out-of-domain mixtures, and evaluated with one fixed set of hyperparameters, the system is the most accurate streaming diarizer on low-overlap benchmarks at sub-second latency, degrades far less than cache-based systems as the number of speakers grows, and runs five times faster than real time on a single CPU thread.
Abstract:Real-time conversational agents require speaker diarization that streams and runs on a CPU. Most systems apply an utterance-level speaker encoder to short, heavily overlapping chunks, which wastes computation and leaves the model optimized for the wrong task. We instead turn a state-of-the-art speaker recognition architecture into a causal frame-level encoder that reads the stream once and emits one embedding every 80~ms from a bounded two-second window of past audio, and pair it with online clustering that gates every update on the self-similarity of the stream. Trained only by distillation from an utterance-level teacher on simulated and out-of-domain mixtures, and evaluated with one fixed set of hyperparameters, the system is the most accurate streaming diarizer on low-overlap benchmarks at sub-second latency, degrades far less than cache-based systems as the number of speakers grows, and runs five times faster than real time on a single CPU thread.
Audio and Speech Processing (eess.AS); Machine Learning (cs.LG); Sound (cs.SD)
arXiv:2610.02941 [eess.AS]
(or arXiv:2610.02941v1 [eess.AS] for this version)
https://doi.org/10.48550/arXiv.2610.02941 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Text integral sub licența deschisă a publicației · 246 cuvinte
Subiecte conexe
**Know Who Spoke When: Build Real-Time, Multi-Speaker AI with NVIDIA Nemotron 3 Diarization**
Hugging Face · 23 sept. 2026
NVIDIA Releases Nemotron 3 Diarization: A 100M-Parameter Open-Weight Model That Tracks 8 Speakers in Real Time
MarkTechPost · 23 sept. 2026
[AINews] DeepSeek v4.1-Flash: 763B-P8B-D16B novel causal Encoder–Decoder architecture with vision marks the Return of the Whale
Latent Space · 12 sept. 2026
VOSSA: Voiceprint Optimization for Streaming Speech Architectures
arXiv · 1 oct. 2026