aiqbench analysis
This research addresses a critical gap in long-context audio reasoning by releasing a synthetic dataset of 8,800 doctor-patient conversations generated entirely with open-weight models. The study provides controlled evaluation metrics for SOAP note generation, revealing that cascaded approaches currently offer performance advantages comparable to end-to-end systems after recent corrections.
Generating Synthetic Doctor-Patient Conversations for Long-form Audio Summarization
arXiv:2604.06138v2 Announce Type: replace-cross Abstract: Long-context audio reasoning is underserved in both training data and evaluation. Existing benchmarks target short-context tasks, and the open-ended generation tasks most relevant to long-context reasoning pose well-known challenges for automatic evaluation. We propose a synthetic data generation pipeline designed to serve both as a training resource and as a controlled evaluation environment, and instantiate it for first-visit doctor-patient conversations with SOAP note generation as the task. The pipeline has three stages, persona-driven dialogue generation, multi-speaker audio synthesis with overlap/pause modeling, room acoustics, and sound events, and LLM-based reference SOAP note production, built entirely on open-weight models. We release 8,800 synthetic conversations with 1.3k hours of corresponding audio and reference notes. Evaluating current open-weight systems, we find that cascaded approaches still substantially outperform end-to-end models.
Abstract:Long-context audio reasoning is underserved in both training data and evaluation. Existing benchmarks target short-context tasks, and the open-ended generation tasks most relevant to long-context reasoning pose well-known challenges for automatic evaluation. We propose a synthetic data generation pipeline designed to serve both as a training resource and as a controlled evaluation environment, and instantiate it for first-visit doctor-patient conversations with SOAP note generation as the task. The pipeline has three stages, persona-driven dialogue generation, multi-speaker audio synthesis with overlap/pause modeling, room acoustics, and sound events, and LLM-based reference SOAP note production, built entirely on open-weight models. We release 8,800 synthetic conversations with 1.3k hours of corresponding audio and reference notes. Evaluating current open-weight systems, we find that cascaded approaches still substantially outperform end-to-end models.
Published at Interspeech 2026. v2 is the published version; results were corrected after a bug fix: end-to-end and cascaded systems now perform comparably (per-claim hallucination 32% vs. 22-24%)
Sound (cs.SD); Artificial Intelligence (cs.AI)
arXiv:2604.06138 [cs.SD]
(or arXiv:2604.06138v2 [cs.SD] for this version)
Proc. Interspeech 2026, 4384-4389
https://doi.org/10.21437/Interspeech.2026-2901 DOI(s) linking to related resources
Full text under the publisher's open licence · 176 words
Related coverage
Evaluate Clinical ASR Models Faster with Agent Skills and NVIDIA Nemotron Speech
NVIDIA · Jun 9, 2026
Bridging the Modality Gap in Long-Form Clinical Audio: A Comparative Study of Lightweight and Heavyweight End-to-End SOAP Generation
arXiv · Sep 16, 2026
Runway’s WorldPrompt and the Engineering of Real-Time Worlds
Latent Space · Sep 25, 2026
A New Era of Discovery: Google Research at I/O 2026
Google Research · May 28, 2026