Virtual Speech Therapist: A Clinician-in-the-Loop AI Agent for Speech Therapy Planning

Abstract:This paper presents a proof of concept of the Virtual Speech Therapist (VST), an intelligent multi-agent based system designed to support stuttering assessment and individualized therapy planning through artificial intelligence (AI) driven workflows. VST integrates deep learning (DL) based stuttering detection fSD with large language model (LLM) based agentic system to assist clinicians in developing evidence informed therapy plans. The VST system begins with the acquisition and feature extraction of given speech samples, followed by the detection of stuttering types. Building on these speech driven information S, VST initiates an agentic reasoning process in which specialized LLM agents autonomously generate, critique, and iteratively refine individualized therapy plans P. A dedicated CriticAgent {\pi}ca acts as a structured self-refinement mechanism, reviewing the generated therapy plans P and identifying areas for potential improvement in consistency, methodological soundness, and alignment with available evidence and professional guidelines. The resulting therapy plan P is presented then as a draft for clinician review. Clinician feedback can then be incorporated to produce a revised plan P, thereby maintaining a clinician-in-the-loop (CITL) paradigm. An initial evaluation, involving expert speech language pathologist (SLP) examines the clinical interpretability and feasibility of the generated therapy plans. Taken together, the results show the potential of VST to augment clinician-led therapy planning and provide a promising basis for future studies evaluating its integration into real world clinical settings. An interactive user interface for the proposed prototype is available online at: this https URL, facilitating interactive stuttering assessment and stuttering therapy planning.

Artificial Intelligence (cs.AI); Computation and Language (cs.CL); Sound (cs.SD); Audio and Speech Processing (eess.AS)

arXiv:2605.01101 [cs.AI]

(or arXiv:2605.01101v3 [cs.AI] for this version)

https://doi.org/10.48550/arXiv.2605.01101 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.