aiqbench analysis
This study demonstrates the feasibility of using large language models to automatically generate personalized therapeutic networks from session transcripts, offering a scalable alternative to data-intensive longitudinal methods for case conceptualization. While the pipeline successfully produced coherent and interpretable clinical structures, inconsistent interrater agreement highlights the need for further validation before these tools can reliably predict treatment outcomes. The research marks a significant step toward integrating generative AI into mental health workflows, though its ultimate efficacy in improving patient care remains to be proven against traditional statistical methods.
Testing the Utility of Using Large Language Models to Create Personalized Networks From Therapy Session Transcripts: A Proof of Concept Study
arXiv:2512.05836v2 Announce Type: replace Abstract: Recent advances in psychotherapy have focused on treatment personalization, such as by selecting treatment modules based on individual networks. However, estimating personalized networks typically requires intensive longitudinal data, which is not always feasible to collect. A solution to increase scalability of network-driven treatment personalization is leveraging large language models (LLMs). In this study, we developed an end-to-end pipeline for automatically generating client networks to support case conceptualization and treatment planning. We annotated 8,028 utterances from 77 therapy transcripts (N = 6). In the first stage of the pipeline, we identified clinically relevant processes (binary classification) and their corresponding dimensions (multi-label classification). Then, we introduced a two-step method that grouped the processes into clinically meaningful clusters and generated labels for the clusters. Finally, we generated connections between clusters. Evaluation results generally supported model utility, however, interrater agreement on model performance metrics was inconsistent, ranging from poor to substantial. Qualitative examination of the networks indicated consistency with original study data. Given coherence and interpretability of the generated networks, developing networks from therapy transcripts using LLMs appears to be feasible. Nonetheless, more research is needed to examine whether these networks improve treatment outcomes, including relative to other methods of treatment personalization, such as statistically estimated networks. Potential use cases and limitations of our pipeline are discussed.
Abstract:Recent advances in psychotherapy have focused on treatment personalization, such as by selecting treatment modules based on individual networks. However, estimating personalized networks typically requires intensive longitudinal data, which is not always feasible to collect. A solution to increase scalability of network-driven treatment personalization is leveraging large language models (LLMs). In this study, we developed an end-to-end pipeline for automatically generating client networks to support case conceptualization and treatment planning. We annotated 8,028 utterances from 77 therapy transcripts (N = 6). In the first stage of the pipeline, we identified clinically relevant processes (binary classification) and their corresponding dimensions (multi-label classification). Then, we introduced a two-step method that grouped the processes into clinically meaningful clusters and generated labels for the clusters. Finally, we generated connections between clusters. Evaluation results generally supported model utility, however, interrater agreement on model performance metrics was inconsistent, ranging from poor to substantial. Qualitative examination of the networks indicated consistency with original study data. Given coherence and interpretability of the generated networks, developing networks from therapy transcripts using LLMs appears to be feasible. Nonetheless, more research is needed to examine whether these networks improve treatment outcomes, including relative to other methods of treatment personalization, such as statistically estimated networks. Potential use cases and limitations of our pipeline are discussed.
Artificial Intelligence (cs.AI)
arXiv:2512.05836 [cs.AI]
(or arXiv:2512.05836v2 [cs.AI] for this version)
https://doi.org/10.48550/arXiv.2512.05836 Focus to learn more arXiv-issued DOI via DataCite
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 305 words
Related coverage
Links #3: 2026/06 Part 1
LessWrong · Jun 15, 2026
AI Futures: Racing to Lose, A sermon
LessWrong · Sep 28, 2026
An unexamined cause of the OpenAI Hugging Face hacking incident: its binary performance metric
LessWrong · Sep 23, 2026
The J-Space Debate, Agent Swarms, and Pacing Frontier AI - Digital Minds Newsletter #4
LessWrong · Sep 18, 2026