aiqbench analysis
This research introduces FORMA, a framework that transforms cognitive disorder models into directed weighted graphs to ensure AI-generated clinical vignettes maintain structural fidelity rather than just fluency. By validating generated content against specific causal links, the system significantly improves expert ratings and reduces demographic bias compared to standard zero-shot generation methods. The work demonstrates that auditable specifications are essential for scaling synthetic text in high-stakes medical contexts.
Generating Clinical Vignettes that Preserve Cognitive Formulations
arXiv:2608.29995v1 Announce Type: cross Abstract: Large language models can generate fluent clinical case vignettes, but fluency alone does not ensure fidelity to a specifiable clinical structure. We introduce FORMA, a theory-grounded framework that compiles a cognitive model of a disorder into a directed weighted graph, samples a person-specific configuration of that graph, and validates whether the generated vignette preserves the specified components and causal links. We instantiate FORMA on Posttraumatic Stress Disorder using the Ehlers and Clark cognitive model, generating 16,500 vignettes across 500 personas, 11 generation models, and three ablation conditions. Evaluation combines an external edge-recovery probe, two clinical experts, a scaled LLM judge, and a clinician user study with 100 licensed practitioners. The cognitive graph is recoverable from full-condition vignettes (MCC = +0.41, AUC = 0.70) but not from zero-shot generation (MCC = +0.01, AUC = 0.50). Experts rate full vignettes substantially higher than zero-shot alternatives, and clinicians perceive them to be human-written 85% of the time, compared with 22% for zero-shot. FORMA also reduces demographic disparity in perceived quality by 1.5-7x. These results show that cognitive formulation can serve as an auditable specification for scalable synthetic clinical text generation. A repository with the data and code is available online: https://github.com/Amit-Oren/FORMA.
Abstract:Large language models can generate fluent clinical case vignettes, but fluency alone does not ensure fidelity to a specifiable clinical structure. We introduce FORMA, a theory-grounded framework that compiles a cognitive model of a disorder into a directed weighted graph, samples a person-specific configuration of that graph, and validates whether the generated vignette preserves the specified components and causal links. We instantiate FORMA on Posttraumatic Stress Disorder using the Ehlers and Clark cognitive model, generating 16,500 vignettes across 500 personas, 11 generation models, and three ablation conditions. Evaluation combines an external edge-recovery probe, two clinical experts, a scaled LLM judge, and a clinician user study with 100 licensed practitioners. The cognitive graph is recoverable from full-condition vignettes (MCC = +0.41, AUC = 0.70) but not from zero-shot generation (MCC = +0.01, AUC = 0.50). Experts rate full vignettes substantially higher than zero-shot alternatives, and clinicians perceive them to be human-written 85% of the time, compared with 22% for zero-shot. FORMA also reduces demographic disparity in perceived quality by 1.5-7x. These results show that cognitive formulation can serve as an auditable specification for scalable synthetic clinical text generation. A repository with the data and code is available online: this https URL.
Accepted to Findings of the Association for Computational Linguistics: EMNLP 2026. Code and data: this https URL
Computation and Language (cs.CL); Artificial Intelligence (cs.AI)
arXiv:2608.29995 [cs.CL]
(or arXiv:2608.29995v1 [cs.CL] for this version)
https://doi.org/10.48550/arXiv.2608.29995 Focus to learn more arXiv-issued DOI via DataCite (pending registration)
arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.
Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.
Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.
Full text under the publisher's open licence · 315 words
Related coverage
Deconfusing Corrigibility
AI Alignment Forum · Oct 17, 2026
DMT-CBT: Longitudinal Therapeutic State Modeling for CBT Counseling
arXiv · Jun 3, 2026
Interpretable Synthetic Medical Tabular Data Generation for Clinical Decision Support Using Fuzzy Cognitive Maps
arXiv · Oct 2, 2026
Explainable and Generalisable LLM-based Cognitive Decline Detection with Spontaneous Speech
arXiv · Sep 30, 2026