Skip to content
aiqbench takeAI Research

aiqbench analysis

The study tests GPT-3.5-turbo on classifying depression severity from user posts, in English and in Greek via machine translation. It reports weak performance in English and uneven results in Greek, so it's a useful caution against assuming LLMs transfer well to clinical-adjacent tasks or lower-resource languages. Readers building multilingual or health-related systems should note that translation pipelines add error, and that the authors call for human oversight.
Source coveringarXiv
What this isA model's reading of the article and related coverage, written for aiqbench. Not the publisher's words.

Leveraging LLMs for Translating and Classifying Mental Health Data

arXiv2 min read

arXiv:2410.12985v3 Announce Type: replace Abstract: Large language models (LLMs) are increasingly used in medical fields. In mental health support, the early identification of linguistic markers associated with mental health conditions can provide valuable support to mental health professionals, and reduce long waiting times for patients. Despite the benefits of LLMs for mental health support, there is limited research on their application in mental health systems for languages other than English. Our study addresses this gap by focusing on the detection of depression severity in Greek through user-generated posts which are automatically translated from English. Our results show that GPT3.5-turbo is not very successful in identifying the severity of depression in English, and it has a varying performance in Greek as well. Our study underscores the necessity for further research, especially in languages with less resources. Also, careful implementation is necessary to ensure that LLMs are used effectively in mental health platforms, and human supervision remains crucial to avoid misdiagnosis.

Abstract:Large language models (LLMs) are increasingly used in medical fields. In mental health support, the early identification of linguistic markers associated with mental health conditions can provide valuable support to mental health professionals, and reduce long waiting times for patients. Despite the benefits of LLMs for mental health support, there is limited research on their application in mental health systems for languages other than English. Our study addresses this gap by focusing on the detection of depression severity in Greek through user-generated posts which are automatically translated from English. Our results show that GPT3.5-turbo is not very successful in identifying the severity of depression in English, and it has a varying performance in Greek as well. Our study underscores the necessity for further research, especially in languages with less resources. Also, careful implementation is necessary to ensure that LLMs are used effectively in mental health platforms, and human supervision remains crucial to avoid misdiagnosis.

Computation and Language (cs.CL)

arXiv:2410.12985 [cs.CL]

(or arXiv:2410.12985v3 [cs.CL] for this version)

https://doi.org/10.48550/arXiv.2410.12985 Focus to learn more arXiv-issued DOI via DataCite

arXivLabs is a framework that allows collaborators to develop and share new arXiv features directly on our website.

Both individuals and organizations that work with arXivLabs have embraced and accepted our values of openness, community, excellence, and user data privacy. arXiv is committed to these values and only works with partners that adhere to them.

Have an idea for a project that will add value for arXiv's community? Learn more about arXivLabs.

Full text under the publisher's open licence · 247 words

Related coverage