New York, NY — Today the Child Mind Institute, in collaboration with Harvard University and the Massachusetts Institute of Technology, released a new research paper that found that AI tools cannot yet provide valid psychological assessments and require a more rigorous testing process. The paper reaffirms that mental health professionals are still best suited to assess and address the needs of people seeking mental health resources. Additionally, researchers introduced a framework for evaluating whether AI systems can measure mental health in a scientifically sound, reliable, and fair way for future development. The paper is in press at PLOS Digital Health.
Every day, people describe their inner lives in text — including crisis hotline messages, social media posts, therapy transcripts, journal entries, and more. As AI becomes increasingly capable of analyzing language, researchers are beginning to use these tools as a new way to conduct psychological assessments.
However, unlike traditional questionnaires used by psychologists, most AI language tools are insufficiently tested. Experts have spent a century building rigorous statistical standards for questionnaires within the field of psychometrics. Does a test measure what it claims? Does it give consistent answers? And does it work equally well for everyone? AI language tools are rarely held to that same standard.
In this paper, researchers introduce the concept of Text Psychometrics, a framework for evaluating whether AI and natural language processing (NLP) tools can measure psychological constructs in text in a scientifically sound way. Rather than asking only whether a model can match human judgments, the co-authors argue that researchers should also examine whether these tools capture the full range of how people express symptoms, remain reliable across different psychological situations, and work fairly across different patient populations.
"This research reaffirms that AI cannot replace mental health professionals to adequately assess and meet the needs of patients and people seeking mental health resources. An AI model may appear accurate while still missing important ways people express psychological symptoms," says Daniel Low, PhD, a research scientist at the Child Mind Institute and the study's lead author. "AI can help researchers and clinicians analyze language at scale and find patterns that would otherwise be very challenging, but these AI models do not currently understand the full approach that experienced clinicians take when making assessments. With our new framework, our goal is not to replace human expertise, but to build tools that are helpful while being more transparent, trustworthy, and scientifically grounded to meet the needs of patients."
To develop the paper and framework, researchers analyzed thousands of crisis counseling conversations and public social media posts. They compared several approaches for identifying mental health concerns, including large language models and other language-analysis methods. The team found that some AI systems performed well, on average, with commonly used accuracy measures. But they still failed to recognize many clear and important expressions of mental health concerns because they were less common and therefore did not affect the general accuracy. In other words, a model could appear successful while overlooking language that a clinician or researcher would consider highly relevant.
To address this issue, the researchers developed a new framework to evaluate language models that includes different types of validation, including what they call "content validity test sets." Rather than asking only whether a model agrees with human judgments, the approach examines whether the model recognizes the full range of ways people express a psychological symptom. For example, to screen for eating disorders, a model would need to not only capture mentions of anorexia and bulimia but also purging, chewing and spitting, and other symptoms.
"A mental health model should not be considered trustworthy simply because it performs well on a single metric," Dr. Low explains. "Researchers should also ask whether it captures the most important expressions of a symptom, performs consistently, protects privacy, and works fairly across different groups of people, among many other desirable properties we describe."
In a second study, the team examined whether language from the beginning of a crisis counseling conversation could predict who would later be sent an emergency service intervention towards the end of the conversation. The researchers found that language patterns associated with suicide risk could provide useful signals, while also revealing which factors out of about 50 were most strongly linked to acute risk. Mentions of lethal means for suicide, substance use and overdose, and having the texter write more than the counselor were the most predictive variables for detecting the need for an emergency service intervention.
Human clinicians consider far more than written words when evaluating a person's mental health, including clinical history, life circumstances, nonverbal cues — if available — and professional judgment. The findings suggest that AI systems are not yet at this level.
The researchers hope Text Psychometrics will help consolidate efforts to improve how AI-based mental health assessments are developed and evaluated. By applying the same rigorous standards used for traditional psychological tests, they believe language-based AI tools can become more reliable, interpretable, and useful for research, clinical care, and public health.
