LLMs for Predicting Identity Diffusion from Clinical Interviews

Abstract

Introduction

Identity diffusion—the fragmented, polarised, and unstable sense of self and others maintained by the primitive defence of splitting—is central to borderline personality organisation [1]. Object Relations Theory (ORT) proposes that individuals who rely on splitting oscillate between perceiving themselves or others as “all good” or “all bad.” Colibazzi et al. [2] showed that such polarised appraisals leave measurable traces in the emotional valence of words used in natural speech.

The Structured Interview of Personality Organization (STIPO) offers clinicians a theory‑grounded tool for assessing identity pathology, yet each interview demands time, expertise, and retains some subjectivity. Automating the analysis of interview responses could supply a complement to expert judgment and provide new insights into how identity pathology is expressed through language.

Past Work

Replicating Colibazzi’s [2] approach on a new dataset of German STIPO identity‑section transcripts (N = 73 adolescents, 13–19  years), we applied word‑level, lexicon‑based sentiment analysis to participants’ replies. We found strong correlations between extracted metrics—such as mean valence and the standard deviation of valence—and the clinician‑assigned pathology severity score on the same STIPO section. This proof of concept demonstrates that linguistic markers of splitting can be captured, but focuses only on individual words, ignoring context.

Objective & Hypotheses

The present study therefore asks whether more advanced context-aware tools could be used for the same application. Specifically, we test whether the Large Language Model (LLM) GPT‑4, prompted with well designed prompts, can approximate clinician scores of identity pathology in the same material. We hypothesise that (H1) GPT‑4 severity estimates will correlate positively with clinician scores and (H2) a regularised regression model trained on GPT‑4‑extracted features will predict identity pathology severity better than chance.

Methodology 

Transcripts from the same 73 interviews will be fed to GPT‑4 via iteratively refined prompts informed by ORT, the STIPO manual, and feedback from experienced interviewers. Prompts will instruct the model to quantify diffusion–related constructs—such as overall emotional valence, shifts in emotional polarity, and narrative contradictions—on numerical scales. Model outputs will be correlated with scores of identity pathology assigned by clinicians during the interviews, using Pearson’s r for the 15-item identity sum score (range: 0–30) and Spearman’s ρ for the 5-point overall identity score. Predictive performance of ridge and lasso regression models trained on GPT‑4‑derived features will be evaluated with leave‑one‑subject‑out cross‑validation.

Expected Results

We anticipate medium‑to‑large correlations (r ≈ 0.50) between GPT‑4 metrics and clinician scores. Success would demonstrate the viability of large language models as adjunct scoring aids for semi‑structured clinical interviews and provide empirical support for psychoanalytic constructs, such as splitting, that are otherwise difficult to quantify.

References 

[1] O. F. Kernberg, Severe Personality Disorders: Psychotherapeutic Strategies. New Haven, CT, USA: Yale Univ. Press, 1984.

[2] T. Colibazzi et al., ‘Identifying Splitting Through Sentiment Analysis’, Journal of Personality Disorders, vol. 37, no. 1, pp. 36–48, Feb. 2023, doi: 10.1521/pedi.2023.37.1.36.