Reading between the lines: Combining pause dynamics and semantic coherence for automated assessment of thought disorder.

Pause dynamics in speech—silent intervals between utterances—add unique information beyond semantic coherence for assessing formal thought disorder (FTD) in schizophrenia spectrum disorders. Across three datasets (naturalistic diaries, picture descriptions, and dream narratives; total 255 participants), models combining pause features with semantic coherence predicted clinician-rated FTD severity more accurately than coherence alone. Late fusion of both feature types raised average Spearman correlation from 0.413 to 0.455. The most informative pause patterns varied by context, indicating that pause dynamics and semantic coherence capture complementary aspects of thought disorganization. This multimodal framework offers a scalable, automated approach to assessing disorganized speech.

Study at a glance

Abstract

Formal thought disorder (FTD), a hallmark of schizophrenia spectrum disorders, manifests as incoherent speech and poses challenges for clinical assessment. Traditional clinical rating scales, though validated, are resource-intensive and lack scalability. Automated speech recognition (ASR) allows for objective quantification of linguistic and temporal features of speech, offering scalable alternatives. Furthermore, ASR-derived utterance timestamps provide access to pause dynamics, which are thought to reflect the cognitive processes underlying speech production. Yet, their added value beyond semantic measures remains insufficiently explored. In this study, we evaluated a scalable multimodal framework that integrates pause features with semantic coherence metrics across three datasets: naturalistic self-recorded diaries (AVH, n = 140 participants), structured picture descriptions (TOPSY, n = 72 participants), and dream narratives (PsyCL, n = 43 participants). Pause-related features were evaluated alongside established coherence measures using support vector regression (SVR) to predict clinical FTD scores. Models using pause features alone robustly predict manually rated FTD severity consistently across datasets. Integrating pause features with semantic coherence metrics enhanced predictive performance compared to coherence-only models, with late fusion yielding the most robust and consistent gains in all three datasets. On average across datasets, Spearman correlation increased from ρ = 0.413 for semantic-only models to ρ = 0.455 with late fusion. The performance gains from semantic and pause features integration held consistently across all contexts, though the nature of the most informative pause patterns was dataset-dependent. These findings suggest that both pause dynamics and semantic coherence reflect complementary aspects of thought disorganization. Our integration framework provides a scalable approach for refining the assessment of disorganized speech to advance automated speech analysis in psychosis.