Ecological Momentary Assessment as a Measure of Intervention Change: Evaluation in 4 Digital Mental Health Trials

Abstract

Ecological momentary assessment (EMA) is increasingly being incorporated into intervention studies to acquire a more fine-grained and ecologically valid assessment of change. The added utility of including relatively burdensome EMA measures in a clinical trial hinges on several psychometric assumptions, including that these measure are: (1) reliable, (2) related but not redundant with conventional self-report measures (convergent and discriminant validity), (3) sensitive to intervention-related change, and (4) associated with a clinically-relevant criterion of improvement (criterion validity) above conventional self-report measures (incremental validity). Using data from 4 app-based meditation trials (N = 412), we examined the reliability, validity, and sensitivity to change of conventional self-report and EMA measures of improvement in rumination. Conventional self-report and EMA measures of rumination were only modestly correlated, particularly with regards to change over time, which may be due to their lower reliability. Changes in rumination were larger for conventional self-report than EMA. Notably, change in both self-report and EMA rumination each accounted for unique variance in depressive symptom improvement, demonstrating incremental predictive validity. Conventional self-report and EMA measures of rumination provide distinct and clinically meaningful information. Researchers using EMA should consider the psychometric properties of their measures and the precise construct they intend to capture.