Validating Psychometric Questionnaires Using Experience-Sampling Data: The Case of Nightmare Distress

The Nightmare Distress Questionnaire (NDQ) is widely used to measure how much distress nightmares cause, but its validity remains debated. In an experience sampling study over 22 days with 92 participants reporting twice daily via smartphones, only two of the three proposed NDQ subscales—general nightmare distress and impact on sleep—showed both convergent and divergent validity. The third subscale, impact on daily reality perception, had unclear validity and was only indirectly linked to nightmare distress. Additionally, all NDQ items failed to distinguish between bad dreams and nightmares, suggesting the questionnaire may measure negative dreams in general rather than nightmares specifically.

Study at a glance

Abstract

Nightmares are a comparatively frequent phenomenon. They are often accompanied by emotional distress and gain clinical relevance when recurrent. To assess how much distress nightmares cause the individual, the Nightmare Distress Questionnaire (NDQ, Belicki, 1992) is probably the most often used measure. However, its validity is still disputed. To analyze the validity of the proposed three NDQ subscales in more detail, we conducted an experience sampling study, gathering data either in real-time or short retrospective timeframes over the course of 22 days twice per day (N = 92 participants). The measurements were implemented via a mobile app using participants’ own smartphones. Besides the dream quality, we assessed concepts on a daily basis that past research found to be related to dreams. These included critical life events, alcohol consumption, eating behavior, and well-being. We found that only the subscales “general nightmare distress” and “impact on sleep” showed convergent as well as divergent validity. The validity of the subscale “impact on daily reality perception” is unclear. If at all, this subscale is rather indirectly associated with nightmare distress. Furthermore, all of the NDQ items did not differentiate between a bad dream and a nightmare, which suggests that the NDQ might rather be a measure of negative dreams in general and not nightmares in particular. Based on the present experience sampling design, we propose to advance the validation process by further possibilities, such as an item-level, person-level, and multi-level approach. This approach seems to be especially fruitful for concepts which are not very salient (e.g., laughter), can hardly be remembered retrospectively (e.g., dream content), or are potentially threatened by recall biases (e.g., alcohol consumption).