The Reliability of Rating via Audio-Recording Using the Mindfulness-Based Interventions: Teaching Assessment Criteria.
Erin Floyd, Shelley R Adler, Rebecca S. Crane, Judson A. Brewer, Patricia J Moran, Robert Richler, Wendy Hartogensis, Willem Kuyken, Frederick M Hecht
Global Advances in Integrative Medicine and Health 2023 DOI: 10.1177/27536130221149966 (opens in new tab)
Study at a glance
AI-extracted from the abstract| Characteristics | Qualitative and quantitative reliability study Peer reviewed |
|---|---|
| Sample size | 21 |
| Population | Mindfulness-Based Stress Reduction teachers whose video recordings had been previously rated |
| Measures | Mindfulness-Based Interventions: Teaching Assessment Criteria (MBI:TAC), intraclass correlation coefficients (ICCs), Bland-Altman plots |
| Topics | Meditation |
| Keywords | Mbi Intervention fidelity Intervention integrity |
| Key findings | Inter-rater reliability of MBI:TAC ratings from audio-only recordings was adequate for many research and clinical purposes when scores were averaged across 3 evaluators (ICCs .53 to .69), but lower for single ratings (.27 to .38). Audio ratings showed little consistent bias relative to video and agreed more closely for higher-rated teachers. Assessors indicated audio-only rating may be more challenging for less experienced teachers. |
Abstract
The Mindfulness-Based Interventions: Teaching Assessment Criteria (MBI:TAC) is an important tool for assessing teacher skill and aspects of the fidelity of mindfulness-based interventions, but prior research on and implementation of the MBI:TAC has used video recordings, which can be difficult to obtain, share for assessments, and which increase privacy concerns for participants. Audio-only recordings might be a useful alternative, but their reliability is unknown. To assess evaluator perception of the rating process and inter-rater reliability of MBI:TAC ratings using audio-only recordings. We prepared audio-only files from video recordings of 21 previously rated Mindfulness-Based Stress Reduction teachers. Each audio recording was rated by 3 trained MBI:TAC assessors drawn from a pool of 12 who had previously participated in rating the video recordings. Teachers were rated by evaluators who had not viewed the video recording and did not know the teacher. We then conducted semi-structured interviews with evaluators. On the 6 MBI:TAC domains, the intraclass correlation coefficients (ICCs) for audio recordings ranged from .53 to .69 using an average across 3 evaluators. Using a single rating resulted in lower ICCs (.27-.38). Bland-Altman plots showed audio ratings had little consistent bias compared to video recordings and agreed more closely for teachers with higher ratings. Qualitative analysis identified 3 themes: video recordings were particularly helpful when rating less skillful teachers, video recordings tended to provide a more complete picture for rating, and audio rating had some positive features. Inter-rater reliability of the MBI:TAC using audio-only recordings was adequate for many research and clinical purposes, and reliability is improved when using an average across several evaluators. Ratings using audio-only recordings may be more challenging when rating less experienced teachers.
Comparable studies
Other qualitative studies on meditation, most cited first.
Citations in the library
Cited by 3
- Intervention Fidelity in Mindfulness-Based Research and Practice: Overview of the Special Collection and State of the Field. Global Advances in Integrative Medicine and Health 2025
- Combining mindfulness and cognitive training in children with attention deficit hyperactivity disorder: study protocol of a pilot randomized controlled trial (the NeuroMind study). Frontiers in Psychology 2024
- Teaching Mindfulness-Based Programs online: perceived costs and benefits. BMC Psychology December 9, 2025
Cites 5
- Efficacy of Mindfulness-Based Cognitive Therapy in Prevention of Depressive Relapse JAMA Psychiatry April 28, 2016
- Competence in Teaching Mindfulness-Based Courses: Concepts, Development and Assessment Mindfulness September 21, 2011
- Intervention Integrity in Mindfulness-Based Research. Mindfulness 2018
- Mindfulness-based stress reduction and health benefits. A meta-analysis. Journal of Psychosomatic Research July 1, 2004
- Effect of Mindfulness-Based Stress Reduction vs Cognitive Behavioral Therapy or Usual Care on Back Pain and Functional Limitations in Adults With Chronic Low Back Pain: A Randomized Clinical Trial. JAMA