Mindfulness-Based Teaching Competency Assessment: Comparing Self-Assessment With Expert Evaluation in the Veterans Administration - Compassionate Awareness Learning Module Program.
J Greg Serpa, Stephen R Shamblen, Kathy Atwood, Aree Sangpukdee, Alison Whitehead, Christiane Wolf
Global Advances in Integrative Medicine and Health 2023 DOI: 10.1177/27536130231197654 (opens in new tab)
Study at a glance
AI-extracted from the abstract| Characteristics | Observational study Peer reviewed |
|---|---|
| Sample size | 39 |
| Population | VA clinicians participating in a national MBI training program |
| Measures | Mindfulness-Based Intervention: Teaching Assessment Criteria (MBI:TAC) |
| Topics | Meditation |
| Keywords | Veterans Whole health |
| Key findings | Self-evaluation scores on the MBI:TAC were not significantly correlated with expert rater scores, and participants rated their competence higher than experts did. Inter-rater reliability between two experts was significant for Guiding Practice (ρ = .83, P = .003) but not for Embodiment of Mindfulness (ρ = .34, P = .186). The authors suggest the MBI:TAC may not produce accurate scores when used for self-assessment and that senior teacher instruction is needed for accurate scoring. |
Abstract
Meditation, including Mindfulness-Based Interventions (MBI), is a required Complementary and Integrative Health intervention at the US Department of Veterans Affairs (VA). Training VA clinicians to provide MBI at scale must address fidelity concerns and the assessment of clinician competency. The psychometric properties of the Mindfulness-Based Intervention: Teaching Assessment Criteria (MBI:TAC), a widely used tool for assessing facilitator competence, continue to be explored. To support the dissemination of MBI, the utility of using the MBI:TAC for self-assessment for clinicians in a national training program was evaluated. In a training cohort of VA clinicians (n = 39), participant self evaluations on 2 domains of the MBI:TAC are compared to the competency scores of 2 expert evaluators as based on the observations of a 10-minute exercise. Additionally, the inter-rater reliability between the 2 experts was explored. Intraclass Correlation for the 2 expert evaluators for Guiding Practice was significant (ρ = .83, P = .003), but was not significant for Embodiment of Mindfulness (ρ = .34, P = .186). Self-evaluation scores were not significantly correlated to expert rater scores such that participants rate their level of competence higher than expert scores. The MBI:TAC, while an essential tool in teacher training, may not produce accurate scores when used for self-assessment. Instruction from a senior teacher is needed for accurate scoring. Interrater reliability may be improved with enhanced domain operationalization and training. Implications for MBI training are explored.
Comparable studies
Other observational and cohort studies on meditation, most cited first.
| Study | Year | Design | Participants |
|---|---|---|---|
| Using Self-Report Assessment Methods to Explore Facets of Mindfulness Undergraduate students | 2006 | Observational cohort | |
| Effectiveness of a meditation-based stress reduction program in the treatment of anxiety disorders Patients with generalized anxiety disorder or panic disorder with or without agoraphobia | 1992 | Observational cohort | n = 22 |
| Meditation experience is associated with increased cortical thickness Participants with extensive Insight meditation experience and matched controls | 2005 | Observational cohort | n = 20 |
| Meditation experience is associated with differences in default mode network activity and connectivity Experienced meditators and matched meditation-naive controls | 2011 | Observational cohort | |
| Long-term meditators self-induce high-amplitude gamma synchrony during mental practice Long-term Buddhist practitioners and controls | 2004 | Observational cohort |
Citations in the library
Cites 8
- Prospects for a clinical science of mindfulness-based intervention. American Psychologist October 1, 2015
- Dispositional mindfulness and depressive symptomatology: Correlations with limbic and self-referential neural activity during rest. Emotion 2010
- Trends and Developments in Mindfulness Research over 55 Years: A Bibliometric Analysis of Publications Indexed in Web of Science Mindfulness July 16, 2021
- Implementing the Mindfulness-Based Interventions; Teaching Assessment Criteria (MBI:TAC) in Mindfulness-Based Teacher Training Global Advances in Health and Medicine February 25, 2021
- Can We Agree What Skilled Mindfulness-Based Teaching Looks Like? Lessons From Studying the MBI:TAC. Global Advances in Health and Medicine October 13, 2020
and 3 more in the library