Skip to Main Content
Follow us

Medical Education Research Grant — Recipients

2017

Ryan Brydges, PhD and
Jeffrey J.H. Cheung, PhD(c)

Co-Principal investigators

Competency by what design? Exploring the impact of integrated instruction on procedural skills transfer


Ryan Brydges, PhD
Scientist & Director of Research
Allan Waters Family Simulation Centre,
St. Michael’s Hospital


Jeffrey J.H. Cheung, PhD(c)
Research Fellow
The Wilson Centre, Toronto General Hospital and
University of Toronto

Abstract

Background:
Competency-based Medical Education (CBME) should be designed to develop trainees who perform expected clinical skills proficiently, and who also shift out of their routines to respond to changing clinical circumstances adaptively. One proposed framework for CBME design is to teach trainees to integrate competencies, however, there is little evidence for which instructional strategies prepare trainees to integrate effectively. Furthermore, few instructional strategies have been linked to improvements in learners’ ability to transfer their learning (i.e., adapt their skills). Researchers in clinical reasoning have established preliminary evidence on the benefits of instruction that integrates relevant types of knowledge (i.e., basic science and clinical features) for learning transfer and future learning outcomes.

Objectives:
To evaluate the effects of integrated instruction during simulation-based training on acquisition, retention, and transfer of lumbar puncture (LP) skills, as well as future learning of thoracentesis skill.

Methods:
Medical students (N=40) will be randomized into two groups for simulation-based LP training with one of two instructional videos. One video will contain a step-by-step demonstration of LP; the second will contain integrated instruction including the same procedural how demonstration plus relevant conceptual why explanations (e.g., relationship of steps to anatomy). Participants will complete an immediate LP post-test, and a one-week delayed transfer test. Participants will then complete a brief thoracentesis training session, followed by a final transfer test. We will develop written tests of LP procedural and conceptual knowledge to assess group differences and to evaluate the relationship of these test scores with performance test scores. Statistical analysis will include t-tests and Pearson’s correlations.

Conclusion:
Our study presents an opportunity to i) expand our list of instructional design features that facilitate integration, ii) test the theory of cognitive integration; and iii) provide empirical evidence to inform the teaching of integrated competencies.

How this work will inform research in medical education
Though competency-based medical education (CBME) frameworks consistently note the need for integrated and adaptive competencies, they do not clarify which instructional designs help trainees develop those competencies. In this project, we propose and test a mechanism we believe will help trainees integrate the foundational knowledge and skills that underlie clinical competencies. Specifically, we have designed videos to provide integrated instruction, which we expect will teach trainees the target skill (lumbar puncture), as well as integrated knowledge they can transfer to learn a novel, related clinical skill in a future training experience (thoracentesis). This work is important to medical education because it represents a program of research aimed at clarifying which instructional strategies support ‘cognitive integration’ and skill transfer. Moreover, the findings will have implications for educators and researchers seeking to develop curricula and instructional sessions that align with the aspirations of CBME frameworks.

Vicki Leblanc, PhD

The Role of Emotions in Clinical Reasoning and Recall


Vicki LeBlanc, PhD
Associate Professor and Chair, Department of Innovation in Medical Education, University of Ottawa Director, University of Ottawa Skills and Simulation centre at The Ottawa Hospital and University of Ottawa

Abstract

Objectives:
In caring for patients, clinicians and trainees are constantly confronted with emotional events. These events can trigger powerful emotions such as stress in situations of uncertainty, anger at perceived injustices or sadness at undesired patient outcomes. In many of these emotion-evoking situations, clinicians and trainees must obtain and interpret information, make judgments regarding treatment options with different benefits and risks, and remember important clinical information. The purpose of the present study is to investigate how different emotions influence medical trainees’ clinical reasoning and recall of a clinical encounter.

Design & Setting:
Twenty Anesthesia residents will manage two simulated clinical scenarios following induction of a target emotion; one in an “angry” condition and one in an “anxious” condition. We will compare their risk perception and preferences in terms of choosing a low resource (i.e. easier) but risky procedure over an approach that requires more resources (i.e. more difficult) but also has less risk. We will also measure their clinical performance and recall of relevant information following the scenarios.

Analysis:
As a manipulation check, the changes in the participants’ emotion self-report scores (post measures minus baseline measures) will be submitted to multivariate general linear model analyses, with mood manipulation (anger, anxiety) and scenario order (1st, 2nd) as independent variables. Participants’ risk judgments and preference scores, performance scores and memory recall scores will be submitted to multivariate general linear model analyses, with the scenario order and mood manipulation as independent variables.

Conclusion:
A greater understanding of the effects of emotions on clinical reasoning and recall is instrumental in preparing health professionals to identify situations in which emotional regulation is called for, as well as identify situations where systems or team-based strategies could reduce unwanted biases that result from the emotional states of health providers.

How this work will inform research in medical education
In caring for patients, clinicians and trainees are constantly confronted with emotional events. These events can trigger powerful emotions such as anxiety in situations of uncertainty, anger at perceived injustices, or sadness at undesired patient outcomes. In many of these emotion-evoking situations, clinicians and trainees must obtain and interpret information, make judgments regarding treatment options with different benefits and risks, and remember important clinical information. The study will investigate how different emotions (anxiety, anger) influence residents’ risk perceptions, clinical performance and recall of relevant information in simulated clinical encounters. A greater understanding of the effects of emotions on clinical reasoning and recall is instrumental in preparing health professionals to identify situations in which emotional regulation is called for, as well as identify situations where systems or team-based strategies could reduce unwanted biases that result from the emotional states of health providers.

Carmen L. Mueller, BSc(H) MD FRCSC MEd FACS

Reciprocal Peer Mentorship for Practicing Surgeons – Modernization of Continuous Professional Development in Surgery


Carmen L. Mueller, BSc (H), MD, FRCSC, Med, FACS
Assistant Professor of Surgery, McGill University

Abstract

Background:
Intensive practice with feedback is crucial to mastery learning and forms a core component of surgical training. However, after training is completed, live, in-situ feedback and coaching opportunities abruptly cease and surgeons in practice are left to refine their skills largely in isolation for the remainder of their careers. Available continuous professional development (CPD) opportunities most commonly take the form of weekend courses, presentations by experts at conferences, video sharing platforms, and peer-reviewed literature: modalities that are expensive, sporadic, remote, provide limited longitudinal learning opportunities and generally offer little if any meaningful feedback. Presently, no formalized program exists to allow for cross-pollination of skills and techniques amongst surgical peers.

Objectives:
(1) To determine the perceived need for, attitudes towards and potential barriers to reciprocal peer mentorship as a means of longitudinal skill refinement for practicing surgeons; (2) To develop and evaluate a reciprocal peer mentorship pilot program for practicing surgeons

Design:
This study will include (a) knowledge synthesis to define the present use of formalized peer mentorship for practicing surgeons; (b) semi-structured interviews using grounded theory methodology and (c) nation-wide survey data to determine attitudes towards peer mentorship of practicing surgeons; and (d) a pilot study to test the feasibility and learning outcomes of such an intervention.

Setting:
The study will be conducted at a university-affiliated hospital but will include surgeons representative of diverse practice settings and subspecialties in Canada.

Statistical Analysis:
Qualitative data will be analyzed according to grounded theory methodology. Quantitative data will be analyzed using descriptive and inferential statistical methods.

Conclusion:
This research program will identify barriers and facilitators of reciprocal peer mentorship for surgeons in practice, a first step in this novel approach that has the potential to greatly impact CPD across all surgical specialties.

How this work will inform research in medical education
Currently, surgeons engage in solitary practice with few opportunities for mentorship and feedback for the duration of their careers. This project seeks to determine attitudes towards, barriers to, and the impact of a reciprocal peer mentorship model for practicing surgeons. Similar to an "exchange program", a reciprocal peer mentorship framework will be implemented to help overcome traditional barriers that prevent surgeons from scrubbing with and learning from each other, even within their own hospitals or departments. In addition to facilitating ongoing surgical skill refinement and the adaptation of new techniques in an era of rapid change, the reciprocal peer mentorship model is expected to help engender a greater culture of cooperation and collaboration between colleagues. It is anticipated that the outcomes of this study will contribute to a paradigm shift in the way surgeons presently hone their skills, adapt to practice changes and adopt new technology over time.

Geoffrey Norman, PhD

Diagnostic Errors: Knowledge Deficit vs Cognitive Bias and Checklists


Geoffrey Norman, PhD
Program for Educational Research and Development, McMaster University

Abstract

Background:
There is an ongoing debate regarding the cause of diagnostic errors. One view is that errors result from unconscious application of innately flawed cognitive heuristics; the alternative is that errors are primarily a consequence of knowledge deficits.

Objective:
To directly compare the effectiveness of checklists based on a) cognitive biases, b) clinical features, in reducing errors in ECG interpretation.

Design:
Three group randomized controlled trial. All residents will have initial instruction related to reviewing ECGS. Group 1 will receive additional instruction related to the role of specific cognitive biases and the use of a checklist identifying biases. Group 2 will receive instruction related to systematic reading of ECGs and use of a feature-based checklist. Group 3 is a no-intervention control. All will be tested with a set of 20 new ECGs, which will be reviewed twice – an initial rapid read and a second review using the group-specific checklist (or none).

Setting:
Postgraduate training programs in emergency and internal medicine, using a total of 60 residents and 30 faculty in Hamilton, Canada, Seattle, USA and Rotterdam, Netherlands.

Outcomes:
Overall accuracy on a new set of 20 ECGS, before and after application of a checklist. Secondary outcome is performance on a subset of 6 ECGs specifically designed to illustrate 2 cognitive biases.

Implications:
Findings have major implications for strategies to reduce diagnostic errors.

How this work will inform research in medical education
Diagnostic errors are a major cause of morbidity and mortality, and have become the focus of much writing in recent years. The dominant view is that errors are a consequence of innate cognitive biases that are a consequence of the nature of human cognition. In turn, there have been many calls to mount courses to educate students about cognitive biases. Surprisingly there is very little evidence of effectiveness. Although over 100 biases have been described, only a handful of studies examined the relationship between expertise and biases, with conflicting results. Moreover, only a few studies have examined the impact of bias teaching on errors; all are negative.

The alternative is a focus on applying the appropriate knowledge by reminding physicians of condition-specific knowledge (diagnoses, signs, tests) they should consider. This study is the first direct test of the two approaches, contrasting a cognitive bias checklist with a condition-specific checklist. The results have major consequences for error reduction strategies.

Nha Voduc, MD, FRCPC

Measuring Development of Competency with the Ontario Bronchoscopy Assessment Tool


Nha Voduc, MD, FRCPC
Associate Professor and Respirology Program Director Faculty of Medicine, University of Ottawa

Abstract

Bronchoscopy is a widely performed diagnostic and therapeutic procedure. Competency in bronchoscopy is a requirement for many residency programs, yet there is virtually no evidence on which to base educational curriculae for bronchoscopy. Basic questions such as the amount of training or experience required to become competent in bronchoscopy have not been answered.

Previously, we developed, validated and published the Ontario Bronchoscopy Assessment Tool (OBAT), the first and only competency-based performance assessment tool for bronchoscopy.

For this study, we will use the OBAT to prospectively follow the development of competency in bronchoscopy, in respirology trainees from multiple academic centres. Trainees will be enrolled at the beginning of their subspecialty training (PGY-4). Their bronchoscopy performance will be scored by the supervising physicians, using the OBAT. Our primary objective is to track OBAT scores during the course of training to develop bronchoscopy learning curves, which estimate the mean, upper and lower quartile OBAT scores over the number of procedures. The curves will allow us to derive the median number of procedures required for trainees to become competent in bronchoscopy as well as provide us with a better understanding of the variation in the development of competency in this procedure. Our secondary objectives will be to identify trainee and training factors which may influence the development of competency and identify which aspects of bronchoscopy are most challenging to learn.

How this work will inform research in medical education
The current focus on competency by design by the Royal College has highlighted the need for a better understanding of the training requirements for attainment of procedural competency. This work is important to medical education because it represents the first attempt to determine the learning curve for bronchoscopy, using a validated, competency based assessment tool.

By determining the learning curve for bronchoscopy, our study will provide training programs with the first ever evidence on which to base their bronchoscopy training curriculum. The learning curve will also serve as a benchmark against which individual trainee performance can be compared.

The multicenter and clinically-based design of our study represents a unique feature for medical education studies in the Respirology field. If successful, our study design can serve as a template for future studies for other medical procedures.

2016

Joanna Bates, MDCM, CCFP, FCFP

Gisèle Bourgeois-Law, MD, FRCSC, MEd (PhD student, Maastricht University)

Kathy Boutis, MD, MSc, FRCPC

Kevin Eva, PhD

Joanne Goldman, PhD

Sandra Monteiro, PhD

Dominique Piquette, MD, MSc, MEd, PhD

Akshay (Shay) Seth, MD MAEd(c)

Kong Eric You-Ten, MD, PhD, FRCPC and Sev Perelman, MD, CCFP-EM

Bin Zheng, MD, PhD