The faces of CBD
Early insights: regular, direct observations
Rob Anderson, MD, FRCPC
Anesthesiology Program Director
Northern Ontario School of Medicine
Clinician Educator Workplace Based Assessment Committee,
Andrew Shanmugarajah, MD
Northern Ontario School of Medicine
With the first two disciplines launching their Competence by Design (CBD) programs last July and six more ready to launch this coming July, we offer a glimpse into one of the new aspects of the hybrid competency-based medical education system — regular, redirect observations.
For this issue of CBD Community Touchpoint, we wanted to look at both sides of this CBD component. We talked with Residency Program Director Rob Anderson and PGY1 resident Andrew Shanmugarajah about their experiences with regular, direct observations in the Department of Anesthesia at the Northern Ontario School of Medicine. Here is what they had to say.
What do you see as the real advantage of regular, direct observations in CBD?
Increasing the number and refining the structure of direct observations combined with coaching feedback in the CBD paradigm leads to an enhanced learning trajectory for all learners. It seemed unfair to me that residents who were struggling had to have remediation, often meaning increased and focussed direct observation and feedback. If it improves the learning of those who are struggling, why not give this advantage to all learners, regardless of where they are on the spectrum? Today’s residents yearn for more frequent, specific and actionable feedback than ever before. CBD can respond to that need.
Regular, direct observations have allowed each bit of feedback I have received to be specific and actionable. Instead of a global assessment of my performance for a given day, week, or month, I have received focused direction daily. Being able to follow my own progress and being asked to reflect on my learning needs has also given me some ownership and control over my clinical learning.
Describe your role in CBD observations.
As a program director in the early days of CBD, my role has been to help create, structure and revise the observations; provide faculty development on accessing the forms and giving high quality feedback; provide resident development on “gently teaching faculty” how to play in a CBD world; and guide the use of the observations. A key aspect of CBD observations is the “not yet competent” ones and providing coaching feedback. This is vital to learner growth. Residents should want to get coaching feedback before becoming competent as much as to document competence. Competence committees need to look favourably on the assessment for learning. This is a mindset change for all and an important part of my job is supporting that change.
As a resident, my role is to determine what observations and (Entrustable Professional Activities) EPAs I need and look at my academic schedule to see how I may achieve them. The onus is also on me to be proactive each day and tell my preceptors which EPAs I would like to be assessed on. I am also expected to develop a learning plan, shared with the competency committee, to ensure that my priorities are appropriate and that I am continuing to maximize the results of my training.
What challenges did you encounter?
Working with outside groups, such as the off-service rotations that are just learning about CBD, can be a challenge, but always achievable. There are certainly glitches along the way. Changing technology, evolving definitions and assessment strategies, creating a competence committee and not an evaluation committee, defining the role of simulation and determining finances are all hurdles that we are struggling with. Having a clear vision and the supportive team, locally and nationally, has made them just problems to be solved, not impassible barriers.
One of our biggest ongoing challenges is collecting observations while on off-service rotations. Certain EPAs are mapped to those rotations, but the transition has been difficult, especially in rotations where you are not interacting regularly with a staff physician.
Did you encounter any surprises?
For me, the biggest surprise is the robust and thoughtful conversations at the competence committee. We have had an evaluation committee for a long time, looking at flagged evaluations and supporting the struggling residents. Our competence committee does an in-depth review of each resident. The collective wisdom and experience have been keys to providing support and feedback to all the residents. I don’t worry that someone may be falling behind or struggling, and we didn’t catch them in time to help them. It has been a very positive experience despite the added workload. It feels like valuable and important work!
I’ve been surprised by how much some preceptors have bought into the CBD assessment strategies, actively thinking about EPAs I can accomplish or filling out assessments on observed skills without my asking them. Their continued efforts have made the transition a lot easier than I expected it to be when I started my residency.
What is important to keep in mind about CBD observations?
The single most important thing is that the intent of observations is to facilitate learning and development, not just to document it. If we are afraid of giving coaching feedback or marking below independent, if the competence committee looks at below competent observations as a concern, if residents don’t ask for observations before they feel they can do it independently … then we risk falling back to just documenting competence and flagging fails. Then we missed adding the value associated with this change.
The observations are meant to reflect residents’ growth. I will still sometimes lose sight of this because it can feel uncomfortable being assessed when you are not yet competent. However, waiting until I feel competent to ask for an assessment defeats the purpose of CBD and wastes opportunities to enhance my growth earlier in my training.
If you could tell physicians in another medical specialty one thing about preparing for CBD observations, what would that be?
I would say the same thing I was told by one of the CBME leads in the Ortho program at U of T. Just get started. Pick two or three aspects to observe that seem easiest and adapt assessments for them that have already been invented. Engage a few leads who will do a great job and start doing some observations. Don’t get too set in your ways, as things will change a lot along the path to the new norm. You can learn a lot from just doing a few, and you can apply that to the rest of your journey.
From my experience, it requires significant buy-in from learners and educators. It is a new style of assessment, requiring a new mindset from both parties. Learners must become comfortable being assessed before they are competent, and assessors must feel comfortable assessing someone as temporarily incompetent. It provides opportunities for teaching earlier and reveals a resident’s growth over time. The process is also enhanced if learners and educators are proactive about knowing what EPAs exist and recognizing opportunities for assessments as they arise.