Follow us

Competence committees: A critical component of competency-based assessment

Inherent in the new Competence by Design (CBD) model is a philosophical shift from “assessment of learning” to “assessment for learning”; in CBD learning is much more personalized and it’s guided by regular, real-time, quality feedback to the resident.

Competency-based assessment in CBD is focused on Entrustable Professional Activities (EPAs) observed in the workplace, with frequent, ‘small stakes’ assessments for learners. Over time, multiple observations by multiple observers will provide a comprehensive image of a physician’s competence and eventually inform promotion decisions.

Resident promotion decisions: The current model vs a competence committee model

The current PGME system in Canada is based on the assumption that the more time a learner spends on an activity, the more the learner absorbs and excels. Generally a resident, who has completed the right rotations, is assumed to have achieved competence and progresses on a yearly basis, unless there are glaring gaps in performance requiring remediation. Evidence suggests however that not all learners achieve mastery at the same rate. For this reason, in the CBD model, leaners will progress through their residency education program at their own pace. With regular feedback and coaching each resident will achieve the EPAs and related milestones within their current stage of training, within a predictable training timeframe.

In CBD, a competence committee uses the EPA’s as a framework to determine that each resident has achieved the appropriate level of competence before moving to the next stage (i.e. being promoted). Promotion decisions on EPA achievement are determined away from the individual teacher-learner interaction, and within the group decision-making process of a competence committee.

The role of a competence committee

The role of a competence committee is to review and make decisions on a learner’s achievement of EPAs and their progression through the stages of training toward the national standards as set by the discipline. The competence committee also provides guidance for training activities to help a resident progress within his/her current stage. Committee decisions and recommendations are made using highly integrative data from multiple EPA and milestone observations as well as feedback from clinical practice.

NOTE: Terms of Reference and Procedures for the committee are determined at a faculty/program level with guidance from the Royal College.

Competence Committee Guidelines

The Royal College and the Postgraduate Deans’ Assessment Advisory Working Group have developed a set of guidelines, which can be used to support local development of competence committees. The guidelines are not mandatory, they were developed in recognition of the fact that some people prefer to have a model upon which to base their efforts. Those who choose to use the guidelines are welcome to do so, but it’s important to emphasize that competence committees must be further defined and implemented at a local scale. If you have questions about committee responsibilities, composition and structure you may find these Guidelines useful;

How would a competence committee work?

There is no single way to set-up and run a competence committee. Different contexts and cultures within programs and institutions mean that there is no one-size fits all approach. For CBD, the Terms of Reference and Procedures for the competence committee are determined at a faculty/program level with guidance from the Royal College.

Guideline Highlights:

  • The Residency Program Committee has such a broad mandate, consider making your Competence Committee a subcommittee of the RPC, or even a separate committee with members from outside the RPC. You may well find that the focused discussion is more efficient and effective.
  • Consider making the size of the committee proportionate to the size of the program (e.g. approximately 5 members for 50 residents).
  • Meet regularly (monthly, quarterly) to discuss specific residents (i.e. not all residents at every meeting)
  • Assign members a limited number of resident files to review to ensure members have time to do a thorough review and to lead a meaningful discussion
  • Video (04:24) on setting up competence committees

Busting the Myths on Competence Committees

In periods of change, some uncertainty is to be expected. Test your current knowledge about competence committees and help bust the myths when you get the opportunity!

True or False?

There is only one way to structure and run a competence committee.


False - There are many ways to structure and run a competence committee!

+ More

- Less

The Royal College has developed a set of suggested guidelines and proposed terms of reference so that institutions have an optional point of reference from which to start.

There is no one-size-fits-all solution – programs will tailor the guidelines to work for the realities of their unique situations.

Competence committees will replace Residency Program committees.


False - Residency Program Committees already have an important and broad mandate. In many cases, it is impractical to expect the RPC to also assume direct responsibility for the review of resident portfolios. For this reason, we’re suggesting that programs set up a competence committee as a subcommittee of the Residency Program Committee or even a separate committee with members from outside the RPC.

+ More

- Less

A competence committee will have a focused mandate (i.e. to review resident assessment data and make progression decisions).

By allocating time that is dedicated to the review of resident progress, which is separate from the broader RPC activities, a program can be sure that its residents get the focused attention they need and deserve.

The size of your competence committee will likely be proportionate to the size of your program. In some very small programs there may be insufficient faculty to have both a Residency Program Committee and a competence committee. In these cases, the programs may schedule additional meetings of the RPC and then dedicate or protect those meeting to focus on competence committee work.

Program Directors should not participate in competence committee meetings.


False - Program Directors are encouraged to be members of the competence committee. Program Directors have valuable insight on their trainees, which can provide valuable context for competence committee discussions.

+ More

- Less

Ideally the competence committee is chaired by someone other than the Program Director. As a member (not chair) of the competence committee, the Program Director can fully participate in the discussions and not be hampered by the need to wear multiple hats during the discussion.

Also, as Chair of the Residency Program Committee, it’s the Program Director’s role to receive the results from the competence committee, facilitate a ratification vote, and advocate for systemic issues in the program overall. Having different leadership between the committees allows for a shared workload and greater clarity in roles.

A competence committee may use individual meetings to focus on specific residents.


True - It’s unlikely that a competence committee will review all trainees every meeting, however, it is expected that each trainee will be reviewed at least twice a year. Those residents that are progressing faster or slower than expected may be discussed more often, to facilitate and support their training.

+ More

- Less

Competence committees will certainly look for those residents that are falling behind, not to be punitive, but to arrange support and find creative ways to coach them to progress (e.g. assigning special mentors, extra readings, or modified rotations).

Competence committee work will be onerous.


False - Each competence committee member will typically be assigned a small number of resident files to review prior to a committee meeting.

+ More

- Less

The size of your program, how often the committee meets and the way the resident files are stored in your program will determine the intensity of the preparations.

One bad observation can prevent a resident from advancing.


False - No promotion decision will ever rely on a single observation, but with that said residents and observers need to be comfortable with the idea that it will take time for residents to develop the competencies inherent in an EPA (i.e. there will be many “bad” observations along the way).

Residents are smart people and they’re used to getting great results. Part of the change ahead is helping residents understand that it is not only ‘normal’ but expected that they will not meet the standards immediately. The goal is to get them there as quickly as possible through coaching that is based on observation and honest feedback.

In many ways, the development of a competence committee is expected to help facilitate the culture change needed to transition effectively to CBD.

+ More

- Less

Promotion decisions are determined using multiple observations, made by multiple observers, in multiple contexts. Together, these observations comprise a comprehensive image of a physician’s competence.

Observations made for an EPA are made by a single observer, at one point in time, in a specific context. The entire EPA does not need to be assessed at one time. They’re often short, formative assessments where feedback provided on various components of an EPA (milestones) will inform the ultimate tracking of the acquisition of the EPA and allow academic advisors/competence committees to see the development of the EPA being achieved.

Each observation should be followed by timely, specific, constructive feedback designed to help learners improve. The feedback residents get has the potential to contribute to their learning. As residents adjust to CBD, they will understand that “in-progress” is the new norm and, as a result, they will expect and appreciate feedback. Residents should not think of faculty as judges; instead they should see you as coaches who are there to help them improve their ‘game’.

The collection of many assessments on a resident over time, will allow the competence committee to evaluate how a learner is progressing and when they are ready to move on to the next stage of training.

There are resources available to make this easier.


Resources

If you have questions or need support please contact us at cbd@royalcollege.ca. We want to hear from you.