Is Competence by Design Working?

The role of Program Evaluation

The implementation of CBD requires considerable time and resources from many different stakeholders in medical education and can have significant impacts; therefore it is important to evaluate it on an ongoing basis, and ensure it is having the desired impact. Program evaluation also signals that we are accountable as an organization to this initiative.

More importantly, it is incumbent upon the Royal College to ensure that implementation of a new educational model does not lead to any harmful effects. First and foremost is considering the impact on patient care, but also important is monitoring the impact on learners, teachers and other stakeholders involved in residency education.

Program evaluation is also important from a scholarly perspective – CBME is increasingly being adopted on a global basis, and there is a desire to share what is being learned as the Royal College implements CBD. Systematically evaluating and disseminating the findings from CBD will help us all learn together.

Recent reports: Understanding the Royal College approach to CBD Program Evaluation


CBD Program Evaluation Recommendations Report

This report summarizes the challenges and recommendations found in the 2019 program evaluation studies conducted separately by the Royal College and the Fédération des médecins résidents du Québec (FMRQ). Actions that the Royal College has taken or will take to address these recommendations are identified.

CBD Implementation Pulse Check

These reports outline the findings of studies conducted by the Royal College to understand how CBD implementation is going on the ground, as well as benefits, challenges, advice for moving forward, and any early outcomes. It primarily studies fidelity and integrity of implementation to monitor trends and inform necessary adjustments. The Pulse Check is conducted 6 months post-launch for each cohort, and annually thereafter.

2019 Annual Report
2019 Launch Disciplines 6 months post-launch
2020 Annual Report

Ready for Launch? Checking off Readiness Factors for CBD Implementation

This report outlines the findings of a study conducted by the Royal College to understand program readiness factors that promote or inhibit a successful implementation of Competence by Design. Measurements include programs’ motivation, general capacity for change, and completion of readiness tasks.

2019 Annual Report
2020 Annual Report

CBD Cost Analysis Report (executive version)

The CBD Cost Analysis Report is a result of the CBD Resource Framework working group’s study on the cost of CBD, focussing on variability, direct and additional costs, anticipated impacts, and the costs of running parallel systems.

Understanding CBD Program Evaluation

These presentation slides explain the Royal College approach to program evaluation. It outlines the program goals, logic model and the significance of the three pillars of program evaluation.

CBME Program Evaluation Forum:

This program evaluation forum holds events for those engaged in CBME program evaluation initiatives to learn and share results, methods, and processes, as well as to create a dialogue on program evaluation and encourage collaboration. For more information, please contact educationstrategy@royalcollege.ca

CBME Program Evaluation Summit

The Competence by Design Program Evaluation operations team hosts an annual CBME Program Evaluation Summit, in conjunction with ICRE. Learn about previous summits or sign up to join the mailing list for the next event.

Join the conversation! To contribute to ongoing collaboration in CBD program evaluation, please contact us at educationstrategy@royalcollege.ca or participate in our CBME Program Evaluation Forum events.