Is Competence by Design Working?
The role of Program Evaluation
The implementation of CBD requires considerable time and resources from many different stakeholders in medical education and can have significant impacts; therefore it is important to evaluate it on an ongoing basis, and ensure it is having the desired impact. Program evaluation also signals that we are accountable as an organization to this initiative.
More importantly, it is incumbent upon the Royal College to ensure that implementation of a new educational model does not lead to any harmful effects. First and foremost is considering the impact on patient care, but also important is monitoring the impact on learners, teachers and other stakeholders involved in residency education.
Program evaluation is also important from a scholarly perspective – CBME is increasingly being adopted on a global basis, and there is a desire to share what is being learned as the Royal College implements CBD. Systematically evaluating and disseminating the findings from CBD will help us all learn together.
Recent reports: Understanding the Royal College approach to CBD Program Evaluation
CBD program evaluation reports and other study findings can also be found on the CBD Program Evaluation Dashboard.
Resident Pulse Check 2022
The Resident Pulse Check Survey was a collaborative effort between Resident Doctors of Canada (RDoC) and the Royal College. This survey aims to understand the resident physicians’ experience with the implementation of Competence by Design.
For a copy of the full report, please email educationstrategy@royalcollege.ca
CBD Program Evaluation Recommendations Report
This report summarizes the challenges and recommendations found in the 2019 program evaluation studies conducted separately by the Royal College and the Fédération des médecins résidents du Québec (FMRQ). Actions that the Royal College has taken or will take to address these recommendations are identified.
CBD Implementation Pulse Check
These reports outline the findings of studies conducted by the Royal College to understand how CBD implementation is going on the ground, as well as benefits, challenges, advice for moving forward, and any early outcomes. It primarily studies fidelity and integrity of implementation to monitor trends and inform necessary adjustments. The Pulse Check is conducted 6 months post-launch for each cohort, and annually thereafter.
2019 Annual Report
2019 Launch Disciplines 6 months post-launch
2020 Annual Report
Ready for Launch? Checking off Readiness Factors for CBD Implementation
This report outlines the findings of a study conducted by the Royal College to understand program readiness factors that promote or inhibit a successful implementation of Competence by Design. Measurements include programs’ motivation, general capacity for change, and completion of readiness tasks.
CBD Cost Analysis Report (executive version)
The CBD Cost Analysis Report is a result of the CBD Resource Framework working group’s study on the cost of CBD, focussing on variability, direct and additional costs, anticipated impacts, and the costs of running parallel systems.
Understanding CBD Program Evaluation
These presentation slides explain the Royal College approach to program evaluation. It outlines the program goals, logic model and the significance of the three pillars of program evaluation.
CBME Program Evaluation Forum:
This program evaluation forum holds events for those engaged in CBME program evaluation initiatives to learn and share results, methods, and processes, as well as to create a dialogue on program evaluation and encourage collaboration. For more information, please contact educationstrategy@royalcollege.ca
CBME Program Evaluation Summit
The Competence by Design Program Evaluation operations team hosts an annual CBME Program Evaluation Summit, in conjunction with ICRE. Learn about previous summits or sign up to join the mailing list for the next event.
Join the conversation! To contribute to ongoing collaboration in CBD program evaluation, please contact us at educationstrategy@royalcollege.ca or participate in our CBME Program Evaluation Forum events.
