Self-Assessment Program Development Template

Section 3 of the Framework of CPD Options of the Maintenance of Certification Program

Background

Written Self-Assessment Programs (SAP) provide specialists with the opportunity to review their knowledge and clinical judgment with current scientific evidence to identify opportunities that can be addressed through the planning of other learning activities.

Self-Assessment Programs must

1. Be developed by a planning committee who are members of a physician organization.
2. Address a defined need within a specific subject area, topic, or problem.
3. Describe or include methods that enable participants to demonstrate or apply knowledge, skills or attitudes.
4. Provide detailed feedback to participants on their performance to enable the identification of any deficit in knowledge and the development of a future learning plan.
5. Be independent from any commercial interests.

The Office of Professional Affairs has created the following template to provide Accredited CPD Providers or physician organizations with a tool that describes the key steps and rationale for the development of a SAP that meets the above standards. Each national or provincial specialty society or physician organization is invited to use this template to create a disease specific Self-Assessment Program.

Creation of an Accredited SAP Program

To facilitate the development of a disease specific SAP program, the following template or blueprint is suggested that defines the steps required.

1. Select a topic, disease, or disorder

   • That is prevalent or important
   • With an established scientific evidence base (such as a clinical practice guideline, meta-analysis or systematic review)
   • Where the scientific evidence base is changing or expanding and
   • Where changes in the application of knowledge would influence management decisions and the potential benefits experienced by patients

   Rationale: Conditions that are prevalent have relevance to a wide variety of patients. Conditions where evidence has been established or summarized (for example a clinical practice guidelines) will assist in ensuring that the assessment of participants knowledge and judgment will be compared with established evidence.

2. Identify the key knowledge areas or themes that will be covered during the Self-Assessment Program. For example a disease-specific Self-Assessment Program could include an assessment regarding;
   • Etiology
   • Pathophysiology
   • Clinical Manifestations
   • Clinical Findings
   • Diagnosis
   • Differential Diagnosis
   • Medical Management (drug and non-drug therapy)
   • Prognosis

   Rationale: Completing a thorough review of the current or evolving evidence within each area of a particular disease provides specialists with an opportunity to evaluate their current knowledge with best evidence to identify opportunities for improvement that can be addressed through additional learning activities. The focus of a SAP is to uncover unidentified deficiencies for future learning.

3. Create the written content for the SAP (optional)

   Many Self-Assessment Programs include a written summary of current evidence organized into sections to explain the basic key knowledge areas or practice recommendations that serve as the basis for the program. The written component can be organized in multiple formats including a case base format.

   If a case or series of cases are presented and a summary of the current evidence is provided then this step should include:

   • Selecting a SAP program development writing committee
   • Dividing the program into sections or themes
   • Establishing learning objectives
   • Assigning individuals to write the content for individual sections.
   • Identifying appropriate references

   Rationale: Reviewing a summary of the key evidence or knowledge across the disease, disorder, or topic will provide participants with an opportunity to learn about best evidence and compare their knowledge with current understanding.

4. Create multiple-choice questions (MCQ), short answer questions (SAQ) or true/false statements for each section or theme with appropriate distracters. For each MCQ, link the correct response to an appropriate reference.

   Rationale: Written Self-Assessment Programs are frequently designed to assess knowledge or the application of knowledge to practice. MCQ’s and SAQ’s are common, highly reliable and valid methods for assessing cognitive domains.

5. Create a scoring key and answer sheet (consider optical scanner technology) or web based assessment tools that will provide individuals with section specific feedback (questions answered correctly; incorrectly with supporting references).

   The answer sheet should include instructions for how to complete and submit the scoring sheet for credits

   Rationale: Providing specific feedback on which answers were correct and incorrect with references enables specialists to determine if there are important gaps in their practice knowledge that should be addressed through engaging in further learning activities. Providing specific feedback (by section or theme) is a required accreditation standard for SAP.

6. Create a SAP evaluation form to be completed by participants. The evaluation form will elicit information about the value or relevance of the program as identified by the participant on completion of the program.

   The evaluation form should include an assessment of the following areas:

   • Relevance or importance to practice
   • Clarity of the content and evaluation questions
   • Thoroughness in assessing content areas
   • Capability in assessing knowledge gaps

   Rationale: The purpose of the evaluation is to assist program planners to modify the program (where required) based on participant’s feedback

7. Create a SAP reflection tool that the participants can use to assess and document the outcomes for them and use this information to develop a learning plan to address any areas where improvement is required.

   The SAP reflection tool should prompt participants to review the feedback provided and:

   • Identify specific areas that require improvement
   • Use the framework of CPD options in the Maintenance of Certification program to a learning to address these identified areas

   Rationale: The purpose of a SAP is not just to identify where knowledge is up-to-date, but to identify areas where further learning needs to be completed.

8. Ensure that the program materials inform participants about the importance of documenting their outcomes or proposed actions on MAINPORT for credit

   Participants must be directed to the self-assessment templates in MAINPORT to complete the documentation of all relevant sections and submit the activity for credit. CPD providers must provide:

   • Instructions for how to get to MAINPORT
   • What sections must be completed
   • The number of hours that have been established for completing all program elements

   Rationale: Completing a Self-Assessment Program is not an end in itself. Each participant needs to be informed of the important of documenting their assessment and identified outcomes or plans on MAINPORT to receive appropriate credit. The focus on the documentation of learning outcomes is one of the goals of the second phase of implementation of the Maintenance of Certification program.

9. Ensure that the Self-Assessment Program was developed independent from any commercial or other conflicts of interest. Independence from commercial or other conflicts of interest is based on the following standards:

   Members of the planning committee have

   1. Complete control regarding the selection of:
      • the area, subject or topic
      • the authors
      • the scientific validity and objectivity of the evidence.
   2. Ensured that:
      • there is consistency and fairness for how drugs names are used throughout the SAP’s written materials
      • there are no drug or product advertisements on any of the SAP written materials.