3/14/2016 Mapping to the Milestones A CONCEPTUAL PRESENTATION ON HOW TO THINK ABOUT LINKING YOUR EVALUATION TOOLS, HAVING YOUR EVALUATION TOOLS WORK FOR YOU AND NOT AGAINST YOU Jennifer W. Swoyer D.O. Osteopathic Program Director/Director of Medical Education Adventist La Grange Family Medicine Residency Program What are the Milestones The Milestones are an ACGME mandated mapping document They must be completed twice yearly on each resident in the program The Milestone document must be completed by a reviewing committee called the CCC committee The document itself is NOT an evaluation The data collected from each program will be reviewed by the ACGME over the course of 10 years… 1
3/14/2016 UGH….what does that all mean As more and more pressure arises to ensure that physician education is standardized/nationalized the Milestone project becomes an opportunity for programs to evaluate each resident against a standardized norm of expected behaviors. These behaviors are the substrates of the general competencies that we have worked to integrate over the last 10+ years. The milestone document gives a standard map of expected physician behaviors that we can map against. This ultimately will highlight the success of our residents and our training programs “ Until we changed our evaluation system, the end of the year evaluations were always due in May. I hated May. I live in ” Chicago, May is the only reason I stay. U NNAMED OVERBURDENED PROGRAM D IRECTOR How Many of you dislike your current evaluation system? “ How many of you want an evaluation system that will work for you? ” Then Stay seated… 2
3/14/2016 So, lets step back and look at the big picture… Let’s remember…Our Ultimate Goal Competent physician So How do we get there? How did we get here? 3
3/14/2016 A Model of Physician Development Medical school graduate Competent in 13 standard EPA’s, ready for residency Residency program curriculum Residency Teaching modalities and rotations Residency Program Evaluation System Milestone Sub-competencies Competencies Competent physician Licensure Continuing Medical Education Maintenance of Certification Board Re-Certification Professional Development JWS-LGFPR 8-15-15 Physician Leadership As Residency Programs we are accountable for documenting and assessing a young physician’s competence A Model of Physician Development Medical school graduate Competent in 13 standard EPA’s, ready for residency Residency program curriculum Residency Teaching modalities and rotations Residency Program Evaluation System Milestone Sub-competencies Competencies Competent physician Licensure Continuing Medical Education Maintenance of Certification Board Re-Certification Professional Development JWS-LGFPR 8-15-15 Physician Leadership 4
3/14/2016 We used to think this meant…. Every evaluator evaluates on all Resident evaluates faculty competencies on all competencies Everyone evaluates faculty evaluates every everyone component everytime Peer evaluations Resident evaluates all rotations Unfortunately our old system had a lot of input and little output 5
3/14/2016 We started by correcting our vision COMPETENCE We Put Competence at the center. Competence is our target. We added the OSTEOPATHIC competencies PRACTICE AND PRINCIPLES COMMUNICATION OMM/OPP C MEDICAL PATIENT KNOWLEDGE Competent CARE Physician MK PC PROFESSIONALISM SYSTEMS BASED P PRACTICE PRACTICE BASED SBP LEARNING PBLI OUR VISION GREW TO INCLUDE THE SUB COMPETENCIES MK2 OMM C PC1 OPP MK1 MK PC2 PC PC3 Competent physician P PC4 SBP PC5 PBLI 6
3/14/2016 Next we layered on the rotations Team eval Team eval and educational Pt eval experiences where we could collect P3 data on resident P4 P2 C1 P1 performance C2 PC5 PROF C3 PC4 IC C4 PC3 PC PC2 Competent PBLI3 physician PC1 PBLI SBP1 PBLI2 SBP SBP2 SBP3 OMM & MK PBLI1 SBP4 OPP MK2 MK1 OMM clinic OP eval ITE IP eval And the work had just begun… Evaluating where we could collect our data was a group process with all of the faculty Our objective was to get multiple data points for each sub- competency When we finished… we had a list of the evaluations we needed to create. Evaluations were linked to sub- competencies C-1: FMC, PEDS, OB, Geri, OSCE, Patient evaluation C-2: FMC, OSCE, shadowing, video, FMS C-3: FMS, PEDS, nursing eval, Patient evaluation C-4: FMS, FMC, nursing eval, Patient evaluation P-1: all rotations, front desk eval, nursing eval, FMS P-2: Coordinator eval, front desk eval, OSCE P-3: FMC, Behavioral health, FMS, OSCE, Patient evaluation P-4: Advisor and PD Review OMM: FMS, FMC, OMT clinic eval 7
3/14/2016 Front desk Evaluation of Resident: P-1, P-2a, P-2b, P-2c Journal Club: PBLI-1 Nursing Evaluation of Resident: C-2, C-1, C-3, P-2, P-3, P-2, PC-4, PBLI-3 Outpatient Clinic Evaluation: C-1, C-2, C-3, MK-2, P-1, P-2, OMM, P-2, P-3, PC-1a, PC-1b, PC-2a, PC-2b, PC-3a, PC-3c, Pc-4, Pc-5, SBP-1, SBP-2, SBP-4, PBLI-1 Before we were done, our bullseye grew Team eval Residency Evaluation System Pt eval Team eval P3 P4 P2 C1 P1 Residency Teaching Modalities and Rotations C2 PC5 C3 PROF PC4 IC C4 PC3 PC Competent PC2 PBLI3 physician PC1 Residency Curriculum and Curricular Objectives PBLI SBP1 PBLI2 SBP SBP2 SBP3 OMM & MK PBLI1 OPP SBP4 MK2 MK1 OMM clinic IP eval ITE OP eval 8
3/14/2016 An evaluation NEW EVALUATIONS plan meant We created a system of standardized performance statements These linked to the sub-competencies Some sub-competencies required several statements to absorb the important elements Milestone: Patient Care (PC-3) “ “Partners with the patient, family and community to improve health through disease ” prevention and health promotion” 9
3/14/2016 We turned this into… PC-3a PC-3b PC-3c Partners with the Tracks and monitors Reconciles the various patient and family to disease prevention and guidelines and overcome barriers to health promotion within screening disease prevention and the practice. recommendations for health promotion. Is health maintenance able to describe cost and applies these benefit ratio in relation appropriately to patient to health promotion care. and disease prevention therapies We completed this process for all of the competencies This gave us a list of evaluation statements that linked to the sub- competencies. With this list we could build our new evaluations tools and link them directly to the sub-competencies and build a model for completing our Milestone documents. There was just one piece missing… One major problem Please raise your hand if you are below average. Physicians by nature are over achievers. We tend to be above average. We are not always great evaluators. 10
3/14/2016 Has anyone here suffered the tears and anguish of an average resident who was given an average evaluation? Let’s say a 3/7 Likert scale? There had to be a better scale… The Milestone system is a 9 point scale. We chose 5 major performance definitions 1 = Beginner : This resident is performing at a level that shows awareness of the medical system but he/she depends on other physicians for the development of patient care plans 2 = Advanced Beginner : This resident is performing at a level that demonstrates awareness of the medical system and patient care. He/she is developing assessments of the patients in his/her care and utilizing attending input for the development of care plans. 3 = Intermediate : This resident is performing at a level that demonstrates independent thought. This resident is able to evaluate, assess and develop a plan of care for most patients. 11
3/14/2016 4 = Independent : This resident is performing at a level that demonstrates the ability to teach others and function in independent practice. This resident is able to evaluate, assess and develop a plan of care for all patients. 5 = Expert : This resident is performing at the highest level, consistent with experts in the field of family medicine. This resident is able to evaluate the medical system and current models of care. This resident contributes to the development of systems that impact patient care within the community. He/she role models active involvement in community education and health policy. “ This scale has significantly reduced grade inflation ” Especially from the specialists Every subcompetency statement was entered in New Innovations 12
3/14/2016 Evaluations were built using New Innovations. We linked each question to the corresponding competency 13
3/14/2016 A new evaluation tool was generated…. Once all of our evaluations were created, we developed a spread sheet to ensure all sub-competencies were measured several times during the course of the year. ARE YOU EXCITED YET??? 14
3/14/2016 Once we were sure… we covered all sub-competencies we had all evaluations written we had all evaluations linked to the Milestones we developed a schedule for evaluation completion Then we had our first CCC meeting. 15
Recommend
More recommend
