Development of viva voce exam to assess clinical reasoning in the - PowerPoint PPT Presentation

Development of viva voce exam to assess clinical reasoning in the paraclinical phase of a BVSc program. Helen Owen, Senior Lecturer in Veterinary Pathology and Jus:ne Gibson, Senior Lecturer in Veterinary Bacteriology and Mycology Dr Tamsin Barnes Associate Professor Joanne Meers Associate Professor Rowland Cobbold Dr Dan Schull Dr Katrina GarreG Associate Professor Jenny Seddon Dr Steven Kopp Dr Frances Shapter Associate Professor Rebecca Traub

Background on Program • The BVSc at UQ is an undergraduate 5 year program • First 3 years consist predominantly of preclinical sciences with the fourth and fi?h years having more of a clinical focus. • Major courses in third year - a year-long course on pathophysiology – stereotypical ways which Fssues respond to injury, prototype diseases - clinical signs, clinical exam and diagnosFc approach to body system - course on infecFous disease (parasitology, virology, bacteriology and mycology).

Development of viva voce exam • In 2012, we were tasked with developing a viva voce oral exam as a program level “hurdle” assessment at the end of third year. ObjecFves, to assess: • applicaFon of theory to case scenarios • diagnosFc clinical reasoning • integraFon of material from both the pathophysiology and infecFous diseases courses • professionalism and oral communicaFon skills • Exposure to oral exam format before 5 th year viva exams

Advantages of oral format • Good for assessing problem-solving abiliFes. • Can provide insight into students’ cogniFve processes. • It allows probing of the depth and extent of students’ knowledge with follow-up quesFons • Unclear or ambiguous quesFons can be re-expressed or immediately clarified • Students can be guided back onto the right track if they stray

Advantages of oral format • Allows judgements about students’ interpersonal competence - confidence, self-awareness and aspects of professionalism. • SimulaFon of clinical pracFce – students need to “think on their feet” • It improves the quality of student learning – students have been reported to strive for thoroughness in understanding in response to the unpredictability and to avoid embarrassing themselves in front of the examiner • It suits some students

Disadvantages of oral format • Generates a lot of anxiety in students and examiners • Time consuming and requires a lot of energy • Lack of anonymity and potenFal for bias – unconscious bias based on charismaFc personality, well dressed, being arFculate • Reliability - Inter-case - different students get different cases to prevent sharing of informaFon. - Inter-rater - Intra-rater reliability

Features of the initial exam • Formulaic series of quesFons based on the diagnosFc process - Provided with image and some informaFon – history and clinical exam findings - body system(s) - possible pathological processes (high weighFng) – using pathophysiology for clinical reasoning - differenFal diagnoses (low weighFng!) - diagnosFc plan (high weighFng) - Interpret results, treatment, control, prognosis • professionalism

Initial exam • All students received one case • Students who failed iniFal exam were given a resit • 10 minutes perusal, 20 minutes oral • 2 examiners per student - increase reliability – consensus marks used • Rubric and model answers - increase reliability

Changes over the years based on examiner and student feedback • 2 cases (usually one large animal, one small) – increases reliability • More examiner training • Removed history and clinical exam secFons • 2-stage pathophysiology • Adjusted rubric - more emphasis on systemaFc approach for pathogenesis and diagnosFc tesFng, forced to prioriFse DDxs • Year-long preparaFon and provision of formaFve feedback.

Exam format – written section (30 minutes) You are presented with a 1 year old en3re male Domes3c Short Hair cat. The owner no3ced that the cat is off its food and ‘quieter’ than usual, this has been the case for about a week. Part A. Descrip:on of case, presen:ng signs (images/ videos)

Exam format – written section • Part B. Outline the pathological processes which may be causing the presentaFon • Students sFll rote learning pathogenesis for parFcular clinical presentaFon without emphasising/ prioriFsing based on case parFculars • Or students using pacern recogniFon and being able to describe one specific pathogenesis but not being able to suggest alternaFves • Break down into two steps, in the first step, they need to demonstrate a systemaFc knowledge structure that includes all possible pathogeneses • Second step, provided with addiFonal informaFon about the case and need to use this to prioriFse DDxs

• Encourage students to consider all possible pathogeneses for a broad clinical sign at this point • Must demonstrate structure, systemaFc approach, encouraged to use schemes

Scheme-inductive knowledge structures Research suggests that expert problem solvers are disFnguished by the way in they organise and understand their knowledge Harasym et al. 2008 Kaohsiung J Med Sci, 24:7; 341-355.

Scheme-inductive knowledge structures • Mandin et al, University of Calgary Faculty of Medicine – “Clinical PresentaFon” curriculum • “The “hypotheFco-deducFve” strategy tradiFonally used for PBL should be replaced by scheme-driven search strategies so that students develop a more organised and logical approach to problem-solving” Schemes have mulFple purposes: • Provide systemaFc approach to decrease the risk of omidng DDxs

• way of organising knowledge for more efficient understanding, serve as a scaffold for new informaFon

The history, physical examinaFon and laboratory invesFgaFon are all driven by the derived scheme

Paraclinical schemes • Probably different content to clinical schemes, more reliance on principles of pathophysiology, less signalment etc? Nodes can consist of: • locaFon e.g. body systems - cardiovascular versus respiratory tract; within body systems - upper versus lower respiratory tract; • a form of general pathological process in the Fssue e.g. myocardiFs, hepaFc necrosis, pulmonary oedema • Specific causaFve agents (prototypic) • Some presentaFons a bit different e.g. anaemia, jaundice • Lots of schemes provided to the students and they are encouraged to improve on them

Mnemonics as a system? • e.g. DAMNITV - DegeneraFve, Anomalous, Metabolic etc • Students o?en prefer this approach • trying to not be too prescripFve, flexibility allowing for different thought processes, clinical presentaFon Don’t work too well in our hands …. • Doesn’t require explicit idenFficaFon of body systems • Students o?en don’t seem to understand pathogenesis • Doesn’t provide structure for systemaFc diagnosFc approach • More appropriate as a memory jogger in clinical years a?er students have demonstrated pathophysiology knowledge? • Useful as part of a mulFple strategy approach?

Exam format – written section You will be provided with addiFonal history or clinical features • You perform a PCV and there is marked anaemia. • IniFally we assume that good students would prioriFse DDxs based on all the features of the case but this rarely happened Part C. Give a problem defini:on • Subacute inappetence, depression, jaundice and anaemia.

Exam format – oral section Part D. Body system(s)? • haematopoieFc • Part E. Use the addiFonal informaFon to idenFfy most likely pathological process(es), describe pathogenesis and provide aeFological examples if appropriate

Exam format – oral section Part F. List 5 differenFal diagnoses Part G. DiagnosFc plan • IniFally found that students would just test and Fck off each of their differenFals • Encourage systemaFc approach, use of scheme to guide, most informaFon from fewest tests • Encourage use of diagnosFcs to guide treatment • Students do relaFvely poorly in this secFon

Exam format – oral section Provide diagnosis, prognosis, treatment plan, control This secFon also poorly done in general

Exam format – oral section • Part I – professionalism – use of professional language, professional behaviour and dress. Outcome of viva exams • This is a must-pass hurdle, a larger number of students fail this compared to failing on basis of theory exams (~5-9/120 versus ~2-3) • O?en about 10 students on pass/ fail line, reluctant to let them go into 4 th year – what should our expectaFons be for these novice diagnosFcians? • Most students who fail viva are weak in theory also but not always

Preparation for viva exams • 6 face-to-face tutorials dispersed through the year • 5 th year students act as tutors for small groups • 3 rd years examining each other in pairs • Include all steps in process, even at the beginning • 3 on-line case-based tutorials where generic feedback given but students can elect to have personal feedback • “diagnosFc work-shop” • Viva “hint-sheet”