Outline What is TURF ? A TURF Model for Critical User Interactions - PowerPoint PPT Presentation

Categorization of Critical User Interactions for Pediatric EHR W E L C O M E Jiajie Zhang, PhD Event or Meeting Title Director, SHARPC Interim Dean, UT School of Biomedical Informatics at Houston And The SHARPC Team

Outline  What is TURF ?  A TURF Model for Critical User Interactions

What is TURF?

TURF - A Unified Framework of EHR Usability  An Acronym for  T ask, U ser, R epresentation, & F unction  A Theory for  defining, describing, explaining, and predicting usability  A Method for  evaluating and measuring usability  designing usability  categorizing usability and safety problems  A Software Tool for  (partially) automating usability evaluation  conducting user testing  building EHR ontology  generating evidence-based designs  conducting usability and patient safety analytics

What is Usability ? 5  Under TURF, usability is defined as how  useful,  usable,  satisfying  a system is for the intended users to accomplish goals in the work domain by performing certain sequences of tasks

TURF Framework for EHR Usability 6 F unctions Useful Intrinsic U sers Complexity Satisfying Extrinsic R epresentations Difficulty Usable Electronic Health System Usability T asks Record Zhang, J., & Walji, M. (2011). TURF: Toward a unified framework of EHR usability. Journal of Biomedical Informatics, 44 (6) , 1056-1067.

Dr. Townshend prescribes 90 day supply of Metformin 500 mg tablets by mouth twice daily to patient John Doe who is a pre-diabetic patient with a glucose level of 110. USERS JOHN DOE manipulate REPRESENTATIONS Dr. Townshend execute Dose Instructions Duration Dose TASKS Dispense Route Find the “Medication" tab Frequency Point to “Medication" tab Click “Medication" tab Pt. Instructions Click “New Med” button Medication Start Dosage / Quantity Refills Wait for the system to show Name/Strength Date Instructions “Medication" window Find the "Search all Meds" support Click "Search All Meds" Wait for the system to show "Order a Med" pop-up window Find “Search" field Point to “Search" field Type “ Metformin" Point to “ Metformin" on the list Click “ Metformin" on the list Wait for the system to show “Dose Instructions" window Extrinsic Difficulty--Usableness Etc. etc. etc.

Dr. Townshend prescribes 90 day supply of Metformin 500 mg tablets by mouth twice daily to patient John Doe who is a pre-diabetic patient with a glucose level of 110. USERS JOHN DOE manipulate REPRESENTATIONS Dr. Townshend execute Goal : Dose Instructions FUNCTIONS Treat high glucose level Duration Dose TASKS Dispense Route Operation : Operation : Operation : Find the “Medication" tab Check drug-drug interaction Prescribe medication Check drug allergy Frequency Point to “Medication" tab Click “Medication" tab Pt. Instructions Object : Object : Object : Click “New Med” button Patient = John Doe Medication = Metformin Diagnosis = diabetes Medication Start Dosage / Quantity Refills Wait for the system to show Name/Strength Date Instructions “Medication" window Find the "Search all Meds" Properties : support Click "Search All Meds" Wait for the system to show Dosage Form Generic name Brand name Strength "Order a Med" pop-up window Find “Search" field Frequency Dose Route Dispense Quantity Point to “Search" field Type “ Metformin" Other Instructions Date Ordered Refill Duration Point to “ Metformin" on the list Click “ Metformin" on the list Wait for the system to show “Dose Instructions" window Intrinsic Complexity--Usefulness Etc. etc. etc.

Measuring Extrinsic Difficulty -- Usableness: Task Time and Usability Problems 9 Task Time (seconds) Task Analysis Representation Analysis Usability Problems

Measuring Extrinsic Difficulty -- Usableness: Mental Workload 10 Physical Mental CPOE ePrescribing Clinical Summary Smoking Status Problem List Growth Chart BMI Vital signs Demographics 0% 20% 40% 60% 80% 100%

Measuring Intrinsic Complexity-- Usefulness: Overhead Function # Overhead Functions in EHR = Overhead in EHR #Domain Functions in EHR + #Overhead Functions in EHR Domain Functions in EHR Overhead Functions in EHR CPOE 2 64% 36% CPOE 1 58% 42% 0% 20% 40% 60% 80% 100%

Measuring Intrinsic Complexity-- Usefulness: Domain Function Completeness 12 #Domain Functions in EHR = Domain Function Completeness #Domain Functions in Entire Work Domain 37  A Small EDR System 46 % 80 (From Chen, 2008)

TURF Software Tool Architecture Users Usability Evaluator Other Users Developer Provider System administrator Presentation layer Analysis & Report Modeling Data Collection Business layer Data Capturing Modeling Model-Driven Analysis Mapping Populate Representation data: Data editor Usability Metrics TURF Screenshot, video, widget integration models Interaction data: Usability Benchmarks Keystroke & mouse movement User data: Usability & safety patterns Modeling engine Profiles and Personas Function Data: EHR domain ontology Repository management Work domain ontology Data Layer Service agents Data access components Data utilities Other usability test/analysis Entity-relational database services Ontology database (e.g., Cogtool, Ulog, Noldus, etc.)

TURF Software Tool: Assess, measure, and improve EHR usability 14  Semi-automate usability expert reviews  Build cognitive models to predict clinical task performance times  Capture user testing data  Build EHR ontology from usage data  Generate evidence-based designs TURF 1 Beta Release: Summer 2012

TURF Model for Critical Risks

A TURF Model of Critical Risks for Pediatric EHR Functions  Dosage support Intrinsic Complexity  Growth chart Function Root Causes  Vaccine schedule supported  Physical characteristics  Medication order  Data accuracy error by o Weight, height, Body Surface Area  Alerts for abnormal values  Data availability error (BSA), Body Mass Index (BMI)  Privacy  Data integrity error  Developmental issues  Other pediatrics-specific functions o Fetal to postnatal to adulthood o Gestational age and actual age act-upon o Change of organ systems with age Users Adverse Events o Age, weight, and height dependent disease states, symptoms, exam perform User Root Causes • Pediatricians findings, lab results, and treatments  Wrong patient action of • Nurses  Complexity of medications  Unintentional • Patients commission o Change of dosage with age, weight, and o Slips (attention) • Parents  Wrong patient action of BSA o Lapse (memory) o Many formulations; liquid can be manipulate omission o Mistake (knowledge) continuum  Intentional  Wrong treatment action of Representations contribute cause o Rounding of dosage commission  Patient identification issues  Navigational structure  Wrong treatment action of o Date of birth, names, temporary  User interface entities omission names, pre-admission identification Representation Root o Objects (information items, e.g.,  Unique information requirements  Wrong medication MRN, text, graph, etc.) Causes o Growth chart  Delay of treatment supported o Operations (actionable items, e.g., o Vaccination history  Unintended or improper by buttons, checkboxes, etc.)  Patient identification error o Parental and sibling information  Organizational structures  Mode error treatment o Information from third person o Spatial layouts  Interpretation error o Graph variables over time o Color, texture, shape, shade,  Truncation o Genetic information contour o privacy Tasks Extrinsic Difficulty Task Root Causes  Task goals  Task sequences  Recall error  Individual operations Pediatric  Feedback error  Temporal constraints Electronic Health Record TURF Components Root Causes

Function Root Causes 17 Description Example Potential Risk/Impact Forced data format Systolic blood pressure Data entry values must be entered errors as 3-digits (060)

Function Root Causes 18 Description Example Potential Risk/Impact Default This pop-up reverts to prior data if a Data entry Values parameter is entered that is not “in errors Sys. BP Dia. BP range” with NO WARNING to the user. Height/Length Inch cm Weight Lbs. oz kg Temperature F C Pulse RR O 2 Sat.

Representation Root Causes 19 Description Examples Potential Risk/Impact Mode error A patient’s weight and height Drug dosage are entered in pounds and miscalculation inches, and then displayed in kilograms and meters.

Representation Root Causes 20 Description Examples Potential Risk/Impact Patient Multiple patients’ data are Wrong patient identification displayed concurrently. procedure error Diagnostic test is ordered for Patient A and thought to be Patient B

Representation Root Causes 21 Description Example Potential Risk/Impact Visibility Does not allow entire growth chart in one Missed error view. No resizing. Requires 2 slider bars diagnosis to mentally visualize the entire chart. Menu Menu Menu Menu Menu Menu Menu