Results Patient safety Morbidity & Closed Specialist Literature event reports Mortality malpractice focus groups search (n=4,352) reports (n=24) claims (n=403) (n=6) 155 diagnostic 75 diagnostic 10 diagnostic 396 diagnostic 355 focus pitfall-related pitfall-related pitfall-related pitfall-related group articles reports reports claims responses 201 204 106 15 711 101 15 625 175 96 DEER RDC DEER DEER DEER RDC RDC RDC DEER RDC TOTAL DEER = 1208 TOTAL RDC = 1041
Results – DEER Taxonomy Errors (n = 1208) 503 500 450 400 Frequency 350 300 260 250 200 154 141 150 101 100 42 50 7 0 Access/ History Physical Tests Assessment Referral/ Follow-up Presentation Exam Consult Diagnostic Process Steps
Results - RDC Taxonomy Issues (n = 1041) 350 314 305 300 250 Frequency 222 200 150 111 89 100 50 0 Challenging Patient factors Testing challenges Stressors Broader disease Challenges presentation Diagnosis Challenges
Results Examples of generic and disease-specific diagnostic pitfalls identified: • Disease A repeatedly mistaken for Disease B • Bipolar disease mistaken for depression • Failure to appreciate test/exam limitations • Pt w/ breast lump and negative mammogram and/or ultrasound • Atypical presentation • Addison’s disease presenting with cognitive difficulties • Presuming chronic disease accounts for new symptoms • Lung cancer: failure to pursue new/unresolving pulmonary sx in patient with pre-existing COPD • Failure to monitor evolving symptom • Cranial imagining can be normal shortly after head injury, but chronic subdural hematoma later develops
Diagnostic Situational Awareness Model
• SUPPLIMENTAL SLIDES
2 Key Improvement Concepts • Situational Awareness • Safety Nets
Diagnostic Risk Situational Awareness • Specialized type of situational awareness • High reliability organizations/theory – High worry anticipation of what can go wrong – Preoccupied w/ risks recognizing/preventing • Appreciation diagnosis uncertainty, limitations – Limitations of tests, systems’ vulnerabilities – Knowing when “over head” need for help • Making failures visible • Don’t miss diagnoses, red flag symptoms • Diagnostic pitfalls – potentially useful construct 35
Diagnostic Risk Safety Nets • Recognizing inherent uncertainties/risks, build in mitigation, protections, recovery structures and processes • Proactive, systematic follow-up, feedback via closed loop systems • Major role for HIT to hard-wire – To automate, ensure reliability, ease burden on staff/memory, ensure loops closed and outliers visible 36
Failure to Consider: Cognitive vs. System Problem? Why did clinician fail to consider? • Lack knowledge, memory recall • Inadequate time • Failure to elect key hx or physical • Competing diagnoses, symptoms • Rare, atypical • Tests threw off • Distractions What are the causes? • Biases; heuristic What are the remedies? 37
Breast Cancer Pitfalls: Malpractice Cases Review Preliminary Findings Pitfall N Example 1. Family History - Failure to obtain family history of breast cancer 4 - Under-weighing family history of breast cancer Issues - Underestimating risk of BC in young symptomatic patients 2. Atypical - Fast-growing cancers arising during MMG interval Presentation/ - 6 Under-weighing complaints of patients with Cognitive psychiatric diagnoses Challenges - Prioritizing chronic medical or social issues over screenings in complex patients - Lump felt to be benign on physical exam 3. False Negative - 2 Bias in wanting to reassure patient, due to low Physical Exam likelihood of BC - Fibrocystic breast tissue can obscure underlying BC in MMG 4. Fibrocystic/Dense - Not recognizing changes in breast density over time 9 - Failure to investigate unilateral fibrocystic changes Breast Dilemmas - Failure to investigate breast lump with FNA in patient with dense breasts and negative U/S Schiff et al. Unpublished data. Coverys/CRICO Closed Claims Review, 2016
Breast Cancer Pitfalls: Malpractice Cases Review Preliminary Findings Pitfall N Example 5. Screening vs. - Ordering/performing a screening MMG, rather than Diagnostic 2 a diagnostic MMG Mammogram Order - False negative MMG in pt with fibrocystic breasts - Failure to reevaluate breast complaints in light of previously negative MMG 6. False Negative - 9 Misreading of MMG by radiologists Mammogram - Failure to follow-up on nipple retraction observed on MMG, attributing it to imaging technique Falsely reassuring negative “additional views” - 7. False Negative - Falsely reassuring negative U/S in pts with breast 2 lump Ultrasound - Failure to refer to breast surgeon 8. Surgical Referral - 4 Breast lump appearing benign to surgeon palpation - Patient failure to follow-up on referral Schiff et al. Unpublished data. Coverys/CRICO Closed Claims Review, 2016
Breast Cancer Pitfalls: Malpractice Cases Review Preliminary Findings Pitfall N Example 9. Biopsy Performance/ - Inability to recognize missed sampling due to 1 bleeding/complications and failure to repeat biopsy Interpretation - Failure to order diagnostic imaging studies (MMG 10. Failure to Order 2 and U/S) Further Studies - Failure to recommend excisional biopsy - Failure to document/ensure pt was receiving 11. Diffusion of screening MMGS and breast exams Responsibility/ 4 - Failed coordination/communication between PCP Coordination Issues and GYN - Failure to follow-up on resolution of mastitis - Failure to pursue etiology of persistent galactorrhea - Pursuing lymphoma as cause of lymphadenopathy 12. Other Symptoms 8 - Axillar lymphadenopathy lost due to fact that not incorporated into BIRADS coding (revised now) - Failure to work up persistent painful cyst Schiff et al. Unpublished data. Coverys/CRICO Closed Claims Review, 2016
With Electronic Medical Records, Doctors Read When They Should Talk Abigail Zuger New York Times 10/13/2014 Will history someday show that the electronic medical record almost did the great state of Texas in? We are in the middle of a simmering crisis in medical data management. Like computer servers everywhere, hospital servers store great masses of trivia mixed with valuable information and gross misinformation, all cut and pasted and endlessly reiterated There is no time to dig and, even worse, no time to do what we were trained to do — slow down, go to the source, and start from the beginning.
43
Role for Electronic Goals and Features of Redesigned Systems Documentation Providing access to Ensure ease, speed, and selectivity of information searches; aid information cognition through aggregation, trending, contextual relevance, and minimizing of superfluous data. Recording and sharing Provide a space for recording thoughtful, succinct assessments, assessments differential diagnoses, contingencies, and unanswered questions; facilitate sharing and review of assessments by both patient and other clinicians. Maintaining dynamic patient Carry forward information for recall, avoiding repetitive pt history querying and recording while minimizing erroneous copying and pasting Maintaining problem lists Ensure that problem lists are integrated into workflow to allow for continuous updating. Tracking medications Record medications patient is actually taking, patient responses to medications, and adverse effects to avert misdiagnoses and ensure timely recognition of medication problems. Tracking tests Integrate management of diagnostic test results into note workflow to facilitate review, assessment, and responsive action as well as documentation of these steps.
Clinical Documentation CYA
Canvass for Your Assessment
-Differential Diagnosis -Weighing Likelihoods Canvass for -Etiology -Urgency Your -Degree of certainty Assessment
Role for Electronic Goals and Features of Redesigned Systems Documentation Ensuring coordination and Aggregate and integrate data from all care episodes and continuity fragmented encounters to permit thoughtful synthesis. Enabling follow-up Facilitate patient education about potential red-flag symptoms; track follow-up. Providing feedback Automatically provide feedback to clinicians upstream, facilitating learning from outcomes of diagnostic decisions. Providing prompts Provide checklists to minimize reliance on memory and directed questioning to aid in diagnostic thoroughness and problem solving. Providing placeholder for Delineate clearly in the record where clinician should resume resumption of work work after interruption, preventing lapses in data collection and thought process. Schiff & Bates NEJM 2010
Open Loop System Water goes on the same time each day, regardless of whether it is raining or lawn is flooded Schiff A J Med 2008 49
50
Feedback – Key Role in Safety • Structural commitment patient role to play • Embodies/conveys message: uncertainty, caring, reassurance, access if needed • Allows deployment of test of time, more conservative diagnosis • Enables differential diagnosis • Emphasizes that disease is dynamic • Reinforces culture of learning & improvement • Illustrates how much disease is self limited • Makes invisible missed diagnoses visible 51
Examples of Feedback Learning Feeding back to upstream hospital Tracking persistent mysteries - spinal epidural abscess Chart correction by patients IVR follow-up post urgent care visit Radiology/pathology - UAB Berner project - systematic second reviews Dedicated Dx Error M&M 2 nd opinion cases - Best Doctors dx changed Autopsy Feedback Linking lab and pharmacy data - 7/32 MDs aware disseminated CMV - to find signal of errors (missed ↑ TSH) ED residents post admission tracking Urgent care Feedback to previous service - call back f/up systems Malpractice 52 - knock on the door
Feedback- Challenges • Effort, time, support required • Discontinuities • Can convey non-reassuring message • Feedback fatigue • Non-response not always good predictor of misdiagnosis as multiple confounders • Tampering – form of availability bias 53
Diagnosis Essentials Checklist 1. Essential Data Elements - Elements of Hx, P.exam, tests data that should be reliably obtained for every pt presenting with given sx. In many situations can reliably be done w/ computer questionnaire. 2. Don’t miss diagnoses – critical dx can present w/ sx that are fatal or have serious consequences if not recognized and rx promptly. These dx should be considered in every patient with that symptom. 3. Red flag symptoms- sx or findings (e.g. back pain with new urinary incontinence in cancer patient) that may indicate serious condition & should lead to heightened suspicion/evaluation for don’t miss dx. Schiff & Leape Acad Med 2012 54 Schiff BMJ Safety & Qual 2012
Fatigue Checklist (27 diagnoses) Obstructive sleep apnea ♠ Depression, anxiety Deconditioning *Drugs (beta blocker, clonidine, alcohol) Chronic fatigue syndrome, fibromyalgia ♠ *Infections, infectious mononucleosis, hepatitis, pneumonia, mastitis ♠ Pregnancy ♠ *Anemia Vitamin D deficiency ♠ Hypothyroidism, hyperthyroidism ♠ Hypokalemia, hyponatremia ♠ *Myocardial infarction ♠ Don’t miss ♠ Celiac disease * Often missed ♠ Disturbance of calcium, phosphorus, magnesium ♠ Polymyalgia rheumatica/Temporal arteritis Prevalence Parkinson disease Hypogonadism Myasthenia gravis ♠ *Heart failure, myocarditis Pulmonary, hepatic, renal failure Restless legs syndrome Multiple sclerosis ♠ Carbon monoxide ♠ Adrenal insufficiency, Addison’s disease B12 deficiency ♠ Botulism Ely Acad Med 2010 Black widow spider bite
Diagnosis Essentials Checklist 4 . Potential drug causes – meds that can cause the symptom. High % sx med side effects, yet infrequently considered. 5. Required referrals - When is specialist expertise or technology needed to adequately and safely evaluate the patient? Includes possible rare conditions that only specialists have sufficient experience or where required testing (biopsy or endoscopy) 6. Patient follow-up instructions and plan - Warnings that patients should receive regarding specific symptoms that should lead them to return or call. These should be in writing and include a time frame. (e.g. call if you develop rash or fever, or if you are not improved in 48 hours) Schiff & Leape Acad Med 2012 56 Schiff BMJ Safety & Qual 2012
3 rd generation Dx support Cerner with Isabel integration
Role for Electronic Goals and Features of Redesigned Systems Documentation Calculating Bayesian Embed calculator into notes to reduce errors and minimize biases probabilities in subjective estimation of diagnostic probabilities. Providing access to Provide instant access to knowledge resources through context- information sources specific “info buttons” triggered by keywords in notes that link user to relevant textbooks and guidelines. Offering second opinion or Integrate immediate online or telephone access to consultants to consultation answer questions related to referral triage, testing strategies, or definitive diagnostic assessments. Increasing efficiency More thoughtful design, workflow integration, easing and distribution of documentation burden could speed up charting, freeing time for communication and cognition. Schiff & Bates NEJM 2010
Role for Patient In Minimizing and Preventing Diagnosis Error and Delay • Push for timely access • Being patient: time & tests • Reliable follow-up, continuity • Recruiting family for support • Keen observer, reporter sx • Respecting limits on staff time, • Proactive on test results society resources • Sharing hunches • Agreeing to disagree • Curiously reading on own • Help in building, maintaining • Meticulously adhering w/ trust and communication • Getting involved with patient empiric trial regimens • Active as co-investigator organizations 60
Role for Patient In Minimizing and Preventing Diagnosis Error and Delay • Push for timely access • Being patient: time & tests • Reliable follow-up, continuity • Recruiting family for support • Keen observer, reporter sx • Respecting limits on staff time, • Proactive on test results society resources • Sharing hunches • Agreeing to disagree • Curiously reading on own • Help in building, maintaining • Meticulously adhering w/ trust and communication • Getting involved with patient empiric trial regimens • Active as co-investigator organizations Key question is : What will it take at the provider and institutional end to support these roles and help them flourish? 61
Diagnostic Pitfalls: A New Paradigm to Understand and Prevent Diagnostic Error Gordon Schiff, MD 1,2 , Mayya Volodarskaya 1 , Harry Reyes Nieva 1,2 , Hardeep Singh, MD, MPH 3,4 , Adam Wright, PhD 1,2 1 Division of General Internal Medicine and Primary Care, BWH 2 Harvard Medical School 3 Health Policy and Quality Program, Houston VA Health Services 4 Baylor College of Medicine
Specific Aims Aim 1: Develop & refine new construct: diagnostic pitfalls Clinical situations vulnerable to repeated errors that lead to missed, delayed, or wrong diagnosis Seeking to overcoming the silo’s of “cognitive” vs. “system” errors
Results Study Definition Clinical situations vulnerable to repeated errors that lead to missed, delayed, or wrong diagnosis
Specific Aims Aim 1: Develop & refine new construct: diagnostic pitfalls Clinical situations vulnerable to repeated errors that lead to missed, delayed, or wrong diagnosis Aim 2: Review multiple data sources to create list of disease-specific and general diagnostic pitfalls Aim 3: Design and test the accuracy of retrospective electronic screens (triggers) for diagnostic pitfalls using EMR data
Methods Data elements collected: • Correct and wrong diagnoses • Presenting signs and symptoms • Breakdown(s) in the diagnostic process • Diagnosis Error Evaluation and Research (DEER) Taxonomy • Reliable Diagnosis Challenges (RDC) Taxonomy
Strengths • Wide range of diseases • Breadth of high-quality data sources • Comprehensive set of diagnostic errors represented • Structured data collection tools • Consistent coder with secondary review by clinician
Limitations • Literature search limitations • Limited to keyword-tagged articles • Over-representation of individual case reports • Malpractice claims & patient safety events • Under/selective reporting • Coding may be subject to reviewer bias • Each case reviewed by research assistant and internist
Conclusions • Diagnostic pitfalls construct is useful for considering the synergies between system and cognitive diagnostic failures • Identified recurring patterns of generic and disease-specific diagnostic pitfalls • Low yield of diagnostic errors found in patient safety event reports • Pitfalls identified were predominantly related to testing issues
71
PROMISES Chart review preliminary results: Number of potential adverse events Intervention practices 140 126 120 Number of events 100 80 60 41 40 20 0 Before After • Potential adverse events in intervention practices declined by almost 70% after participation in the PROMISES program 72
PROMISES Chart review preliminary results: Number of serious potential adverse events Intervention practices 25 27 20 Number of events 15 10 10 5 0 Before After • Serious potential adverse events in intervention practices declined by 57% after participation in the PROMISES program 73
ONLY ~50-50 chance this order results in colonoscopy actually being performed ! 74
Primary Drivers Secondary Drivers Harvard Center for Primary Care Academic Engage diverse group of institutional leaders and stakeholders across the organization Improvement Ensure Organizational Collaborative Communicate how this work builds on and aligns with other initiatives across the organization Alignment Create clear organization-wide consensus for CRC screening and guidelines Colorectal Seek to understand and reduce barriers to scheduling, bowel prep, and day-of-test navigation Cancer Driver Reduce barriers for patients to communicate with care team around new/concerning Patient and Family symptoms or for help with navigating care system Diagram Engagement Engage patient partners in improvement processes 10/2014 Seek regular formal and informal patient feedback on process Develop clear protocols and algorithms, integrated into care workflow and HIT Clearly define roles, responsibilities and handoffs/interactions within care team Engage and partner with specialists Optimized Teamwork Promote culture of collaboration and teamwork Empanel patients To Reduce Missed At-Risk Patient Address risk at office visits and Delayed Identification and Identify and manage patient risk factors Diagnoses of Tracking Identify and track patients who are symptomatic, high-risk and/or overdue for screening Colorectal Cancer Develop clear care pathways for screening and diagnosis Ensure needed referral access and capacity Ensure coordinated system for scheduling, tracking referrals and tests through to referral partner Closed Loops for Develop reliable processes to support patient education around bowel prep Referrals and Tests Track and develop systems to reduce and f/up on no-shows/failure to schedule Ensure reliable and timely communication of test results to patients Develop system for timely, reliable follow-up of abnormal test results Leverage Health Ensure structured data capture and reliable update of family history, diagnoses and Information Technology symptoms. and Population-based Create population-based outreach and tracking systems. Management and Develop reports to identify and notify patients due for screening and patients that are hard to reach. Outreach Identify and provide needed resources for population management
10 Principles for Engaging Dx Error 1. Diagnosis as a multi-dimensional construct 2. reliance on human memory 3. Leverage HIT: clinical documentation, feedback/f/up 4. Co-production of Dx (patient; team) 5. Need for new science and culture of uncertainty 6. Culture/praxis of conservative dx 7. Foundation of trusting, continuous relationships 8. Linkages Dx & Rx (diagnosing what matters) 9. Special role/responsibility iatrogenic dx 10. S ynergies with “disease specific” issues, pitfalls Schiff, Institute of Medicine Testimony 8/2014 76
Tampering • Reflex actions in response to errors • Need to understanding/diagnose difference between special cause vs. common cause variation • Responding to special cause as if it was common cause analogous to availability bias – where fail to weigh true incidence, instead overweigh more vividly recalled event. 77
Suboptimization How to recognize and avoid • Suboptimization refers to the process of optimizing one element of the system at the expense of the other parts of the system and the larger whole. – Every lab perfecting own ordering, reporting system – Every unit in hospital its own system – Ditto every practice and doctor • Workarounds as both symptoms of and contributor to problems 78
Workarounds • Most diagnostic processes developed in an ad hoc fashion over time; filled with workarounds and unnecessary steps and opportunities for error. • Workaround=bypass problems – Often creative, innovative, successful – But temporary, suboptimal to fixing problem – Can mask embedded problems, inhibit solving – Worse yet, may introduce new problems 79
Redundancy • Duplication of critical components of a system with the intention of increasing reliability of the system, usually in the case of a backup or fail- safe, or parallel systems • However to extent redundancy increases complexity, dilutes responsibility and even encourages risk taking, should be questioned as safety strategy. • Redundant systems can be costly, using valuable resources that could be freed for more reliable, productive system. 80
The question that drives safety work in a just culture is not who is responsible for failure, rather, it asks what is responsible for things going wrong. What is the set of engineered and organized circumstances that is responsible for putting people in a position where they end up doing things that go wrong?” ― Sidney Dekker, Just Culture
Confidence vs. Accuracy – Not Well Calibrated? Shynkaruk and Thompson Memory & Cognition 2006
Over- Inadequate confidence Feedback Arrogance Spotty Follow-up Defensive- ness Time Pressures Malpractice Fail to Share Fears Uncertainties
Results: Most common DEER errors (n=1208) DEER Taxonomy Frequency Subcategory % (N) Failure in ordering needed test(s) 17% (164) Failure to consider correct diagnosis 12% (112) Failed/delayed follow-up of abnormal test result 9% (83) Failure in weighing critical piece of history data 8% (75) Failure/delay in ordering referral 6% (62)
Results: Most common RDC barriers (n=1041) RDC Taxonomy Subcategory Frequency Test Follow-Up Issues 12% (103) Recognition of Acuity/Severity 9% (73) Test Performance/Interpretation 7% (62) Diagnosis of Underlying Cause 6% (51) Fragmentation of Care 6% (48)
Results: Top 10 Missed or Delayed Diagnoses Diagnosis by Diagnosis by Frequency Frequency disease system Colorectal cancer 38 Oncology 225 Lung cancer 36 Neurology 89 Breast cancer 20 Cardiology 50 Myocardial infarction 20 Infectious disease 46 Prostate cancer 18 Other 40 Stroke 15 Dermatology 37 Sepsis 13 Gastroenterology 35 Bladder cancer 10 Pulmonology 33 Pulmonary embolism 9 Rheumatology 29 8 Brain Hemorrhage Orthopedics 16
Older ways thinking Newer, better ways to think about Diagnosis, Error about Dx and Improvement Good diagnosticians get it Inexact science Goal: ↓ error & cycle times via right 1st time 100% time more reliable systems and f/up Skillful diagnostician lore Quality Dx based on finely tuned distributed network/team of people and processes Dx-- the doctor’s job Co-production Dx between clinicians (lab/radiol) & patient Pts seen as anxious, Patients as allies. exaggerating, demanding, Need to engage, negotiate, unreasonable expectations address fears Schiff BMJ Qual Safety 9/13
Clinical Practice Clinical data-more the better Targeted, well organized data key Order lots of test to avoid Appreciation test limitations, missing Dx judicious ordering More specialty referrals. Pull systems to lower barriers for Utilization barriers (co-pays, raising questions, real time virtual prior auth) consults Frequent empirical drug Conservative use of drug trails to trials when uncertain avoid confusing the picture MD attention to screening Automating, delegating clerical and preventive care functions, teamwork Schiff BMJ Qual Safety 9/13
New Thinking About Diagnosis, Errors Mostly cognitive miscues Many rooted in processes, system failures Errors either system or Multifactorial w/ interacting cognitive cognitive--system factors Reactive approach learning Proactively seeking out high risk about errors situations Clinicians reactions: denial, Culture of actively seeking to defensive, others to blame learn from each other and share and they making errors too Dreading diagnostic Welcoming intellectual & dilemmas professional challenge Dx as distinct labels, events Dx as fuzzy, multifactorial, evolving over time Schiff BMJ Qual Safety 9/13
Documentation Communication Documentation-dumb, Documentation-tool, friend, time consuming, CYA canvass for assessment to share assessments, uncertainty, reflect Say & write as little as Share uncertainties to maximize possible as could be used communication and engagement again you legally Don’t let pts know about Pts often find out about errors errors lest they sue, feel anyway; we should anticipate, guilty engage questions Call if not better; no news is Systemic proactive good news feedback and follow-up Eschew/hide uncertainty Leverage, disclose learn from uncertainty Schiff BMJ Qual Safety 9/13
Global Solutions Vigilance, defensive Drive out fear. Making Dx practice stimulated by fear challenges fun again malpractice suits More accountability, Clinician engagement, trust, incentives, P4P culture changes, Yet to be defined best practices More rules, requirements Standardization, flexibility Exhortations to have "high Situational awareness- index of suspicion" appreciation limitations tests, memory, diseases Redundancies, double Highly reliable systems safer checks than multiple halfway systems Reflex changes in response Avoiding tampering; understand to errors special common cause variation
Ricardo Levins Morales Art Studio 97
So what, concretely should I do? 1. Diagnosis error case conferences /M&M’s -Extreme safety culture & learning climate 2. Hard wire feedback/follow-up, closed loops -Proactive systematic follow-up; Open Notes 3. Engineering uncertainty into diagnosis -Differential diagnosis, sharing uncertainties w/ pt 4. HIT/Electronic documentation redesign -Get out of the way; get (helpfully) into the fray -Eliminate useless info/noise; voice in real info Involve patients in all of the above
Principles for Leveraging, Learning, Lessening Diagnostic Error in Medicine • Reliance on memory • New science of diagnostic uncertainty • Linkages Dx & Rx • Leverage HIT • Re-engineering diagnosis as a system • Diagnostician of future
IOM Quality Reports
Recommend
More recommend
