Patient Safety through Intelligent Procedures in medication Human Factors Methods to Support the Experts’ Review of Automatically Detected Adverse Drug Events Nicolas LEROY Brian BJØRN , Adrian BACEANU, Marie-Catherine BEUSCART-ZEPHIR 1 MIE 2009 – Sarajevo, August 31 2009
PSIP: a research project (7 th FP – ICT) Consortium: 13 partners 1/ Hospitals France, Denmark With / without CPOE 2/ Industry : Oracle, IBM, Medasys (CPOE editors) Vidal (pharmaceutical Kbase) 3/ Academic teams Data & Semantic mining, Decision Support Systems, Human Factors Engineering Duration: 40 months (Jan 08 April 2011) 2 MIE 2009 – Sarajevo, August 31 2009
Validation of rules in PSIP PSIP Rule generation Data mining 3 MIE 2009 – Sarajevo, August 31 2009
Validation of rules in PSIP PSIP Rule Validation of the rule generation Knowledge based • Literature review Data mining • Knowledge about drugs chart review Validation in the clinical context 4 MIE 2009 – Sarajevo, August 31 2009
Validation of rules in PSIP PSIP Rule Validation of the rule Integration in a generation CDSS Module Knowledge based Detection of • Literature review dangerous Data mining • Knowledge about situation drugs Feed back to chart review clinician Validation in the clinical context 5 MIE 2009 – Sarajevo, August 31 2009
Validation of rules in PSIP PSIP Rule Validation of the rule Integration in a generation CDSS Module Knowledge based Expected benefit : Detection of Help clinicians to improve the patient • Literature review dangerous Data mining safety • Knowledge about situation drugs Potential Risk : Over alerting Feed back to chart review clinician Validation in the clinical context 6 MIE 2009 – Sarajevo, August 31 2009
Validation of rules in PSIP PSIP Rule creation Validation of the rule Integration in a CDSS Module Knowledge based Detection of • Literature review dangerous Data mining • Knowledge about situation drugs Feed back to chart review clinician Validation in the clinical context 7 MIE 2009 – Sarajevo, August 31 2009
Validation of rules in PSIP PSIP Rule creation Validation of the rule Integration in a The Expert review requires : CDSS Module Knowledge based • Human experts review medical records • Decide whether the rules properly Detection of • Literature review explained the observed abnormality dangerous Data mining • Knowledge about situation drugs semantic Feed back to mining chart review clinician Validation in the clinical context 8 MIE 2009 – Sarajevo, August 31 2009
Validation of rules in PSIP PSIP Rule creation Validation of the rule Integration in a Our objective: CDSS Module Knowledge based • Develop a methodology supporting the evaluation and the improvement of the Detection of • Literature review PSIP rules dangerous Data mining • Knowledge about • Involve the end users in this evaluation situation drugs process semantic Feed back to mining chart review clinician Validation in the clinical context 9 MIE 2009 – Sarajevo, August 31 2009
Chart review - Method The chart review is performed in two different hospitals : – The Region H hospital (Copenhagen – Denmark) • Two physicians of the patient safety unit • Work in progress – The hospital of Denain (France) • The Head Pharmacist • The head clinician of the internal medicine department • Review completed Data: 80 hospital stays, 40 “abnormal” (detected by a PSIP rule) vs. 40 “control” 10 MIE 2009 – Sarajevo, August 31 2009
Chart review - Method Chart review Consultation of the files 11 MIE 2009 – Sarajevo, August 31 2009
Chart review - Method Chart review Consultation of the files 12 MIE 2009 – Sarajevo, August 31 2009
Chart review - Method Rule evaluation in the Chart review clinical context Consultation Analysis Evaluation of the files of the of the rule relevance 13 MIE 2009 – Sarajevo, August 31 2009
Chart review - Method Agree with the rule ? Rule evaluation in the Chart review clinical context Consultation Analysis Evaluation of the files of the of the rule relevance 14 MIE 2009 – Sarajevo, August 31 2009
Chart review - Method Agree with the rule ? Rule evaluation in the validated Chart review clinical context Yes Yes Consultation Analysis Evaluation of the files of the of the rule relevance 15 MIE 2009 – Sarajevo, August 31 2009
Chart review - Method Agree with the rule ? Rule evaluation in the validated Chart review clinical context Yes Yes No Not validated Consultation Analysis Evaluation of the files of the of the (this rule could not explain rule relevance the case under review) 16 MIE 2009 – Sarajevo, August 31 2009
Chart review - Method Agree with the rule ? Rule evaluation in the validated Chart review clinical context Yes Yes No Not validated Consultation Analysis Evaluation of the files of the of the do not rule relevance Uncertain know 17 MIE 2009 – Sarajevo, August 31 2009
Chart review - Method Agree with the rule ? Rule evaluation in the validated Chart review clinical context Yes Yes No Not validated Consultation Analysis Evaluation of the files of the of the do not rule relevance Uncertain know Think aloud methodology 18 MIE 2009 – Sarajevo, August 31 2009
Think aloud - Method • Objective : track the user mental and behavioral action with a system • Instructions :The experts were asked to “think-aloud”. • Technical device : A recording system allows to track all the experts’ actions with the Expert Explorer application • Transcription : All the experts’ verbalizations are typewritten and coded. 19 MIE 2009 – Sarajevo, August 31 2009
Coding of the verbal protocols Creation of the coding scheme • Two ergonomists read the comments of the experts • They listed all the categories of explanation of rule rejection • 11 categories were identified and organized in a two dimensional coding framework. Coding of the verbalizations 1. Each ergonomist coded independently all the verbalizations 2. A debriefing session allowed to clear the rare disagreements 20 MIE 2009 – Sarajevo, August 31 2009
Results The hospital of Denain (France) 21 MIE 2009 – Sarajevo, August 31 2009
Results – Evaluation of the rules 22 MIE 2009 – Sarajevo, August 31 2009
Results – Evaluation of the rules Independently of the clinical context, Do you agree with the rule ? 23 MIE 2009 – Sarajevo, August 31 2009
Results – Evaluation of the rules Independently of the clinical context, Do you agree with the rule ? No 2% Do not know 19% Do not know 12% Yes 86% Yes 81% Expert 1 Expert 2 24 MIE 2009 – Sarajevo, August 31 2009
Results – Evaluation of the rules Do you think that the rule applies to the case under review ? 25 MIE 2009 – Sarajevo, August 31 2009
Results – Evaluation of the rules Do you think that the rule applies to the case under review ? No 48% No 51% Do not know 12% Do not know 33% Yes 40 % Yes 16 % Expert 1 Expert 2 26 MIE 2009 – Sarajevo, August 31 2009
Results – Evaluation of the rules Do you think that the rule applies to the case under review ? No 48% No 51% 94% 79% Do not know 12% Do not know 33% Yes 40 % Yes 16 % Expert 1 Expert 2 Jha Hwang 2008 2008 27 MIE 2009 – Sarajevo, August 31 2009
Results – Evaluation of the rules Do you think that the rule applies to the case under review ? No 48% No 51% 94% 79% Do not know 12% Do not know 33% Yes 40 % Yes 16 % Expert 1 Expert 2 Jha Hwang 2008 2008 28 MIE 2009 – Sarajevo, August 31 2009
Results: Coding scheme Categorization of comments Effect Cause Independent of the clinical context Depending of the clinical context Problems with Data 29 MIE 2009 – Sarajevo, August 31 2009
Results: Coding scheme Categorization of comments Effect Cause Independent of the clinical context Depending of the clinical context Problems with Data 30 MIE 2009 – Sarajevo, August 31 2009
Results: Coding scheme Categorization of comments Effect Cause Independent of the clinical context Depending of the clinical context Problems with Data 31 MIE 2009 – Sarajevo, August 31 2009
Results: Coding scheme Categorization of comments Effect Cause Independent of the clinical context Depending of the clinical context Problems with Data 32 MIE 2009 – Sarajevo, August 31 2009
Results: Coding scheme Categorization of comments Effect Cause Independent of the clinical context Depending of the clinical context Problems with Data 33 MIE 2009 – Sarajevo, August 31 2009
Results: Coding scheme Categorization of comments Effect Cause Independent of the clinical context Depending of the clinical context Problems with Data 34 MIE 2009 – Sarajevo, August 31 2009
Examples of verbalizations 35 MIE 2009 – Sarajevo, August 31 2009
Recommend
More recommend
