Global Trigger Tool An assessment of ADHB experience Rob - PowerPoint PPT Presentation

Global Trigger Tool An assessment of ADHB experience Rob Ticehurst, Principal Pharmacist Medication Safety (on behalf of Colin McArthur, Medical Advisor – Quality & Safety) Auckland District Health Board

Setting the scene  ADHB: 3400 admissions/month  NZ Adverse Events study (Davis et al)  Case note review of 6500 patients • all harms: 12.9% • significant harm: ~4% of admissions = ~130/month for ADHB • Preventable significant harms: ~45/month  ADHB Reported significant harm events (SAC 1/2) ~6/month   GTT  IHI suggest AEs in ~30% of admissions • = ~1000/month for ADHB

There’s quite a range!  440 harms (Davis)  1000 harms (IHI)  6 reported (ADHB SAC1/2) Can’t review every patient Sample size is going to be important….

Global Trigger Tool: Methods ADHB ~3400 discharges per month  Review 20-40 discharges (>24h admission) per month  ~1% ADHB discharges  “ unintended physical injury resulting from or contributed to by medical care that requires additional monitoring, treatment or hospitalisation, or results in death ”  Acts of commission (not omission)  Irrespective of preventability

Global Trigger Tool: Outputs  Adverse events  Unintended physical injury caused or contributed to by treatment (from patient’s viewpoint)  Classification of event type and severity of harm  Multiple (separate) events counted  Events per 1000 patient-days (typically 90)  Events per 100 discharges (typically 40)  % of discharges with any events (typically 30%)

Presenting the results – Run chart Simple time plot of number of harms identified from a small sample presented as harms per 1000 pt days.

Same Data

 A very brief introduction to control charts4.flv

A closer look… Statistical Process Control  Measure current / optimised process  mean ± standard deviation (SD)  Regular small samples (n)  Sample means normally distributed  Control limits usually 3 x SD

Adverse event “sampling” An example  1000 discharges  Random sample of n=20  Believe our AE rate 20-80% How much variation is there from sample to sample? Is our sample representative of the population?

Example 1 - 30% AE rate 60% 50% 40% 30% 20% 10%

Example 2 - 30% AE rate 60% 50% 40% 30% 20% 10%

Events 100.0 120.0 20.0 40.0 60.0 80.0 0.0 25/07/2011 Adverse Events per 8/08/2011 1000 patient-days 22/08/2011 5/09/2011 19/09/2011 3/10/2011 17/10/2011 31/10/2011 Date 14/11/2011 28/11/2011 12/12/2011 26/12/2011 9/01/2012 23/01/2012 6/02/2012 20/02/2012 5/03/2012

% admissions with adverse events 100% 90% 80% 70% 60% 50% 40% 30% 27% 20% 10% 0% 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

Cummulative data accuracy 95% confidence limits 60% 50% 40% 30% 20% 10% 0% 2 4 6 8 10 12 14 16 18 20 22 24 26 28 30 32 34 36 38 40 42 44 46 48 50 52 Week Fortnightly samples of 20, >1000 admissions

Sample size required  Trust me on this…. Margin of Sample size error Suspected AE rate = 30% 1% 8281 5% 364 With a sample size of 10 10% 100 we can be 95% confident 20% 29 30% 14 that our actual value is 40% 8 between 0% and 65% (+/- 50% 5 35%) How much uncertainty are you prepared to accept? How much precision do you require?

Conclusion 1  Small sample size  Wide confidence limits – our actual AE rate could be anywhere between 0 and 65%  Wide limits on control chart  Control charts of overall AE rate highly unlikely to be useful to demonstrate the effectiveness of an intervention  IHI suggest otherwise….

What if you aggregate the data? ADHB -196 adverse events from 512 admissions over 20 months

Drill down further Medication adverse n Nosocomial infections n events Wound 14 Hypotension/bradycardia 12 Pneumonia 9 Sedation & delirium 10 CLAB 4 Nausea & vomiting 10 Urinary tract 6 Constipation 7 Bacteraemia 7 Chemo complications 8 Other 10 Acute kidney injury 6 Procedure - related n Hypoglycaemia 2 Bleeding 10 Other 10 Ileus 5 Other 27 ADHB - 196 adverse events from 512 admissions over 20 months

Adverse Event Severity  E: Temporary harm to the patient and required intervention – 58%  F: Temporary harm and required initial or prolonged hospitalization – 38%  G: Permanent patient harm – 0.5%  H: Intervention to sustain life – 2%  I: Patient death – 1.5% ADHB - 196 adverse events from 512 admissions over 20 months

High frequency, low severity 795 admissions to 3 large tertiary USA hospitals Classen et al, Health Affairs 2011; 30(4):581-589

Conclusion 2  Aggregated data can give us an idea of what is happening  Identifies areas for further investigation  GTT not suitable for demonstrating outcomes of any interventions

Summary  Small random samples provides trend over time (but with wide control limits)  Large short term variability  Accurate population rate can only be estimated over long periods (12 months +)  Statistically significant change will take years to demonstrate

Summary  Run charts are not useful  GTT identifies high frequency “minor” events not detected by other methods  Interventions require more targeted/accurate measures (eg CLAB, surgical site infection)  Future:  Feed into improvement programme priorities?  Intermittent larger sampling?