supply-demand balance at the bedside: An exploration of data sources - PowerPoint PPT Presentation

1 Toward dynamic assessment of supply-demand balance at the bedside: An exploration of data sources and methods Dana Womack, PhD, RN Department of Medical Informatics Oregon Health & Science University

2 “When I’m really stretched I’ll stop offering help to others. I’ll stop trying to build rapport with my patients. I’ll prioritize the essential things… I’ll defer assessments on the patient who is not as sick, or defer clean-ups. If someone is having a blood pressure crisis, the other patient might have to wait. And it feels bad, but that’s just what has to happen .” — Intensive care RN

3 Hospital clinician overload Clinician overload can lead to • elevated patient risk • clinician burnout • staff turnover Hospitals cannot staff to peak demand, as labor costs already represent ~ 60% of the operating budget Need: Proactive & targeted response to overload “hot spots” as they emerge 3 3

4 Workplaces are increasingly quantified Can we harness ambient data for good? Image: www.omnicell.com Image: www.vocera.com Image: www.kronos.com 4 Image: www.wnyc.org/story/213967-pager

Common pattern of adaptive system failure: 5 Decompensation Escalating Disturbance Target system performance Compensation Extra effort exerted to sustain target performance Decompensation Escalating Degraded Control Performance declines as system adaptive capacity is performance exhausted Adapted from Woods D, Branlat M. Basic patterns in how adaptive systems fail. 5 In: Resilience Engineering in Practice: A Guidebook; 2011. p. 127-44.

6 Is it windy today? 6 Image: https://loveandworktour.files.wordpress.com/2015/08/walking-against-the-wind.jpg

7 Yes! Adaptation 7 Image: https://loveandworktour.files.wordpress.com/2015/08/walking-against-the-wind.jpg

8 Study questions 1. How do RNs recognize strain today? 2. Do work environments contain “digital echoes” of strain? 3. It is feasible to provide early warning of work system strain? 8

9 Methods CRISP-DM Data Mining Process Business understanding Data understanding Modeling Data preparation Evaluation • Activity feature • Environmental scan Focus ‒ Definition groups • Data acquisition & ‒ Generation association ‒ Selection Frontline RNs • Classification of shifts by unplanned overtime Time & attendance (proxy measure of strain) Medications Data Gathering: Interview guide Phone Analysis • Development of coding template by Nurse call three qualitative researchers • Content coding in Quirkos software 9

10 Setting: Two patient care units in an academic medical center Per shift Per year Unit Beds RNs/shift Ratio Distinct RNs, Distinct (mean) (day shift) including floats patients Medical 16 11 1:1-2 156 1,464 ICU Medical- 20 6 1:4 105 1,485 surgical Scope: Day shift 10

11 1. Business Understanding 11

What environmental changes occur 12 during times of strain? Empty nurses’ station ‒ RNs ping-ponging between rooms ‒ Increase in phone call volume ‒ Frequent task switching ‒ Many PRN medication requests ‒ Delayed response to call lights ‒ Multiple patients with delirium, ‒ No one sitting down to chart or 2+ person assist 12

13 How do RNs adapt to increasing strain? Increasing strain Degraded system performance 13

14 2. Data Understanding 14

15 Ambient activity data is not yet integrated Ambient data Electronic health record Medication dispensing Pharmacy Phone Paging Imaging Lab Team Patient Nurse Others Others Surgery call Time & Documentation attendance Non-integrated Integrated 2 patient care units 366 shifts >400,000 records 15

16 3. Data Preparation 16

Raw log files shift & staff-centric data 17 Disparate staff IDs Consistent staff IDs Medication log files Shift ID Phone transactions 17 Exemplary data from a single medical-surgical work shift

Data required significant preprocessing Nurse Phone call • Room number Meds • Date-time of call • RN name • Call type • Date-time of call Time • Call type No patient-room link • Device unit Manual ID mapping No RN-room link • Patient name • Date-time of dispenses • Unit • Dispensing RN name • Employee ID • Drug name RN name RN name only – outdated, inconsistent format • Dates worked • Total hours worked Dispensing role not indicated • Start & end (float RNs only) • Pay codes Duplicate records 18 Lack of start/end time for regular staff

19 Data exploration: Observable shift rhythms Medication count Hour of day Medical-surgical unit 366 day shifts 19

20 Adaptive strategy: Help passing meds within a single shift Key: Gave med help Medication count Received med help Meds given to RN’s own pts Nurse RN 1 RN 2 RN 3 RN 4 RN 5 RN 6 Managers do not have access to this info today due to lack of data integration 20

21 4. Modeling 21

22 Unplanned overtime: Binary proxy measure of work system strain Overtime >3 hours: Overtime ≤ 30 min. Unplanned overtime likely an extra shift excluded from definition >30 min. to <3 hours Count, RN instances Overtime duration in hours, MICU Frequency of unplanned overtime by unit, FY 2016 Unit % Hours % Shifts MICU 1.16% 52% Med-surg 0.17% 8% 22

23 Visualization of differences across outcomes Cross-assignment meds (help) Aggregate minutes spent on phone No overtime Unplanned overtime No overtime Unplanned overtime Source: MICU 23 Shifts: 366 day shifts

24 MICU selected features Activity & workplace characteristics % IV push medications % patients assigned to same RN as yesterday Differences in activity across RNs, patients % Experienced, non-float RNs Adaptive Strategies Aggregate minutes on phone # cross-assignment med admin (helping) 24 Image: https://loveandworktour.files.wordpress.com/2015/08/walking-against-the-wind.jpg

25 5. Evaluation 25

Repeatable process for insight creation 26 Feature definition, generation Feature selection Machine learning classification Train & test a Present Define features Identify features Predict support vector that reflect activity unplanned that differ by machine (SVM) overtime & adaptation outcome state Absent algorithm 4-fold cross validation train & test 75% data for training 25% Test X 4 92 shifts 274 MICU shifts Repeat for multiple Hour 0-4 timeframes, to 0-6 assess ability to 0-8 predict outcome during a work shift 0-10 0-12 Full shift 26

27 Classification of workplace strain – MICU 4-fold train-test Overall results (mean) MICU Sensitivity | 1 2 3 4 Accuracy Conf. Int. Kappa Shift hour Specificity Hours 1-4 53.7% 54.2% 51.6% 50% 52 52.4% .4% .41 .41 – .62 .62 .05 .05 .46 .46 | | .59 .59 7am – 11 am Shift hours Hours 1 - 6 61.1% 57.7% 59.7% 58.1 59.2% 59 .2% .47 .47 – .68 .68 .16 .16 .57 .57 | | .60 .60 7am – 1 pm Hours 1 – 8 60% 59.4% 58.9% 52.5% 57 57.7% .7% .46 .46 – .67 .67 .15 .15 .52 .52 |.63 |.63 7am – 3 pm Hours 1 – 10 64.9% 59.1% 66.5% 54.6% 61 61.3% .3% .50 .50 – .71 .71 .22 .22 .58 .58 | | .65 .65 7am – 5 pm Hours 1 – 12 63.3% 70.3% 59.4% 61.1% 63.5% .5% .55 .55 – .74 .74 .30 .30 .58 .58 – .68 .68 7am – 7pm Closed training: 72 72.3% .3% .68 - .77 .68 .77 .45 .45 .64 .64 | .77 .77 27

28 Conclusions 28

29 Key Findings  Ambient data contains echoes of signs of strain ‒ Environmental & behavioral  Ambient data is underutilized for purposes of care improvement ‒ Produced in real-time, provides granular observability of work  It is feasible to use ambient data to characterize system strain ‒ Unplanned overtime predicted 8 – 10 hours into a shift (MICU) 29

30 Limitations  As is true for any recall methodology, discussion of past workplace events carries potential for participant recall bias  Study conducted on 2 units at a single facility, findings may not generalize to other units or facilities  Derived and manually associated ambient data may contain inaccuracies 30

31 Implications  Dynamic workplace monitoring could augment charge nurse decision making re: mid-shift resource requests  Integration of ambient data is needed • Create staff & team-centric views of data • Facilitate use of ambient data in healthcare delivery research  Data exploration of temporal patterns of activity can identify improvement opportunities • Level-loading work • Development of strain-relieving interventions 31

32 Recommendations for future studies 1. Expand the collection of computable signs of strain • Additional data sources, feature types & time granularities Employ higher fidelity outcome data 2. • Hourly RN to patient ratios, periodic subjective ratings • Additional outcomes e.g. missed care, delayed discharge, others Transition to field-based, participatory research 3. • Engage frontline staff in refining activity features, developing real-time data displays • Develop early warning systems & organizational interventions 32