Using Voice-Activated Conversational Interfaces for Reporting Patient Safety Incidents: a Technical Feasibility and Pilot Usability Study OWEN SUN JESSICA CHEN & FARAH MARABI 30 th July 2018 Centre for Health Informatics AUSTRALIAN INSTITUTE OF HEALTH INNOVATION
“Alexa, Play ‘Africa’ by Toto” AUSTRALIAN INSTRITUTE OF HEALTH INNOVATION 2
A “Voice - Activated Conversational Interface” Voice-Activated Conversational Interface – Voice-activated computer systems that interact on a turn-by-turn basis through spoken language. N. Fraser 1997 AUSTRALIAN INSTRITUTE OF HEALTH INNOVATION 3
Background Patient Safety Incident Reports are the “cornerstone” of initiatives to improve health service safety. Pham et al. 2011 System quality of incident reporting is the second largest factor in both facilitating and preventing effective reporting. Archer et al. 2017 Voice-activated systems reduce turnaround times in healthcare settings. Johnson et al. 2014 AUSTRALIAN INSTRITUTE OF HEALTH INNOVATION 4
Objectives 1. Assess technical feasibility of speech recognition software in incident reporting 2. Conduct a pilot study of the application’s usability in clinical contexts AUSTRALIAN INSTRITUTE OF HEALTH INNOVATION 5
Method – Technical Feasibility A Voice-Activated Conversational interface was built: ― Modified online template ― Javascript, HTML, CSS & Mozilla’s “ SpeechRecognition ” interface Responses stored as text files in Javascript Object Notation (JSON) on MongoDB’s database Hosted online through Heroku AUSTRALIAN INSTRITUTE OF HEALTH INNOVATION 6
Method – Pilot Study • Two participants: Specialist Medical Doctor & Pharmacist • Reported two mock scenarios ― Simple vs. Complex • Usability rated on the System Usability Scale ( SUS ) ― 10 item questionnaire that reliably captures usability ratings • Additional comments recorded AUSTRALIAN INSTRITUTE OF HEALTH INNOVATION 7
Simple Scenario 1. Describe the incident when you were using your computer and its peripherals (e.g. printer)? Tramadol is prescribed for a patient on a selective serotonin reuptake inhibitor (SSRI). 2. What was the result? Describe actual and potential consequences. The patient developed symptoms of serotonin syndrome: sweating, tremor, confusion and was noted to be hypertensive, tachycardic and hyper-reflexive. 3. What did you do to fix the problem? Advised the patient to stop intake of Tramadol immediately. Alternative SSRI compatible medication was considered. 4. Why did this incident happen? The prescribing software did not provide any warning about the interaction. 5. How could the incident have been prevented? The prescribing software should have provided a warning about the interaction. After Magrabi et al. 2015 AUSTRALIAN INSTRITUTE OF HEALTH INNOVATION 8
Conversational Interface AUSTRALIAN INSTRITUTE OF HEALTH INNOVATION 9
Results – Technical Feasibility Application was successfully created ― Responses recorded and stored on an online database AUSTRALIAN INSTRITUTE OF HEALTH INNOVATION 10
Verdict: Easy to use, but… • Participants found the system easy to use , quick to learn and not unnecessarily complex • SUS Scores: 97.5, 77.5 (above average of 68) • Regarding Graphical User Interface ( GUI ): ― Minimal design ― Minimal Interaction • Suited for general purpose use in health contexts without specific training AUSTRALIAN INSTRITUTE OF HEALTH INNOVATION 11
Issues Encountered 1. Voice Recognition 2. Socio-Technical 3. Individual AUSTRALIAN INSTRITUTE OF HEALTH INNOVATION 12
Issues with Voice Recognition • Specific technical terms were not picked up by the interface: ― e.g. “ nephrectomy ” • General speech misrecognition ― “a patient” was written as “APT” ? AUSTRALIAN INSTRITUTE OF HEALTH INNOVATION 13
Field Example Describe the incident when you were using last night at about 11 o'clock APT on the mental ward was given Tramadol when it your computer when there are some very high dose of ssris and its peripherals (e.g. printer) What was the add round midnight the patient develops symptoms of serotonin syndrome we result? Describe observe sweating tremor and confusion upon further investigation when I was that actual & potential she was hypertensive tachycardic and hyperreflexive consequences. OFFICE | FACULTY | DEPARTMENT 14
Socio-Technical Issues • Sensitive patient info and clinical errors or system problems being discussed out loud • Background noise, especially in loud and busy healthcare environment AUSTRALIAN INSTRITUTE OF HEALTH INNOVATION 15
Challenges to the Reporter • Having to plan out a response before speaking makes it more difficult to give a coherent response ― Partially mitigated by going back to edit with keyboard and mouse AUSTRALIAN INSTRITUTE OF HEALTH INNOVATION 16
Discussion • Task level incompatibility with speech recognition and processing information ― Incident reporting as story telling vs. clinical notes • Voice recognition issues, socio-technical barriers and challenges to reporting ― These must be addressed before voice-activated reporting systems are implemented • Future work into voice recognition should focus on these issues first • Further applications can be made from open-sourced tools AUSTRALIAN INSTRITUTE OF HEALTH INNOVATION 17
Acknowledgements Special thanks to: Associate Professor Farah Magrabi Ms Jessica Chen Additional thanks: The medical doctor and pharmacist for their assistance with pilot testing; Mr Tony Wu and Mr Vitaliy Kim for their assistance in programming the conversational-interface. AUSTRALIAN INSTRITUTE OF HEALTH INNOVATION 18
Thank You Owen Sun B Advanced Science (Psychology/Statistics), UNSW o.sun@student.unsw.edu.au
Recommend
More recommend
