2018 Quro: Facilitating user symptom check using a personalised chatbot- oriented dialogue system Shameek Ghosh, Sammi Bhatia, Abhi Bhatia July 2018
Outline • Understanding a patient’s journey • Background of the Problem • Facilitation of symptom check using Quro • Solution Description • Experimental Results • Conclusion
A Patient’s Journey TODAY Patient books an appointment or Patient experiences Doctor offers a diagnosis walks in, describes the symptoms, concern about and explains context. symptoms. certain conditions. Assessment Prescription Online Search Feel Sick Appointment Diagnosis Patient jumps online, Doctor asks about medical Doctor prescribes starts searching about history, conducts medications, refers to symptoms, gets anxious examination. specialist, requests tests from incorrect generic or other responses results. Medius – Technology Nurturing Humanity Driven By Patient Driven By Doctor Driven By Patient
Problem Pre Consultation 84% of users of the internet incorrectly search for health related information online 1 During Consultation 75% of all GP visits are for minor ailments, repeat prescriptions, referrals - a heavy burden on the healthcare system 1 . Post Consultation $1.2b per year is the cost to Australian healthcare system due to non adherence of medication or treatment regimes - triggering readmissions 2 . 1. https://www.healthdirect.gov.au/health-information-online-facts-or-fiction 2. http://6cpa.com.au/wp-content/uploads/National-trial-to-test-strategies-to-improve-medication- compliance-in-a-community-pharmacy-setting-Full-Final-Report-.pdf
3 rd Leading Cause of Death No of doctors per 1000 population diagnostic error 251,000 4.78 Leading causes 3.9 3.59 3.31 of 2.15 death cancer 1.1 0.64 0.59 585,000 heart disease India South China United United Australia Germany Spain 611,000 Africa Kingdom States of America
Primary Healthcare is crippled with myriad problems • Overburdened Doctors. • Limited time with patients. • Delayed Diagnosis. • Unsatisfied Patients. • Treatment Non-Adherence.
The problem with rule-based models
Symptom check by Quro • A personal health assistant pow ered by Artificial Intelligence and developed by Medius Health. • Quro is a goal-directed conversational - bot for primary care. • Quro explores user ’s symptoms, identifies likely causes of conditions and helps them decide w hat to do next and w here to go.
Large-scale Data Extraction and Careful Curation 7M+ data points describing the relationships between … • 8K+ Symptoms • 2K+ Diseases • 4K+ Causes & Risk Factors
Salient Features of the Quro Chatbot and Triage System Online instant medical triage: Online users want to know if their conditions require going to emergency care, visiting the GP, or remain at home and rest Improving engagement through an easy-to-use user interface and better user experience Using natural language processing to make sense of the user’s demands followed by sequential symptom question answering using a medical knowledge graph Ensemble models involving disease text embedding models for generation of a shareable pre-assessment report for a user for sharing with a GP
Overall architecture of Quro
Word Embedding Models Represent words in a continuous vector space For each word, the vector could reflect syntactic and semantic patterns such as the degree of similarity between words Neural Network is used to generate a vector matrix for word text in the corpus Visualizing and clustering medical text
Continuous evaluation process for condition pre-assessment: Current status Evaluation Criteria At least 1 of the top 3 reported conditions is a correct assessment 2 out of 3 reported conditions were expected conditions by our in-house clinical experts Datasets used for testing 30 clinical vignettes curated by internal experts from primary case notes On-going evaluations across 10 diseases
Evaluation Results for triage pre-assessment: Current status Initial evaluation using 30 patient vignettes in two different test criteria, showed an accurate outcome in 25 out of 30 cases (83.3%) and in 20 out of 30 cases (66.6%).
On-going Study: List of diseases for word embeddings Investigated Disease Infectious Gastroenteristis (IG) Cholecystitis (CTS) Pelvic Inflammatory Disease (PID) Benign Prostatic Hyperplasia (BHP) Celiac Disease (CD) Ulcerative Colitis (UC) Menopause (MNP) Gastroesophageal Reflux Disease (GERD) Polycystic Ovarian Syndrome (PCOS). Irritable Bowel Syndrome (IBS) Urinary Tract Infectious (UTI)
Initial Multi-class prediction results using Embedding Models Predict/True IG UTI IBS BPH GERD PCOS CTS PID CD UC MNP Precision Recall IG 12 0 0 1 0 0 1 0 0 0 0 0.667 0.857 UTI 0 9 1 1 0 0 1 0 0 2 0 0.818 0.643 IBS 4 0 10 0 0 0 0 0 0 0 0 0.833 0.714 BPH 0 1 0 12 0 1 0 0 0 0 0 0.75 0.857 GERD 0 0 0 0 13 0 1 0 0 0 0 0.928 0.929 PCOS 0 1 0 0 0 10 0 1 1 0 1 0.909 0.714 CTS 1 0 0 0 0 0 12 0 0 1 0 0.706 0.857 PID 0 0 0 0 0 0 1 11 0 1 0 0.846 0.846 CD 1 0 0 0 0 0 0 1 10 2 0 0.833 0.714 UC 0 0 1 0 1 0 1 0 1 10 0 0.625 0.714 MNP 0 0 0 2 0 0 0 0 0 0 12 0.923 0.857 Accuracy 0.791
Future Work: Planned immediate improvements Scale context embedding model to multi-hundred-disease prediction for 150 diseases activated in the Quro system Integration of Quro knowledge graph with context embedding models Further evaluations using expanded set of clinical vignettes across 150 diseases
Takeaways Consumer focussed engagement for understanding user symptoms, their needs, and provide valuable information to physicians for further inquiry Proactive dynamic collection of illness narrative over time, prior to doctor appointments Use of NLP entity recognition and relation extraction algorithms to determine initial entry points in knowledge graph Graph reasoning engine for optimal sequential question answering Automated data collection and expert driven primary case collection for development of disease prediction models
Thank you Head Office Level 25, Tower Three, International Towers Sydney, Barangaroo Phone +61 430 450 204 shameek.ghosh@mediushealth.org https://mediushealth.org/
Recommend
More recommend
