Automated Summarisation for Evidence Based Medicine Diego Moll a - PowerPoint PPT Presentation

Automated Summarisation for Evidence Based Medicine Diego Moll´ a Centre for Language Technology, Macquarie University HAIL, 22 March 2012

Evidence Based Medicine Our Corpus for Summarisation Applications Contents Evidence Based Medicine Our Corpus for Summarisation Structure of our Corpus How we Created the Corpus Statistics Applications Possible Uses Single-document Summarisation Evidence Grading EBM Summarisation Diego Moll´ a 2/60

Evidence Based Medicine Our Corpus for Summarisation Applications About Us: Research Group on Natural Language Processing of Medical Texts http://web.science.mq.edu.au/~diego/medicalnlp/ Active Members Diego Moll´ a Senior lecturer at Macquarie University. C´ ecile Paris Senior principal research scientist at CSIRO ICT Centre. Abeed Sarker PhD student at Macquarie University. Sara Faisal Shash Masters student. Past Members Mar´ ıa Elena Santiago-Mart´ ınez Research programmer. Patrick Davis-Desmond Masters student. Andreea Tutos Masters student. EBM Summarisation Diego Moll´ a 3/60

Evidence Based Medicine Our Corpus for Summarisation Applications Contents Evidence Based Medicine Our Corpus for Summarisation Structure of our Corpus How we Created the Corpus Statistics Applications Possible Uses Single-document Summarisation Evidence Grading EBM Summarisation Diego Moll´ a 4/60

Evidence Based Medicine Our Corpus for Summarisation Applications Evidence Based Medicine http://laikaspoetnik.wordpress.com/2009/04/04/evidence-based-medicine-the-facebook-of-medicine/ EBM Summarisation Diego Moll´ a 5/60

Evidence Based Medicine Our Corpus for Summarisation Applications EBM and Natural Language Processing http://hlwiki.slais.ubc.ca/index.php?title=Five_steps_of_EBM EBM Summarisation Diego Moll´ a 6/60

Evidence Based Medicine Our Corpus for Summarisation Applications PICO for Asking the Right Question EBM Summarisation Diego Moll´ a 7/60

Evidence Based Medicine Our Corpus for Summarisation Applications Where to search for external evidence? 1. Evidence-based Summaries (Systematic Reviews): ◮ EBM Online (http://ebm.bmj.com). ◮ UptoDate (http://www.uptodate.com). ◮ The Cochrane Library (http://www.thecochranelibrary.com/). ◮ . . . EBM Summarisation Diego Moll´ a 8/60

Evidence Based Medicine Our Corpus for Summarisation Applications Where to search for external evidence? 1. Evidence-based Summaries (Systematic Reviews): ◮ EBM Online (http://ebm.bmj.com). ◮ UptoDate (http://www.uptodate.com). ◮ The Cochrane Library (http://www.thecochranelibrary.com/). ◮ . . . 2. Search the Medical Literature: ◮ E.g. PubMed (http://www.ncbi.nlm.nih.gov/pubmed/). EBM Summarisation Diego Moll´ a 8/60

Evidence Based Medicine Our Corpus for Summarisation Applications Searching Cochrane EBM Summarisation Diego Moll´ a 9/60

Evidence Based Medicine Our Corpus for Summarisation Applications Searching PubMed EBM Summarisation Diego Moll´ a 10/60

Evidence Based Medicine Our Corpus for Summarisation Applications Searching the Trip Database EBM Summarisation Diego Moll´ a 11/60

Evidence Based Medicine Our Corpus for Summarisation Applications Appraising the Evidence The SORT Taxonomy Level A Consistent and good-quality patient-oriented evidence. Level B Inconsistent or limited-quality patient-oriented evidence. Level C Consensus, usual practise, opinion, disease-oriented evidence, or case series for studies of diagnosis, treatment, prevention, or screening. EBM Summarisation Diego Moll´ a 12/60

Evidence Based Medicine Our Corpus for Summarisation Applications Levels of Evidence Study quality Diagnosis Treatment / prevention / Prognosis screening Level 1: Validated clinical decision SR/meta-analysis of RCTs SR/meta-analysis of good- good-quality rule; SR/meta-analysis of with consistent findings; quality cohort studies; patient-oriented high-quality studies; high- high-quality individual prospective cohort study evidence quality diagnostic cohort RCT; all-or-none study with good follow-up study Level 2: Unvalidated clinical SR/meta-analysis of lower- SR/meta-analysis of lower- limited-quality decision rule; SR/meta- quality clinical trials or of quality cohort studies or patient-oriented analysis of lower-quality studies with inconsistent with inconsistent results; evidence studies or studies with findings; lower-quality clin- retrospective cohort study inconsistent findings; ical trial; cohort study; or prospective cohort study lower-quality diagnostic case-control study with poor follow-up; case- cohort study or diagnostic control study; case series case-control study Level 3: other Consensus guidelines, extrapolations from bench research, usual practice, opinion, evidence disease-oriented evidence (intermediate or physiologic outcomes only), or case series for studies of diagnosis, treatment, prevention, or screening EBM Summarisation Diego Moll´ a 13/60

Evidence Based Medicine Our Corpus for Summarisation Applications Where can NLP Help? ◮ Questions: ◮ Help to formulate answerable questions. ◮ Question analysis and classification. EBM Summarisation Diego Moll´ a 14/60

Evidence Based Medicine Our Corpus for Summarisation Applications Where can NLP Help? ◮ Questions: ◮ Help to formulate answerable questions. ◮ Question analysis and classification. ◮ Search: ◮ Retrieve and rank relevant literature. ◮ Extract the evidence-based information. ◮ Summarise the results. EBM Summarisation Diego Moll´ a 14/60

Evidence Based Medicine Our Corpus for Summarisation Applications Where can NLP Help? ◮ Questions: ◮ Help to formulate answerable questions. ◮ Question analysis and classification. ◮ Search: ◮ Retrieve and rank relevant literature. ◮ Extract the evidence-based information. ◮ Summarise the results. ◮ Appraisal: Classify the evidence. EBM Summarisation Diego Moll´ a 14/60

Evidence Based Medicine Our Corpus for Summarisation Applications Contents Evidence Based Medicine Our Corpus for Summarisation Structure of our Corpus How we Created the Corpus Statistics Applications Possible Uses Single-document Summarisation Evidence Grading EBM Summarisation Diego Moll´ a 15/60

Evidence Based Medicine Our Corpus for Summarisation Applications Where’s the Corpus for Summarisation? Summarisation Systems ◮ CENTRIFUSER/PERSIVAL: Developed and tested using user feedback (iterative design). ◮ SemRep: Evaluation based on human judgement. ◮ Demner-Fushman & Lin: ROUGE on original paper abstracts. ◮ Fiszman: Factoid-based evaluation. EBM Summarisation Diego Moll´ a 16/60

Evidence Based Medicine Our Corpus for Summarisation Applications Where’s the Corpus for Summarisation? Summarisation Systems ◮ CENTRIFUSER/PERSIVAL: Developed and tested using user feedback (iterative design). ◮ SemRep: Evaluation based on human judgement. ◮ Demner-Fushman & Lin: ROUGE on original paper abstracts. ◮ Fiszman: Factoid-based evaluation. Corpora ◮ Several corpora of questions/answers available. ◮ Answers lack explicit pointers to primary literature. ◮ Medical doctors want to know the primary sources. EBM Summarisation Diego Moll´ a 16/60

Evidence Based Medicine Our Corpus for Summarisation Applications Contents Evidence Based Medicine Our Corpus for Summarisation Structure of our Corpus How we Created the Corpus Statistics Applications Possible Uses Single-document Summarisation Evidence Grading EBM Summarisation Diego Moll´ a 17/60

Evidence Based Medicine Our Corpus for Summarisation Applications Journal of Family Practice’s “Clinical Inquiries” EBM Summarisation Diego Moll´ a 18/60