Radiology in the Era of Artificial Intelligence November 11, 2019 - PowerPoint PPT Presentation

Radiology in the Era of Artificial Intelligence November 11, 2019 Curtis P. Langlotz, MD, PhD Professor of Radiology and Biomedical Informatics, Stanford University Director, Center for Artificial Intelligence in Medicine & Imaging Associate Chair, Information Systems, Department of Radiology Medical Informatics Director, Stanford Health Care @curtlanglotz

Disclosures Shareholder and advisor: • whiterabbit.ai • Nines.ai • GalileoCDS, Inc • Bunker Hill, Inc • Department research support: • Philips • Siemens Healthineers • GE Healthcare • School of Medicine research support: • Google • Board of directors, Radiological Society of North America •

Variability in Response Assessment Schwartz LH et al. Variability in Response Assessment in Solid McNitt-Gray MF et al. Computed Tomography Assessment of Response to Therapy: Tumors: Effect of Number of Lesions Chosen for Measurement. Tumor Volume Change Measurement, Truth Data, and Error. Transl Oncol . 2009 Dec 1; Clin Cancer Res . 2003 Oct 1;9(12):4318-23. 2(4): 216–222.

http://www.image-net.org/ • 14 million images • 21,841 distinct labels: • 856 types of bird • 993 types of tree • 157 musical instruments • Russakovsky O, Deng J, Su H, et al. ImageNet Large Scale Visual Recognition Challenge. Int J Comput Vis. 2015;115(3):211-252. • https://www.economist.com/news/special-report/21700756-artificial-intelligence-boom-based-old-idea-modern-twist-not • http://karpathy.github.io/2014/09/02/what-i-learned-from-competing-against-a-convnet-on-imagenet/

Dramatic Advances in Accuracy of Deep Learning ImageNet Visual Recognition Error Rates 30% 25% 20% 15% 10% Human error rate 5% 0% 2010 2011 2012 2013 2014 2015 2016 2017 5 Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, et al. ImageNet Large Scale Visual Recognition Challenge. Int J Comput Vis. 2015;115: 211–252.

http://www.radiologyassistant.nl/ Karpathy, Andrej & Li, Fei Fei. Deep Visual-Semantic Alignments for Generating Image Descriptions, CVPR, 2015

Advances in Computation ASR 33 Teletype, 1976 DEC PDP-8, 1965 Apple A12 Processor, 2019 75 lbs, 110 baud 0.333 MHz, $18,500 2.49 GHz x 6, $1000 https://www.pdp8.net/asr33/asr33.shtml

Advances in Storage Seagate 4TB HD, 2019 IBM 5MB HD, 1956 $3200/month lease $99, Amazon.com

Neural Network A flexible and powerful form of machine learning. Positive Cases Malignant Negative Cases Benign https://hackernoon.com/challenges-in-deep-learning-57bbf6e73bb

“Deep” Neural Networks: Tens of Millions of Parameters

Learning Object Recognition Faces Cars Elephants Chairs Courtesy of Andrew Ng

Machine Learning Research in Medical Imaging Source “Raw” Data Labeled New Decision Training Data Image Support Actionable Recon New Systems Advice New Methods Machine Machine Learning Learning Explanation Methods Methods New Image Labeling Methods CT scan icon by Sergey Demushkin from the Noun Project

• Federated learning • Generative adversarial networks • Semi-supervised learning • Deep contextualized word representations • Multi-task learning • Reinforcement learning • Explainable AI models • Homomorphic encryption

Foundational AI Research in Radiology: Ingredients for Success Healthcare Academic AI Established Delivery Professional Health Ingredient for Success Startups Companies Systems Societies Systems Deep technical knowledge High performance computing Interdisciplinary teams Ongoing source of labeled images Infrastructure for prospective evaluation Market dissemination channel =available =can acquire =difficult to acquire

• Anesthesia • Bioengineering • Biomedical Data Science • Cardiothoracic Surgery • Computer Science • Dermatology • Emergency Medicine • Genetics • Medicine • Neurology & Neurological Sciences • Neurosurgery • Ophthalmology • Pathology • Pediatrics • Psychiatry & Behavioral Sciences • Psychology • Radiation Oncology • Radiology • Surgery • Urology Key C Compute ter S Science F Faculty C Collaborators Andrew Ng, PhD Fei Fei Li, PhD Chris Manning, PhD Deep Learning Pioneer Creator of ImageNet Stanford AI Lab Director

AIMI Diagnostic Radiology Projects Brain tumors Cervical spine fracture Brain age 14 chest abnormalities Brain abnormalities PE classification Pneumonia Coronary calcium scoring Brasfield scoring Renal cell carcinoma Lung cancer Normal/abnormal Fractures Bone age Renal abnormalities Knee cartilage Appendicitis Knee menisci and ligaments Deep vein thrombosis Fractures MR CT US XR PT

AI Throughout The Imaging Life Cycle Select precise treatment Enable patient self- with global reach scheduling Disease Image classification protocoling Prevent/detect disease Lower radiation dose Computer Image recon- early among populations and imaging time aided- struction and detection enhancement Imaging Image triage sequence optimization Decrease length of stay Answer the clinical question Image quality control https://aimi.stanford.edu/ Reduce repeat imaging 17

Infrastructure for AI in Medical Imaging Clinical Practice Laboratory DICOM Clinical EMR, RIS, Research Data Scanner Router PACS, Reporting Warehouse (STARR) Data Clinical Clinical Migration images images Cohort selection Image labels Cohort Management “Raw” Clinical Imaging • Role-based access Other images detector Reporting • Image viewing clinical data for AI for AI System data • Image annotation/ processing processing labeling AI results Machine-learning- AI AI ready data set results results Containerized UPSTREAM DOWNSTREAM Algorithm Algorithm Training & AI AI Prospective Evaluation and Deployment Platform Testing Vendor 18

Research Opportunities and Infrastructure Development Requirements for Translation https://doi.org/10.1016/j.jacr.2019.04.014 • Data sets for training, testing, and validating AI algorithms • Standards for clinical integration of AI algorithms • Software use cases with common data elements • Balanced regulatory framework to ensure safety and efficacy

Publicly-Released Labeled Radiology Datasets

Publicly-Released Labeled Radiology Datasets https://aimi.stanford.edu/ 21

QIBA = Reproducibility Measure = 7 ± 6 Measure = 7 ± 2 Problem Solution Variability in: QIBA Profile specifying: Treat • Patient handling • Calibration • Acqisition protocols • Patient Preparation ? • Image reconstruction • Acquisition Parameters • Segmentation • Reconstruction Parameters • Image processing • Resolution Wait • Processing Parameters • Segmentation Adapted from Kevin O’Donnell and Daniel Sullivan

Thank You Engage: @curtlanglotz @StanfordAIMI LinkedIn: aimi-stanford Join the mailing list: https://aimi.stanford.edu/engage/subscribe Apply to become affiliated faculty: https://aimi.stanford.edu/engage/affiliate-faculty