gtc
play

GTC M ARCH 2018 Mike Tilkin ACR Chief Information Officer and EVP - PowerPoint PPT Presentation

Nov 12, 2022 •2.71k likes •3.23k views

AI I N R ADIOLOGY : R EGULATORY , Q UALITY , AND I MPLEMENTATION I SSUES GTC M ARCH 2018 Mike Tilkin ACR Chief Information Officer and EVP for Technology R EALIZING THE P OTENTIAL OF AI I MAGING AI C RITICAL S UCCESS F ACTORS Addressing the

  1. AI I N R ADIOLOGY : R EGULATORY , Q UALITY , AND I MPLEMENTATION I SSUES GTC M ARCH 2018 Mike Tilkin ACR Chief Information Officer and EVP for Technology

  2. R EALIZING THE P OTENTIAL OF AI I MAGING AI C RITICAL S UCCESS F ACTORS • Addressing the “Right” Problem • Verifying Safety and Efficacy in a high-stakes environment • Integrating into the Clinical Workflow • Monitoring, Adapting, and Communicating Results

  3. BACKGROUND

  4. The Role of the ACR • Founded in 1924, the American College of Radiology has been at the forefront of radiology evolution • More than 38,000 radiologists, radiation oncologists, nuclear medicine physicians and medical physicists. • Core Purpose: To serve patients and society by empowering members to advance the practice, science and professions of radiological care.

  5. Q UALITY A ND S AFETY R EGISTRIES AND A CCREDITATION A PPROPRIATENESS C RITERIA T ECHNICAL S TANDARDS A ND P RACTICE P ARAMETERS EDUCATION A MERICAN I NSTITUTE FOR R ADIOLOGIC P ATHOLOGY ACR E DUCATION C ENTER O NLINE L EARNING I NFORMATICS T ECHNOLOGY S TANDARDS - DICOM C LINICAL D ECISION S UPPORT Mammography 8252 C OMPUTER A SSISTED R EPORTING ACR accreditation helps Clinical Decision Support for Order Entry has been MRI 7099 adopted by over 500 health systems covering 2,000 assure your patients that E CONOMICS CT 6991 facilities which process over 5 million decision support you provide the highest Ultrasound 4970 transactions monthly. CPT C ODING level of image quality and Nuclear Medicine 3558 V ALUATION O F P HYSICIAN S ERVICES AND P RACTICE E XPENSE safety. Our process Breast Ultrasound 2222 MACRA M ETRICS A ND P AYMENT M ODELS documents that your facility PET 1542 G OVERNMENT R ELATIONS meets requirements for Stereotactic Breast Biopsy 1473 equipment, medical Breast MRI 1612 C ONGRESS personnel and quality Radiation Oncology 678 HHS TOTAL 38,397 assurance.

  6. AI and Next Generation Technology • The ACR Data Science Institute established May 2017 • Core Purpose: ACR Data Science Institute (DSI) empowers the advancement, validation, and implementation of artificial intelligence in medical imaging and the radiological sciences for the benefit of our patients, society, and the profession

  7. ACR DSI R EGULATORY C OLLABORATIONS R EGU GULATORY C ONSID IONS (F (FDA) IDERATIO • Objectives • Protect the public health • Help speed safe and effective innovation • Medical Device Classification • Based on Risk • Based on Intended Use (what does your label say) • Based on Indications for Use (under what conditions will the product be used) Class I Class III Class II LOW HIGH RISK RISK General Controls General Controls General Controls + Special Controls + Pre Market Approval

  8. Where Does AI fall? • CADe - Detection Devices intended to identify, mark, highlight, or in any other manner direct attention to portions of an image, or aspects of radiology device data, that may reveal specific abnormalities during interpretation of patient radiology images or patient radiology device data by the clinician • CADx – Diagnosis Devices go beyond CADe and include those that are intended to provide an assessment of disease or other conditions in terms of the likelihood of the presence or absence of disease, or are intended to specify disease type (i.e., specific diagnosis or differential diagnosis), severity, stage, or intervention recommended • 9/17 – Ruling classified CADx with AI as Class II. Vendors with similar products can apply for 510k clearance and avoid Pre-Market Approval (PMA)

  9. Opportunities to Accelerate the Process • Software as a Medical Device (SaMD) • 21st Century Cures Act provides guidance of medical device software • FDA is developing guidance for implementation • Medical Device Development Tools • Promotes innovation in medical device development and regulatory science to help bridge the gap between research of medical devices and the delivery of devices to patients. • National Evaluation System For Health technology (NEST) • Intended to shorten the time to market for new technology health care products by developing a system for more robust post-market surveillance

  10. FDA R EVIEW P ATHWAYS F OR AI D EVICES Establi lishin ing NEST Wil ill Enable le Th The Pre-Post Mark rket Sh Shift ift INFORMATION FLOW “Real Expedited World” Access Premarket Decision Data Pathway National Postmarket Premarket Surveillance Evaluation Review System Benefit “Safety Risk Net” TIME TO MARKET Graphic courtesy of Greg Pappas, Assistant Director FDA NEST

  11. NEST Demonstration Project: Lung-RADS Assist

  12. L UNG C ANCER S CREENING USING L OW D OSE CT UNITED STATES PREVENTATIVE SERVICES TASK FORCE L UNG C ANCER Leading cause of cancer related deaths in men and women: - 1.59 Million worldwide (2012) - 158,000 United States (2016) - 75% present symptomatically with incurable disease USPSTF RECOMMENDATION Annual screening for lung cancer with low-dose computed tomography (LDCT) in adults aged 55 to 80 years who have a 30 pack-year smoking history and currently smoke or have quit within the past 15 years. U NITED S TATES E LIGIBLE P OPULATION 20 Million individuals require annual screening

  13. Lung Nodule Detection Algorithms Clinical Machine Data AI Model 3 4 2 1 Validation Learning Management Inferencing Data Acquisition Ground Truth Algorithm Training Clinical Validation Nodule – Description

  14. Lung Nodule Detection Algorithms Data Acquisition Ground Truth Training Validation M ODEL 1 Nodule Description 1 Inferencing Validation Data Acquisition Ground Truth Training M ODEL 2 Nodule Description 2 Data Acquisition Ground Truth Training Validation M ODEL 3 Nodule Description 3

  15. VALIDATION, CERTIFICATION & COMPLIANCE

  16. Challenges in the AI Life Cycle Train Test Deploy Acquire Implicit Use FDA Provider Case Model Model Model Data • How generalizable is the inference model? • Is there hidden sample bias? • What is the appropriate threshold for clinical use? • How do we ensure ongoing performance? • How robust is the model to changes in the environment?

  17. Challenges in the AI Life Cycle Train Test Acquire Deploy Implicit Use FDA Provider Case Model Model Data Model Deploy Train Test Acquire Implicit Use FDA Model Case Model Model Data Train Test Acquire Deploy Implicit Use Case Model Model FDA Data Model • Do models solving the same problem yield consistent, comparable outputs? • Does the customer understand potential differences in the implicit use cases? • How do we establish standard, consistent performance metrics?

  18. Establish Standards & Certification Criteria Train Test Acquire Implicit Use Touch-AI Use (reference) Case Model Model Data Case Deploy Train Test Acquire Implicit Use Certify FDA Provider Case Model Model Data Model Train Test Acquire Implicit Use Case Model Model Data Well-qualified Data to Reference Use Case • Establish common expectations for addressing specific clinical scenarios (e.g. BI-RADS) • Create well-qualified data sets that address explicit concerns about bias • Define standard performance metrics that establish a quality threshold • Validate models that address a specific clinical condition against these standards

  19. Monitoring and Communication Train Test Acquire Implicit Use Touch-AI Use (reference) Case Model Model ACR Data ACR ACR Case Deploy Train Test Acquire Implicit Use Certify Assess FDA Case Model Model Data Performance Model TOUCH-AI Assess-AI Certify-AI Train Test Acquire Implicit Use Case Model Model Data Well-qualified Data to Provider Reference Use Case • Monitor Ongoing Performance to Ensure Ongoing Quality and Safety • Provide Feedback Loop to Providers, Regulators, Vendors, Content Creators • Match continuous learning with continuous assessment, monitoring, and feedback

  20. TOUCH-AI

  21. Detecting Lisfranc Joint Injury Lisfranc joint injury is common and easily missed. AI that segments and detects abnormality would prove valuable and help reduce false negative rate, patient risk, and medical-legal risk for the radiologists.

  22. DSI Use Cases Clinical Guidance for Developers Example: Lisfranc Joint Injury Expected Clinical Inputs/Outputs Conditions for launch Data Considerations for Training/Testing

  23. ACR DSI U SE C ASE D EVELOPMENT – ACR DSI U SE C ASE P ANELS TBI-RADS LUNG-RADS WORKGROUP WORKGROUP Common Use Case Framework TOUCH-AI (Technically Oriented Use Cases for Healthcare-AI) Li-RADS BONE AGE WORKGROUP WORKGROUP ACR DSI Use Case Creation Process Abdominal Interventional Breast Imaging Neuroimaging Thoracic Imaging Pediatric Imaging Imaging Radiology AI USE CASE AI USE CASE AI USE CASE AI USE CASE AI USE CASE AI USE CASE PANEL PANEL PANEL PANEL PANEL PANEL USE CASE PANELS P ATIE Oncology Musculoskeletal Cardiac Imaging Quality, Safety Imaging RO And Cancer NT AI USE CASE AI USE CASE AI USE CASE AI USE CASE PANEL PANEL PANEL PANEL

  24. Use Case Develo lopment Status • All ACR DSI Subspecialty Data Panels underway - 19 Use Cases in drafting stage - 9 Use Cases in the review stage • Examples of use cases under development - Pediatric Bone Age classification - Lisfranc fracture detection and classification - Colon polyp detection - TBI-RADS • Industry collaborations

Recommend

More recommend