mitosis Image-based profiling using deep learning Juan C. Caicedo Ph.D Broad Institute of MIT and Harvard
Images can be quantified for all kinds of phenotypes Muscle structure Patient biopsy tissue Image Mass Spec David Thomas Margaret Shipp/Scott Rodig Michael Angelo 3D Muscle structure Control human iPS Isogenic Duchenne-like iPS Allen Institute for Cell Science Olivier Pourquie
Screen for specific phenotypes using images DMSO SU6656 DNA stain with outlines identifying the nuclei Clinical trials underway for Alisertib in adults with AMKL. Wen Q, et al. (2012). Cell 150(3):575-89
TKK 0.1 uM
AZ138 - 0.01uM
What is image-based profiling? Caicedo J.C., Singh S., Carpenter A. "Applications of Image-Based Profiling of Perturbations". Current Opinion in Biotechnology - 2016.
Cell Painting assay Gustafsdottir, et al. PLOS ONE 2013 Bray, et al. Nature Protocols 2016
Image-based Profiling 1. Raw images 4. Population profiles of treatments 5. Downstream statistical analysis 2. Segmented images 3. Single-cell feature matrices Are treatments significantly different / effective? Caicedo, J.C., et al. "Data-analysis strategies for image-based cell profiling." Nature Methods 14.9, 2017
Automatic Identification of Cells Segmentation
Neural Networks for Segmentation Training Applying Labeled objects Labeled objects Manual annotation New image Example Image Train Run Model Model
Neural Nets produce fewer segmentation errors Caicedo, J.C., et al. "Evaluation of Deep Learning Strategies for Nucleus Segmentation in Fluorescence Images." BioRxiv (2019): 335216.
More data improves performance
Diversity of imaging studies
Can we create a single tool to detect cell nuclei in any light microscopy image?
Universal nucleus segmentation 3 3,634 65,333 months teams experiments
Dataset • 37,333 annotated nuclei • 841 images • 30 biological experiments • 22 cell types • 5 image groups
Performance of participants a b Distribution of scores in second-stage evaluation Accuracy of top-3 models 1.0 40 Number of participant teams CellProfiler reference* Accuracy: F1-score 0.8 30 0.6 5 - Inom Mirzaev 4 - Nuclear Vision 20 0.4 1st place 3 - Deep Retina 2nd place 3rd place 2 - jacobkie 0.2 reference 10 1 - [ods.ai] topcoders 0.0 0.5 0.6 0.7 0.8 0.9 0.1 0.2 0.3 0.4 0.5 0.6 0.0 Competition score Intersection over Union Threshold
Top 3 solutions [ods.ai] topcoders jacobkie Deep Retina 1st place 2nd place 3rd place
Performance by image type a Accuracy by image type b Dataset distribution 0.9% 2.4% 1st place Accuracy: F1-score @ 0.7 IoU 2nd place 15.5% 0.8 3rd place reference 0.6% 0.6 Training 0.4 80.6% 0.2 11.3% 0.9% 0.0 Small Purple Pink and purple Large Grayscale 15.1% fluorescent tissue tissue fluorescent tissue Test 4.7% 67.9%
More accurate and takes no time Small Purple Pink and purple Large Grayscale fluorescent tissue tissue fluorescent tissue Caicedo et al. 2019 Nature Methods. In Press.
Lung Adenocarcinoma A Cell Painting study
Patient with metastatic breast cancer ● She is diagnosed in 2016 ● Doctors observe tennis-ball sized tumors in several parts of her body ● They give her 3 months life expectancy The patient accepts an experimental treatment called immunotherapy .
Train the immune system to fight cancer
The patient fully recovered Taken from https://www.bbc.co.uk/news/health-44338276
Why did the treatment work? Tumor sequencing revealed 62 mutations They knew treatment for 7 of them The experiment worked only with 4 mutations
Cell Painting LUAD dataset CONTROL EGFR_WT ARAF_WT CTNNB1_WT FBXW7_WT KEAP1_WT KRAS_WT MAPK7_WT RIT1_WT STK11_WT Cell line: A549 Over-expression 8 replicates 50 million+ single cells Juan Mohammad Shantanu Anne Caicedo Rohban Singh Carpenter
Morphology-based VIP ARAF_p.S214F
Image-based Profiling 1. Raw images 4. Population profiles of treatments 5. Downstream statistical analysis 2. Segmented images 3. Single-cell feature matrices Are treatments significantly different / effective? Caicedo, et al. 2017 Nature Methods
Cell Feature Extraction Engineer measurements Define and compute useful properties Area Shape Color Nuclei size distribution
Cell Feature Extraction Learn features to solve a task Train a deep neural network classification
Weakly supervised learning Batch 1 Batch 2 A1 B1 C1 A3 D2 C3 A2 B2 C2 D1 B3 D3 Mechanistic Class X Mechanistic Class Y Discover associations Compound A Compound C Compound B Compound D
Weakly supervised learning Auxiliary task: Main goal: Single-cell treatment classification Treatment-level profiling softmax CNN Caicedo, J. C., et al. "Weakly supervised learning of single-cell feature embeddings". Computer Vision and Pattern Recognition, IEEE CVPR 2018
Deep Learning for Image-based Profiling CellProfiler Weakly Supervised Learning Control TP53.WT STK11.WT NFE2L2.WT MDM2.WT KRAS_p.G12V KEAP1.WT EGFR_p.T790M EGFR_p.L858R EGFR.WT Transfer learning In all plots, x-axis is t-SNE1 and y-axis is t-SNE2 of Caicedo et al. CVPR 2018 the projected phenotypic space.
Determining variant impact EGFR_p.S645C Control EGFR Wild Type EGFR Mutant
Single-cell morphological analysis Variant impact: 66.9% Control EGFR WT EGFR MUT EGFR_p.S645C
Prioritizing variants by impact EGFR_p.T790M, p.L858R.o EGFR_p.L858R EGFR_p.K754E EGFR_p.Q102H EGFR_p.S654C EGFR_p.R222L Find more results at: https://broad.io/cp-luad
Prioritizing variants by impact EGFR_p.Q102H EGFR_p.S654C EGFR_p.T790M, p.L858R.o EGFR_p.R222L EGFR_p.L858R EGFR_p.K754E 0.0% 20.0% 40.0% 60.0% 80.0% Find more results at: https://broad.io/cp-luad Morphological Variant Impact Score (%)
Conclusions Imaging is a rich source of information. Computer vision has powerful tools for image analysis. Many computer vision tasks can be fully automated. Imaging data can be connected to other sources of data.
Gratitude Broad Imaging Platform Anne Carpenter Profiling group: Hamdah Abbasi Shantanu Singh Jeanelle Ackerman Tim Becker Beth Cimini Marzieh Haghighi Minh Doan Matt Smith Allen Goodman Many thanks to our many biology collaborators! Recent major funding for this work provided by: ● NIH NIGMS: R01 GM089652 ● NIH NIGMS: MIRA R35 GM122547 ● NSF CAREER: DBI 1148823 ● NSF/BBSRC: DBI 1458626
Recommend
More recommend
![Improving Robustness of Deep-Learning-Based Image Reconstruction Ankit Raj [1] , Yoram Bresler](https://c.sambuz.com/785835/improving-robustness-of-deep-learning-based-image-s.jpg)
