Paper number: 230 Training deep segmenta.on networks on texture- - PowerPoint PPT Presentation

Oct 02, 2022 •338 likes •476 views

Paper number: 230 Training deep segmenta.on networks on texture- encoded input: applica.on to neuroimaging of the developing neonatal brain AE AE Fe Fe&t, J Cupi., T Kart, D Rueckert The shape hypothesis in deep CNNs cat

Paper number: 230 Training deep segmenta.on networks on texture- encoded input: applica.on to neuroimaging of the developing neonatal brain AE AE Fe Fe&t, J Cupi., T Kart, D Rueckert
The ‘shape hypothesis’ in deep CNNs ‘cat’ Low-level Mid-level High-level Classifier features features features e.g. lines, e.g. circles, e.g. ears, edges.. triangles.. paws.. Low level shape features are combined in increasingly complex hierarchies un.l the object can be readily classified or detected Work suppor)ng: Zeiler and Fergus, 2014; LeCun et al., 2015; RiOer et al., 2017.
Textural bias in deep CNNs Geirhos et al., 2019.
Segmenta.on of the developing brain with CNNs White Matter Cortical Grey Matter CSF Ventricles Deep Grey Matter
Challenge: Varia.on in both shape and texture 32 weeks 34 weeks 35 weeks 38 weeks 40 weeks
Context: Developmental brain mapping e.g. The Developing Human Connectome Project (DHCP) aims to make major scien&fic progress by crea&ng the first 4D connectome map of early life. It is imp mportant to be.er understand the role of visual texture when developing CNNs on heterogeneous neonatal neuroima maging data.
Our approach: Encoding with local textural paOerns T2-weighted: LBP, 1 pixel: LBP, 10 pixels: Ground-truth:
Experimental set-up Total data To 558, 3D T2-weighted neonatal MRI scans, publicly available by DHCP. Cl Classes es 1. Background, 2. CSF, 3. CGM, 4. WM, 5. Background bordering brain &ssue, 6. Ventricles, 7. Cerebellum, 8. DGM, 9. Brainstem, 10. Hippocampus. Labels Labels Segmenta&on maps available, output of the DHCP structural pipeline. Model-developme ment set 450 for training, PMA 24.7- 42.1 weeks. 20 for valida&on, PMA 27.6 - 42.2 weeks. Held Held-ou -out t tes est se set 88 for tes&ng, PMA 24.3 – 42 weeks. CNN CNN 3D architecture developed with DeepMedic.
CNN architecture • 3D modeling using DeepMedic Kamnitsas et al. 2016. • Three parallel pathways: • normal resolu&on • downsampled by 3 • downsampled by 5 • 8 layers per pathway • Training batch size was set to 5 • Learning rate followed a pre-defined schedule.
Three 3D CNNs 2 3 1 T2-weighted LBP-encoded: LBP-encoded: 1 pixel distance 10 pixel distance The goal is to train CNNs 2 and 3 on explicit textural representations generated from the T2-weighted images, and to evaluate performance in a complex tissue segmentation task.
Summary of results Classes: 1. Background, 2. CSF, 3. CGM, 4. WM, 5. Background bordering brain tissue, 6. Ventricles, 7. Cerebellum, 8. DGM, 9. Brainstem, 10. Hippocampus.
Conclusion The study is the first to show on (neonatal) neuroimaging data that CNNs can indeed be trained on explicit textural representations of the data to achieve segmentation performance that is comparable to models trained on the original T2-weighted scans.
Paper number: 230 Thank you! Questions?

Recommend

Architecture Perceptron Highway Networks Highway Networks Allows training very deep

Highway Networks for Visual Question Answering Aaditya Prakash PhD advisor: James Storer Brandeis University Architecture Perceptron Highway Networks Highway Networks Allows training very deep networks Srivastava et al trained

174 views • 16 slides

Deep Learning: Training Juhan Nam Training Deep Neural Networks Forward (hidden unit

GCT634/AI613: Musical Applications of Machine Learning (Fall 2020) Deep Learning: Training Juhan Nam Training Deep Neural Networks Forward (hidden unit activation) Parametric Gradient-based Learning Backward Non-parametric (gradient

650 views • 21 slides

No Spurious Local Minima in Training Deep Quadratic Networks Abbas Kazemipour Conference on

No Spurious Local Minima in Training Deep Quadratic Networks Abbas Kazemipour Conference on Mathematical Theory of Deep Neural Networks October 31, 2019 New York City, NY Need for New Optimization Theory The mystery of deep neural networks

549 views • 17 slides

Normalization Techniques in Training of Deep Neural Networks Lei Huang ( ) State Key

Normalization Techniques in Training of Deep Neural Networks Lei Huang ( ) State Key Laboratory of Software Development Environment, Beihang University Mail:huanglei@nlsde.buaa.edu.cn August 17 th , 2017 Outline Introduction to Deep

845 views • 47 slides

FLEET: Flexible Efficient Ensemble Training for Heterogenous Deep Neural Networks Hui Guan,

FLEET: Flexible Efficient Ensemble Training for Heterogenous Deep Neural Networks Hui Guan, Laxmikant Kishor Mokadam, Xipeng Shen, Seung-Hwan Lim, Robert Patton 1 Build an image classifier? Deep Neural Network (DNN) CPU GPU Training

957 views • 41 slides

Adversarial Training Attacks on Deep Networks and Generative Adversarial Networks Erkut Erdem

images from Geris Game (Pixar, 1997) Adversarial Training Attacks on Deep Networks and Generative Adversarial Networks Erkut Erdem Aykut Erdem Levent Karacan Computer Vision Lab, Hacettepe University Outline Part 1: Attacks on

1.11k views • 72 slides

Paper number: 136 A deep learning approach to segmentation of the developing cortex in fetal

Paper number: 136 A deep learning approach to segmentation of the developing cortex in fetal brain MRI with minimal manual labeling AE Fetit , A Alansary, L Cordero-Grande, J Cupitt, AB Davidson, AD Edwards, JV Hajnal, E Hughes, K Kamnitsas, V

676 views • 11 slides

Parameter efficient training of deep convolutional neural networks by dynamic sparse

Parameter efficient training of deep convolutional neural networks by dynamic sparse reparameterization Hesham Mostafa (Intel AI) Xin Wang (Intel AI, Cerebras Systems) Easy : post-training (sparse) compression Hard : direct training of sparse

727 views • 9 slides

Training Neural Networks: Normalization, Regularization etc. Intro to Deep Learning, Fall 2020 1

Training Neural Networks: Normalization, Regularization etc. Intro to Deep Learning, Fall 2020 1 Quick Recap: Training a network Divergence between desired output and Average over all actual output of net for a given input training instances

1.75k views • 140 slides

Paper Reading Paper HetConv: Heterogeneous Kernel-Based Convolutions for Deep

Paper Reading Paper HetConv: Heterogeneous Kernel-Based Convolutions for Deep CNNs, CVPR, 2019 Drop an Octave: Reducing Spatial Redundancy in Convolutional Neural Networks with Octave Convolution EfficientNet: Rethinking

424 views • 21 slides

COMP 762: Paper Presentation (Technical Details) Alexander Nicholson Are Deep Neural Networks

COMP 762: Paper Presentation (Technical Details) Alexander Nicholson Are Deep Neural Networks the Best Choice for Modeling Source Code? Authors: Vincent J. Hellendoorn, Premkumar Devanbu Overview: Big question in the title Creates a

799 views • 18 slides

Deep Neural Networks CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu Deep learning slides credit:

Deep Neural Networks CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu Deep learning slides credit: Vlad Morariu Training (Deep) Neural Networks Computational graphs Improvements to gradient descent Stochastic gradient descent Momentum

814 views • 21 slides

Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks

Cluster-GCN: An Efficient Algorithm for Training Deep and Large Graph Convolutional Networks Wei-Lin Chiang 1 , Xuanqing Liu 2 , Si Si 3 , Yang Li 3 , Samy Bengio 3 , Cho-Jui Hsieh 23 1 National Taiwan University, 2 UCLA, 3 Google Research Graph

616 views • 23 slides

and Hash Coding with Deep Neural Networks Presenter: MinKu Kang 1 Why did I choose this paper?

Simultaneous Feature Learning and Hash Coding with Deep Neural Networks Presenter: MinKu Kang 1 Why did I choose this paper? Efficient image retrieval via binary encoding of images - efficient bitwise operations. - space-efficient

606 views • 36 slides

Concise Introduction to Deep Neural Networks Outline: Classification problems Motivating

Concise Introduction to Deep Neural Networks Outline: Classification problems Motivating Deep (large) Neural Network (DNN) classifiers Neurons and DNN architectures Numerical training of DNNs (supervised deep learning) Spiking

630 views • 33 slides

Neural Networks Neural networks arise from attempts to model Neural Networks human/animal

Feed-forward Networks Network Training Error Backpropagation Deep Learning Feed-forward Networks Network Training Error Backpropagation Deep Learning Neural Networks Neural networks arise from attempts to model Neural Networks

380 views • 9 slides

Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations

Deep Learning: Theory and Practice Deep Learning - Practical 02-04-2020 Considerations deeplearning.cce2020@gmail.com Deep Networks Intuition Neural networks with multiple hidden layers - Deep networks [Hinton, 2006] Deep Networks Intuition

1.33k views • 28 slides

How to Construct Deep Recurrent Neural Networks AUTHORS: R. PASCANU, C. GULCEHRE, K. CHO, Y.

How to Construct Deep Recurrent Neural Networks AUTHORS: R. PASCANU, C. GULCEHRE, K. CHO, Y. BENGIO PRESENTATION: HAROUN HABEEB PAPER: HTTPS://ARXIV.ORG/ABS/1312.6026 This presentation Motivation Formal RNN paradigm Deep RNN designs

395 views • 13 slides

Compressing DMA Engine: Leveraging Activation Sparsity For Training Deep Neural Networks Minsoo

(C) Minsoo Rhu 1 Compressing DMA Engine: Leveraging Activation Sparsity For Training Deep Neural Networks Minsoo Rhu , Mike OConnor * , Niladrish Chatterjee * , Jeff Pool * , Youngeun Kwon , and Stephen W. Keckler * POSTECH and

2.04k views • 66 slides

Learning Face Recognition from Limited Training Data using Deep Neural Networks Xi Peng * , Nalini

Learning Face Recognition from Limited Training Data using Deep Neural Networks Xi Peng * , Nalini Ratha ** , Sharathchandra Pankanti ** * Rutgers University, New Jersey, 08854 ** IBM T. J. Watson Research Center, New York 10598 Motivation

630 views • 30 slides

Reduced-memory training and deployment of deep residual networks by stochastic binary quantization

Reduced-memory training and deployment of deep residual networks by stochastic binary quantization Mark D. McDonnell 1 , Ruchun Wang 2 and Andr van Schaik 2 cls-lab.org 2 BENS Laboratory 1 Computational Learning Systems Laboratory MARCS

256 views • 23 slides

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan Zelener Machine Learning Reading Group January 7 th 2016 The Graduate Center, CUNY Objectives Explain some of the trends of deep learning and

680 views • 44 slides

Training Neural Networks: Optimization Intro to Deep Learning, Fall 2020 1 Quick Recap

Training Neural Networks: Optimization Intro to Deep Learning, Fall 2020 1 Quick Recap Gradient descent, Backprop 2 Quick Recap: Training a network Divergence between desired output and Average over all actual output of net for a given

1.21k views • 118 slides

Deep Networks Andrea Passerini passerini@disi.unitn.it Machine Learning Deep Networks Need for

Deep Networks Andrea Passerini passerini@disi.unitn.it Machine Learning Deep Networks Need for Deep Networks Perceptron Can only model linear functions Kernel Machines Non-linearity provided by kernels Need to design appropriate kernels

563 views • 43 slides