Tensor Networks for Medical Image Classification Presented at MIDL, - PowerPoint PPT Presentation

university of copenhagen Tensor Networks for Medical Image Classification Presented at MIDL, 2020 Raghav endra Selvan & Erik B. Dam Dept. of Computer Science, University of Copenhagen raghav@di.ku.dk @SuperVoxel Code: https://github.com/raghavian/lotenet_pytorch/

u n i v e r s i t y o f c o p e n h a g e n Overview 1 Motivation 2 Background 3 Locally orderless Tensor Networks 4 Experiments 5 Summary & Conclusion Slide 2 — Raghavendra Selvan — Tensor Networks for Medical Image Classification —

u n i v e r s i t y o f c o p e n h a g e n How far can we push linear decision boundaries? Slide 3 — Raghavendra Selvan — Tensor Networks for Medical Image Classification —

u n i v e r s i t y o f c o p e n h a g e n Decision boundaries in low/high dimensions Kernelizing to higher dimensions (SVMs) Non-linear decisions in lower dimensions (NNs) Tensor networks Linear decision boundaries in exponentially high dimensional spaces. Slide 4 — Raghavendra Selvan — Tensor Networks for Medical Image Classification —

u n i v e r s i t y o f c o p e n h a g e n One slide introduction to tensor notation Slide 5 — Raghavendra Selvan — Tensor Networks for Medical Image Classification —

u n i v e r s i t y o f c o p e n h a g e n Linear model in high dimensions: Feature Maps Consider input image with N pixels, flattened as a vector x ∈ [0 , 1] N Φ i 1 , i 2 ,... i N ( x ) = φ i 1 ( x 1 ) ⊗ φ i 2 ( x 2 ) ⊗ · · · φ i N ( x N ) (1) φ i j ( · ) is d -dimensional local feature map acting on pixel x j : φ i j ( x j ) = [cos( π 2 x j ) , sin( π 2 x j )] . (2) High dimensional feature maps Dimensionality of the joint feature map Φ( x ) is d N due to tensor products i.e, an order N tensor Slide 6 — Raghavendra Selvan — Tensor Networks for Medical Image Classification —

u n i v e r s i t y o f c o p e n h a g e n Linear model in high dimensions: Decision Rule Decision rule for a multi-class classification task: m f m ( x ) , f ( x ) = arg max (3) where m = [0 , 1 , . . . M − 1] are the M classes, f m ( x ) = W m · Φ( x ) . (4) W has M · d N tunable weights With a gray scale image of size 100 × 100 as input and d = 2 W has 2 · 2 10000 ≈ 10 3010 parameters. Slide 7 — Raghavendra Selvan — Tensor Networks for Medical Image Classification —

u n i v e r s i t y o f c o p e n h a g e n Approximate tensor dot product with MPS Matrix Product State (MPS) is a type of Tensor Network Factorisation of order N tensor into chain of order 3 tensors Reduces computation complexity from d N to N · β 3 · d (linear in N ) Slide 8 — Raghavendra Selvan — Tensor Networks for Medical Image Classification —

u n i v e r s i t y o f c o p e n h a g e n Approximate tensor dot product with MPS Matrix Product State (MPS) is a type of Tensor Network Factorisation of order N tensor into chain of order 3 tensors Reduces computation complexity from d N to N · β 3 · d (linear in N ) Slide 8 — Raghavendra Selvan — Tensor Networks for Medical Image Classification —

u n i v e r s i t y o f c o p e n h a g e n Overview 1 Motivation 2 Background 3 Locally orderless Tensor Networks 4 Experiments 5 Summary & Conclusion Slide 9 — Raghavendra Selvan — Tensor Networks for Medical Image Classification —

u n i v e r s i t y o f c o p e n h a g e n Tensor Networks for Medical Images MPS is defined for 1-d inputs 2-d images are flattened in existing literature Loss of spatial structure Flattening discards useful information in medical images Proposed idea Flatten small regions assuming local orderlessness. Aggregate at multiple resolutions. Slide 10 — Raghavendra Selvan — Tensor Networks for Medical Image Classification —

u n i v e r s i t y o f c o p e n h a g e n Locally orderless Tensor Network: LoTeNet Extending Tensor Networks to medical images 1. Partition image into small patches 2. Squeeze patches to retain spatial information 3. Perform MPS contraction at patch level 4. Aggregate and perform squeeze + MPS at next resolution 5. Output decision boundary Slide 11 — Raghavendra Selvan — Tensor Networks for Medical Image Classification —

u n i v e r s i t y o f c o p e n h a g e n LoTeNet: Partition and Squeeze Squeeze operation with stride k = 2. A 4 × 4 × 1 image patch is reshaped into 2 × 2 × 4 stack which then yields a vector of size 4 with feature dimension d=4. Slide 12 — Raghavendra Selvan — Tensor Networks for Medical Image Classification —

u n i v e r s i t y o f c o p e n h a g e n LoTeNet: Partition and Squeeze 6 8 16 5 7 1 2 3 4 14 2 4 13 15 12 1 3 5 6 7 8 10 9 11 9 10 11 12 13 14 15 16 6 8 14 16 5 7 13 15 2 4 10 12 1 3 9 11 Squeeze operation with stride k = 2. A 4 × 4 × 1 image patch is reshaped into 2 × 2 × 4 stack which then yields a vector of size 4 with feature dimension d=4. Slide 12 — Raghavendra Selvan — Tensor Networks for Medical Image Classification —

u n i v e r s i t y o f c o p e n h a g e n LoTeNet: Patch level MPS Squeeze operation with stride k = 2. A 4 × 4 × 1 image patch is reshaped into 2 × 2 × 4 stack which then yields a vector of size 4 with feature dimension d=4. Slide 13 — Raghavendra Selvan — Tensor Networks for Medical Image Classification —

u n i v e r s i t y o f c o p e n h a g e n LoTeNet: The Final Model Optimized using backpropagation. Slide 14 — Raghavendra Selvan — Tensor Networks for Medical Image Classification —

u n i v e r s i t y o f c o p e n h a g e n Overview 1 Motivation 2 Background 3 Locally orderless Tensor Networks 4 Experiments 5 Summary & Conclusion Slide 15 — Raghavendra Selvan — Tensor Networks for Medical Image Classification —

u n i v e r s i t y o f c o p e n h a g e n Model Evaluation: PCam Dataset The PatchCamelyon (PCam) dataset Binary classification Positive label indicates ≥ One pixel with tumour Image patches of size 96 × 96 px 220 k patches for training-validation (80 : 20) 57 . 5 k test patches Slide 16 — Raghavendra Selvan — Tensor Networks for Medical Image Classification —

u n i v e r s i t y o f c o p e n h a g e n Model Evaluation: LIDC Dataset 128 × 128 px image patches 15 k patches. 60 : 20 : 20 splits for training/validation/test Annotated by 4 radiologists. Originally a segmentation dataset Slide 17 — Raghavendra Selvan — Tensor Networks for Medical Image Classification —

u n i v e r s i t y o f c o p e n h a g e n Model Evaluation: Results Slide 18 — Raghavendra Selvan — Tensor Networks for Medical Image Classification —

u n i v e r s i t y o f c o p e n h a g e n Model Evaluation: Results Slide 18 — Raghavendra Selvan — Tensor Networks for Medical Image Classification —

u n i v e r s i t y o f c o p e n h a g e n Model Evaluation: Results Slide 18 — Raghavendra Selvan — Tensor Networks for Medical Image Classification —

u n i v e r s i t y o f c o p e n h a g e n Model Evaluation: Results Slide 18 — Raghavendra Selvan — Tensor Networks for Medical Image Classification —

u n i v e r s i t y o f c o p e n h a g e n Conclusion + Fully linear decision boundary + Single model hyperparameter ( β ) + Squeeze operation helps retain structure + LoTeNet performs competitively + Massive reduction in GPU utilization - Tendency to overfit - Not optimized for efficiency, yet Slide 19 — Raghavendra Selvan — Tensor Networks for Medical Image Classification —

u n i v e r s i t y o f c o p e n h a g e n Summary Proposed LoTeNet for medical image classification Different paradigm compared to feed-forward NNs or CNNs Low GPU memory requirement ( < 10%) New and exciting applications are to be expected Slide 20 — Raghavendra Selvan — Tensor Networks for Medical Image Classification —

u n i v e r s i t y o f c o p e n h a g e n Questions Thanks to Jacob Miller for TorchMPS 1 Model and data are available here: https://github.com/raghavian/lotenet_pytorch raghav@di.ku.dk 1 https://github.com/jemisjoky/TorchMPS Slide 21 — Raghavendra Selvan — Tensor Networks for Medical Image Classification —