applying deep neural network techniques for lartpc data
play

Applying Deep Neural Network Techniques for LArTPC Data - PowerPoint PPT Presentation

Nov 24, 2023 •182 likes •585 views

Applying Deep Neural Network Techniques for LArTPC Data Reconstruction Laura Domine (Stanford University / SLAC) Fermilab Machine Learning Group meeting - 11/7/18 Plan 1. LArTPC & Deep Learning 2. Examples of applications: UResNet &

  1. Applying Deep Neural Network Techniques for LArTPC Data Reconstruction Laura Domine (Stanford University / SLAC) Fermilab Machine Learning Group meeting - 11/7/18

  2. Plan 1. LArTPC & Deep Learning 2. Examples of applications: UResNet & PPN networks 3. Sparse convolutions

  3. LArTPC & Deep Learning

  4. Liquid Argon Time Projection Chamber (LArTPC) Neutrino detectors Ex: MicroBooNE @ Fermilab, 150 tons 2D or 3D data Bigger and bigger! (DUNE) Neutrinos.

  5. Deep Neural Networks (DNN) & Computer Vision Picture from Martin Görner

  6. Deep Neural Networks (DNN) & Computer Vision Object detection & classification Semantic segmentation

  7. Towards a full reconstruction chain with DNN Currently: Lots of heuristic algorithms ● Goal: Replace them with a set of DNN ● algorithms which ideally will Run faster ○ Have a better ○ performance

  8. Towards a full reconstruction chain with DNN Steps: 1. Point detection (track edge) Non-contractual picture - Actual product may differ

  9. Towards a full reconstruction chain with DNN Steps: 1. Point detection (track edge) PPN Non-contractual picture - Actual product may differ

  10. Towards a full reconstruction chain with DNN Steps: 1. Point detection (track edge) PPN 2. Pixel-wise labeling (particle track vs electromagnetic shower) Non-contractual picture - Actual product may differ

  11. Towards a full reconstruction chain with DNN Steps: 1. Point detection (track edge) PPN 2. Pixel-wise labeling (particle track vs electromagnetic shower) UResNet Non-contractual picture - Actual product may differ

  12. Towards a full reconstruction chain with DNN Steps: 1. Point detection (track edge) PPN 2. Pixel-wise labeling (particle track vs electromagnetic shower) UResNet 3. Clustering of energy deposits and instance segmentation Non-contractual picture - Actual product may differ

  13. Towards a full reconstruction chain with DNN Steps: 1. Point detection (track edge) PPN 2. Pixel-wise labeling (particle track vs electromagnetic shower) UResNet 3. Clustering of energy deposits and instance segmentation Work in progress! Non-contractual picture - Actual product may differ

  14. Towards a full reconstruction chain with DNN Steps: 1. Point detection (track edge) PPN 2. Pixel-wise labeling (particle track vs electromagnetic shower) UResNet 3. Clustering of energy deposits and instance segmentation Work in progress! 4. Particle identification and energy estimate 5. Hierarchical reconstruction Non-contractual picture - Actual product may differ

  15. Examples of applications: UResNet and PPN networks

  16. Semantic Segmentation: UResNet

  17. Semantic Segmentation: UResNet Decoder Encoder Residual connections

  18. Semantic Segmentation: UResNet Data Physicist’s label Network’s output A Deep Neural Network for Pixel-Level Electromagnetic Particle Identification in the MicroBooNE Liquid Argon Time Projection Chamber. https://arxiv.org/abs/1808.07269

  19. Point-finding: PPN Inspired by Faster-RCNN architecture Region Proposal Network detects ● regions of interest Replace regions with points = ● Pixel Proposal Network (PPN) Why not Mask-RCNN? Computations expensive ○ Our features topology is ○ different (track, shower)

  21. PPN proposals

  22. PPN needs post-processing + scores…!

  23. PPN needs post-processing Option 1: DBSCAN Density estimation algorithm ● No prior on the number of clusters. ● Option 2: NMS (Non-Maximal Suppression) Popular post-processing method for object ● detection Order by score and prune boxes with too much ● overlap

  26. ZOOM NB: independently of DBSCAN vs NMS, these plots also benefit from debugged ground truth pixels position.

  27. Together

  28. 6mm/voxel UResNet + PPN 3D Analysis 24cm 2 4 c m

  29. 6mm/voxel UResNet + PPN 3D Analysis 24cm 24cm

  30. Sparse UResNet

  31. How do we handle sparse data? Sparse Dense

  32. Naive approach Input: dense 3D matrix of energy deposits. Crop your data ● Run the network on small cropped images ● Stitch together results ● Many cropping algorithms possible Compromises to make: Maximize the number of overlapping boxes (accuracy) ● Minimize the number of boxes (computation time) ●

  33. Sparse Convolutions Many possible definitions and implementations of ‘sparse convolutions’... Submanifold Sparse Convolutions : https://github.com/facebookresearch/SparseConvNet Submanifold? “input data with lower effective dimension than the space in which it lives” Ex: 1D curve in 2+D space, 2D surface in 3+D space Our case: the worst! 1D curve in 3D space ...

  34. Sparse Convolutions Submanifold Sparse Convolutions : https://github.com/facebookresearch/SparseConvNet 3D Semantic Segmentation with Submanifold Sparse Convolutional Networks

  35. Sparse UResNet Input: list of points coordinates and their features (e.g. energy deposition) With UResNet architecture: >99.9% accuracy in 3D ● Faster training (less computations!): only a few hours ● Much lower memory usage ● Example in larcv-viewer

  36. Summary Extract interesting / useful features with deep neural networks: ● Points of interest with PPN ○ Pixel-wise classification track vs shower with UResNet ○ Currently working on clustering and instance segmentation ● (particle type, particle instances) Sparse techniques are very exciting! ● Join DeepLearnPhysics group! Technical discussion on ML applied to experimental physics data ● Data + code sharing for reproducibility ●

  37. Thank you!

  38. Backup slides

  39. PPN Loss: details

Recommend

Intelligent Low-level Signal Detection and Zero- Suppession in Raw LArTPC Waveforms through Deep

Intelligent Low-level Signal Detection and Zero- Suppession in Raw LArTPC Waveforms through Deep

Intelligent Low-level Signal Detection and Zero- Suppession in Raw LArTPC Waveforms through Deep Learning Techniques Lorenzo Uboldi Supervisor Michael Wang Final report 24 September 2019 DUNE Deep Underground Neutrino Experiment is a LArTPC

492 views • 19 slides
Applying Deep Learning to Financial Markets with News Data Executive Summary Large amounts of

Applying Deep Learning to Financial Markets with News Data Executive Summary Large amounts of

Applying Deep Learning to Financial Markets with News Data Executive Summary Large amounts of Use deep neural Predict market news data networks to build direction of predictive models relevant stocks Dealing with News Original Workflow

480 views • 7 slides
for Speech Synthesis and Sensor Data Augmentation Deep Generative Neural Network Speech Text

for Speech Synthesis and Sensor Data Augmentation Deep Generative Neural Network Speech Text

Deep Generative Modeling for Speech Synthesis and Sensor Data Augmentation Deep Generative Neural Network Speech Text Praveen Narayanan Ford Motor Company PROJECT DESCRIPTION Use of DNNs increasingly prevalent as a solution for many data

893 views • 53 slides
Normalization Techniques in Training of Deep Neural Networks Lei Huang ( ) State Key

Normalization Techniques in Training of Deep Neural Networks Lei Huang ( ) State Key

Normalization Techniques in Training of Deep Neural Networks Lei Huang ( ) State Key Laboratory of Software Development Environment, Beihang University Mail:huanglei@nlsde.buaa.edu.cn August 17 th , 2017 Outline Introduction to Deep

845 views • 47 slides
Deep Learning Primer Nishith Khandwala Neural Networks Overview Neural Network Basics

Deep Learning Primer Nishith Khandwala Neural Networks Overview Neural Network Basics

Deep Learning Primer Nishith Khandwala Neural Networks Overview Neural Network Basics Activation Functions Stochastic Gradient Descent (SGD) Regularization (Dropout) Training Tips and Tricks Neural Network (NN)

828 views • 45 slides
Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference

Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference

Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference Engine Song Han CVA group, Stanford University Apr 7, 2016 1 Intro about me and my advisor Fourth year PhD with Prof. Bill Dally at Stanford.

2.06k views • 70 slides
Deep Learning (jkim@bi.snu.ac.kr) 2015/05/7

Deep Learning (jkim@bi.snu.ac.kr) 2015/05/7

Deep Learning (jkim@bi.snu.ac.kr) 2015/05/7 1 History of Neural Network Research Neural network Deep belief net Back propagation Science

1.27k views • 108 slides
Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference

Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference

Deep Compression and EIE: Deep Neural Network Model Compression and Efficient Inference Engine Song Han CVA group, Stanford University Jan 6, 2015 A few words about us Fourth year PhD with Prof. Bill Dally at Stanford.

966 views • 54 slides
Concise Introduction to Deep Neural Networks Outline: Classification problems Motivating

Concise Introduction to Deep Neural Networks Outline: Classification problems Motivating

Concise Introduction to Deep Neural Networks Outline: Classification problems Motivating Deep (large) Neural Network (DNN) classifiers Neurons and DNN architectures Numerical training of DNNs (supervised deep learning) Spiking

630 views • 33 slides
Deep Neural Network for Automatic Speech Recognition: from the Industrys View Jinyu Li

Deep Neural Network for Automatic Speech Recognition: from the Industrys View Jinyu Li

Deep Neural Network for Automatic Speech Recognition: from the Industrys View Jinyu Li Microsoft September 13, 2014 at Nanyang Technological University Speech Modeling in an SR System Training Acoustic Model Acoustic data base

822 views • 80 slides
Neural Networks: What can a network represent Deep Learning, Fall 2020 1 Recap : Neural

Neural Networks: What can a network represent Deep Learning, Fall 2020 1 Recap : Neural

Neural Networks: What can a network represent Deep Learning, Fall 2020 1 Recap : Neural networks have taken over AI Tasks that are made possible by NNs, aka deep learning Tasks that were once assumed to be purely in the human domain

1.92k views • 150 slides
Neural Network Meets DCN: Traffic-driven Topology Adaptation with Deep Learning Moewi Wang, Yong

Neural Network Meets DCN: Traffic-driven Topology Adaptation with Deep Learning Moewi Wang, Yong

Neural Network Meets DCN: Traffic-driven Topology Adaptation with Deep Learning Moewi Wang, Yong Cui, Shihan Xiao, Xin wang, Dan Yang, Kai Chen, Jun Zhu Introduction Conventional wired data centers generally adopt a static network topology

338 views • 14 slides
Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Learning in Neural Networks w 1,3 w 3,5 1 3 5 w w 1,4 3,6 w w 2,3 4,5 2 4 6 w w 2,4 4,6 Class #18: Back-Propagation; } A neural network can learn a classification function by Tuning Hyper-Parameters adjusting its weights to

259 views • 4 slides
Neural Networks: What can a network represent Deep Learning, Spring 2018 1 Recap : Neural

Neural Networks: What can a network represent Deep Learning, Spring 2018 1 Recap : Neural

Neural Networks: What can a network represent Deep Learning, Spring 2018 1 Recap : Neural networks have taken over AI Tasks that are made possible by NNs, aka deep learning 2 Recap : NNets and the brain In their basic form, NNets

2.14k views • 145 slides
IMPLEMENTATION OF A PARALLEL BATCH TRAINING ALGORITHM FOR DEEP NEURAL NETWORK YUPING LIN IFLYTEK

IMPLEMENTATION OF A PARALLEL BATCH TRAINING ALGORITHM FOR DEEP NEURAL NETWORK YUPING LIN IFLYTEK

IMPLEMENTATION OF A PARALLEL BATCH TRAINING ALGORITHM FOR DEEP NEURAL NETWORK YUPING LIN IFLYTEK LABORATORY FOR NEURAL COMPUTING FOR MACHINE LEARNING DEPARTMENT OF ELECTRICAL ENGINEERING AND COMPUTER SCIENCE YORK UNIVERSITY, TORONTO NOVEMBER

212 views • 20 slides
Bias Also Matters: Bias Attribution for Deep Neural Network Explanation Shengjie Wang*, Tianyi

Bias Also Matters: Bias Attribution for Deep Neural Network Explanation Shengjie Wang*, Tianyi

Bias Also Matters: Bias Attribution for Deep Neural Network Explanation Shengjie Wang*, Tianyi Zhou*, Jeff A. Bilmes University of Washington, Seattle Explain DNNs as a linear model per data point DNN with piecewise linear activations like

250 views • 9 slides
Lecture 2: Deeper Neural Network 1 Objective In the second lecture, you will see How to

Lecture 2: Deeper Neural Network 1 Objective In the second lecture, you will see How to

Lecture 2: Deeper Neural Network 1 Objective In the second lecture, you will see How to deepen your neural network to learn more complex functions Several techniques to fasten your learning process The most popular artificial

423 views • 40 slides
15-388/688 - Practical Data Science: Deep learning J. Zico Kolter Carnegie Mellon University

15-388/688 - Practical Data Science: Deep learning J. Zico Kolter Carnegie Mellon University

15-388/688 - Practical Data Science: Deep learning J. Zico Kolter Carnegie Mellon University Fall 2019 1 Outline Recent history in machine learning Machine learning with neural networks Training neural networks Specialized neural network

1.05k views • 39 slides
EN. 601.467/667 Introduc3on to Human Language Technology Deep Learning I Shinji Watanabe 1

EN. 601.467/667 Introduc3on to Human Language Technology Deep Learning I Shinji Watanabe 1

EN. 601.467/667 Introduc3on to Human Language Technology Deep Learning I Shinji Watanabe 1 Todays agenda Introduction of deep neural network Basics of neural network 2 Short bio Research interests Automatic speech

1.71k views • 62 slides
CS6220: DATA MINING TECHNIQUES Image Data: Classification via Neural Networks Instructor: Yizhou

CS6220: DATA MINING TECHNIQUES Image Data: Classification via Neural Networks Instructor: Yizhou

CS6220: DATA MINING TECHNIQUES Image Data: Classification via Neural Networks Instructor: Yizhou Sun yzsun@ccs.neu.edu November 19, 2015 Methods to Learn Matrix Data Text Set Data Sequence Time Series Graph & Images Data Data

310 views • 28 slides
How to Evaluate Efficient Deep Neural Network Approaches Vivienne Sze ( @eems_mit)

How to Evaluate Efficient Deep Neural Network Approaches Vivienne Sze ( @eems_mit)

How to Evaluate Efficient Deep Neural Network Approaches Vivienne Sze ( @eems_mit) Massachusetts Institute of Technology In collaboration with Yu-Hsin Chen, Joel Emer, Yannan Wu, Tien-Ju Yang, Google Mobile Vision Team Slides available at

692 views • 48 slides
and Augmentation for Deep Neural Network Training Trevor Gale Steven Eliuk Cameron Upright

and Augmentation for Deep Neural Network Training Trevor Gale Steven Eliuk Cameron Upright

High-Performance Data Loading and Augmentation for Deep Neural Network Training Trevor Gale Steven Eliuk Cameron Upright tgale@ece.neu.edu steven.eliuk@gmail.com c.upright@samsung.com Roadmap 1. The General-Purpose Acceleration

507 views • 36 slides
Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan Zelener Machine Learning Reading Group January 7 th 2016 The Graduate Center, CUNY Objectives Explain some of the trends of deep learning and

680 views • 44 slides
Learning from Limited Data The University of Tokyo / RIKEN AIP Tatsuya Harada Deep Neural

Learning from Limited Data The University of Tokyo / RIKEN AIP Tatsuya Harada Deep Neural

GTC March 18, 2019 Learning from Limited Data The University of Tokyo / RIKEN AIP Tatsuya Harada Deep Neural Networks for Visual Recognition Deep Neural Networks Applications A yellow train on the tracks near a train station. cellphone cup

661 views • 53 slides

More recommend