FROM SCRATCH TO PRODUCTION Andrew Liu, Ryan Shen Deep Learning - PowerPoint PPT Presentation

DEFECT INSPECTION FROM SCRATCH TO PRODUCTION Andrew Liu, Ryan Shen Deep Learning Solution Architect

Defect Inspection and its challenges NGC Docker images AGENDA Model set up - Unet Data preparation - DAGM Production - Speed up with TensorRT 2

INDUSTRIAL DEFECT INSPECTION 3

INDUSTRIAL DEFECT INSPECTION 4

NGC DOCKER IMAGES 5

Benefits for Deep Learning Workflow High Level Benefits and Feature Set Develop once, deploy Scale across teams of anywhere Single software stack practitioners Developer, DevOp, QC

Defect inspection Workflow from scratch to production within container Training Inference TENSORRT  GRE NGC optimized docker image docker pull docker pull nvcr.io/nvidia/ tensorflow nvcr.io/nvidia/ tensorrt : :18.02-py3 18.02-py2 7

MODEL SET UP 8

WAYS FOR DEFECT INSPECTION Depends on domain adaptation Image Image Object Detection Segmentation Classification Label: 0 / 1 Label: Bounding-Box Label: Polygons Mask (Defect / Non Defect) 9

CNN STRUCTURE LeNet Source: Design of Deep Convolutional Neural Network Architectures for Automated Feature Extraction in Industrial Inspection, D. Weimer et al, 2016 10

convnets perform classification < 1 millisecond “defect class1” 1000-dim vector end-to-end learning https://arxiv.org/abs/1605.06211 11

lots of pixels, little time? ~1/10 second ??? end-to-end learning https://arxiv.org/abs/1605.06211 12

U-Net structure 32 16 16 1 16 16 512 2 512 2 512 2 512 2 512 2 512 2 16 32 32 64 32 32 256 2 256 2 256 2 256 2 256 2 256 2 32 64 64 64 64 128 3X3 Conv2d+ReLU 128 2 128 2 128 2 128 2 128 2 128 2 2X2 MaxPool 2X2 Conv2dTranspose 128 128 128 64 256 128 copy and concatenate 64 2 64 2 64 2 64 2 64 2 64 2 128 256 256 32 2 32 2 32 2

KERAS IMPLEMENTATION Convolution 14

DECODING deconvolution 15

Image segmentation on medical images Same process among various use cases Data Science BOWL Data Science BOWL Data Science BOWL 2016 2017 2018 MRI image CT image Image Left ventricle Nodule Nuclei heart disease Lung cancer Drug discovery 16

DATA PREPARATION 17

INDUSTRIAL OPTICAL INSPECTION German Association for Pattern Recognition • http://resources.mpi-inf.mpg.de/conferences/dagm/2007/prizes.html 18

INDUSTRIAL OPTICAL INSPECTION German Association for Pattern Recognition Pass NG Pass NG Pass NG NG Pass 19

DATA DETAILS Original images are 512 x 512 grayscale format • Output is a tensor of size 512 x 512 x 1 • Each pixel belongs to one of two classes • • Training set consist of 100 images Validation set consist of 50 images • 20

MORE EXAMPLES 21

IMBALANCE DATA 22

Dice Metric (IOU) Metric to compare the similarity of two samples: • 2 𝐵 𝑜𝑚 ________________________________ 𝐵 𝑜 + 𝐵 𝑚 Where: • A n is the area of the contour predicted by the network • A l is the area of the contour from the label • A nl is the intersection of the two • The area of the contour that is predicted correctly by the network • 1.0 means perfect score. • • More accurately compute how well we’re predicting the contour against the label We can just count pixels to give us the respective areas • 23

24 LOSS FUNCTION WITH KERAS 24

25 LEARNING CURVES LOSS 25

APPLICATION: INDUSTRIAL INSPECTION NVIDIA 1000 defect-free, 150 defect images • • Challenges : Not all deviations from the texture are necessarily defects. 26

PRODUCTION 27

FINAL DECISION Plus Human Logic Size, Position, … etc 28

INFERENCE PIPELINE Inference Post-processing and decision making DGX Server DataCenter TensorRT+GRE Determine threshold Domain Criteria Camera  Defect Pattern Ratio Precision/ Recall  Defect Level  Defect region size  Defect counts Result Composite Detectors/Classifiers/Segment … Metadata Fetch image Machine Edge TensorRT+GRE 29 P40

Development flow Called UFF, Universal Framework Format ONNX 30

Defect inspection Workflow from scratch to production within container Training Inference TENSORRT  GRE NGC optimized docker image docker pull docker pull nvcr.io/nvidia/ tensorflow nvcr.io/nvidia/ tensorrt : :18.02-py3 18.02-py3 31

SUMMARY Challenges Delivers Training , inference environment is hard to build, maintain, share. Using NGC Docker images. Model optimizations and speed up throughput. TensorRT + GRE SDK So many deep learning model out there, how to choose the right If your dataset, demand requirement fit the scenario model? like we do. U-Net model is great choice for segmentation task. 32

Thank You 33