CS 4803 / 7643: Deep Learning Topics: Structured representations - PowerPoint PPT Presentation

CS 4803 / 7643: Deep Learning Topics: – Structured representations with graph networks Zsolt Kira Georgia Tech

Deep Learning (C) Dhruv Batra & Zsolt Kira 2 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 3 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 4 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 5 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 6 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 7 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 8 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 9 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 10 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 11 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 12 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 13 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 14 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 15 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 16 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 17 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 18 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 19 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 20 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 21 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 22 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 23 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 24 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 25 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 26 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 27 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 28 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 29 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 30 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 31 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 32 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 33 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 34 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 35 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 36 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 37 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 38 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 39 Slide Credit: Thomas Kipf

Problem Description • Application: Component detection on PCBs. • Input: high resolution (eg. 6000 x 4000) PCB images. • Output: component types (eg. resistor, capacitor, IC, etc) and bounding boxes. • Challenge 1: expensive to build a large- scale dataset for training • ~500 components per PCB. • Require professional labeler. • Very few high-quality datasets available on the Internet. Data-Efficient Graph Embedding Learning for PCB Component Detection Chia-Wen Kuo, Jacob Ashmore, David Huggins, Zsolt Kira (C) Dhruv Batra & Zsolt Kira 40

Problem Description • Challenge 2: data distribution • Unbalanced data distribution: 100+ resistors and capacitors, 10+ ICs, and only a few switches. • High intra-class variance (eg. connector). • Low inter-class variance (eg. resistor, led, capacitor). (C) Dhruv Batra & Zsolt Kira 41

Graph Network (GN) • Capture the spatial feature of component layout. • Capture the structure of feature manifold. • Capture the global statistics of the whole board. • => Refine the feature of object proposals based on these additional sources of information. • => Everything is learned and jointly optimized including graph edges and node features. (C) Dhruv Batra & Zsolt Kira 42

Similarity Prediction Network (C) Dhruv Batra & Zsolt Kira 43

Pipeline (C) Dhruv Batra & Zsolt Kira 44

Results • Significant improvement in mAP with graph network (GN) within a board – Triplet loss used to train similarity prediction. – Propagation of label features in few labeled examples further improves results. (C) Dhruv Batra & Zsolt Kira 45

Results Blue : correct type and precise bounding box location. Cyan : imprecise bounding box location. Magenta : miss-detected component. Yellow : precise bounding box location but wrong type. • Leveraging the local, spatial, and global structure of PCB boards results in significant improvements over standard detection pipelines. • Connectivity can be initialized via learned similarity and jointly optimized to learn the structure to maximize accuracy. (C) Dhruv Batra & Zsolt Kira 46

(C) Dhruv Batra & Zsolt Kira 47 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 48 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 49 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 50 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 51 Slide Credit: Thomas Kipf

(C) Dhruv Batra & Zsolt Kira 52 Slide Credit: Thomas Kipf