re thinking cnn frameworks for time sensitive autonomous
play

Re-thinking CNN Frameworks for Time- Sensitive Autonomous-Driving - PowerPoint PPT Presentation

Jul 01, 2023 •192 likes •1.04k views

Re-thinking CNN Frameworks for Time- Sensitive Autonomous-Driving Applications: Addressing an Industrial Challenge Ming Yang 1 , Shige Wang 2 , Joshua Bakita 1 , Thanh Vu 1 , F. Donelson Smith 1 , James H. Anderson 1 , and Jan-Michael Frahm 1 1

  1. Re-thinking CNN Frameworks for Time- Sensitive Autonomous-Driving Applications: Addressing an Industrial Challenge Ming Yang 1 , Shige Wang 2 , Joshua Bakita 1 , Thanh Vu 1 , F. Donelson Smith 1 , James H. Anderson 1 , and Jan-Michael Frahm 1 1 The University of North Carolina at Chapel Hill 2 General Motors Research

  2. Re-thinking CNN Frameworks for Time- Sensitive Autonomous-Driving Applications: Addressing an Industrial Challenge Ming Yang 1 , Shige Wang 2 , Joshua Bakita 1 , Thanh Vu 1 , F. Donelson Smith 1 , James H. Anderson 1 , and Jan-Michael Frahm 1 1 The University of North Carolina at Chapel Hill 2 General Motors Research

  3. � 3 Ming Yang - RTAS 2019

  4. � 3 Ming Yang - RTAS 2019

  5. � 3 Ming Yang - RTAS 2019

  6. � 3 Ming Yang - RTAS 2019

  7. https://blogs.nvidia.com/blog/2016/01/04/ automotive-nvidia-drive-px-2/ � 3 Ming Yang - RTAS 2019

  8. https://blogs.nvidia.com/blog/2016/01/04/ automotive-nvidia-drive-px-2/ Icons made by Freepik from � 3 Ming Yang - RTAS 2019 Flaticon is licensed by CC 3.0 BY

  9. https://blogs.nvidia.com/blog/2016/01/04/ automotive-nvidia-drive-px-2/ 1. Response time Icons made by Freepik from � 3 Ming Yang - RTAS 2019 Flaticon is licensed by CC 3.0 BY

  10. https://blogs.nvidia.com/blog/2016/01/04/ automotive-nvidia-drive-px-2/ 1. Response time 2. Accuracy Icons made by Freepik from � 3 Ming Yang - RTAS 2019 Flaticon is licensed by CC 3.0 BY

  11. https://blogs.nvidia.com/blog/2016/01/04/ automotive-nvidia-drive-px-2/ 1. Response time 2. Accuracy 3. Throughput Icons made by Freepik from � 3 Ming Yang - RTAS 2019 Flaticon is licensed by CC 3.0 BY

  12. Our focus https://blogs.nvidia.com/blog/2016/01/04/ automotive-nvidia-drive-px-2/ 1. Response time 2. Accuracy 3. Throughput � 4 Ming Yang - RTAS 2019

  13. Our focus https://blogs.nvidia.com/blog/2016/01/04/ automotive-nvidia-drive-px-2/ 1. Response time Hardware 2. Accuracy resources are 3. Throughput constrained and expensive . � 4 Ming Yang - RTAS 2019

  14. Our focus https://blogs.nvidia.com/blog/2016/01/04/ automotive-nvidia-drive-px-2/ 1. Response time Hardware 2. Accuracy resources are 3. Throughput constrained and expensive . CNN software underutilizes the hardware. � 4 Ming Yang - RTAS 2019

  15. Current CNN frameworks CPU GPU � 5 Ming Yang - RTAS 2019

  16. Current CNN frameworks CPU GPU � 5 Ming Yang - RTAS 2019

  17. Current CNN frameworks CPU GPU � 5 Ming Yang - RTAS 2019

  18. Current CNN frameworks Gaps CPU GPU � 5 Ming Yang - RTAS 2019

  19. Current CNN frameworks Gaps CPU GPU Cycles not utilized � 5 Ming Yang - RTAS 2019

  20. Current CNN frameworks Gaps CPU GPU Cycles not utilized Single CNN underutilizes the hardware. � 5 Ming Yang - RTAS 2019

  21. Traditional Multiple-Camera Processing Setup 0 Private CNN … … … Private CNN C-1 � 6 Ming Yang - RTAS 2019

  22. Traditional Multiple-Camera Processing Setup 0 Private CNN … … … Private CNN C-1 Issues: � 6 Ming Yang - RTAS 2019

  23. Traditional Multiple-Camera Processing Setup 0 Private CNN … … … Private CNN C-1 Issues: 1. Memory requirements multiply, limiting the number of instances. � 6 Ming Yang - RTAS 2019

  24. Traditional Multiple-Camera Processing Setup 0 Private CNN … … … Private CNN C-1 Issues: 1. Memory requirements multiply, limiting the number of instances. 2. Context switches on GPU cause overheads. � 6 Ming Yang - RTAS 2019

  25. Traditional Multiple-Camera Processing Setup 0 Private CNN … … … Private CNN C-1 Issues: 1. Memory requirements multiply, limiting the number of instances. 2. Context switches on GPU cause overheads. 3. Fast synchronization between cameras becomes harder. � 6 Ming Yang - RTAS 2019

  26. Traditional Multiple-Camera Processing Setup 0 Private CNN … … … Private CNN C-1 Parallelism through multi-process isn’t helping. Issues: 1. Memory requirements multiply, limiting the number of instances. 2. Context switches on GPU cause overheads. 3. Fast synchronization between cameras becomes harder. � 6 Ming Yang - RTAS 2019

  27. Proposed Solutions Part I: Parallel Execution for CNN frameworks Multi-camera Composite Images to provide Part II: high throughput for multiple cameras. � 7 Ming Yang - RTAS 2019

  28. Proposed Solutions Part I: Parallel Execution for CNN frameworks Multi-camera Composite Images to provide Part II: high throughput for multiple cameras. � 8 Ming Yang - RTAS 2019

  29. Let’s re-think the design of CNN frameworks Layer 1 Layer n Part I: Parallel Execution … Part II: Multi- camera Composite Images � 9 Ming Yang - RTAS 2019 Ming Yang - RTAS 2019

  30. Let’s re-think the design of CNN frameworks Layer 1 • CNN models are graphs of Layer n Part I: layers . Parallel Execution … Part II: Multi- camera Composite Images � 9 Ming Yang - RTAS 2019 Ming Yang - RTAS 2019

  31. Let’s re-think the design of CNN frameworks Layer 1 • CNN models are graphs of Layer n Part I: layers . Parallel Execution • Processing of images can be … independent , e.g., object detection. Part II: Multi- camera Composite Images � 9 Ming Yang - RTAS 2019 Ming Yang - RTAS 2019

  32. Let’s re-think the design of CNN frameworks Layer 1 • CNN models are graphs of Layer n Part I: layers . Parallel Execution • Processing of images can be … independent , e.g., object detection. Part II: Multi- camera Composite Images We enable parallel execution for CNN frameworks and shared CNN for multiple cameras. � 9 Ming Yang - RTAS 2019 Ming Yang - RTAS 2019

  33. Stage 0 Stage 1 Stage N-1 … • Generalize concept of 퓁 퓀 r + e y 퓁 layers into stages a r L e y a L … � 10 Ming Yang - RTAS 2019

  34. Stage 0 Stage 1 Stage N-1 Queue 1 Queue N-1 Queue 0 1 3 2 … • Generalize concept of Bookkeeping Frames layers into stages data 0 0 • Communicate data 1 Data for 1 Frame 1 2 between stages using Data for 2 Frame 2 3 Data for PGM RT (a processing 3 Frame 3 graph management tool) � 11 Ming Yang - RTAS 2019

  35. Shared CNN Cameras Detection box results … Stage 0 Stage 1 Stage N-1 0 Queue 1 Queue N-1 Queue 0 1 3 2 … … C-1 • Generalize concept of Bookkeeping Frames layers into stages data 0 0 • Communicate data 1 Data for 1 Frame 1 2 between stages using Data for 2 Frame 2 3 Data for PGM RT (a processing 3 Frame 3 graph management tool) • Share CNN among multiple cameras � 12 Ming Yang - RTAS 2019

  36. Different Execution Methods Part I: Parallel Execution Part II: Multi- camera Composite Images � 13 Ming Yang - RTAS 2019 Ming Yang - RTAS 2019

  37. Different Execution Methods S ERIAL private CNN in one process Part I: Parallel Execution Part II: Multi- camera Composite Images � 13 Ming Yang - RTAS 2019 Ming Yang - RTAS 2019

  38. Different Execution Methods S ERIAL private CNN in one process Part I: Parallel Execution P IPELINE shared CNN that has one thread per stage Part II: Multi- camera Composite Images � 13 Ming Yang - RTAS 2019 Ming Yang - RTAS 2019

  39. Different Execution Methods S ERIAL private CNN in one process Part I: Parallel Execution P IPELINE shared CNN that has one thread per stage Part II: Multi- camera Composite P ARALLEL shared CNN that has multiple threads per stage Images � 13 Ming Yang - RTAS 2019 Ming Yang - RTAS 2019

  40. CPU S ERIAL GPU Part I: CPU P IPELINE Parallel Execution GPU Part II: Multi- camera CPU Composite P ARALLEL Images GPU � 14 Ming Yang - RTAS 2019 Ming Yang - RTAS 2019

  41. CPU S ERIAL GPU Part I: CPU P IPELINE Parallel Execution GPU Part II: Multi- camera CPU Composite P ARALLEL Images GPU � 14 Ming Yang - RTAS 2019 Ming Yang - RTAS 2019

  42. CPU S ERIAL GPU Part I: CPU P IPELINE Parallel Execution GPU Part II: Multi- camera CPU Composite P ARALLEL Images GPU � 14 Ming Yang - RTAS 2019 Ming Yang - RTAS 2019

  43. CPU S ERIAL GPU Part I: CPU P IPELINE Parallel Execution GPU Part II: Multi- camera CPU Composite P ARALLEL Images GPU � 14 Ming Yang - RTAS 2019 Ming Yang - RTAS 2019

  44. CPU S ERIAL GPU Part I: CPU P IPELINE Parallel Execution GPU Part II: Multi- camera CPU Composite P ARALLEL Images GPU � 14 Ming Yang - RTAS 2019 Ming Yang - RTAS 2019

  45. CPU S ERIAL GPU Part I: CPU P IPELINE Parallel Execution GPU Part II: Multi- camera CPU Composite P ARALLEL Images GPU � 14 Ming Yang - RTAS 2019 Ming Yang - RTAS 2019

  46. CPU S ERIAL GPU Part I: CPU P IPELINE Parallel Execution GPU Part II: Multi- camera CPU Composite P ARALLEL Images GPU � 14 Ming Yang - RTAS 2019 Ming Yang - RTAS 2019

  47. CPU S ERIAL GPU Concurrency Part I: CPU P IPELINE Parallel Execution GPU Part II: Multi- camera CPU Composite P ARALLEL Images GPU � 14 Ming Yang - RTAS 2019 Ming Yang - RTAS 2019

  48. CPU S ERIAL GPU Concurrency Part I: CPU P IPELINE Parallel Execution GPU Part II: Multi- camera CPU Composite P ARALLEL Images GPU � 14 Ming Yang - RTAS 2019 Ming Yang - RTAS 2019

  49. CPU S ERIAL GPU Concurrency Part I: CPU P IPELINE Parallel Execution GPU Part II: Multi- camera CPU Composite P ARALLEL Images GPU � 14 Ming Yang - RTAS 2019 Ming Yang - RTAS 2019

Recommend

The AGEE Forum Geneva: 17 May 2013 Four key questions: What priority do we give to thinking

The AGEE Forum Geneva: 17 May 2013 Four key questions: What priority do we give to thinking

The AGEE Forum Geneva: 17 May 2013 Four key questions: What priority do we give to thinking about the future? How much space and time do we give for long-term strategic thinking? How sensitive are our policies to global

109 views • 7 slides
Introduction Supporting Time-Sensitive Applications on a Commodity OS by: Ashvin Goel, Luca

Introduction Supporting Time-Sensitive Applications on a Commodity OS by: Ashvin Goel, Luca

Introduction Supporting Time-Sensitive Applications on a Commodity OS by: Ashvin Goel, Luca Abeni, Charles Krasic, Jim Snow, Jonathan Walpole presented by: Dora Raymaker Background: Real-Time and Time Sensitive Applications Real-Time

549 views • 31 slides
Heuristic List Scheduler for Time-Triggered traffic in Time- Sensitive Networks Maryam Pahlevan,

Heuristic List Scheduler for Time-Triggered traffic in Time- Sensitive Networks Maryam Pahlevan,

Heuristic List Scheduler for Time-Triggered traffic in Time- Sensitive Networks Maryam Pahlevan, Nadra Tabassam and Roman Obermaisser University of Siegen Siegen, Germany Background- Time Sensitive Networking Standard Ethernet Provides

442 views • 16 slides
Thinking about child poverty: Concepts, frameworks, measures, trends and targets Bryan

Thinking about child poverty: Concepts, frameworks, measures, trends and targets Bryan

Thinking about child poverty: Concepts, frameworks, measures, trends and targets Bryan Perry Ministry of Social Development, NZ Oranga Tamariki Seminar, 30 Nov 2017 Material wellbeing part of wider wellbeing Starting point for

307 views • 13 slides
Time Sensitive Application Transport Mohammad Shafahi University of Amsterdam Faculty of

Time Sensitive Application Transport Mohammad Shafahi University of Amsterdam Faculty of

Time Sensitive Application Transport Mohammad Shafahi University of Amsterdam Faculty of Science Introduction Time sensitive applications are becoming more and more demanded The demand of TSAs has introduced discussions in the network

617 views • 41 slides
A Probabilistic Framework for Time-Sensitive Search Dhruv Gupta & Klaus Berberich {dhgupta,

A Probabilistic Framework for Time-Sensitive Search Dhruv Gupta & Klaus Berberich {dhgupta,

A Probabilistic Framework for Time-Sensitive Search Dhruv Gupta & Klaus Berberich {dhgupta, kberberi}@mpi-inf.mpg.de June 9, 2016 1 1 Motivation 2 Building Blocks for Time-Sensitive Search 3 Temporal Intent Disambiguation 4 Temporally

818 views • 38 slides
Evaluation of Online Schedule Synthesis Algorithms for Time-based Scheduled Time Sensitive

Evaluation of Online Schedule Synthesis Algorithms for Time-based Scheduled Time Sensitive

Chair of Network Architectures and Services Department of Informatics Technical University of Munich Evaluation of Online Schedule Synthesis Algorithms for Time-based Scheduled Time Sensitive Networks Intermediate talk for the Masters Thesis

214 views • 17 slides
Evaluation of Online Schedule Synthesis Algorithms for Time-based Scheduled Time Sensitive

Evaluation of Online Schedule Synthesis Algorithms for Time-based Scheduled Time Sensitive

Chair of Network Architectures and Services Department of Informatics Technical University of Munich Evaluation of Online Schedule Synthesis Algorithms for Time-based Scheduled Time Sensitive Networks Final talk for the Masters Thesis by

298 views • 25 slides
A6: Sensitive Data Exposure A6 Sensitive Data Exposure Sensitive data stored or transmitted

A6: Sensitive Data Exposure A6 Sensitive Data Exposure Sensitive data stored or transmitted

A6: Sensitive Data Exposure A6 Sensitive Data Exposure Sensitive data stored or transmitted insecurely Failure to protect all sensitive data Usernames, passwords, password hashes, credit-card information, identity info Session

415 views • 17 slides
An Examination of the Run- time Performance of GUI Creation Frameworks Christopher J. Howell

An Examination of the Run- time Performance of GUI Creation Frameworks Christopher J. Howell

An Examination of the Run- time Performance of GUI Creation Frameworks Christopher J. Howell Gregory M. Kapfhammer Robert S. Roos Principles and Practices of Programming in Java Kilkenny City, Ireland June 16-18, 2003

467 views • 13 slides
Cognitive Technologies, Thinking and Time Alvin F. Poussaint, MD Visiting Lectureship December 6

Cognitive Technologies, Thinking and Time Alvin F. Poussaint, MD Visiting Lectureship December 6

Cognitive Technologies, Thinking and Time Alvin F. Poussaint, MD Visiting Lectureship December 6 2016 M-J. Seplveda, MD FACP FAAP IBM Fellow Alvin F. Poussaint, MD 2 Disclaimer Purpose: stimulate your thinking Discussion threads time

712 views • 29 slides
On 802.1 Time Sensitive Networking Norm Finn Cisco 1 1 1 1 This is an overview, with

On 802.1 Time Sensitive Networking Norm Finn Cisco 1 1 1 1 This is an overview, with

On 802.1 Time Sensitive Networking Norm Finn Cisco 1 1 1 1 This is an overview, with lots of pointers to more detailed documents. One slide has market justification. The rest of the deck is about making it happen, with an emphasis on

436 views • 18 slides
Web Frameworks Web Frameworks Banned for homework assignments Now that you're starting

Web Frameworks Web Frameworks Banned for homework assignments Now that you're starting

Web Frameworks Web Frameworks Banned for homework assignments Now that you're starting your project where you can use these Let's talk about it Web Frameworks There are many common tasks that every web developer must accomplish on

632 views • 22 slides
We spend a lot of time designing the bridge, but not enough time thinking about the people who are

We spend a lot of time designing the bridge, but not enough time thinking about the people who are

We spend a lot of time designing the bridge, but not enough time thinking about the people who are crossing it. Dr. Prabhjot Singh U-SENTRIC COMPANY PROFILE U-Sentric was founded to narrow the gap between companies and the people they serve.

639 views • 26 slides
Historically Forward Thinking April 8, 2013 Historically Forward Thinking County ED Mission

Historically Forward Thinking April 8, 2013 Historically Forward Thinking County ED Mission

Historically Forward Thinking April 8, 2013 Historically Forward Thinking County ED Mission Economic Diversification Competitive Posture ED Goals, Strategies & Time Frames Historically Forward Thinking Department formed in

1.16k views • 89 slides
How to work with people* who spend less time than you do thinking about behaviour (*Everyone)

How to work with people* who spend less time than you do thinking about behaviour (*Everyone)

How to work with people* who spend less time than you do thinking about behaviour (*Everyone) Chris Atherton, Netlife / @finiteattention It was at this point I realised I had a problem Visual design hygiene is a no- brainer when we're making

872 views • 72 slides
Cognitive De-biasing Strategies for Critical, Time Sensitive Decision- making in Austere

Cognitive De-biasing Strategies for Critical, Time Sensitive Decision- making in Austere

Cognitive De-biasing Strategies for Critical, Time Sensitive Decision- making in Austere Environmental Emergencies What I Learned Spending 14 hours Peeing In My Wetsuit - Andrew Munoz Objectives Identify cognitive centers Isolate biases

601 views • 33 slides
Frameworks Concepts set of cooperating classes Frameworks extending some class

Frameworks Concepts set of cooperating classes Frameworks extending some class

Frameworks Concepts set of cooperating classes Frameworks extending some class inversion of control Examples from java API Horstmann ch.8-8.4 swing, applet, collection Constructing a framework graph editor

570 views • 9 slides
Thinking and acting globally The role of recognition, quality assurance and qualifications

Thinking and acting globally The role of recognition, quality assurance and qualifications

Thinking and acting globally The role of recognition, quality assurance and qualifications frameworks for student and talent mobility: challenges and opportunities Stig Arne Skjerven, Director of foreign education, NOKUT; ENIC bureau president

288 views • 16 slides
TSNsched: Automated Schedule Generation for Time Sensitive Networking Aellison Cassimiro T. dos

TSNsched: Automated Schedule Generation for Time Sensitive Networking Aellison Cassimiro T. dos

TSNsched: Automated Schedule Generation for Time Sensitive Networking Aellison Cassimiro T. dos Santos * , Ben Schneider , Vivek Nigam * * Universidade Federal da Paraba, fortiss FMCAD 2019 Oct 23, 2019 SUMMARY 1. TSN:

687 views • 27 slides
Selective Coflow Completion for Time-sensitive Distributed Applications with Poco Shouxi Luo

Selective Coflow Completion for Time-sensitive Distributed Applications with Poco Shouxi Luo

Selective Coflow Completion for Time-sensitive Distributed Applications with Poco Shouxi Luo Joint work with Pingzhi Fan, Huanlai Xing, and Hongfang Yu Outline Coflow patterns in DCN Existing solutions Two trade-offs Poco: key

1.04k views • 40 slides
MANAGEMENT PRESENTATION May 1, 2017 DISCLAIMER This presentation includes time- sensitive

MANAGEMENT PRESENTATION May 1, 2017 DISCLAIMER This presentation includes time- sensitive

MANAGEMENT PRESENTATION May 1, 2017 DISCLAIMER This presentation includes time- sensitive information that may be accurate only as of todays date, May 1, 2017. Estimates of future net income per share, funds from operations per share,

602 views • 48 slides
Using the Network as a Reliable Platform for Time-Sensitive Systems Henrik Austad

Using the Network as a Reliable Platform for Time-Sensitive Systems Henrik Austad

Using the Network as a Reliable Platform for Time-Sensitive Systems Henrik Austad haustad@cisco.com Cisco Systems https://xkcd.com/927/ Berlin, Oct 13, 2016 This talk will dive into what TSN actually is, what it can provide and its

778 views • 33 slides
LMIs and autonomous work 1 From autonomous work to discontinuous career paths Autonomous

LMIs and autonomous work 1 From autonomous work to discontinuous career paths Autonomous

LMIs and autonomous work 1 From autonomous work to discontinuous career paths Autonomous work raises many questions in terms of continuity of income, skills development, access to the social security system, integration into the social

613 views • 14 slides

More recommend