a framework for new generation ai research
play

a framework for new-generation AI Research Soumith Chintala, Adam - PowerPoint PPT Presentation

May 27, 2023 •358 likes •1.03k views

a framework for new-generation AI Research Soumith Chintala, Adam Paszke, Sam Gross & Team Facebook AI Research Paradigm shifts in AI research Active Research & Tools for AI Today's AI Future AI keeping up with change Today's AI

  2. a framework for new-generation AI Research Soumith Chintala, Adam Paszke, Sam Gross & Team Facebook AI Research

  3. Paradigm shifts in AI research

  4. Active Research & Tools for AI Today's AI Future AI keeping up with change

  5. Today's AI DenseCap by Justin Johnson & group https://github.com/jcjohnson/densecap Future AI Tools for AI Today's AI

  6. Today's AI DeepMask by Pedro Pinhero & group Future AI Tools for AI Today's AI

  7. Today's AI Machine Translation Future AI Tools for AI Today's AI

  8. Today's AI Text Classification (sentiment analysis etc.) Text Embeddings Graph embeddings Machine Translation Ads ranking Future AI Tools for AI Today's AI

  9. Today's AI Model BatchNorm Train Model Conv2d Objective ReLU Data Future AI Tools for AI Today's AI

  10. Today's AI Model BatchNorm Train Model Conv2d Objective ReLU Data BatchNorm New Conv2d Prediction Deploy & Use ReLU Data Future AI Tools for AI Today's AI

  11. Today's AI Static datasets + Static model structure BatchNorm Train Model Conv2d Objective ReLU Data BatchNorm New Conv2d Prediction Deploy & Use ReLU Data Future AI Tools for AI Today's AI

  12. Today's AI Static datasets + Static model structure BatchNorm Train Model Conv2d Objective ReLU Data O ffl ine Learning BatchNorm New Conv2d Prediction Deploy & Use ReLU Data Future AI Tools for AI Today's AI

  13. Current AI Research / Future AI Self-driving Cars Future AI Tools for AI Today's AI

  14. Current AI Research / Future AI Agents trained in many environments Cars Video games Internet Future AI Tools for AI Today's AI

  15. Current AI Research / Future AI Dynamic Neural Networks self-adding new memory or layers changing evaluation path based on inputs Future AI Tools for AI Today's AI

  16. Current AI Research / Future AI BatchNorm Live Conv2d Prediction ReLU data Continued Online Learning Future AI Tools for AI Today's AI

  17. Prediction Data-dependent change in model structure Tools for AI Current AI Research / Future AI ReLU R e L U BatchNorm BatchNorm Conv2d C o n v 2 d ReLU ReLU BatchNorm BatchNorm Future AI Conv2d Conv2d ReLU ReLU B a t m c h r o N N o r h m c t a B Conv2d Conv2d ReLU BatchNorm Conv2d Today's AI Sample -1

  18. Current AI Research / Future AI BatchNorm Conv2d ReLU ReLU C o n v BatchNorm 2 B a d t c h N o r R m e Conv2d BatchNorm L ReLU U Conv2d ReLU BatchNorm Conv2d Sample -2 Prediction B a t c h N Conv2d o r m ReLU Data-dependent change in model structure Future AI Tools for AI Today's AI

  19. Prediction Change in model-capacity at runtime Tools for AI Current AI Research / Future AI ReLU ReLU ReLU BatchNorm BatchNorm BatchNorm Conv2d Conv2d Conv2d Future AI ReLU ReLU ReLU BatchNorm BatchNorm BatchNorm Conv2d Conv2d Conv2d ReLU BatchNorm Conv2d Today's AI Sample

  20. Prediction Change in model-capacity at runtime Tools for AI Current AI Research / Future AI ReLU ReLU ReLU ReLU ReLU BatchNorm BatchNorm BatchNorm BatchNorm BatchNorm Conv2d Conv2d Conv2d Conv2d Conv2d Future AI ReLU ReLU ReLU ReLU ReLU BatchNorm BatchNorm BatchNorm BatchNorm BatchNorm Conv2d Conv2d Conv2d Conv2d Conv2d ReLU BatchNorm Conv2d Today's AI Sample

  21. The need for a dynamic framework • Interop with many dynamic environments - Connecting to car sensors should be as easy as training on a dataset - Connect to environments such as OpenAI Universe • Dynamic Neural Networks - Change behavior and structure of neural network at runtime • Minimal Abstractions - more complex AI systems means harder to debug without a simple API Future AI Tools for AI Today's AI

  22. Tools for AI research and deployment Many machine learning tools and deep learning frameworks Future AI Tools for AI Today's AI

  23. Tools for AI research and deployment Static graph frameworks Dynamic graph frameworks define-by-run define-and-run Future AI Tools for AI Today's AI

  24. Dynamic graph Frameworks • Model is constructed on the fly at runtime • Change behavior, structure of model • Imperative style of programming Future AI Tools for AI Today's AI

  26. Overview automatic di ff erentiation Ndarray library gradient based engine with GPU support optimization package Deep Learning Numpy-alternative Reinforcement Learning

  27. ndarray library with GPU support

  28. ndarray library • np.ndarray <-> torch.Tensor • 200+ operations, similar to numpy • very fast acceleration on NVIDIA GPUs

  29. ndarray library PyTorch Numpy

  30. ndarray / Tensor library

  31. ndarray / Tensor library

  32. ndarray / Tensor library

  33. ndarray / Tensor library

  34. NumPy bridge

  35. NumPy bridge Zero memory-copy very e ffi cient

  36. NumPy bridge

  37. NumPy bridge

  38. Seamless GPU Tensors

  39. Seamless GPU Tensors A full suite of high performance Tensor operations on GPU

  40. GPUs are fast • Buy $700 NVIDIA 1080Ti • 100x faster matrix multiply • 10x faster operations in general on matrices

  41. automatic di ff erentiation engine for deep learning and reinforcement learning

  42. Deep Learning Frameworks - Provide gradient computation - Gradient of one variable w.r.t. any variable in graph MM MM Add Tanh

  43. Deep Learning Frameworks - Provide gradient computation - Gradient of one variable w.r.t. any variable in graph d(i2h)/d(W_h) MM MM Add Tanh

  44. Deep Learning Frameworks - Provide gradient computation - Gradient of one variable w.r.t. any variable in graph - Provide integration with high performance DL libraries like CuDNN MM MM d(h2h)/d(W_h) Add Tanh

  45. PyTorch Autograd from torch.autograd import Variable

  46. PyTorch Autograd from torch.autograd import Variable x = Variable(torch.randn(1, 10)) prev_h = Variable(torch.randn(1, 20)) W_h = Variable(torch.randn(20, 20)) W_x = Variable(torch.randn(20, 10))

  47. PyTorch Autograd from torch.autograd import Variable x = Variable(torch.randn(1, 10)) MM MM prev_h = Variable(torch.randn(1, 20)) W_h = Variable(torch.randn(20, 20)) W_x = Variable(torch.randn(20, 10)) i2h = torch.mm(W_x, x.t()) h2h = torch.mm(W_h, prev_h.t())

  48. PyTorch Autograd from torch.autograd import Variable x = Variable(torch.randn(1, 10)) MM MM prev_h = Variable(torch.randn(1, 20)) W_h = Variable(torch.randn(20, 20)) W_x = Variable(torch.randn(20, 10)) i2h = torch.mm(W_x, x.t()) h2h = torch.mm(W_h, prev_h.t()) next_h = i2h + h2h

  49. PyTorch Autograd from torch.autograd import Variable x = Variable(torch.randn(1, 10)) MM MM prev_h = Variable(torch.randn(1, 20)) W_h = Variable(torch.randn(20, 20)) W_x = Variable(torch.randn(20, 10)) Add i2h = torch.mm(W_x, x.t()) h2h = torch.mm(W_h, prev_h.t()) next_h = i2h + h2h

  50. PyTorch Autograd from torch.autograd import Variable x = Variable(torch.randn(1, 10)) MM MM prev_h = Variable(torch.randn(1, 20)) W_h = Variable(torch.randn(20, 20)) W_x = Variable(torch.randn(20, 10)) Add i2h = torch.mm(W_x, x.t()) h2h = torch.mm(W_h, prev_h.t()) next_h = i2h + h2h Tanh next_h = next_h.tanh()

  51. PyTorch Autograd from torch.autograd import Variable x = Variable(torch.randn(1, 10)) MM MM prev_h = Variable(torch.randn(1, 20)) W_h = Variable(torch.randn(20, 20)) W_x = Variable(torch.randn(20, 10)) Add i2h = torch.mm(W_x, x.t()) h2h = torch.mm(W_h, prev_h.t()) next_h = i2h + h2h Tanh next_h = next_h.tanh() next_h.backward(torch.ones(1, 20))

  52. side by side: TensorFlow and PyTorch

  53. High performance • Integration of: • CuDNN v6 • NCCL • Intel MKL • 200+ operations, similar to numpy • very fast acceleration on NVIDIA GPUs Upcoming feature: 
 Distributed PyTorch

  54. Planned Feature: JIT Compilation

  55. Compilation benefits Out-of-order Automatic Kernel fusion execution work placement Node 1 ReLU CPU 1 2 3 3 1 2 Node 1 GPU 1 Node 0 BatchNorm GPU 0 Node 1 Node 0 GPU 0 GPU 1 Conv2d Node 0 GPU 0

  56. JIT Compilation • Possible in define-by-run Frameworks • The key idea is deferred or lazy evaluation - y = x + 2 - z = y * y - # nothing is executed yet, but the graph is being constructed - print(z) # now the entire graph is executed: z = (x+2) * (x+2) • We can do just in time compilation on the graph before execution

  57. Lazy Evaluation from torch.autograd import Variable x = Variable(torch.randn(1, 10)) prev_h = Variable(torch.randn(1, 20)) MM MM W_h = Variable(torch.randn(20, 20)) W_x = Variable(torch.randn(20, 10)) i2h = torch.mm(W_x, x.t()) Add h2h = torch.mm(W_h, prev_h.t()) next_h = i2h + h2h next_h = next_h.tanh() Tanh next_h.backward(torch.ones(1, 20))

  58. Lazy Evaluation from torch.autograd import Variable x = Variable(torch.randn(1, 10)) prev_h = Variable(torch.randn(1, 20)) MM MM W_h = Variable(torch.randn(20, 20)) W_x = Variable(torch.randn(20, 10)) i2h = torch.mm(W_x, x.t()) Add h2h = torch.mm(W_h, prev_h.t()) next_h = i2h + h2h Graph built but not actually executed next_h = next_h.tanh() Tanh

Recommend

Generation of political priority for global health initiatives: A research program, framework,

Generation of political priority for global health initiatives: A research program, framework,

Generation of political priority for global health initiatives: A research program, framework, and case study of maternal mortality* Jeremy Shiffman, Ph.D. Associate Professor of Public Administration The Maxwell School of Syracuse University

780 views • 45 slides
ICTs in education Moving from 1 Generation to 2 Generation models a framework for program

ICTs in education Moving from 1 Generation to 2 Generation models a framework for program

ICTs in education Moving from 1 Generation to 2 Generation models a framework for program success Gurumurthy K IT for Change, Bangalore February 2013 ICT Models in Business First generation computer applications in business - Simple

356 views • 8 slides
a European Framework for the generation of guarantees of origin for green hydrogen CertiHy

a European Framework for the generation of guarantees of origin for green hydrogen CertiHy

CertifHy Developing a European Framework for the generation of guarantees of origin for green hydrogen CertiHy event: October 19th 2016, Brussels Project co-finance by the FCH JU The research leading to these results has received under

895 views • 42 slides
Science Framework, And Next Generation Science Standards Context Education is US is a

Science Framework, And Next Generation Science Standards Context Education is US is a

Science Framework, And Next Generation Science Standards Context Education is US is a controlled at the state and local level. Common Core Math and Language Arts -- 47+ states choosing common standards Next Generation Science

534 views • 24 slides
A Framework for Automatic Generation A Framework for Automatic Generation of Configuration Files

A Framework for Automatic Generation A Framework for Automatic Generation of Configuration Files

A Framework for Automatic Generation A Framework for Automatic Generation of Configuration Files for a Custom of Configuration Files for a Custom Hardware/Software RTOS Hardware/Software RTOS Jaehwan Lee* Lee* Jaehwan Kyeong Keol Keol Ryu

702 views • 29 slides
A Reinforcement Learning Framework for Natural Question Generation using Bi-discriminators

A Reinforcement Learning Framework for Natural Question Generation using Bi-discriminators

A Reinforcement Learning Framework for Natural Question Generation using Bi-discriminators Presenter: Ji, Lu Zhihao Fan 1 , Zhongyu Wei 1 , Siyuan Wang 1 , Yang Liu 2 , Xuanjing Huang 3 1 School of Data Science, Fudan University, China 2

573 views • 22 slides
the EU framework programme for research and innovation DG Research and Innovation The

the EU framework programme for research and innovation DG Research and Innovation The

the EU framework programme for research and innovation DG Research and Innovation The Multiannual Financial Framework 2014-2020: Commissions proposals of 29 June 2011 Smart &amp; inclusive growth ( 491bn) 1. Horizon 2020 Education,

360 views • 15 slides
the EU framework programme for research and innovation The Multiannual Financial Framework

the EU framework programme for research and innovation The Multiannual Financial Framework

the EU framework programme for research and innovation The Multiannual Financial Framework 2014-2020: Commissions proposals of 29 June 2011 1. Smart &amp; inclusive growth (491bn) Horizon Education, Competitive Connecting Cohesion

639 views • 19 slides
MontiCore A Framework for DSL-Development Efficient Editor Generation for Compositional DSLs in

MontiCore A Framework for DSL-Development Efficient Editor Generation for Compositional DSLs in

MontiCore A Framework for DSL-Development Efficient Editor Generation for Compositional DSLs in Eclipse Holger Krahn, Bernhard Rumpe, Steven Vlkel Software Systems Engineering Technische Universitt Braunschweig http://www.sse-tubs.de/

265 views • 22 slides
and Use of Data Implementing the ROMA Cycle in the Next Generation Performance Management

and Use of Data Implementing the ROMA Cycle in the Next Generation Performance Management

Introduction to the Analysis and Use of Data Implementing the ROMA Cycle in the Next Generation Performance Management Framework ROMA Next Generation Video Series Barbara Mooney Frederick Richmond Courtney Kohler ROMA Next Generation

482 views • 34 slides
EEN 336 COMMUNICATION SYSTEMS Research Project Supervised By: Dr. Montasir Qasymeh By:

EEN 336 COMMUNICATION SYSTEMS Research Project Supervised By: Dr. Montasir Qasymeh By:

EEN 336 COMMUNICATION SYSTEMS Research Project Supervised By: Dr. Montasir Qasymeh By: Muhammad Obaidullah - 1030313 Overview Introduction 1 st Generation 2 nd Generation 3 rd Generation 4 th Generation The Future Conclusion

477 views • 18 slides
Column Generation Method Frdric Giroire FG Simplex 1/38 Column Generation in Two Words

Column Generation Method Frdric Giroire FG Simplex 1/38 Column Generation in Two Words

Column Generation Method Frdric Giroire FG Simplex 1/38 Column Generation in Two Words A framework to solve large problems. Main idea: not considering explicitly the whole set of variables. decompose the problem into a

610 views • 40 slides
CoCoSim, a code generation framework for control/command applications An overview of CoCoSim for

CoCoSim, a code generation framework for control/command applications An overview of CoCoSim for

CoCoSim, a code generation framework for control/command applications An overview of CoCoSim for multi-periodic discrete Simulink models Hamza Bourbouh, Pierre-Loc Garoche, Thomas Loquen, Eric Noulard and Claire Pagetti January 31 st 2020

395 views • 23 slides
Framework for a Next- Generation Accountability System 01 Accountability design components 02

Framework for a Next- Generation Accountability System 01 Accountability design components 02

Framework for a Next- Generation Accountability System 01 Accountability design components 02 Relative component accountability percentile 03 Criterion referenced component target AGENDA setting 04 Use of subgroup data for

557 views • 13 slides
Techno-economics for next generation fixed access networks Methodology and framework

Techno-economics for next generation fixed access networks Methodology and framework

Techno-economics for next generation fixed access networks Methodology and framework some key assumptions; Major results of the fixed broadband access evolution scenarios Author: Thomas Monath, T-Systems Enterprise Services GmbH

458 views • 23 slides
A New GDML Generation Framework KEVIN WIERMAN PNNL FNAL K. Wierman, LArSoft Geometry December

A New GDML Generation Framework KEVIN WIERMAN PNNL FNAL K. Wierman, LArSoft Geometry December

A New GDML Generation Framework KEVIN WIERMAN PNNL FNAL K. Wierman, LArSoft Geometry December 6, 2016 1 The Current GDML Generation Scheme Python based soluGons Perl-based GDML Script (uBooNE)

469 views • 11 slides
Convergence of Policy based Resource Management Framework in Next Generation Networks Dong Sun

Convergence of Policy based Resource Management Framework in Next Generation Networks Dong Sun

Convergence of Policy based Resource Management Framework in Next Generation Networks Dong Sun Bell Laboratories, Alcatel-Lucent Agenda Challenges Review of current works Converged model Summary 2 Challenges The Internet

704 views • 21 slides
Pegasus: a framework for sound continuous invariant generation Andrew Sogokon 2 , 1 , Stefan

Pegasus: a framework for sound continuous invariant generation Andrew Sogokon 2 , 1 , Stefan

Pegasus: a framework for sound continuous invariant generation Andrew Sogokon 2 , 1 , Stefan Mitsch 1 , Yong Kiam Tan 1 , Katherine Cordwell 1 , e Platzer 1 and Andr 1 Carnegie Mellon University, USA 2 University of Southampton, UK FM 2019, 3rd

398 views • 39 slides
Substra: a Framework for Automatic Generation of Integration Tests Hai Yuan Tao Xie Department of

Substra: a Framework for Automatic Generation of Integration Tests Hai Yuan Tao Xie Department of

Computer Science Substra: a Framework for Automatic Generation of Integration Tests Hai Yuan Tao Xie Department of Computer Science North Carolina State University, USA http://ase.csc.ncsu.edu/ 1 Motivation Software components

406 views • 22 slides
Shaping research for the years to come the next EU Framework Programme for R&amp;I and other

Shaping research for the years to come the next EU Framework Programme for R&amp;I and other

Shaping research for the years to come the next EU Framework Programme for R&amp;I and other policies GSSI 17.6.20 LC outline/timeline evaluation of the previous FPs HE support from Framework&amp;Spe EIC towards the expert

1.15k views • 71 slides
An Ethical Framework for Thinking about Canine Research and Animal Research More Generally

An Ethical Framework for Thinking about Canine Research and Animal Research More Generally

An Ethical Framework for Thinking about Canine Research and Animal Research More Generally DAVID DEGRAZIA, PH.D. SENIOR RESEARCH FELLOW, DEPARTMENT OF BIOETHICS, NIH ELTON PROFESSOR OF PHILOSOPHY, GWU Disclaimer &amp; Disclosure I speak

480 views • 46 slides
Outline of the Presentation Research Context and Objectives Theoretical Framework

Outline of the Presentation Research Context and Objectives Theoretical Framework

Outline of the Presentation Research Context and Objectives Theoretical Framework Research Questions Research Design Main Findings and Discussion Significance of Research Social, cultural and political English

183 views • 17 slides
Operator Language: A Program Generation Framework for Fast Kernels Franz Franchetti, Frdric

Operator Language: A Program Generation Framework for Fast Kernels Franz Franchetti, Frdric

Carnegie Mellon Operator Language: A Program Generation Framework for Fast Kernels Franz Franchetti, Frdric de Mesmay, Daniel McFarlin, Markus Pschel Electrical and Computer Engineering Carnegie Mellon University Sponsors: DARPA DESA

655 views • 26 slides
Comparing NAEP and the Next Generation Science Standards (NGSS) Teresa Neidorf, Yan Wang, Alka

Comparing NAEP and the Next Generation Science Standards (NGSS) Teresa Neidorf, Yan Wang, Alka

Comparing NAEP and the Next Generation Science Standards (NGSS) Teresa Neidorf, Yan Wang, Alka Arora, and Kim Gattis American Institutes for Research June 24, 2015 CCSSO/NCSA San Diego, CA The NGSS and NAEP Frameworks K-12 Framework

1.14k views • 75 slides

More recommend