SYNTHETIC DATA / AI Rev Lebaredian - Vice President, Simulation - PowerPoint PPT Presentation

SYNTHETIC DATA / AI Rev Lebaredian - Vice President, Simulation Technolgy

NEURAL NETWORKS NEED DATA And Labels! Deep Learning is amazing! Require huge amounts of quality data Data needs labeling For some problems, data + labeling is available Good data doesn’t exists for most problems Image-Net (http://www.image-net.org/) example 2

DOMINOS, ANYONE? Isaac at SIGGRAPH 2017 3

NO EXISTING DATASET 4

1min per object Labeled Data 5

SYNTHETIC DATA Real data is expensive, sometimes dangerous Synthetic labels are automatic and accurate Useful for validation, in addition to training 7

AV SYNTHETIC DATASETS Virtual KITTI Adrien Gaidon, Qiao Wang, Yohann Cabon, Eleonora Vig: Virtual Worlds as Proxy for Multi-Object Tracking Analysis IEEE CVPR 2016 8

DOMAIN RANDOMIZATION Explore the gap using random cars, textures, camera, distractors, etc 9

DOMAIN RANDOMIZATION Example Scenes 10

STRUCTURED DOMAIN RANDOMIZATION Putting it all together 11

NVIDIA DRIVE END-TO-END PLATFORM COLLECT DATA TRAIN MODELS SIMULATE DRIVE Cars Pedestrians Path Cars Pedestrians Path Lanes Signs Lights Lanes Signs Lights 12

13

14

REINFORCEMENT LEARNING SUCCESSES AlphaZero OpenAI Five OpenAI, 2018 Deepmind, 2018 15

APPLICATIONS Reinforcement Learning Locomotion/Animation Liang, Makoviychuk, Handa etc, 2018 NVIDIA 16

APPLICATIONS Robotics Sim2Real Robotics Chebotar, Handa, Makoviychuk, etc, 2018 NVIDIA 17

EXAMPLES Locomotion 18

EXAMPLES Locomotion + Physics 19

EXAMPLES Locomotion + Physics 20

ISAAC GYM Toolkit for Parallel AI Learning Experiments 21

PHYSICS PhysX 22

PHYSICS FleX Multi-physics • • Rigid and FEM soft bodies Cloth, ropes • Liquids • • Two-way coupling and force propagation between different phases 23

CABLE ROBOT 24

ROBOTS SEE, HEAR, TOUCH LEARN ACT & PLAN 25

26

27

BARTENDING ROBOT Trained in Isaac Gym 28

ISAAC SIM 29

30

31

32

Artificial Intelligence Human Reality Virtual Reality Our Reality 33