deep reinforcement learning for robotics using dianne
play

Deep Reinforcement Learning for Robotics Using DIANNE Tim Verbelen, - PowerPoint PPT Presentation

Deep Reinforcement Learning for Robotics Using DIANNE Tim Verbelen, Steven Bohez, Elias De Coninck, Sam Leroux, Pieter Van Molle Bert VanKeirsbilck, Pieter Simoens, Bart Dhoedt sam.leroux@ugent.be PUBLIC How can we build robots that are able to


  1. Deep Reinforcement Learning for Robotics Using DIANNE Tim Verbelen, Steven Bohez, Elias De Coninck, Sam Leroux, Pieter Van Molle Bert VanKeirsbilck, Pieter Simoens, Bart Dhoedt sam.leroux@ugent.be PUBLIC

  2. How can we build robots that are able to execute complex tasks without programming them explicitly ?

  3. Kuka Youbot Gripper 5 axis arm Length: 66 cm Battery operated Embedded PC Omnidirectional wheels Max speed: 0.8 m/s 3

  4. Kuka soft gripper Hokuyo Laser rangefinder

  5. Reinforcement learning Agent Environment 5

  6. Reinforcement learning Agent Environment Observation 6

  7. Reinforcement learning Action Agent Environment 7

  8. Reinforcement learning Reward Agent Environment 8

  9. Deep Reinforcement learning The actor needs to process high dimensional observations to determine the next action. ● Our favorite processing block: deep neural networks ● Observation Action 9

  10. How can we train without destroying our robot ?

  11. 11

  12. V-REP simulator 12

  13. Multiple simulator instances gathering experience on CPU 13

  14. Multiple simulator instances gathering experience on CPU GPU system training the model 14

  15. Abstraction layer with ROS Base Arm Sensor 15

  16. How can we evaluate our models on the robot ?

  17. Brain transplantation ! 17

  18. How can we connect the different components ?

  19. 19

  20. Dianne • Modular software framework for designing, training and evaluating neural networks. • Distributed training and evaluation • Java based • Easy integration (service based architecture) • GUI • Open source (AGPL 3) 20

  21. Deployed Deployed agent agent 21

  22. Experience Pool Deployed Deployed agent agent 22

  23. Experience Pool Deployed Deployed Training agent agent Repository 23

  24. Experience Pool Deployed Deployed Training agent agent Repository 24

  25. Deep Reinforcement learning algorithms

  26. DQN “Playing Atari with Deep Reinforcement Learning” (Mnih et al, 2013) Q Values raw laser scanner measurements Expected future return (512 values) for each possible action 26

  27. 27

  28. DDPG Continuous control with Deep Reinforcement Learning (Lillicrap, et al. 2015) Actor network Continuous action raw laser scanner measurements (512 values) Expected future return Critic network 28

  29. 29

  30. Visit dianne.intec.ugent.be for more information 30

  31. PUBLIC

Recommend


More recommend