recent advances in reinforcement learning with a focus on
play

Recent Advances in Reinforcement Learning (with a focus on - PowerPoint PPT Presentation

Dec 14, 2022 •560 likes •704 views

01/29/2020 Recent Advances in Reinforcement Learning (with a focus on ) Patrick Scholz Division of Computer Assisted Medical Interventions Author Division Taxonomic position of RL 01/28/2020 | Page2 01/29/2020 |

  1. 01/29/2020 Recent Advances in Reinforcement Learning (with a focus on ) Patrick Scholz Division of Computer Assisted Medical Interventions

  2. Author Division Taxonomic position of RL 01/28/2020 | Page2 01/29/2020 |

  3. Author Division Basics of RL 01/28/2020 | Page3 Markov Decision Process S – States A – Possible Actions P – Transition Probability R – Immediate Reward Policy Cumulative reward 01/29/2020 |

  4. Author Division Deep RL within the last years wrt 01/28/2020 | Page4 AlphaGo AlphaZero MuZero Zero AlphaGo 2015 2016 2017 2018 2019 01/29/2020 |

  5. Author Division “Deep” Learning and Reinforcement learning 01/28/2020 | Page5 Mnih, V., Kavukcuoglu, K., Silver, D. et al. ‘Human-level control through deep reinforcement learning’. Nature 518, 529–533 (2015). https://doi.org/10.1038/nature14236 01/29/2020 |

  6. Author Division „Go“ as the next holy grail 01/28/2020 | Page6 Using expert moves for Playing against earlier versions supervised learning to generate data Defeated Lee Sedol (world champion) in a regular match 4:1 (using 48 TPUs) Silver, D., Huang, A., Maddison, C. et al. ‘Mastering the game of Go with deep neural networks and tree search’. Nature 529, 484–489 (2016). https://doi.org/10.1038/nature16961 01/29/2020 |

  7. Author Division „Go“ as the next holy grail 01/28/2020 | Page7 Monte Carlo Tree Search Silver, D., Huang, A., Maddison, C. et al. ‘Mastering the game of Go with deep neural networks and tree search’. Nature 529, 484–489 (2016). https://doi.org/10.1038/nature16961 01/29/2020 |

  8. Author Division Dropping initial human input 01/28/2020 | Page8 Major design changes: ● using MCTS action distribution to train ● combining policy and value network ● switching to ResNet architecture ● no hand-crafted input features any more Defeated AlphaGo after 72h under same conditions 100:0 (using 4 TPUs) Silver, D., Schrittwieser, J., Simonyan, K. et al. ‘Mastering the game of Go without human knowledge’. Nature 550, 354–359 (2017). https://doi.org/10.1038/nature24270 01/29/2020 |

  9. Author Division Generalizing input/output representation 01/28/2020 | Page9 Major design changes: ● including draws ● no augmentation exploitation any more ● continuously updating instead of choosing a winner after iteration ● always same hyper- parameters Silver, David, et al. ‘A General Reinforcement Learning Algorithm That Masters Chess, Shogi, and Go through Self-Play’. Science, vol. 362, no. 6419, Dec. 2018, pp. 1140–44. 01/29/2020 |

  10. Author Division Leaving perfect information environments 01/28/2020 | Page10 representation function h prediction function f dynamics function g A: planning B: acting C: training Schrittwieser, Julian, et al. ‘Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model’. ArXiv:1911.08265 [Cs, Stat], Nov. 2019. arXiv.org, http://arxiv.org/abs/1911.08265. 01/29/2020 |

  11. Author Division Leaving perfect information environments 01/28/2020 | Page11 learns all game rules on its own Compared against: Stockfish (chess), Elmo (Shogi), AlphaZero (Go), R2D2 (Atari) Schrittwieser, Julian, et al. ‘Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model’. ArXiv:1911.08265 [Cs, Stat], Nov. 2019. arXiv.org, http://arxiv.org/abs/1911.08265. 01/29/2020 |

  12. Author Division Some other advances 01/28/2020 | Page12 Hide and Seek AlphaStar approx. Starcraft Chess Go values II 10 26 breadth 35 250 Multiple agents in an open environment depth 80 150 1000s 01/29/2020 |

  13. Author Division Thank you for your attention! 01/28/2020 | Page13 Any questions? 01/29/2020 |

Recommend

Demystifying the efficiency of reinforcement learning: A few recent stories Yuxin Chen EE,

Demystifying the efficiency of reinforcement learning: A few recent stories Yuxin Chen EE,

Demystifying the efficiency of reinforcement learning: A few recent stories Yuxin Chen EE, Princeton University Acknowledgement 2/ 74 Reinforcement learning (RL) 3/ 74 RL challenges In RL, an agent learns by interacting with an environment

1.54k views • 129 slides
Recent Advances in Photonic Recent Advances in Photonic effect employing IP- based distributed

Recent Advances in Photonic Recent Advances in Photonic effect employing IP- based distributed

Development of Photonic Network Optical burst switching Next Generation Enhancement of Networks Dynamic (Adaptive) Optical Path Control statistical >10Tb/s multiplexing Photonic MPLS Network Recent Advances in Photonic Recent Advances

762 views • 8 slides
RL Overview of topics About Reinforcement Learning The Reinforcement Learning Problem

RL Overview of topics About Reinforcement Learning The Reinforcement Learning Problem

Introduction to Reinforcement Learning RL Overview of topics About Reinforcement Learning The Reinforcement Learning Problem Inside an RL agent Temporal difference learning Many faces of Reinforcement Learning What is

551 views • 35 slides
Reinforcement Learning by the People and for the People: With a Focus on Lifelong / Meta /

Reinforcement Learning by the People and for the People: With a Focus on Lifelong / Meta /

Reinforcement Learning by the People and for the People: With a Focus on Lifelong / Meta / Transfer Learning Emma Brunskill Stanford CS234 Winter 2018 Quiz Information Monday, in class See piazza for room information (Released by

942 views • 81 slides
Recent Advances in Machine Learning for Mathematical Reasoning Steven Van Vaerenbergh

Recent Advances in Machine Learning for Mathematical Reasoning Steven Van Vaerenbergh

Recent Advances in Machine Learning for Mathematical Reasoning Steven Van Vaerenbergh Universidad de Cantabria Symposium on Artificial Intelligence for Mathematics Education CIEM, Castro Urdiales, February 2020 Motivation Machine Learning

318 views • 31 slides
Recent Advances in Adversarial Machine Learning Nicholas Carlini Google Research Recent

Recent Advances in Adversarial Machine Learning Nicholas Carlini Google Research Recent

Recent Advances in Adversarial Machine Learning Nicholas Carlini Google Research Recent Advances in Adversarial (Examples in) Machine Learning Nicholas Carlini Google Research The Year is 2014 Someone tells you they have a new algorithm to

1.52k views • 139 slides
Recent Advances in Biomolecular NMR Lucia Banci CERM University of Florence Recent Advances

Recent Advances in Biomolecular NMR Lucia Banci CERM University of Florence Recent Advances

Recent Advances in Biomolecular NMR Lucia Banci CERM University of Florence Recent Advances in Biomolecular NMR Protonless NMR for the characterization of Unfolded proteins, Large protein assemblies, Paramagnetic systems I n cell

1.23k views • 70 slides
Recent Advances in Biomolecular NMR Lucia Banci CERM University of Florence Recent Advances

Recent Advances in Biomolecular NMR Lucia Banci CERM University of Florence Recent Advances

Recent Advances in Biomolecular NMR Lucia Banci CERM University of Florence Recent Advances in Biomolecular NMR Protonless NMR for the characterization of Unfolded proteins, Large protein assemblies, Paramagnetic systems I n cell

635 views • 42 slides
Some Recent Algorithmic Questions in Deep Reinforcement Learning CS 285 Instructor: Aviral

Some Recent Algorithmic Questions in Deep Reinforcement Learning CS 285 Instructor: Aviral

Some Recent Algorithmic Questions in Deep Reinforcement Learning CS 285 Instructor: Aviral Kumar UC Berkeley What Will We Discuss Today? So far, we have gone over several interesting RL algorithms, and some theoretical aspects in RL

394 views • 27 slides
Reinforcement Learning Reinforcement Learning Reinforcement Learning in a nutshell g Imagine

Reinforcement Learning Reinforcement Learning Reinforcement Learning in a nutshell g Imagine

Reinforcement Learning Reinforcement Learning Reinforcement Learning in a nutshell g Imagine playing a new game whose rules you dont know; after a hundred or so moves your don t know; after a hundred or so moves, your opponent announces, You

511 views • 30 slides
, , Weakly Supervised Classification Robust Learning and More: Robust Learning and More:

, , Weakly Supervised Classification Robust Learning and More: Robust Learning and More:

The Second Korea-Japan Machine Learning Workshop, Jeju, Korea Feb. 23, 2019 , , Weakly Supervised Classification Robust Learning and More: Robust Learning and More: Overview of Our Recent Advances Overview of Our Recent Advances Masashi

484 views • 29 slides
Doing More with More: Recent Achievements in Large-Scale Deep Reinforcement Learning Compiled by:

Doing More with More: Recent Achievements in Large-Scale Deep Reinforcement Learning Compiled by:

Doing More with More: Recent Achievements in Large-Scale Deep Reinforcement Learning Compiled by: Adam Stooke, Pieter Abbeel (UC Berkeley) March 2019 Atari, Go, and beyond Algorithms & Frameworks (Atari Legacy) A3C / DQN

453 views • 24 slides
Em ploying Recent Advances in Machine Learning for Opinion Sum m arization Claire Cardie

Em ploying Recent Advances in Machine Learning for Opinion Sum m arization Claire Cardie

Em ploying Recent Advances in Machine Learning for Opinion Sum m arization Claire Cardie Department of Computer Science Cornell University CERATOPS Center for Extraction and Summarization of Events and Opinions in Text Janyce Wiebe, U.

472 views • 22 slides
Recent Advances in Machine Learning And Their Application to Networking David Meyer

Recent Advances in Machine Learning And Their Application to Networking David Meyer

Recent Advances in Machine Learning And Their Application to Networking David Meyer dmm@{brocade.com,uoregon.edu,1-4-5.net,..} http://www.1-4-5.net/~dmm/talks/2015/thursday_lunch_ietf93.pptx IETF 93 23 July 2015 Prague, Czech Republic Goals

386 views • 38 slides
Recent Advances and Key Challenges Russ Salakhutdinov Machine Learning Department Carnegie Mellon

Recent Advances and Key Challenges Russ Salakhutdinov Machine Learning Department Carnegie Mellon

Recent Advances and Key Challenges Russ Salakhutdinov Machine Learning Department Carnegie Mellon University Canadian Institute for Advanced Research Key Challenges Multimodal Learning Reasoning, Attention and Memory Natural Language

749 views • 37 slides
Reinforcement Learning You can think of supervised learning as the teacher providing answers

Reinforcement Learning You can think of supervised learning as the teacher providing answers

Reinforcement Learning You can think of supervised learning as the teacher providing answers (the class labels) Reinforcement learning: In reinforcement learning, the agent learns Markov Decision Processes based on a punishment/reward

89 views • 6 slides
A Robotic Auto-Focus System based on Deep Reinforcement Learning Xiaofan Yu, Runze Yu, Jingsong

A Robotic Auto-Focus System based on Deep Reinforcement Learning Xiaofan Yu, Runze Yu, Jingsong

A Robotic Auto-Focus System based on Deep Reinforcement Learning Peking University A Robotic Auto-Focus System based on Deep Reinforcement Learning Xiaofan Yu, Runze Yu, Jingsong Yang, Xiaohui Duan* School of EECS, Peking University Beijing,

340 views • 21 slides
Tensor Network Representation for Machine Learning - Recent Advances and Perspectives Qibin ZHAO

Tensor Network Representation for Machine Learning - Recent Advances and Perspectives Qibin ZHAO

Tensor Network Representation for Machine Learning - Recent Advances and Perspectives Qibin ZHAO Tensor Learning Unit RIKEN AIP AIP Symposium (Mar. 19, 2019) Tensor Learning Unit - Members Postdoctoral Researchers (2) Ming Hou, Chao Li

543 views • 28 slides
Examples of Reinforcement Learning Robocup Soccer Teams Stone & Veloso, Reidmiller et al.

Examples of Reinforcement Learning Robocup Soccer Teams Stone & Veloso, Reidmiller et al.

Reinforcement Learning Slides by Rich Sutton Mods by Dan Lizotte Refer to Reinforcement Learning: An Introduction by Sutton and Barto Alpaydin Chapter 16 Up until now we have been Supervised Learning Classifying, mostly

891 views • 31 slides
CS885 Reinforcement Learning Module 2: June 6, 2020 Maximum Entropy Reinforcement Learning

CS885 Reinforcement Learning Module 2: June 6, 2020 Maximum Entropy Reinforcement Learning

CS885 Reinforcement Learning Module 2: June 6, 2020 Maximum Entropy Reinforcement Learning Haarnoja, Tang et al. (2017) Reinforcement Learning with Deep Energy Based Policies, ICML . Haarnoja, Zhou et al. (2018) Soft Actor-Critic: Off-Policy

683 views • 24 slides
Reinforcement Learning Reinforcement Learning Now that you know a little about Optimal Control

Reinforcement Learning Reinforcement Learning Now that you know a little about Optimal Control

Reinforcement Learning Reinforcement Learning Now that you know a little about Optimal Control Theory, you actually have some knowledge in RL. RL shares the overall goal with OCT: solving for a control policy such that the cumulative cost

610 views • 22 slides
Reinforcement Learning CS 4100: Artificial Intelligence Reinforcement Learning II Still

Reinforcement Learning CS 4100: Artificial Intelligence Reinforcement Learning II Still

Reinforcement Learning CS 4100: Artificial Intelligence Reinforcement Learning II Still assume a Marko kov decision process (MDP): A se set o of st states s s S A se set o of a actions ( s (per st state) A A mo model

96 views • 5 slides
R i f R i f Reinforcement Learning III Reinforcement Learning III t L t L i i III III Dec

R i f R i f Reinforcement Learning III Reinforcement Learning III t L t L i i III III Dec

R i f R i f Reinforcement Learning III Reinforcement Learning III t L t L i i III III Dec 03 2008 1 Large State Spaces h When a problem has a large state space we can not longer represent the U or Q functions as explicit tables explicit

436 views • 18 slides
Reinforcement Learning UMaine COS 470/570 Introduction to AI Why reinforcement learning?

Reinforcement Learning UMaine COS 470/570 Introduction to AI Why reinforcement learning?

Reinforcement Learning<br/><br/> 4/25/19, 8*06 PM Reinforcement Learning<br/><br/> 4/25/19, 8*06 PM Reinforcement Learning UMaine COS 470/570 Introduction to AI Why reinforcement learning? Spring 2019 Created:

371 views • 15 slides

More recommend