Mastering the game of Go with deep neural networks and tree search - PowerPoint PPT Presentation

Nov 06, 2023 •46 likes •556 views

Mastering the game of Go with deep neural networks and tree search Nature, Jan, 2016 Roadmap What this paper is about? Deep Learning Search problem How to explore a huge tree (graph) AlphaGo Video

Mastering the game of Go with deep neural networks and tree search Nature, Jan, 2016
Roadmap What this paper is about? • Deep Learning • Search problem • How to explore a huge tree (graph)
AlphaGo Video https://www.youtube.com/watch?v=53YLZBSS0cc https://www.youtube.com/watch?v=g-dKXOlsf98
rank Al AlphaGo vs vs*European*Champion*(Fan*Hui 27Da Dan) * October$5$– 9,$2015 <Official$match> I Time+limit:+1+hour I AlphaGo Wins (5:0)
Al AlphaGo vs vs*Wo World*Champion*(Lee*Se Sedol 97Da Dan) March$9$– 15,$2016 <Official$match> I Time+limit:+2+hours Venue :+ Seoul ,+Four+Seasons+Hotel
Lee*Sedol wiki Photo+source: Maeil+Economics 2013/04
Lee Sedol
=$multiple$machines European$champion
The Game
Go Elo Ranking http://www.goratings.org/history/
Lee Sedol VS Ke Jie
How about Other Games?
Tic Tac Toe
Chess
Chess (1996)
Deep Blue (1996)
AlphaGo is the Skynet?
Go Game
Simple Rules
High Complexity
High Complexity
Different Games
Search Problem (the search space)
Tic Tac Toe
Tic Tac Toe
The “Tree” in Tic Tac Toe
The “Tree” of Chess
The “Tree” of Go Game
Search Problem (how to search)
MiniMax in Tic Tac Toe
Adversarial"Search"–"MiniMax"" 1" 1" 1" 0" 0" J1" J1" 0" J1" 0" J1" J1" 5"
Adversarial"Search"–"MiniMax"" J1" 1" J1" 0" 1" 1" 0" J1" J1" J1" 1" 1" 0" 1" 0" J1" J1" 0" J1" 0" J1" J1" 6"
What is the problem? 1. Generate the Search Tree 2. use MinMax Search
The Size of the Tree Tic Tac Toe: b = 9, d =9 Chess: b = 35, d =80 Go: b = 250, d =150 b : number of legal move per position d : its depth (game length)
One Grain of Rice https://www.youtube.com/watch?v=byk3pA1GPgU
The “Space” of GO Game
How about other Games? • Flappy bird? • Angry Bird? Tic Tac Toe: • Starcraft? b = 9, d =9 • learning a language Chess: b = 35, d =80 • Write a paper Go: • Get a MS/PhD degree b = 250, d =150 • Finding a job • Life
How to solve?
Chess (1996)
Monte Carlo
Las Vegas
Monte"Carlo"Tree"Search" Tree"search" ……." ……." ……." ……." Monte"Carlo"search" ……." ……." ……." ……." ……." 7"
Monte"Carlo"Tree"Search" • Tree"Search"+"Monte"Carlo"Method"" – SelecIon" white"wins"/"total" 3/5" – Expansion" – SimulaIon" 2/3" 1/2" – BackJPropagaIon" 1/1" 1/2" 1/1" 0/1" 1/1" 0/1" 8"

Recommend

Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree

Deep Reinforcement Learning [Mastering the Game of Go with Deep Reinforcement Learning and Tree Search, Nature 2016] CS 486/686 University of Waterloo Lecture 21: July 12, 2017 Outline AlphaGo Supervised Learning of Policy Networks

541 views • 15 slides

Mastering the game of Go with deep neural networks and tree search Article overview by

David Silver et al. from Google DeepMind Mastering the game of Go with deep neural networks and tree search Article overview by Reinforcement Learning Seminar Ilya Kuzovkin University of Tartu, 2016 T HE G AME OF G O B OARD B OARD S TONES B

907 views • 66 slides

THE MOD METHOD with VESPERS MASTERING In this Module What mastering can do & what it

THE MOD METHOD with VESPERS MASTERING In this Module What mastering can do & what it cant Self-mastering vs. third-party mastering Picking a mastering engineer Mastering work fl ow Audio artifacts & fi delity:

520 views • 16 slides

Mastering the Game of Go without Human Knowledge 06/15/18 Presented by: Henry Chen CS885

Mastering the Game of Go without Human Knowledge 06/15/18 Presented by: Henry Chen CS885 Reinforcement Learning Introduction Image source: https://medium.com/syncedreview/alphago-zero-approaching-perfection-d8170e2b4e48 PAGE 2 Introduction

455 views • 22 slides

Analogs of Linguistic Structure in Deep Representations Jacob Andreas and Dan Klein A game for

Analogs of Linguistic Structure in Deep Representations Jacob Andreas and Dan Klein A game for humans everything but the blue shapes orange square and non-squares [FitzGerald et al. 2013] 2 A game for RNNs 1.0 2.3 -0.3 0.4

898 views • 44 slides

Mastering the Gospel P resentation Welcome to the CMF Training page on Mastering a Gospel

Mastering the Gospel P resentation Welcome to the CMF Training page on Mastering a Gospel Presentation. This may be the most important thing that you do and you need to be able to do this as you grow in your Christian walk. You may be the only

290 views • 6 slides

From Deep Blue to Monte Carlo: An Update on Game

From Deep Blue to Monte Carlo: An Update on Game Tree Research Akihiro Kishimoto and Mar0n Mller AAAI-14 Tutorial 6: State of

737 views • 29 slides

e-Bug Junior Game Junior Game Game Style Game Process Demo Game Mechanics and

e-Bug Junior Game Junior Game Game Style Game Process Demo Game Mechanics and Learning Outcomes Feedback to date and evaluation plans Game Style Game style encompasses two genres of game o Platform game fast action

575 views • 22 slides

e-Bug Senior Game Senior Game Game Style Game Process Demo Game Puzzles and

e-Bug Senior Game Senior Game Game Style Game Process Demo Game Puzzles and Learning Outcomes Feedback to date and evaluation plans Game Style Game is a detective game where the player investigates microbial problems.

572 views • 19 slides

Mastering Your Mindset Mastering Your Money Focus: How to Focus on Earning More Income, and

Mastering Your Mindset Mastering Your Money Focus: How to Focus on Earning More Income, and Managing & Investing Your Money with John Assaraf What Is Money? Money Is A VERY Powerful Idea Money Is A Means Of Exchange Money Is

955 views • 40 slides

Mastering Great Performances in Public Presentation The need to personally communicate is

Mastering Great Performances in Public Presentation The need to personally communicate is ever-increasing. Mastering personal appearances on a whole spectrum of channels and formats - TV, radio, YouTube, presentations or speeches - is vital in

272 views • 5 slides

Mastering Data with Spark and ML Strata London 2019 About Me IIT Delhi, 1998 Founder and CEO,

Mastering Data with Spark and ML Strata London 2019 About Me IIT Delhi, 1998 Founder and CEO, Nube Technologies Strata Data San Jose Program Committee Speaker at Spark Summit, Strata, GIDS etc Nube India based startup Deep technical

484 views • 32 slides

Deep Learning for Video Game Playing Authors: Niels Justesen, Philip Bontrager, Julian Togelius,

Deep Learning for Video Game Playing Authors: Niels Justesen, Philip Bontrager, Julian Togelius, Sebastian Risi Presented by: Runsheng (Benson) Guo 1 Outline Background Methods History Open Challenges Recent Advances 2

667 views • 21 slides

Towards a Foundation of Deep Learning: SGD, Overparametrization, and Generalization Jason D. Lee

Towards a Foundation of Deep Learning: SGD, Overparametrization, and Generalization Jason D. Lee University of Southern California January 29, 2019 Jason Lee Successes of Deep Learning Game-playing (AlphaGo, DOTA, King of Glory) Computer

665 views • 48 slides

The Composition Method Wolfgang Thomas Francqui Lecture, Mons, April 2013 Mastering compositions

The Composition Method Wolfgang Thomas Francqui Lecture, Mons, April 2013 Mastering compositions Wolfgang Thomas Overview 1. Motivation 2. m -equivalence and the EF-game 3. Applications 4. m -types 5. Monadic types Wolfgang Thomas

1.06k views • 40 slides

1 What a Game is Not Outline A bunch of cool features What is a Game? Necessary, but

Outline What is a Game? The Game Development Genres Process: Genres What is a Game? (1 of 4) What is a Game? (2 of 4) Movie? Movie? No interaction , outcome fixed Toy? Toy? Puzzle? No goal , but still fun!

512 views • 4 slides

From%Deep%Blue%to%Monte%Carlo:%% ! ! An%Update%on%Game%Tree%Research%

From%Deep%Blue%to%Monte%Carlo:%% ! ! An%Update%on%Game%Tree%Research% Akihiro!Kishimoto!and!Mar0n!Mller! ! AAAI514!Tutorial!5:!! Monte!Carlo!Tree!Search! ! Presenter:!! Mar0n!Mller,!University!of!Alberta! !

1.17k views • 95 slides

Rally-Owl Overview of Rally-Owl Game This game is based off of Rally-X The goal of the game is

Rally-Owl Overview of Rally-Owl Game This game is based off of Rally-X The goal of the game is to collect all 10 flags before losing all your lives The game makes use of a horizontally and vertically scrolling map This game implements a

101 views • 8 slides

Deep Learning Olexandr Isayev, Ph.D. University of North Carolina at Chapel Hill @olexandr

Mastering Computational Chemistry with Deep Learning Olexandr Isayev, Ph.D. University of North Carolina at Chapel Hill @olexandr olexandr@unc.edu http://olexandrisayev.com ANI-1: An extensible DL potential with DFT accuracy at force field

280 views • 26 slides

Game interoperability with functors functor AgsFun (structure Game : GAME) :> sig structure

Game interoperability with functors functor AgsFun (structure Game : GAME) :> sig structure Game : GAME val bestmove : Game.config -> Game.Move.move option val forecast : Game.config -> Player.outcome end where type Game.Move.move =

247 views • 22 slides

X4 Nick Maggini X4 - The Game The classic game of Connect Four or Four-in-a-Row. Game

X4 Nick Maggini X4 - The Game The classic game of Connect Four or Four-in-a-Row. Game board is 7x6 Two player game, each get a color Players alternate until four tokens of the same color make a line Stalemates are possible

891 views • 13 slides

How to log into Mastering Astronomy when you have a NEW textbook you should have a cardboard

How to log into Mastering Astronomy when you have a NEW textbook you should have a cardboard folder that looks like this The course ID is on your syllabus Your CU Identikey ID Pull back the tab to get a bunch of letters like this - do not

288 views • 9 slides

The name of the game: BASKIN ITALY https://www.youtube.com/watch?v=yttF1D_C9ok Game Basics Type

The name of the game: BASKIN ITALY https://www.youtube.com/watch?v=yttF1D_C9ok Game Basics Type of the game: Ball game Character of the game: Competitive Aim of the game: To score more points and goals than the opponent team. Number of players:

333 views • 6 slides

Pre-Game More Than the Game Don Harkey, CEO People Centric Consulting Group Springfield, MO

Pre-Game More Than the Game Don Harkey, CEO People Centric Consulting Group Springfield, MO Pre-Game Last Chance U Season 5 Pre-Game Dior Walker-Scott Laney College Star WR Pre-Game Great work ethic Team player Great

746 views • 61 slides