Open-ended learning in symmetric zero-sum games David Balduzzi, - PowerPoint PPT Presentation

Oct 26, 2022 •98 likes •218 views

Open-ended learning in symmetric zero-sum games David Balduzzi, Marta Garnelo, Yoram Bachrach, Wojciech M. Czarnecki, Julien Perolat, Max Jaderberg, Thore Graepel Long ago and far away (mid-1800s in Cambridge, England): First tutor: I'm

Open-ended learning in symmetric zero-sum games David Balduzzi, Marta Garnelo, Yoram Bachrach, Wojciech M. Czarnecki, Julien Perolat, Max Jaderberg, Thore Graepel
Long ago and far away (mid-1800s in Cambridge, England): First tutor: “I'm teaching the most brilliant boy in Britain” Second tutor: “Well, I'm teaching the best test-taker” Depending on the version of the story, the first boy was either Lord Kelvin or James Clerk Maxwell . The second boy indeed scored highest on the Mathematical Tripos, but is otherwise long forgotten.
Long ago and far away (mid-1800s in Cambridge, England): First tutor: “I'm teaching the most brilliant boy in Britain” Second tutor: “Well, I'm teaching the best test-taker” Depending on the version of the story, the first boy was either Lord Kelvin or James Clerk Maxwell . The second boy indeed scored highest on the Mathematical Tripos, but is otherwise long forgotten. Modern learning algorithms are outstanding test-takers But intelligence is about more than taking tests It’s also about formulating useful problems
Where do problems come from? Answer #1: Someone packages a dataset into a loss function e.g. ImageNet, CIFAR, MNIST, …
Where do problems come from? Answer #1: Someone packages a dataset into a loss function e.g. ImageNet, CIFAR, MNIST, … Answer #2: Someone builds a task (that is, an environment sprinkled with rewards) e.g. Arcade Learning Environment, DM-Lab, Open AI gym, …
Where do problems come from? Answer #3: Self-play in symmetric zero-sum games The agent is the task -- create an outer loop that bends deep RL on itself
(Naive) self-play is an open-ended learning algorithm It’s pretty amazing
(Naive) self-play is an open-ended learning algorithm but … there are really simple examples where it completely breaks down It’s not a general purpose learning algorithm, not even for zero-sum games
On the varieties of zero-sum games transitive: “relative skill cyclic: “every strategy determines who wins” has a counter-strategy”
Theorem: Any symmetric two-player zero-sum game decomposes into [ transitive ] + [ cyclic ] components transitive: skill determines outcome cyclic: every strategy has a counter-strategy
The paper: How to formulate useful objectives in non-transitive games New tools: ● Gamescapes (generalize landscapes, but represent many objectives) Population-level performance measures ● ● Population-level training algorithms

Recommend

Game and Learn: An Introduction to Educational Gaming 8. Games and Players Ruben R. Puentedura,

Game and Learn: An Introduction to Educational Gaming 8. Games and Players Ruben R. Puentedura, Ph.D Emotion and the Fun of Games Four Keys to Emotion in Games (Lazzaro) game fiero curiosity Easy Hard Fun Fun open goal ended player

274 views • 10 slides

Models of Language Evolution Evolutionary game theory & signaling games Michael Franke

Models of Language Evolution Evolutionary game theory & signaling games Michael Franke Topics for today 1 (flavors of) game theory 2 signaling games (& conversion into symmetric form) 3 Nash equilibrium (in symmetric games) 4 evolutionary

421 views • 29 slides

R i f Reinforcement Learning in L i i Board Games Board Games G E O R G E T U C K E R G E

R i f Reinforcement Learning in L i i Board Games Board Games G E O R G E T U C K E R G E O R G E T U C K E R Paper Background Reinforcement learning in board games g g Imran Ghory 2004 Surveys progress in last decade

820 views • 27 slides

Computing Pure Nash Equilibria in Symmetric Action Graph Games Albert Xin Jiang Kevin

AGG Computing Pure Nash Equilibria Symmetric AGGs Algorithm Conclusions Computing Pure Nash Equilibria in Symmetric Action Graph Games Albert Xin Jiang Kevin Leyton-Brown Department of Computer Science University of British Columbia {

962 views • 36 slides

Iterated learning in an open-ended meaning space Jon W. Carr Language Evolution and Computation

Iterated learning in an open-ended meaning space Jon W. Carr Language Evolution and Computation Research Unit School of Philosophy, Psychology and Language Sciences University of Edinburgh Categorical structure ! " # $ % & ' ( ) * + ,

470 views • 36 slides

Open Ended, Real-World, Student-Centered Learning Modules for Nutrition and Pharmacy Jessica

Open Ended, Real-World, Student-Centered Learning Modules for Nutrition and Pharmacy Jessica Rocheleau Assistant Professor of Biology Inspiration: KEEN workshop Skills that employers seek

415 views • 30 slides

Digital Games for Learning: Beyond edutainment and serious games Esplora, Villa Bighi, Triq

Digital Games for Learning: Beyond edutainment and serious games Esplora, Villa Bighi, Triq Marina, Il-Kalkara 17th January2018 Dr Gearid Silleabhin Department of Technology Enhanced Learning Overview Computer and Video

881 views • 41 slides

5 Reputation and Repeated Games with Symmetric Information 5 October 2009 Eric Rasmusen,

5 Reputation and Repeated Games with Symmetric Information 5 October 2009 Eric Rasmusen, Erasmuse@indiana.edu. Http://www.rasmusen.org. The Chainstore Paradox Suppose that we repeat Entry Deterrence I 20 times in the context of a chainstore

544 views • 38 slides

5 Reputation and Repeated Games with Symmetric Information January 27, 2014 Eric Rasmusen,

5 Reputation and Repeated Games with Symmetric Information January 27, 2014 Eric Rasmusen, Erasmuse@indiana.edu. Http://www.rasmusen.o 1 The Chainstore Paradox Suppose that we repeat Entry Deterrence I 20 times in the context of a chainstore

461 views • 42 slides

Symmetric Key Cryptosystems Modern Symmetric Key Cryptosystems Public key cryptosystems (PKCs) and

Modern Block Ciphers DES Games with Block Ciphers Modern Block Ciphers DES Games with Block Ciphers Symmetric Key Cryptosystems Modern Symmetric Key Cryptosystems Public key cryptosystems (PKCs) and their applications are the primary focus

521 views • 5 slides

Games, Auctions, Learning, and the Price of Anarchy va Tardos Cornell University Games and

Games, Auctions, Learning, and the Price of Anarchy va Tardos Cornell University Games and Quality of Solutions Rational selfish action can lead to outcome bad for everyone Model: Value for each cow decreasing function of

526 views • 31 slides

Introducing open games Jules Hedges Joint work with Neil Ghani Viktor Winschel Philipp Zahn

Intro to game theory Compositionality Open games Introducing open games Jules Hedges Joint work with Neil Ghani Viktor Winschel Philipp Zahn University of Oxford February 24, 2017 Intro to game theory Compositionality Open games

978 views • 45 slides

Correlated-Q Learning and Cyclic Equilibria in Markov games Haoqi Zhang Correlated-Q Learning

Correlated-Q Learning and Cyclic Equilibria in Markov games Haoqi Zhang Correlated-Q Learning Greeenwald and Hall (2003) Setting: general sum Markov games Goal: convergence (reach equilibrium), payoff Means: CE-Q Results:

461 views • 24 slides

On Symmetric Losses for Learning from Corrupted Labels Nontawat Charoenphakdee 1,2 , Jongyeong Lee

On Symmetric Losses for Learning from Corrupted Labels Nontawat Charoenphakdee 1,2 , Jongyeong Lee 1,2 and Masashi Sugiyama 2,1 The University of Tokyo 1 RIKEN Center for Advanced Intelligence Project (AIP) 2 2 Supervised learning Predict output

893 views • 27 slides

Structuring Direct Lending Funds: Open-Ended vs. Private Equity/Closed-Ended Style Structures;

Presenting a live 90-minute webinar with interactive Q&A Structuring Direct Lending Funds: Open-Ended vs. Private Equity/Closed-Ended Style Structures; Valuation; Tax and ERISA; Regulatory Considerations WEDNESDAY, SEPTEMBER 27, 2017 1pm

842 views • 65 slides

How to Use the 5 Senses Self-Checking Games The games can be used on a computer, tablet, or

How to Use the 5 Senses Self-Checking Games The games can be used on a computer, tablet, or smartphone. They can also be shared with students via Google Classroom for distance learning! Heres how to get it set up for your child(ren): 1. Click

317 views • 10 slides

Statistical Learning and Inference Methods for Reasoning in Games March 30, 2005 Brian Mihok

Statistical Learning and Inference Methods for Reasoning in Games March 30, 2005 Brian Mihok Michael Terry Draper Laboratory CSAIL, MIT 1 Outline Intro to Games Texas Hold em Demo Hidden Markov Models Structure Learning

1.46k views • 55 slides

OPEN SCHOOL: An Open E-Learning Platform for Khmer-speaking Users Sokunthearith Makara Learning

OPEN SCHOOL: An Open E-Learning Platform for Khmer-speaking Users Sokunthearith Makara Learning Design and Technology Problem Access to quality education Solution Purpose statement: Develop and evaluate the ease of use of an open

1.02k views • 90 slides

Game and Learn: An Introduction to Educational Gaming 4. Games and Learning Ruben R. Puentedura,

Game and Learn: An Introduction to Educational Gaming 4. Games and Learning Ruben R. Puentedura, Ph.D Learning from Games Active Learning Gamers Learn From: 1. Doing and reflecting critically Active Learning Example: Europa Universalis III

473 views • 11 slides

Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild

Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wild Shangzhe Wu Christian Rupprecht Andrea Vedaldi V ISUAL G EOMETRY G ROUP , U NIVERSITY OF O XFORD Agenda v Problem Introduction v Method Overview v Results

3.49k views • 59 slides

Open-Ended Questions GESIS Survey Guidelines Cornelia Zll These slides are based on the GESIS

Open-Ended Questions GESIS Survey Guidelines Cornelia Zll These slides are based on the GESIS Survey Guideline paper about open- ended questions: Zll, C. (2016). Open-Ended Questions. GESIS Survey Guidelines. Mannheim, Germany: GESIS

566 views • 14 slides

Code Games Or How I Learned to Stop Worrying and Love to Code Jacob Wilkins Topics Learning New

Code Games Or How I Learned to Stop Worrying and Love to Code Jacob Wilkins Topics Learning New Languages Esolangs Code Puzzles Code Golf Actual Computer Games Code Games | May 2017 | 2/19 New Languages Learning a new programming language

560 views • 19 slides

The use of Topic Modeling to Analyze Open-Ended Survey Items W. Holmes Finch Maria E. Hernndez

The use of Topic Modeling to Analyze Open-Ended Survey Items W. Holmes Finch Maria E. Hernndez Finch Constance E. McIntosh Claire Braun Ball State University Open ended survey items Researchers making use of surveys for data collection

563 views • 27 slides

What is Tagtiv8? Tagtiv8 games promote Active Learning to develop learners' confidence with:

What is Tagtiv8? Tagtiv8 games promote Active Learning to develop learners' confidence with: phonics spelling words numbers other areas of mathematics. What are the benefits of Tagtiv8 games? Tagtiv8 games: have an

601 views • 45 slides