dungeons and dqns
play

Dungeons and DQNs Toward Reinforcement Learning Agents that Play - PowerPoint PPT Presentation

Sep 19, 2022 •149 likes •328 views

Dungeons and DQNs Toward Reinforcement Learning Agents that Play Tabletop Roleplaying Games LARA J. MARTIN, SRIJAN SOOD, MARK O. RIEDL 2 Its an exciting time for AI Dungeons & DQNs Martin, Sood, & Riedl AAAI Intelligent

  1. Dungeons and DQNs Toward Reinforcement Learning Agents that Play Tabletop Roleplaying Games LARA J. MARTIN, SRIJAN SOOD, MARK O. RIEDL

  2. 2 It’s an exciting time for AI Dungeons & DQNs – Martin, Sood, & Riedl – AAAI Intelligent Narrative Technologies Workshop November 13 & 14, 2018

  3. How do we push the limits of AI? 3 I’m sorry, Dave. I’m afraid I can’t do that. Dungeons & DQNs – Martin, Sood, & Riedl – AAAI Intelligent Narrative Technologies Workshop November 13 & 14, 2018

  4. Games! 4 Chess - 1997 Atari Games - 2015 Go - 2016 Doom - 2016 DOTA 2 - 2018

  5. What about Dungeons & Dragons? 5  Players create characters to play & describe their character’s actions  Characters exist in a shared imaginary world  Game/Dungeon Master (GM/DM) mediates and sets up scenarios — or campaigns Dungeons & DQNs – Martin, Sood, & Riedl – AAAI Intelligent Narrative Technologies Workshop November 13 & 14, 2018

  6. Why Dungeons and Dragons? 6  Unlimited actions (discourse)  Actions can have unexpected consequences and/or DM can get unexpected player actions  Actions cannot cleanly map to states (model of the world changes as game progresses)  Distributed game world (across players and DM)  Players receive intrinsic reward for actions (unclear win condition)  Collaborative Dungeons & DQNs – Martin, Sood, & Riedl – AAAI Intelligent Narrative Technologies Workshop November 13 & 14, 2018

  7. Outline 7  TRPGs compared to:  Interactive Fiction  Experience Management  Automated Story Generation  Our starting point:  Genre Expectation Model + Commonsense Rules Model  Deep Q-Learning Dungeons & DQNs – Martin, Sood, & Riedl – AAAI Intelligent Narrative Technologies Workshop November 13 & 14, 2018

  8. TRPGs vs the World 8 Medium Comparison to TRPGs Interactive Fiction (IF) Playing Use puzzles to uncovers pre- • existing story Often simplified grammar • Experience Management Intervenes in storyline to • (Used in Interactive Narrative) keep things “on track” for quality Often fixed set of actions • Automatic Story Generation Generates new story • Uses planners to create • actions for characters for well-defined domains

  9. Outline 9  TRPGs compared to:  Interactive Fiction  Experience Management  Automated Story Generation  Our starting point:  Genre Expectation Model + Commonsense Rules Model  Deep Q-Learning Dungeons & DQNs – Martin, Sood, & Riedl – AAAI Intelligent Narrative Technologies Workshop November 13 & 14, 2018

  10. Assumptions 10  No dice rolling (i.e. no combat, etc.)  Agent is always in character  GMs aren’t refereeing Dungeons & DQNs – Martin, Sood, & Riedl – AAAI Intelligent Narrative Technologies Workshop November 13 & 14, 2018

  11. The Proposed System (Training) 11 Reward Updated State or <None> Exploit Environment Policy (Rule Engine Event Selection or Update & Current State & State Updater) Explore Updated State Seq2Seq or <None> Distribution of (Genre) Next Events

  12. World Model 12 1. Genre Expectation Model  Seq2Seq network generates next event in the story  Trained on relevant genre 2. Commonsense Rules Model  Things that aren’t mentioned in stories (see: Principle of Minimal Departure)  Temporal & physical rules Dungeons & DQNs – Martin, Sood, & Riedl – AAAI Intelligent Narrative Technologies Workshop November 13 & 14, 2018

  13. The Proposed System Pipeline 13 TRPG Agent Event to Natural Agent’s Language Turn Human Selected Player’s Turn Next Event Natural Update Action DQN Language State Selection Event Current to Event State Dungeons & DQNs – Martin, Sood, & Riedl – AAAI Intelligent Narrative Technologies Workshop November 13 & 14, 2018

  14. Back to Games! 14 Atari Games - 2015 Go - 2016 Doom - 2016 DOTA 2 - 2018

  15. Conclusion 15  TRPGs are the next AlphaGo Dungeons & DQNs – Martin, Sood, & Riedl – AAAI Intelligent Narrative Technologies Workshop November 13 & 14, 2018

  16. 16 Thank you! LARA MARTIN LJMARTIN@GATECH.EDU Dungeons & DQNs – Martin, Sood, & Riedl – AAAI Intelligent Narrative Technologies Workshop November 13 & 14, 2018

More recommend