CS 480: GAME AI MACHINE LEARNING IN GAMES 5/31/2012 Santiago Ontan - PowerPoint PPT Presentation

CS 480: GAME AI MACHINE LEARNING IN GAMES 5/31/2012 Santiago Ontañón santi@cs.drexel.edu https://www.cs.drexel.edu/~santi/teaching/2012/CS480/intro.html

Reminders • Check BBVista site for the course regularly • Also: https://www.cs.drexel.edu/~santi/teaching/2012/CS480/intro.html • Project 4 due June 7 th • Final exam, next week: June 5 th • Tactic & Strategy • Board Games • Learning

Outline • Student Presentations: • “Game AI as Storytelling” • “Computational Approaches to Story-telling and Creativity” • “Procedural Level Design for Platform Games” • Very Brief Introduction to Machine Learning • Machine Learning in Games • Course Summary

Outline • Student Presentations: • “Game AI as Storytelling” • “Computational Approaches to Story-telling and Creativity” • “Procedural Level Design for Platform Games” • Very Brief Introduction to Machine Learning • Machine Learning in Games • Course Summary

Machine Learning • Branch of AI that studies how to: • Infer general knowledge from data : supervised learning • Infer behavior from data : learning from demonstration • Find hidden structure in data : unsupervised learning • Infer behavior from trial an error ( data ): reinforcement learning • Underlying principle is inductive inference: • E.g. after seeing 100 times that objects fall down, we can infer by induction the general law of gravity. • We cannot deduce gravity from observation, we can only induce it.

Supervised Learning • Given labeled (supervised) training data: (x 1 ,y 1 ) (x 2 ,y 2 ) (x 3 ,y 3 ) Examples … (x n ,y n ) • Find the general function that maps x to y: • f(x) = y

Supervised Learning Example • Learning when to cross the street: • X is defined as a list of features: (“cars coming?”, “pedestrian traffic light color”, “time”) • Y defined as: cross, stay • Examples: • x 1 = (no, green, 3pm) y 1 = cross • x 2 = (no, red, 8am) y 2 = don’t cross • x 3 = (yes, red, 5pm) y 3 = don’t cross • x 4 = (yes, green, 9pm) y 4 = don’t cross • x 5 = (no, green, 10am) y 5 = cross • Outcome of machine learning: • f(x) = “cross if no cars coming and light color green”

Supervised Learning • Examples can be: • Fixed-length vector of attributes: • Numerical • Categorical • Structured representations (graphs, trees, logical clauses) • Machine learning works better with: • Few representative attributes • Lots of data

Simplest Method: Nearest Neighbor • Examples are fixed vectors of real numbers • To predict f(x): • f(x) = y i where (x i ,y i ) is the example with x i most similar to x (nearest neighbor)

Simplest Method: Nearest Neighbor • Examples are fixed vectors of real numbers • To predict f(x): • f(x) = y i where (x i ,y i ) is the example with x i most similar to x (nearest neighbor) x

Simplest Method: Nearest Neighbor • Examples are fixed vectors of real numbers • To predict f(x): • f(x) = y i where (x i ,y i ) is the example with x i most similar to x (nearest neighbor) Nearest neighbor f(x) = gray x

Machine Learning Methods: • K-Nearest Neighbor • Decision Trees (ID3, C4.5) • Linear Regression • Bayesian Models (Naïve Bayes) • Boosting (AdaBoost) • Kernel Methods (SVM) • Neural Networks (MLP) • Inductive Logic Programming • etc.

Supervised Learning Methods: • K-Nearest Neighbor • Decision Trees (ID3, C4.5) • Linear Regression • Bayesian Models (Naïve Bayes) • Boosting (AdaBoost) • Kernel Methods (SVM) Which one is the best? • Neural Networks (MLP) • Inductive Logic Programming • etc.

Supervised Learning Methods: • K-Nearest Neighbor • Decision Trees (ID3, C4.5) • Linear Regression • Bayesian Models (Naïve Bayes) • Boosting (AdaBoost) • Kernel Methods (SVM) Which one is the best? • Neural Networks (MLP) • Inductive Logic Programming • etc. NONE! Each one has its own biases

Outline • Student Presentations: • “Game AI as Storytelling” • “Computational Approaches to Story-telling and Creativity” • “Procedural Level Design for Platform Games” • Very Brief Introduction to Machine Learning • Machine Learning in Games • Course Summary

Why Learning in Games? Benefit for Games: Benefit for AI: • Adaptive, believable Game AI • Games are great test beds (perfect instrumentation and • Find good player matches measurements) • Realistic player movement • Reduced cost • Reduce development cost (e.g. • Extremely challenging domains fine-tuning parameters) • Great way to motivate students!

Uses of Learning in Games • Driving Games: • Learn good driving behavior from observing humans • Fine-tune parameters of physics simulation, or of car handling parameters • RTS Games: • Automatically balance the game • RPG/FPS Games: • Believable movements • Others: • Specific game genres possible only with learning (training games) • Automatically adapt to player strategies • Learning player models

Example Usages: Black & White

Learning in Black & White Desires Intention: Abstract Opinions Plan Beliefs Specific Plan Action List Belief-Desire-Intention Architecture

Learning in Black & White Represented as hand- Desires crafted perceptrons (single Intention: layer neural networks) Abstract Opinions Plan Given the current situation, they activate more or less, Beliefs triggering certain desires. Specific Example: hunger Plan The structure of the perceptrons is hardcoded, but the parameters are learned Action List Belief-Desire-Intention Architecture

Learning in Black & White Desires Represented as learned Intention: decision trees, one per Abstract Opinions desire. Plan They capture towards which Beliefs object the creature should express each desire. Specific Plan Example: hunger The creature will learn a decision tree from player’s feedback of what can be eaten Action List Belief-Desire-Intention Architecture

Learning in Black & White Desires Intention: Abstract Opinions Plan Beliefs are just lists of Beliefs properties of the objects in the game, used for learning opinions. Specific Plan Action List Belief-Desire-Intention Architecture

Learning in Black & White Desires Intention: Abstract Opinions Plan Beliefs Specific Which desire will be Plan attempted, and towards which object: e.g. “destroy town” or “eat human” Action List Belief-Desire-Intention Architecture

Learning in Black & White Desires Intention: Abstract Opinions Plan Beliefs Specific Desire, and list of objects Plan that will be used. E.g. “Destroy town by throwing a stone” Action List Belief-Desire-Intention Architecture

Learning in Black & White Desires Intention: Abstract Opinions Plan Beliefs Specific Plan Specific list of actions to execute the plan Action List Belief-Desire-Intention Architecture

Learning in Black & White • BDI model to simulate character personality • Learning is deployed at specific players to adapt the BDI model to player preferences • Quite advanced, for commercial game standards!

Example Usages: Forza Motorsport

Example Usages: Forza Motorsport • Forza Motorsport lets you create “Drivatars” • Drivatars use machine learning to “learn” to drive like the player. • Supervised machine learning: • Input: the track just ahead of the car • Output: car trajectory

Track Segments • Define a set of “track segments”. All the tracks are made up of these • For each segment, define the different ways in which it can be traversed Segments a 1 a 2 a 3 a 4

Learning from the Player • When the player plays: 1. For each segment the player traverses, Drivatar remembers the trajectory used (out of the ones it has stored), speed, and other parameters. 2. Drivatar only remembers the last data seen, and forgets old data (to adapt to newer driving styles of the player) 3. When playing the game, it selects the trajectories learned form the player for each track segment, and combines them into a minimal curvature line, which is the path taken by the AI

Minimal Curvature Line

Learning in Drivatars: Key Ideas • Learning to drive is very complex! Too Many parameters! • Solution: simplify the problem! • Simplifications: • Divide the track in segments • Identify a finite set of possible trajectories for each segment (this defines a discrete and finite vocabulary for the machine learning method) • The goal of AI in games is to make the game fun, not to create advanced AI! (at least from a game company perspective J )

Other Games Using Learning • Colin McRae Rally 2.0: Neural networks to learn how to drive a rally car

Other Games Using Learning • Command & Conquer Renegade: learning new paths by observing the player (e.g. player breaks a window and creates a new path)

Other Games Using Learning • Re-Volt: genetic algorithm tunes the parameters of the steering behaviors for cars, to optimize lap time

Ways to Fake “Learning” in Games • Preprogram different levels of difficulty and change between them at run time • Rule-based system: have a large pool of rules, and add them bit by bit to the AI • Change parameters during game play: • Increase the accuracy of shooting • Decrease reaction time of enemies • Etc.