Probabilistic Inference Applied to Game Heuristics Karl Cronburg karl@cs.tufts.edu Tufts University Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 1 / 28
Motivation Want to judge the strength & limitations of a Primary Goal: game-play heuristic Domain experts model ‘winning‘ strategies Want to then visualize the probabilistic effects of changing the values of parameters in the state space A probabilistic map of the state space where the domain expert specifies interesting variables to plot distributions over Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 2 / 28
MCMC Sampling Methods Rejection Sampling - known model for joint distribution Random Walk ◮ Metropolis-Hastings - known model for joint distribution ◮ Gibbs Sampling - for hard-to-sample joint distribution Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 3 / 28
Probabilistic Languages Hakaru: an EDSL in Haskell ◮ Powerful host language support (fully embedded) ◮ Somewhat lacking in domain-specific features (limited API) BLOG: a Java-based DSL ◮ Poweful model of computation (many-worlds modeling) ◮ Lacking in domain-specific tool support and language features Our discussion could be extended to Church, Figaro, Fun, others languages discussed in class... Focus on Hakaru for this presentation Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 4 / 28
Sampling - Hakaru 1 test :: IS.Measure Double 2 test = IS.unconditioned (normal (-3) 1) 3 test2 = IS.unconditioned (uniform 1 5) 4 5 do 6 t <- IS.sample test [] 7 let s = take 1000 $ map fst t 8 t2 <- IS.sample test2 [] 9 let s2 = take 1000 $ map fst t2 10 return (makeHistogram 30 (V.fromList (s ++ s2)) "Histogram") Can arbitrarily compose Hakaru with Haskell code - single syntax Conditioned sampling on recursive models? 1Example taken from http://indiana.edu/ ppaml/HakaruTutorial.html Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 5 / 28
Case Study: Deckbuilding Games We chose Dominion 1 to model in a probabilistic language The Process: ◮ Write sampling model based on game mechanics Dominion ◮ Determine interesting state variables to condition on ◮ Run Hakaru MCMC ◮ Plot resulting joint distributions for domain expert to explore 1by D. X. Vaccarino Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 6 / 28
Case Study: Dominion Mechanics λ > runGreedy (0.5, 0.5) Player1: name = "Greedy1" hand = [ESTATE, GOLD, PROVINCE, PROVINCE, SILVER] inPlay = [] deck = [SILVER, PROVINCE, COPPER, SILVER, COPPER ,ESTATE, COPPER, COPPER, VILLAGE, VILLAGE ,PROVINCE, ESTATE, SILVER, COPPER] dscrd = [SILVER, SILVER, SILVER, COPPER, COPPER, PROVINCE] buys=1, actions=1, money=0 Player2: name = "Greedy2" hand = [COPPER, COPPER, COPPER, COPPER, VILLAGE] inPlay = [] deck = [SILVER, SILVER, GOLD, COPPER, COPPER, ESTATE ,GOLD, ESTATE, GOLD, ESTATE, PROVINCE, VILLAGE ,PROVINCE, GOLD] dscrd = [SILVER, COPPER, SILVER, SILVER, SILVER, PROVINCE] buys=1, actions=1, money=0 Trash: [] Supply: [(COPPER,60), (CELLAR,10), (MOAT,10), (ESTATE,8) ,(SILVER,27), (VILLAGE,6), (WOODCUTTER,10), (WORKSHOP,10) ,(MILITIA,10), (REMODEL,10), (SMITHY,10), (MARKET,10) ,(MINE,10), (DUCHY,8), (GOLD,25), (PROVINCE,0)] Turn #: 30 Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 7 / 28
Case Study: Choice of Parameters Initially focus on binary choices in card-buying-heuristic ◮ If we have 3 treasure on a turn... ⋆ Buy VILLAGE with probability p 0 ⋆ Buy CHANCELLOR with probability p 1 = 1 − p 0 ◮ Condition on number of turns in a game ◮ What does the distribution P ( p 0 | turns = t ) for some t look like? 1card content by D. X. Vaccarino Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 8 / 28
Case Study: Greedy vs Greedy Dominion Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 9 / 28
Case Study: Rejection Sampling Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 10 / 28
Case Study: Rejection Sampling Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 11 / 28
Case Study: Rejection Sampling Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 12 / 28
Case Study: Rejection Sampling Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 13 / 28
Case Study: Rejection Sampling Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 14 / 28
Case Study: Rejection Sampling Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 15 / 28
Case Study: Rejection Sampling Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 16 / 28
Case Study: Rejection Sampling Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 17 / 28
Case Study: Rejection Sampling Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 18 / 28
Case Study: Rejection Sampling Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 19 / 28
Case Study: Rejection Sampling Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 20 / 28
Case Study: Rejection Sampling Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 21 / 28
Case Study: Rejection Sampling Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 22 / 28
Case Study: Rejection Sampling Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 23 / 28
Case Study: Rejection Sampling Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 24 / 28
Case Study: Rejection Sampling Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 25 / 28
Case Study: Future For paper: ◮ New game heuristic model(s) Metropolis Hastings with conditioning on other heuristics ◮ # Points ◮ Average hand value on a given turn Model-Learner on public data sets Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 26 / 28
Technical Difficulties & Future Haskell monad transformers - Hakaru needs more monadic support Random numbers - Hakaru needs cleaner randomness support Need graceful handling of runtime errors - preferably none Difficult to distinguish Hakaru bugs from model bugs Card shuffling bug: *** Exception: [extractTree] impossible λ > runMetrop 25 100 [(0.9799143491179355,1.0),(0.9799143491179355,1.0),(0.9799143491179355,1.0),... Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 27 / 28
Acknowledgements Norman stackoverflow learnyouahaskell.com � Hakaru Code � P | Type Checks Runs Correctly Title text: This is roughly equivalent to ’number of times I’ve picked up a seashell at the ocean’ / ’number of times I’ve picked up a seashell’, which in my case is pretty close to 1, and gets much closer if we’re considering only times I didn’t put it to my ear. Karl Cronburg karl@cs.tufts.edu (Tufts University) Probabilistic Inference Applied to Game Heuristics 28 / 28
Recommend
More recommend
