Die Entwicklung der Spielprogrammierung: Von John von Neumann bis zu den hochparallelen Schachmaschinen Alexander Reinefeld Zuse Institut Berlin Humboldt-Universität zu Berlin „Themen der Informatik im historischen Kontext“ Ringvorlesung an der HU Berlin, 02.06.2005 Alexander Reinefeld 1
Spiel n. `nicht auf Nutzen ausgerichtete, vergnügliche, mit Ernst betriebene Tätigkeit, Zeitvertreib, Vergnügen, Wettkampf.‘ ... Das Substantiv (speel, spil, spel) ist nur kontinentalwestgerm. bezeugt.9. Jh. Etym. Wörterbuch des Deutschen, DTV, 2000. o das Spiel verderben o die Hand mit im Spiel haben o ins Spiel bringen o abspielen o zuspielen o sich aufspielen o Anspielung o Spielraum Alexander Reinefeld 2
Outline • Gam es • Searching Gam e Trees Minimax, Negamax, Alphabeta, o • History From Minimax to Alphabeta o • Searching Gam e Trees ( cont.) NegaScout, SSS* , … o • Analysis empirical o all leaf permutations o strict dominance o • Sum m ary Alexander Reinefeld 3
W hy Gam es? Games are 1. well defined by a set of simple rules, 2. scalable to a large complexity, 3. their outcome is measurable, and they are fun. „It is not that the games and mathematical problems are chosen „It is not that the games and mathematical problems are chosen because they are clear and simple; rather it is that they give us, because they are clear and simple; rather it is that they give us, for the smallest initial structures, the greatest complexity“ for the smallest initial structures, the greatest complexity“ M. Minsky, 1968 M. Minsky, 1968 Alexander Reinefeld 4
Tw o-Person Zero-Sum Gam e TW O-PERSON 2 parties involved, each trying to maximize his outcome ZERO-SUM the gain of the one player is the loss of the other GAME simple and well-defined rules scalable to larger complexity finite time (necessary to determine the outcome) Alexander Reinefeld 5
Alexander Reinefeld 6
Alexander Reinefeld 7
� erschienen in: Mathematische Annalen 100:295-320, 1928 J.v.N was only 23 years old. Häufiger zitiert, aber viel später veröffentlicht: J. v. Neumann, O. Morgenstern. The Theory of Games and Economic Behavior, Princeton Univ. Press, 1944. Alexander Reinefeld 8
Alexander Reinefeld 9
? Alexander Reinefeld 10
Properties The complete status (all choices and all consequences) of the game is given. Just by knowing the rules, any novice is able to play an optimal game. So what’s the problem? The size of the search space! Checkers: ~ 10 78 [ A.L. Samuel, “Some studies in machine learning using the game of checkers, 1963] Chess: ~ 10 120 [ C.E. Shannon, “Programming a computer for playing chess”, 1950] Go: ~ 10 170 Alexander Reinefeld 11
AND Trees OR Trees AND/ OR Trees To solve a problem, all sub- To solve a problem, one To solve a problem, an optimal problem s must be solved, strategy must be found solution (path) must be found. i.e. all nodes must be explored. goal node a strategy Exam ples Exam ples Exam ples • • • problem decomposition combinatorial optim. theorem proving • constraint satisfaction • robotics, ... Algorithm s Algorithm s Algorithm s • • • divide and conquer A* , IDA* AO* paradigm (no search) • • Branch&Bound SSS* , Dual* • AlphaBeta, NegaScout Alexander Reinefeld 12
A Gam e Tree A Game Tree is a AND/OR tree MAX‘s move with minimax backpropagation MIN‘s move . . . Leaf Leaf Node Nodes Values 4 8 3 5 1 7 2 9 3 9 2 3 5 4 2 8 Note: We use a binary tree for illustration only. A typical chess tree has a branching factor of b = 35 Alexander Reinefeld 13
Minim ax Value v( J) f(J) if J is a leaf node { v(J) = max { v(J.j) | 1 ≤ j ≤ w } if J is an interior MAX node min { v(J.j) | 1 ≤ j ≤ w } if J is an interior MIN node f(J) = leaf value Alexander Reinefeld 14
Minim ax max 3 min 2 3 max 4 2 3 4 min 4 3 1 2 3 2 4 2 4 8 3 5 1 7 2 9 3 9 2 3 5 4 2 8 Where is the principal variation? Alexander Reinefeld 15
Negam ax Value u( J) By observing that min { a, b } = - max { -a, -b } we can simplify the minimax function. { g(J) if J is a leaf node u(J) = max { - v(J.j) | 1 ≤ j ≤ w } if J is an interior node { f(J) if J is a MAX leaf node g(J) = -f(J) if J is a MIN leaf node Alexander Reinefeld 16
computes the maximum of Negam ax the negated successor values = max (2, 3) 3 -2 -3 = max (-3, -4) 4 2 3 4 = max (4, 2) -4 -3 -1 -2 -3 -2 -4 -2 4 8 3 5 1 7 2 9 3 9 2 3 5 4 2 8 principal variation Alexander Reinefeld 17
A MAX Strategy describes all possible replies of MIN 3 for one choice of MAX all successors at MIN nodes one successor at MAX nodes 2 3 4 2 3 4 4 3 1 2 3 2 4 2 4 8 3 5 1 7 2 9 3 9 2 3 5 4 2 8 Alexander Reinefeld 18
Proof that the MAX Strategy is Optim al 3 MAX Strategy must check all MIN strategies 2 3 MIN Strategy 4 2 3 4 4 3 1 2 3 2 4 2 4 8 3 5 1 7 2 9 3 9 2 3 5 4 2 8 Alexander Reinefeld 19
Four w ays to „solve“ a Gam e Tree Minimax Alphabeta • compute the m inim ax (resp. negamax) value NegaScout SSS* enumerate all MAX strategies and chose the best one • DUAL* • determine „maximum word“ in the corresponding context free gram m ar • solve the disjunctive norm al form • resistor/ capacitor network • maximal flow problem • ... (many more) Alexander Reinefeld 20
Alphabeta • computes the minimax value without examining all nodes Branch-and-bound (Land & Doig 1960, Lawler & Wood 1966, … ) o Alphabeta (1963, 1975) o • Examine nodes with a search w indow start with search window (- ∞,∞ ) o α is minimal value (lower bound) known so far for MAX ο β is maximal value (upper bound) known so far for MIN ο disregard all nodes ≤ α and ≥ β o Alexander Reinefeld 21
Minimax Version of Alphabeta int AB_MAX(position p; int α , β ); { if (p == LEAF) return(Evaluate(p)); val = α ; for (i = 1; i ≤ w; i++) { val = max (val, AB_MIN (p i , val, β )); if (val ≥ β ) return (val); /* beta cut-off */ } return (val); } int AB_MIN(position p; int α , β ); { if (p == LEAF) return(Evaluate(p)); val = β ; for (i = 1; i ≤ w; i++) { val = min (val, AB_MAX (p i , α , val); if (val ≤ α ) return (val); /* alpha cut-off */ } return (val); } Alexander Reinefeld 22
Alphabeta (-oo, +oo) 5 (5, +oo) (-oo,+oo) 4 5 α cutoff (-oo,oo) (-oo,5) (5,+oo) 2, 5 9 1, 4 β cutoff 2 5 9 1 4 Alexander Reinefeld 23
Shallow and Deep Cutoff (-oo, +oo) (-oo,+oo) (5, +oo) 5 shallow α cutoff (5, +oo) 3 (5,+oo) 4 deep α cutoff 3 Alexander Reinefeld 24
Brief History Alexander Reinefeld 25
From Minim ax to Alphabeta 1 9 5 6 : McCarthy was the first who noticed that not all branches need to be examined (personal conversation, see Knuth & Moore, 1975) 1 9 5 8 : Newell, Shaw and Simon used branch-and-bound in their chess program 1 9 6 3 : Brudno published the first paper on alpha-beta (in Russian) 1 9 7 5 : Knuth & Moore published the full alpha-beta, including deep cut-offs Alexander Reinefeld 26
C.E. Shannon, Programming a Computer for Playing Chess, Philosophical Magazine 41 , 7 (1950), 256-275. Alexander Reinefeld 27
A. Newell, J.C. Shaw and H.A. Simon, Chess Playing Programs and the Problem of Complexity, IBM J. of Research and Development 4 , 2 (1958), 320-335. � first used the term “cut-off” Alexander Reinefeld 28
A.L. Brudno, Bounds and Valuations for Abridging the Search of Estimates, Problems of Cybernetics 10 , (1963), 225-241. Translation of Russian original in Problemy Kibernetiki 10, 141-150 (May 1963). submitted 31 May 1962 Alexander Reinefeld 29
A.L. Samuel, Some Studies in Machine Learning Using the Game of Checkers, IBM J. of Research and Development 3 , (1959), 210-229. Paper on machine learning, coined the word “ply” – meaning a half move. A.L. Samuel, Some Studies in Machine Learning Using the Game of Checkers II – Recent Progress, IBM J. of Research and Development 11 (1967) 601-17. deep cut-offs Alexander Reinefeld 30
D.E. Knuth and R.W. Moore, An Analysis of Alpha-beta Pruning, Artificial Intelligence 6 , 4 (1975), 293-326. Alexander Reinefeld 31
Alexander Reinefeld 32
Knuth & Moore‘s Function F2 ( aka AlphaBeta) Alexander Reinefeld 33
Alexander Reinefeld 34
Recommend
More recommend
