Adversarial Search CE417: Introduction to Artificial Intelligence - PowerPoint PPT Presentation

Adversarial Search CE417: Introduction to Artificial Intelligence Sharif University of Technology Spring 2017 Soleymani “ Artificial Intelligence: A Modern Approach ” , 3 rd Edition, Chapter 5

Outline  Game as a search problem  Minimax algorithm  𝛽 - 𝛾 Pruning: ignoring a portion of the search tree  Time limit problem  Cut off & Evaluation function 2

Games as search problems  Games  Adversarial search problems (goals are in conflict)  Competitive multi-agent environments  Games in AI are a specialized kind of games (in the game theory) 3

Primary assumptions  Common games in AI:  T wo-player  Turn taking  agents act alternately  Zero-sum  agents ’ goals are in conflict: sum of utility values at the end of the game is zero or constant  Deterministic  Perfect information  fully observable

Game as a kind of search problem  Initial state 𝑇 0 , set of states (each state contains also the turn), 𝐵𝐷𝑈𝐽𝑃𝑂𝑇(𝑡) , 𝑆𝐹𝑇𝑉𝑀𝑈𝑇 𝑡, 𝑏 like standard search  𝑄𝑀𝐵𝑍𝐹𝑆𝑇(𝑡) : Defines which player takes turn in a state  𝑈𝐹𝑆𝑁𝐽𝑂𝐵𝑀_𝑈𝐹𝑇𝑈(𝑡) : Shows where game has ended  𝑉𝑈𝐽𝑀𝐽𝑈𝑍(𝑡, 𝑞) : utility or payoff function 𝑉: 𝑇 × 𝑄 → ℝ (how good is the terminal state 𝑡 for player 𝑞 )  Zero-sum (constant-sum) game: the total payoff to all players is zero (or constant) for every terminal state  We have utilities at end of game instead of sum of action costs 5

Game tree (tic-tac-toe)  Two players: 𝑄 1 and 𝑄 2 ( 𝑄 1 is now searching to find a good move)  Zero-sum games: 𝑄 1 gets 𝑉(𝑢) , 𝑄 2 gets 𝐷 − 𝑉(𝑢) for terminal node 𝑢 𝑄 𝑄 1 : 𝑌 1 𝑄 2 : 𝑃 1-ply = half move 𝑄 2 𝑄 1 𝑄 2 Utilities from the point of view of 𝑄 1 6

Game tree (tic-tac-toe)  Two players: 𝑄 1 and 𝑄 2 ( 𝑄 1 is now searching to find a good move)  Zero-sum games: 𝑄 1 gets 𝑉(𝑢) , 𝑄 2 gets 𝐷 − 𝑉(𝑢) for terminal node 𝑢 1-ply = half move Utilities from the point of view of 𝑄 MAX 1 7

Optimal play  Opponent is assumed optimal  Minimax function is used to find the utility of each state.  MAX/MIN wants to maximize/minimize the terminal payoff MAX gets 𝑉(𝑢) for terminal node 𝑢 8

Minimax 𝑉𝑈𝐽𝑀𝐽𝑈𝑍(𝑡, 𝑁𝐵𝑌) 𝑗𝑔 𝑈𝐹𝑆𝑁𝐽𝑂𝐵𝑀_𝑈𝐹𝑇𝑈(𝑡) 𝑛𝑏𝑦 𝑏∈𝐵𝐷𝑈𝐽𝑃𝑂𝑇 𝑡 𝑁𝐽𝑂𝐽𝑁𝐵𝑌(𝑆𝐹𝑇𝑉𝑀𝑈(𝑡, 𝑏)) 𝑄𝑀𝐵𝑍𝐹𝑆 𝑡 = 𝑁𝐵𝑌 𝑁𝐽𝑂𝐽𝑁𝐵𝑌 𝑡 = 𝑛𝑗𝑜 𝑏∈𝐵𝐷𝑈𝐽𝑃𝑂𝑇 𝑡 𝑁𝐽𝑂𝐽𝑁𝐵𝑌(𝑆𝐹𝑇𝑉𝑀𝑈 𝑡, 𝑏 ) 𝑄𝑀𝐵𝑍𝐹𝑆 𝑡 = 𝑁𝐽𝑂 Utility of being in state s  𝑁𝐽𝑂𝐽𝑁𝐵𝑌(𝑡) shows the best achievable outcome of being in state 𝑡 (assumption: optimal opponent) 3 3 2 2 9

Minimax (Cont.)  Optimal strategy: move to the state with highest minimax value  Best achievable payoff against best play  Maximizes the worst-case outcome for MAX  It works for zero-sum games 10

Minimax algorithm Depth first search function 𝑁𝐽𝑂𝐽𝑁𝐵𝑌_𝐸𝐹𝐷𝐽𝑇𝐽𝑃𝑂(𝑡𝑢𝑏𝑢𝑓) returns 𝑏𝑜 𝑏𝑑𝑢𝑗𝑝𝑜 𝑏∈𝐵𝐷𝑈𝐽𝑃𝑂𝑇(𝑡𝑢𝑏𝑢𝑓) 𝑁𝐽𝑂_𝑊𝐵𝑀𝑉𝐹(𝑆𝐹𝑇𝑉𝑀𝑈(𝑡𝑢𝑏𝑢𝑓, 𝑏)) max return function 𝑁𝐵𝑌_𝑊𝐵𝑀𝑉𝐹(𝑡𝑢𝑏𝑢𝑓) returns 𝑏 𝑣𝑢𝑗𝑚𝑗𝑢𝑧 𝑤𝑏𝑚𝑣𝑓 if 𝑈𝐹𝑆𝑁𝐽𝑂𝐵𝑀_𝑈𝐹𝑇𝑈(𝑡𝑢𝑏𝑢𝑓) then return 𝑉𝑈𝐽𝑀𝐽𝑈𝑍(𝑡𝑢𝑏𝑢𝑓) 𝑤 ← −∞ for each 𝑏 in 𝐵𝐷𝑈𝐽𝑃𝑂𝑇(𝑡𝑢𝑏𝑢𝑓) do 𝑤 ← 𝑁𝐵𝑌(𝑤, 𝑁𝐽𝑂_𝑊𝐵𝑀𝑉𝐹(𝑆𝐹𝑇𝑉𝑀𝑈𝑇(𝑡𝑢𝑏𝑢𝑓, 𝑏))) return 𝑤 function 𝑁𝐽𝑂_𝑊𝐵𝑀𝑉𝐹(𝑡𝑢𝑏𝑢𝑓) returns 𝑏 𝑣𝑢𝑗𝑚𝑗𝑢𝑧 𝑤𝑏𝑚𝑣𝑓 if 𝑈𝐹𝑆𝑁𝐽𝑂𝐵𝑀_𝑈𝐹𝑇𝑈(𝑡𝑢𝑏𝑢𝑓) then return 𝑉𝑈𝐽𝑀𝐽𝑈𝑍(𝑡𝑢𝑏𝑢𝑓) 𝑤 ← ∞ for each 𝑏 in 𝐵𝐷𝑈𝐽𝑃𝑂𝑇(𝑡𝑢𝑏𝑢𝑓) do 𝑤 ← 𝑁𝐽𝑂(𝑤, 𝑁𝐵𝑌_𝑊𝐵𝑀𝑉𝐹(𝑆𝐹𝑇𝑉𝑀𝑈𝑇(𝑡𝑢𝑏𝑢𝑓, 𝑏))) return 𝑤 11

Properties of minimax  Complete?Yes (when tree is finite)  Optimal?Yes (against an optimal opponent)  Time complexity: 𝑃(𝑐 𝑛 )  Space complexity: 𝑃(𝑐𝑛) (depth-first exploration)  For chess, 𝑐 ≈ 35 , 𝑛 > 50 for reasonable games  Finding exact solution is completely infeasible 12

Pruning  Correct minimax decision without looking at every node in the game tree  α - β pruning  Branch & bound algorithm  Prunes away branches that cannot influence the final decision 13

α - β pruning example 14

α - β pruning example 15

α - β pruning example 16

α - β pruning example 17

α - β pruning example 18

α - β progress 19

α - β pruning  Assuming depth-first generation of tree  We prune node 𝑜 when player has a better choice 𝑛 at (parent or) any ancestor of 𝑜  Two types of pruning (cuts):  pruning of max nodes ( α -cuts)  pruning of min nodes ( β -cuts) 20

Why is it called α - β?  α : Value of the best (highest) choice found so far at any choice point along the path for MAX  𝛾 : Value of the best (lowest) choice found so far at any choice point along the path for MIN  Updating α and 𝛾 during the search process  For a MAX node once the value of this node is known to be more than the current 𝛾 ( v ≥ 𝛾 ), its remaining branches are pruned.  For a MIN node once the value of this node is known to be less than the current 𝛽 ( v ≤ 𝛽 ), its remaining branches are pruned. 21

α - β pruning (an other example) 3 ≤ 2 3 2 ≥ 5 5 1 22

function 𝐵𝑀𝑄𝐼𝐵_𝐶𝐹𝑈𝐵_𝑇𝐹𝐵𝑆𝐷𝐼(𝑡𝑢𝑏𝑢𝑓) returns 𝑏𝑜 𝑏𝑑𝑢𝑗𝑝𝑜 𝑤 ← 𝑁𝐵𝑌_𝑊𝐵𝑀𝑉𝐹(𝑡𝑢𝑏𝑢𝑓, −∞, +∞) return the 𝑏𝑑𝑢𝑗𝑝𝑜 in 𝐵𝐷𝑈𝐽𝑃𝑂𝑇(𝑡𝑢𝑏𝑢𝑓) with value 𝑤 function 𝑁𝐵𝑌_𝑊𝐵𝑀𝑉𝐹(𝑡𝑢𝑏𝑢𝑓, 𝛽, 𝛾) returns 𝑏 𝑣𝑢𝑗𝑚𝑗𝑢𝑧 𝑤𝑏𝑚𝑣𝑓 if 𝑈𝐹𝑆𝑁𝐽𝑂𝐵𝑀_𝑈𝐹𝑇𝑈(𝑡𝑢𝑏𝑢𝑓) then return 𝑉𝑈𝐽𝑀𝐽𝑈𝑍(𝑡𝑢𝑏𝑢𝑓) 𝑤 ← −∞ for each 𝑏 in 𝐵𝐷𝑈𝐽𝑃𝑂𝑇(𝑡𝑢𝑏𝑢𝑓) do 𝑤 ← 𝑁𝐵𝑌(𝑤, 𝑁𝐽𝑂_𝑊𝐵𝑀𝑉𝐹(𝑆𝐹𝑇𝑉𝑀𝑈𝑇(𝑡𝑢𝑏𝑢𝑓, 𝑏), 𝛽, 𝛾)) if 𝑤 ≥ 𝛾 then return 𝑤 𝛽 ← 𝑁𝐵𝑌(𝛽, 𝑤) return 𝑤 function 𝑁𝐽𝑂_𝑊𝐵𝑀𝑉𝐹(𝑡𝑢𝑏𝑢𝑓, 𝛽, 𝛾) returns 𝑏 𝑣𝑢𝑗𝑚𝑗𝑢𝑧 𝑤𝑏𝑚𝑣𝑓 if 𝑈𝐹𝑆𝑁𝐽𝑂𝐵𝑀_𝑈𝐹𝑇𝑈(𝑡𝑢𝑏𝑢𝑓) then return 𝑉𝑈𝐽𝑀𝐽𝑈𝑍(𝑡𝑢𝑏𝑢𝑓) 𝑤 ← +∞ for each 𝑏 in 𝐵𝐷𝑈𝐽𝑃𝑂𝑇(𝑡𝑢𝑏𝑢𝑓) do 𝑤 ← 𝑁𝐽𝑂(𝑤, 𝑁𝐵𝑌_𝑊𝐵𝑀𝑉𝐹(𝑆𝐹𝑇𝑉𝑀𝑈𝑇(𝑡𝑢𝑏𝑢𝑓, 𝑏), 𝛽, 𝛾)) if 𝑤 ≤ 𝛽 then return 𝑤 𝛾 ← 𝑁𝐽𝑂(𝛾, 𝑤) return 𝑤 23

Order of moves  Good move ordering improves effectiveness of pruning ? 𝑛 2 )  Best order: time complexity is 𝑃(𝑐 3𝑛 4 )  Random order: time complexity is about 𝑃(𝑐 for moderate 𝑐  α - β pruning just improves the search time only partly 24

Computational time limit (example)  100 secs is allowed for each move (game rule)  10 4 nodes/sec (processor speed)  We can explore just 10 6 nodes for each move  b m = 10 6 , b=35 ⟹ m=4 (4-ply look-ahead is a hopeless chess player!) 25

Computational time limit: Solution  We must make a decision even when finding the optimal move is infeasible.  Cut off the search and apply a heuristic evaluation function  cutoff test: t urns non-terminal nodes into terminal leaves  Cut off test instead of terminal test ( e.g., depth limit)  evaluation function: estimated desirability of a state  Heuristic function evaluation instead of utility function  This approach does not guarantee optimality. 26

Heuristic minimax 𝐼 𝑁𝐽𝑂𝐽𝑁𝐵𝑌 𝑡,𝑒 = 𝐹𝑊𝐵𝑀(𝑡, 𝑁𝐵𝑌) 𝑗𝑔 𝐷𝑉𝑈𝑃𝐺𝐺_𝑈𝐹𝑇𝑈(𝑡, 𝑒) 𝑛𝑏𝑦 𝑏∈𝐵𝐷𝑈𝐽𝑃𝑂𝑇 𝑡 𝐼_𝑁𝐽𝑂𝐽𝑁𝐵𝑌(𝑆𝐹𝑇𝑉𝑀𝑈 𝑡, 𝑏 , 𝑒 + 1) 𝑄𝑀𝐵𝑍𝐹𝑆 𝑡 = MAX 𝑛𝑗𝑜 𝑏∈𝐵𝐷𝑈𝐽𝑃𝑂𝑇 𝑡 𝐼_𝑁𝐽𝑂𝐽𝑁𝐵𝑌(𝑆𝐹𝑇𝑉𝑀𝑈 𝑡, 𝑏 , 𝑒 + 1) 𝑄𝑀𝐵𝑍𝐹𝑆 𝑡 = MIN 27

Evaluation functions  For terminal states, it should order them in the same way as the true utility function.  For non-terminal states, it should be strongly correlated with the actual chances of winning.  It must not need high computational cost. 28