Bayesian networks in Mastermind Ji r Vomlel - PowerPoint PPT Presentation

Bayesian networks in Mastermind Jiˇ r´ ı Vomlel http://www.utia.cas.cz/vomlel/ 1

Contents • Bayesian networks, tasks solved by them, and the junction tree propagation. • An application of BNs - adaptive testing • The game of Mastermind as an example of adaptive test • Bayesian networks in the game of Mastermind • Efficient inference using Bayesian networks for Mastermind 2

P ( X 1 ) P ( X 2 ) X 1 X 2 P ( X 3 | X 1 ) P ( X 4 | X 2 ) X 3 X 4 P ( X 5 | X 1 ) X 5 P ( X 6 | X 3 , X 4 ) X 6 P ( X 9 | X 6 ) X 7 X 8 X 9 P ( X 7 | X 5 ) P ( X 8 | X 7 , X 6 ) P ( X 1 , . . . , X 9 ) = = P ( X 9 | X 8 , . . . , X 1 ) · P ( X 8 | X 7 , . . . , X 1 ) · . . . · P ( X 2 | X 1 ) · P ( X 1 ) = P ( X 9 | X 6 ) · P ( X 8 | X 7 , X 6 ) · P ( X 7 | X 5 ) · P ( X 6 | X 4 , X 3 ) · P ( X 5 | X 1 ) · P ( X 4 | X 2 ) · P ( X 3 | X 1 ) · P ( X 2 ) · P ( X 1 ) 3

Typical use of Bayesian networks • to model and explain a domain. • to update beliefs about states of certain variables when some other variables were observed, i.e., computing conditional probability distributions, e.g., P ( X 23 | X 17 = yes , X 54 = no ) . • to find most probable configurations of variables • to support decision making under uncertainty • to find good strategies for solving tasks in a domain with uncertainty. 4

Bayesian networks: junction tree propagation (1) (2) X 1 X 2 X 1 X 2 X 3 X 4 X 3 X 4 X 5 X 5 X 6 X 6 X 7 X 8 X 9 X 7 X 8 X 9 (3) (4) X 1 X 2 X 1 , X 3 , X 5 X 3 , X 5 , X 7 X 2 , X 4 X 3 X 4 X 3 , X 6 , X 7 X 3 , X 4 , X 6 X 5 X 6 X 6 , X 7 , X 8 X 6 , X 9 X 7 X 8 X 9 5

A simple example of an adaptive test wrong answer No knowledge Easy question wrong answer Low knowledge correct answer Question of medium difficulty wrong answer Medium knowledge correct answer Difficult question Good knowledge correct answer 6

The game of Mastermind T j , H j ... colors on the j th position in the guess and in the hidden code. Let δ ( A , B ) equals one if A = B and zero otherwise. 4 ∑ = δ ( H j , i ) C i = δ ( T j , H j ) P j = min ( C i , G i ) M i j = 1 � � 4 6 ∑ ∑ 4 = = − P P P j C M i ∑ = δ ( T j , i ) G i j = 1 i = 1 j = 1 7

Probability over the codes Q ( H 1 , . . . , H 4 ) ... the probability distribution over the possible codes. At the beginning of the game this distribution is uniform, i.e. 1 1 Q ( H 1 = h 1 , . . . , H 4 = h 4 ) = 6 4 = 1296 During the game we update probability Q ( H 1 , . . . , H 4 ) using the obtained evidence e and compute the conditional probability  1 if ( h 1 , . . . , h 4 ) is a possible code  n ( e ) Q ( H 1 = h 1 , . . . , H 4 = h 4 | e ) = 0 otherwise,  where n ( e ) is the total number of codes that are possible candidates for the hidden code. 8

A measure of uncertainty - the Shannon entropy A criteria suitable to measure the uncertainty about the hidden code is the Shannon entropy Q ( H 1 = h 1 , . . . , H 4 = h 4 | e ) ∑ H ( Q ( H 1 , . . . , H 4 | e )) = , · log Q ( H 1 = h 1 , . . . , H 4 = h 4 | e ) h 1 ,..., h 4 where 0 · log 0 is defined to be zero. Note that the Shannon entropy is zero if and only if the code is known. 9

Optimal Mastermind strategies Different criteria: • minimal expected length minimal sum over all suggested sequences of length of a sequence × probability of this sequence Koyama, Lai (1993): A minimal strategy with 5625/1296 = 4.340 guesses. • minimal depth minimal number of guesses in the worst case Koyama, Lai (1993): A different strategy with depth of 5 guesses. • most informative within a limited number of guesses minimal sum over all suggested sequences of entropy after a sequence × probability of this sequence 10

Bayesian network for the probabilistic Mastermind C ′ C M 1 M 2 M 3 M 4 M 5 M 6 C 1 C 2 C 3 C 4 C 5 C 6 H 1 H 2 H 3 H 4 P 1 T 1 G 1 G 2 P 2 T 2 G 3 G 4 P 3 T 3 G 5 P P 4 T 4 G 6 P ′ 11

Bayesian network after inserting evidence C ′ C ′ after moralization (before triangulation) C C M 1 M 1 M 2 M 2 M 3 M 3 M 4 M 4 M 5 M 5 M 6 M 6 C 1 C 2 C 3 C 4 C 5 C 6 C 1 C 2 C 3 C 4 C 5 C 6 H 1 H 2 H 3 H 4 H 1 H 2 H 3 H 4 P 1 P 1 P 2 P 2 P 3 P 3 P P 4 P P 4 P ′ P ′ 12

Transformation by introducing an auxiliary variable to the model Savicky, Vomlel (2004) C C B M 1 M 1 M 2 M 2 M 3 M 3 M 4 M 4 M 5 M 5 M 6 M 6 = ⇒ P P 13

Bayesian network after the suggested transformation C ′ C B M 1 M 2 M 3 M 4 M 5 M 6 C 1 C 2 C 3 C 4 C 5 C 6 H 1 H 2 H 3 H 4 P 1 Junction tree size: • without the suggested transfor- P 2 mation > 20,526,445 P 3 P P 4 • after the suggested transforma- P ′ tion 214,775 14

Summary • The game of Mastermind is an example of a adpative test. • In order to use Bayesian networks for computations we need to exploit functional dependences in the model. • The suggested transformation substantially decreses computational demands. 15