Artificial Intelligence Time and Uncertainty CS 444 – Spring 2019 Dr. Kevin Molloy Department of Computer Science James Madison University
Time and Uncertainty The world changes; we need to track and predict it Examples: Diabetes management , vehicle diagnosis Basic idea : copy state and evidence variables for each time step X t = set of unobservable state variables at time t e.g. BloodSugar t , StomachContents t , etc. E t = set of observable evidence variables at time t e.g. MeasuredBloodSugar t , PulseRate t , FoodEaten t This assumes discrete time; step size depends on problem. Notation: X a:b = X a , X A+1 , … , X b-1 , X b
Markov processes (Markov chains) Construct a Bayes net from these variables: parents? Markov assumption: X t depends on bounded subset of X 0:t -1 First-order Markov process: P(X t | X 0:t-1 ) = P(X t | X t-1 ) Second-order Markov process: P(X t | X 0:t-1 ) = P(X t | X t-1, X t-2 ) First-order X t-2 X t-1 X t X t+1 X t+1 Second-order X t-2 X t-1 X t X t+1 X t+1 Sensor Markov assumption: P(E t | X 0:t , E 0:t-1 ) = P(E t | X t ) Stationary process: transition model P(X t | X t-1 ) and sensor model P(E t | X t ) fixed for all t .
Example Transition Probabilities T (i,j) = P(X k+1 =j | Z k = i) (i,j ∈ m). Called the transition or stochastic matrix First-order Markov assumption not exactly true in the real world. Emission probabilities Possible fixes: (called the sensor model in the • Increase order of Markov process textbook) • Augment state , e.g., Add temp, pressure, etc. Example: robot motion: Augment position and velocity with Battery
Inference tasks Filtering: P(X t | e 1:t ) Belief state – input to the decision process of a rational agent Prediction: P(X t+k | e 1:t ) for k > 0 Evaluation of possible action sequences; like filtering without the evidence Smoothing: P(X k | e 1:t ) for 0 ≤ k < t Better estimate of past states, essential for learning Most likely explanation: ARGMAX x 1:t P(x 1:t | e 1:t ) Speech recognition, decoding with a noisy channel
Filtering Goal: compute the belief state – the posterior distribution over the most recent state – given all the evidence seen to date. Aim: devise a recursive state estimate algorithm: P(X t+1 | e 1:t+1 ) = f(e t+1 , P(X t | e 1:t )) P(X t+1 | e 1:t+1 ) = P(X t+1 | e 1:t , e t+1 ) divide evidence variables = 𝛽 P(e t+1 | X t+1 , e 1:t ) P(X t+1 | e 1:t ) using Bayes' rule = 𝛽 P(e t+1 | X t+1 ) P(X t+1 | e 1:t ) Markov assumption i.e., prediction + estimation. Prediction by summing out and conditioning on X t : P(X t+1 | e 1:t+1 ) = 𝛽 P(e t+1 | X t+1 ) ∑ $ % 𝑄 𝑌 ()* 𝑦 ( , 𝑓 *:( )𝑄(𝑦 ( , 𝑓 *:( ) = 𝛽 P(e t+1 |X t+1 ) ∑ $ % 𝑄 𝑌 ()* 𝑦 ( )𝑄(𝑦 ( , 𝑓 *:( ) f 1:t+1 = Forward( f 1:t , e t+1 ) where f 1:t = P(X t | e 1:t ) Time and space constant (independent of t ) !!!
Filtering Example Day 0: All we have are the beliefs (priors) Day 1: Umbrella appears. P(R 1 ) = ∑ 1 2 𝑄 𝑆 * 𝑠 5 )𝑄(𝑠 6 ) = ⟨ 0.7,0.3 ⟩ x 0.5 + ⟨ 0.3, 0.7 ⟩ x 0.5 = ⟨ 0.5, 0.5 ⟩ Update based on evidence (Umbrella) P(R 1 | u 1 ) = 𝛽P( u 1, R 1 ) 𝑄 𝑆 * = 𝛽⟨ 0.9,0.2 ⟩ x ⟨ 0.5, 0.5 ⟩ = 𝛽⟨ .45, 0.1 ⟩ ≈ ⟨ 0.818, 0.182 ⟩ Day 2: Umbrella appears. R t-1 P(R t ) R t P(U t ) P(R 2 | u 1 ) = ∑ 1 : 𝑄 𝑆 ; 𝑠 * )𝑄 𝑠 * 𝑣 * ) t 0.7 t 0.9 = ⟨ 0.7,0.3 ⟩ x 0.818 + ⟨ 0.3, 0.7 ⟩ x 0.182 ≈ ⟨ 0.627, 0.373 ⟩ f 0.3 f 0.2 Update: P(R 2 | u 1 , u 2 ) = 𝛽 P(u 2 | R 2 )P(R 2 | u 1 ) = 𝛽⟨ 0.9, 0.2 ⟩⟨ 0.627, 0.373 ⟩ = 𝛽⟨ 0.565, -.0075 ⟩ ≈ ⟨ 0.883, 0.117 ⟩
Smoothing P(X k | e 1:t ) = P(X k | e 1:k , e k+1:t ) = 𝛽 P(X k | e 1:k )P(e k+1:t | X k , e 1:k ) Divide evidence e 1:t into e 1:k , e k+1:t = 𝛽 P(X k | e 1:k ) P(e k+1:t | X k ) = 𝛽 P(X k | e 1:k ) P(e k+1:t | X k ) = 𝛽 f 1:k b k+1:t Backward message computed by a backwards recursion: P(e 1+1:t | X k ) = ∑ $ =>: 𝑄 𝑓 ?)*:( 𝑌 ? , 𝑌 ?)* ) 𝑄 𝑦 ?)* 𝑌 ? ) = ∑ $ =>: 𝑄 𝑓 ?)*:( 𝑦 ?)* ) 𝑄 𝑦 ?)* 𝑌 ? ) = ∑ $ =>: 𝑄 𝑓 ?)*:( 𝑦 ?)* ) 𝑄 𝑦 ?);:( 𝑌 ?)* ) 𝑄 𝑦 ?)* 𝑌 ? )
Smoothing Example Forward-backward algorithm Time linear in t (polytree inference) space is O(t|f|) P(R 1 | u 1 , u 2 ) = 𝛽𝑄 𝑆 * 𝑣 * )𝑄 𝑣 ; 𝑆 * ) P(u 2 | R 1 ) = ∑ 1 @ 𝑄 𝑣 ; 𝑠 ; ) 𝑄 𝑠 ; 𝑆 * ) = (0.9 x 1 x ⟨ 0.7, 0.3 ⟩ ) + (0.2 x 1 x ⟨ 0.3, 0.7 ⟩ = ⟨ 0.69, 0.41) P(R 1 | u 1 , u 2 ) = 𝛽⟨ 0.818,0.182 ⟩ x ⟨ 0.69, 0.41 ⟩ ≈ ⟨ 0.883, 0.117 ⟩
Most likely explanation Most likely sequence ≠ sequence of most likely states!!! Most likely path to each x t+1 = most likely path to some x t plus one more step $ : …$ % 𝑄(𝑦 * , … , 𝑦 ( , 𝑌 ()* 𝑓 *:()* max = 𝑄 𝑓 ()* 𝑌 ()* ) max 𝑄 𝑌 ()* 𝑦 ( $ : … $ %G: 𝑄 𝑦 * , … , 𝑦 (H* , 𝑦 ( 𝑓 *:( ) ) max F % = 𝑄 𝑓 ()* 𝑌 ()* ) max 𝑄 𝑌 ()* 𝑦 ( $ : … $ %G: 𝑄 𝑦 * , … , 𝑦 (H* , 𝑦 ( 𝑓 *:( ) max F % Identical to filtering, except f 1:t replaced by 𝑛 *:( = $ : …$ %G: 𝑄 𝑦 * , … , 𝑦 (H* , 𝑌 ( 𝑓 *:( ) max i.e., m 1:t (i) gives the probability of the most likely path to state i. Update has sum replaced by max, giving the Viterbi algorithm. 𝑛 *:()* = P 𝑓 ()* 𝑌 ()* ) max 𝑄 𝑌 ()* |𝑦 ( 𝑛 *:( $ %
Hidden Markov Model X t is a single, discrete variable (usually E t is too). Domain of X t is {1, …., S} Transition matrix T ij = P(X t = j | X t-1 = i), e.g. 0.7 0.3 0.3 0.7 Sensor matrix O t for each time step, diagonal elements P(e t | X t = i) e.g. With U 1 = true, O 1 = 0.9 0 0 0.2 Forward and backward messages as column vectors *:()* = 𝛽𝑃 ()* 𝑈 T 𝑔 𝑔 *:( 𝑐 ?)*:( = 𝑈𝑃 ?)* 𝑐 ?);:( Forward-backward algorithm needs time O(S 2 t) and space O(St)
Country Dance Algorithm Can avoid storing all forward messages in smoothing by running forward algorithm backwards: V:(W* = 𝛽𝑃 ()* 𝑈 ( 𝑔 𝑔 *:( H* 𝑔 *:()* = 𝛽𝑈 ( 𝑔 𝑃 ()* *:( 𝛽 X 𝑈 T H* 𝑃 ()* H* 𝑔 *:()* = 𝑔 *:( Algorithm: forward pass computes f t , backward pass foes f i , b i
Country Dance Algorithm Can avoid storing all forward messages in smoothing by running forward algorithm backwards: V:(W* = 𝛽𝑃 ()* 𝑈 ( 𝑔 𝑔 *:( H* 𝑔 *:()* = 𝛽𝑈 ( 𝑔 𝑃 ()* *:( 𝛽 X 𝑈 T H* 𝑃 ()* H* 𝑔 *:()* = 𝑔 *:( Algorithm: forward pass computes f t , backward pass foes f i , b i
Country Dance Algorithm Can avoid storing all forward messages in smoothing by running forward algorithm backwards: V:(W* = 𝛽𝑃 ()* 𝑈 ( 𝑔 𝑔 *:( H* 𝑔 *:()* = 𝛽𝑈 ( 𝑔 𝑃 ()* *:( 𝛽 X 𝑈 T H* 𝑃 ()* H* 𝑔 *:()* = 𝑔 *:( Algorithm: forward pass computes f t , backward pass foes f i , b i
Country Dance Algorithm Can avoid storing all forward messages in smoothing by running forward algorithm backwards: V:(W* = 𝛽𝑃 ()* 𝑈 ( 𝑔 𝑔 *:( H* 𝑔 *:()* = 𝛽𝑈 ( 𝑔 𝑃 ()* *:( 𝛽 X 𝑈 T H* 𝑃 ()* H* 𝑔 *:()* = 𝑔 *:( Algorithm: forward pass computes f t , backward pass foes f i , b i
Country Dance Algorithm Can avoid storing all forward messages in smoothing by running forward algorithm backwards: V:(W* = 𝛽𝑃 ()* 𝑈 ( 𝑔 𝑔 *:( H* 𝑔 *:()* = 𝛽𝑈 ( 𝑔 𝑃 ()* *:( 𝛽 X 𝑈 T H* 𝑃 ()* H* 𝑔 *:()* = 𝑔 *:( Algorithm: forward pass computes f t , backward pass foes f i , b i
Country Dance Algorithm Can avoid storing all forward messages in smoothing by running forward algorithm backwards: V:(W* = 𝛽𝑃 ()* 𝑈 ( 𝑔 𝑔 *:( H* 𝑔 *:()* = 𝛽𝑈 ( 𝑔 𝑃 ()* *:( 𝛽 X 𝑈 T H* 𝑃 ()* H* 𝑔 *:()* = 𝑔 *:( Algorithm: forward pass computes f t , backward pass foes f i , b i
Country Dance Algorithm Can avoid storing all forward messages in smoothing by running forward algorithm backwards: V:(W* = 𝛽𝑃 ()* 𝑈 ( 𝑔 𝑔 *:( H* 𝑔 *:()* = 𝛽𝑈 ( 𝑔 𝑃 ()* *:( 𝛽 X 𝑈 T H* 𝑃 ()* H* 𝑔 *:()* = 𝑔 *:( Algorithm: forward pass computes f t , backward pass foes f i , b i
Country Dance Algorithm Can avoid storing all forward messages in smoothing by running forward algorithm backwards: V:(W* = 𝛽𝑃 ()* 𝑈 ( 𝑔 𝑔 *:( H* 𝑔 *:()* = 𝛽𝑈 ( 𝑔 𝑃 ()* *:( 𝛽 X 𝑈 T H* 𝑃 ()* H* 𝑔 *:()* = 𝑔 *:( Algorithm: forward pass computes f t , backward pass foes f i , b i
Country Dance Algorithm Can avoid storing all forward messages in smoothing by running forward algorithm backwards: V:(W* = 𝛽𝑃 ()* 𝑈 ( 𝑔 𝑔 *:( H* 𝑔 *:()* = 𝛽𝑈 ( 𝑔 𝑃 ()* *:( 𝛽 X 𝑈 T H* 𝑃 ()* H* 𝑔 *:()* = 𝑔 *:( Algorithm: forward pass computes f t , backward pass foes f i , b i
Country Dance Algorithm Can avoid storing all forward messages in smoothing by running forward algorithm backwards: V:(W* = 𝛽𝑃 ()* 𝑈 ( 𝑔 𝑔 *:( H* 𝑔 *:()* = 𝛽𝑈 ( 𝑔 𝑃 ()* *:( 𝛽 X 𝑈 T H* 𝑃 ()* H* 𝑔 *:()* = 𝑔 *:( Algorithm: forward pass computes f t , backward pass foes f i , b i
Recommend
More recommend
