em hidden markov models
play

EM & Hidden Markov Models CMSC 691 UMBC Recap from last time - PowerPoint PPT Presentation

Feb 26, 2024 β€’593 likes β€’1.72k views

EM & Hidden Markov Models CMSC 691 UMBC Recap from last time Expectation Maximization (EM) 0. Assume some value for your parameters Two step, iterative algorithm 1. E-step: count under uncertainty, assuming these parameters 2. M-step:

  1. 2 State HMM Likelihood π‘ž π‘Š| start π‘ž π‘Š| π‘Š π‘ž π‘Š| π‘Š π‘ž π‘Š| π‘Š … z 1 = z 2 = z 3 = z 4 = V V V V π‘ž π‘Š| 𝑂 π‘ž π‘Š| 𝑂 π‘ž π‘Š| 𝑂 π‘ž 𝑂| π‘Š π‘ž 𝑂| π‘Š π‘ž 𝑂| π‘Š π‘ž 𝑂| start π‘ž 𝑂| 𝑂 π‘ž 𝑂| 𝑂 π‘ž 𝑂| 𝑂 … z 1 = z 2 = z 3 = z 4 = N N N N π‘ž π‘₯ 3 |π‘Š π‘ž π‘₯ 4 |π‘Š π‘ž π‘₯ 1 |π‘Š π‘ž π‘₯ 2 |π‘Š π‘ž π‘₯ 4 |𝑂 π‘ž π‘₯ 1 |𝑂 π‘ž π‘₯ 2 |𝑂 π‘ž π‘₯ 3 |𝑂 w 1 w 2 w 3 w 4 N V end w 1 w 2 w 3 w 4 start .7 .2 .1 N .7 .2 .05 .05 N .15 .8 .05 V .2 .6 .1 .1 V .6 .35 .05

  2. 2 State HMM Likelihood π‘ž π‘Š| start π‘ž π‘Š| π‘Š π‘ž π‘Š| π‘Š π‘ž π‘Š| π‘Š … z 1 = z 2 = z 3 = z 4 = V V V V π‘ž π‘Š| 𝑂 π‘ž π‘Š| 𝑂 π‘ž π‘Š| 𝑂 π‘ž 𝑂| π‘Š π‘ž 𝑂| π‘Š π‘ž 𝑂| π‘Š π‘ž 𝑂| start π‘ž 𝑂| 𝑂 π‘ž 𝑂| 𝑂 π‘ž 𝑂| 𝑂 … z 1 = z 2 = z 3 = z 4 = N N N N π‘ž π‘₯ 3 |π‘Š π‘ž π‘₯ 4 |π‘Š π‘ž π‘₯ 1 |π‘Š π‘ž π‘₯ 2 |π‘Š π‘ž π‘₯ 4 |𝑂 π‘ž π‘₯ 1 |𝑂 π‘ž π‘₯ 2 |𝑂 π‘ž π‘₯ 3 |𝑂 w 1 w 2 w 3 w 4 Q: What’s the probability of N V end (N, w 1 ), (V, w 2 ), (V, w 3 ), (N, w 4 )? w 1 w 2 w 3 w 4 start .7 .2 .1 N .7 .2 .05 .05 N .15 .8 .05 V .2 .6 .1 .1 V .6 .35 .05

  3. 2 State HMM Likelihood π‘ž π‘Š| π‘Š z 1 = z 2 = z 3 = z 4 = V V V V π‘ž π‘Š| 𝑂 π‘ž 𝑂| π‘Š π‘ž 𝑂| start z 1 = z 2 = z 3 = z 4 = N N N N π‘ž π‘₯ 3 |π‘Š π‘ž π‘₯ 2 |π‘Š π‘ž π‘₯ 4 |𝑂 π‘ž π‘₯ 1 |𝑂 w 1 w 2 w 3 w 4 Q: What’s the probability of N V end (N, w 1 ), (V, w 2 ), (V, w 3 ), (N, w 4 )? w 1 w 2 w 3 w 4 start .7 .2 .1 N .7 .2 .05 .05 A: (.7*.7) * (.8*.6) * (.35*.1) * (.6*.05) = N .15 .8 .05 V .2 .6 .1 .1 0.0002822 V .6 .35 .05

  4. 2 State HMM Likelihood π‘ž π‘Š| start π‘ž π‘Š| π‘Š π‘ž π‘Š| π‘Š π‘ž π‘Š| π‘Š … z 1 = z 2 = z 3 = z 4 = V V V V π‘ž π‘Š| 𝑂 π‘ž π‘Š| 𝑂 π‘ž π‘Š| 𝑂 π‘ž 𝑂| π‘Š π‘ž 𝑂| π‘Š π‘ž 𝑂| π‘Š π‘ž 𝑂| start π‘ž 𝑂| 𝑂 π‘ž 𝑂| 𝑂 π‘ž 𝑂| 𝑂 … z 1 = z 2 = z 3 = z 4 = N N N N π‘ž π‘₯ 3 |π‘Š π‘ž π‘₯ 4 |π‘Š π‘ž π‘₯ 1 |π‘Š π‘ž π‘₯ 2 |π‘Š π‘ž π‘₯ 4 |𝑂 π‘ž π‘₯ 1 |𝑂 π‘ž π‘₯ 2 |𝑂 π‘ž π‘₯ 3 |𝑂 w 1 w 2 w 3 w 4 Q: What’s the probability of N V end (N, w 1 ), (V, w 2 ), (N, w 3 ), (N, w 4 )? w 1 w 2 w 3 w 4 start .7 .2 .1 N .7 .2 .05 .05 N .15 .8 .05 V .2 .6 .1 .1 V .6 .35 .05

  5. 2 State HMM Likelihood z 1 = z 2 = z 3 = z 4 = V V V V π‘ž π‘Š| 𝑂 π‘ž 𝑂| π‘Š π‘ž 𝑂| start π‘ž 𝑂| 𝑂 z 1 = z 2 = z 3 = z 4 = N N N N π‘ž π‘₯ 2 |π‘Š π‘ž π‘₯ 4 |𝑂 π‘ž π‘₯ 1 |𝑂 π‘ž π‘₯ 3 |𝑂 w 1 w 2 w 3 w 4 N V end Q: What’s the probability of w 1 w 2 w 3 w 4 (N, w 1 ), (V, w 2 ), (N, w 3 ), (N, w 4 )? start .7 .2 .1 N .7 .2 .05 .05 A: (.7*.7) * (.8*.6) * (.6*.05) * (.15*.05) = N .15 .8 .05 V .2 .6 .1 .1 0.00007056 V .6 .35 .05

  6. Agenda HMM Detailed Definition HMM Parameter Estimation EM for HMMs General Approach Expectation Calculation

  7. Estimating Parameters from Observed Data π‘ž π‘Š| π‘Š z 1 = z 2 = z 3 = z 4 = π‘ž 𝑂| π‘Š V V V V Transition Counts π‘ž 𝑂| start π‘ž π‘Š| 𝑂 N V end z 1 = z 2 = z 3 = z 4 = N N N N start π‘ž π‘₯ 4 |𝑂 N π‘ž π‘₯ 1 |𝑂 π‘ž π‘₯ 3 |π‘Š π‘ž π‘₯ 2 |π‘Š V w 1 w 2 w 3 w 4 Emission Counts w 1 w 2 W 3 w 4 z 1 = z 2 = z 3 = z 4 = N V V V V V π‘ž π‘Š| 𝑂 π‘ž 𝑂| π‘Š π‘ž 𝑂| start end emission not shown π‘ž 𝑂| 𝑂 z 1 = z 2 = z 3 = z 4 = N N N N π‘ž π‘₯ 2 |π‘Š π‘ž π‘₯ 4 |𝑂 π‘ž π‘₯ 1 |𝑂 π‘ž π‘₯ 3 |𝑂 w 1 w 2 w 3 w 4

  8. Estimating Parameters from Observed Data π‘ž π‘Š| π‘Š z 1 = z 2 = z 3 = z 4 = π‘ž 𝑂| π‘Š V V V V Transition Counts π‘ž 𝑂| start π‘ž π‘Š| 𝑂 N V end z 1 = z 2 = z 3 = z 4 = N N N N start 2 0 0 π‘ž π‘₯ 4 |𝑂 N 1 2 2 π‘ž π‘₯ 1 |𝑂 π‘ž π‘₯ 3 |π‘Š π‘ž π‘₯ 2 |π‘Š V 2 1 0 w 1 w 2 w 3 w 4 Emission Counts w 1 w 2 W 3 w 4 z 1 = z 2 = z 3 = z 4 = N 2 0 1 2 V V V V V 0 2 1 0 π‘ž π‘Š| 𝑂 π‘ž 𝑂| π‘Š π‘ž 𝑂| start end emission not shown π‘ž 𝑂| 𝑂 z 1 = z 2 = z 3 = z 4 = N N N N π‘ž π‘₯ 2 |π‘Š π‘ž π‘₯ 4 |𝑂 π‘ž π‘₯ 1 |𝑂 π‘ž π‘₯ 3 |𝑂 w 1 w 2 w 3 w 4

  9. Estimating Parameters from Observed Data π‘ž π‘Š| π‘Š z 1 = z 2 = z 3 = z 4 = π‘ž 𝑂| π‘Š V V V V Transition MLE π‘ž 𝑂| start π‘ž π‘Š| 𝑂 N V end z 1 = z 2 = z 3 = z 4 = N N N N start 1 0 0 π‘ž π‘₯ 4 |𝑂 N .2 .4 .4 π‘ž π‘₯ 1 |𝑂 π‘ž π‘₯ 3 |π‘Š π‘ž π‘₯ 2 |π‘Š V 2/3 1/3 0 w 1 w 2 w 3 w 4 Emission MLE w 1 w 2 W 3 w 4 z 1 = z 2 = z 3 = z 4 = N .4 0 .2 .4 V V V V V 0 2/3 1/3 0 π‘ž π‘Š| 𝑂 π‘ž 𝑂| π‘Š π‘ž 𝑂| start end emission not shown π‘ž 𝑂| 𝑂 z 1 = z 2 = z 3 = z 4 = N N N N π‘ž π‘₯ 2 |π‘Š π‘ž π‘₯ 4 |𝑂 π‘ž π‘₯ 1 |𝑂 π‘ž π‘₯ 3 |𝑂 w 1 w 2 w 3 w 4

  10. Estimating Parameters from Observed Data π‘ž π‘Š| π‘Š z 1 = z 2 = z 3 = z 4 = π‘ž 𝑂| π‘Š V V V V Transition MLE π‘ž 𝑂| start π‘ž π‘Š| 𝑂 N V end z 1 = z 2 = z 3 = z 4 = N N N N start 1 0 0 π‘ž π‘₯ 4 |𝑂 N .2 .4 .4 π‘ž π‘₯ 1 |𝑂 π‘ž π‘₯ 3 |π‘Š π‘ž π‘₯ 2 |π‘Š V 2/3 1/3 0 w 1 w 2 w 3 w 4 Emission MLE w 1 w 2 W 3 w 4 z 1 = z 2 = z 3 = z 4 = N .4 0 .2 .4 V V V V V 0 2/3 1/3 0 π‘ž π‘Š| 𝑂 π‘ž 𝑂| π‘Š π‘ž 𝑂| start end emission not shown π‘ž 𝑂| 𝑂 z 1 = z 2 = z 3 = z 4 = N N N N smooth these π‘ž π‘₯ 2 |π‘Š values if π‘ž π‘₯ 4 |𝑂 π‘ž π‘₯ 1 |𝑂 π‘ž π‘₯ 3 |𝑂 needed w 1 w 2 w 3 w 4

  11. What If We Don’t Observe 𝑨 ? Approach: Develop EM algorithm Goal: Estimate π‘ž 𝑒 𝑑 β€² 𝑑) and π‘ž 𝑓 𝑀 𝑑) Why: Compute 𝔽 𝑨 𝑗 =𝑑→𝑨 𝑗+1 =𝑑 β€² 𝑑 𝑑 β†’ 𝑑 β€² 𝔽 𝑨 𝑗 =𝑑→π‘₯ 𝑗 =𝑀 𝑑 𝑑 β†’ 𝑀

  12. Expectation Maximization (EM) 0. Assume some value for your parameters Two step, iterative algorithm 1. E-step: count under uncertainty, assuming these parameters 2. M-step: maximize log-likelihood, assuming these uncertain counts estimated counts

  13. Expectation Maximization (EM) p obs (w | s) 0. Assume some value for your parameters p trans (s’ | s) Two step, iterative algorithm 1. E-step: count under uncertainty, assuming these parameters 2. M-step: maximize log-likelihood, assuming these uncertain counts estimated counts

  14. Expectation Maximization (EM) p obs (w | s) 0. Assume some value for your parameters p trans (s’ | s) Two step, iterative algorithm 1. E-step: count under uncertainty, assuming these parameters π‘ž βˆ— 𝑨 𝑗 = 𝑑, 𝑨 𝑗+1 = 𝑑 β€² π‘₯ 1 , β‹― , π‘₯ 𝑂 ) = π‘ž βˆ— 𝑨 𝑗 = 𝑑 π‘₯ 1 , β‹― , π‘₯ 𝑂 ) = π‘ž(𝑨 𝑗 = 𝑑, π‘₯ 1 , β‹― , π‘₯ 𝑂 ) π‘ž(𝑨 𝑗 = 𝑑, 𝑨 𝑗+1 = 𝑑 β€² , π‘₯ 1 , β‹― , π‘₯ 𝑂 ) π‘ž(π‘₯ 1 , β‹― , π‘₯ 𝑂 ) π‘ž(π‘₯ 1 , β‹― , π‘₯ 𝑂 ) 2. M-step: maximize log-likelihood, assuming these uncertain counts estimated counts

  15. M-Step β€œ maximize log-likelihood, assuming these uncertain counts ” 𝑑(𝑑 β†’ 𝑑 β€² ) π‘ž new 𝑑 β€² 𝑑) = Οƒ 𝑑 β€²β€² 𝑑(𝑑 β†’ 𝑑 β€²β€² ) if we observed the hidden transitions…

  16. M-Step β€œ maximize log-likelihood, assuming these uncertain counts ” 𝔽 𝑑→𝑑 β€² [𝑑 𝑑 β†’ 𝑑 β€² ] π‘ž new 𝑑 β€² 𝑑) = Οƒ 𝑑 β€²β€² 𝔽 𝑑→𝑑 β€²β€² [𝑑 𝑑 β†’ 𝑑′′ ] we don’t observe the hidden transitions, but we can approximately count

  17. M-Step β€œ maximize log-likelihood, assuming these uncertain counts ” 𝔽 𝑑→𝑑 β€² [𝑑 𝑑 β†’ 𝑑 β€² ] π‘ž new 𝑑 β€² 𝑑) = Οƒ 𝑑 β€²β€² 𝔽 𝑑→𝑑 β€²β€² [𝑑 𝑑 β†’ 𝑑′′ ] we don’t observe the hidden transitions, but we can approximately count we compute these in the E-step

  18. Expectation Maximization (EM) p obs (w | s) 0. Assume some value for your parameters p trans (s’ | s) Two step, iterative algorithm 1. E-step: count under uncertainty, assuming these parameters π‘ž βˆ— 𝑨 𝑗 = 𝑑, 𝑨 𝑗+1 = 𝑑 β€² π‘₯ 1 , β‹― , π‘₯ 𝑂 ) = π‘ž βˆ— 𝑨 𝑗 = 𝑑 π‘₯ 1 , β‹― , π‘₯ 𝑂 ) = π‘ž(𝑨 𝑗 = 𝑑, π‘₯ 1 , β‹― , π‘₯ 𝑂 ) π‘ž(𝑨 𝑗 = 𝑑, 𝑨 𝑗+1 = 𝑑 β€² , π‘₯ 1 , β‹― , π‘₯ 𝑂 ) π‘ž(π‘₯ 1 , β‹― , π‘₯ 𝑂 ) π‘ž(π‘₯ 1 , β‹― , π‘₯ 𝑂 ) 2. M-step: maximize log-likelihood, assuming these uncertain counts Baum-Welch estimated counts

  19. Estimating Parameters from Unobserved Data Expected Transition Counts N V end βˆ— π‘Š| start βˆ— π‘Š| π‘Š βˆ— π‘Š| π‘Š βˆ— π‘Š| π‘Š π‘ž start π‘ž π‘ž π‘ž z 1 = z 2 = z 3 = z 4 = N V V V V βˆ— π‘Š| 𝑂 βˆ— π‘Š| 𝑂 V βˆ— π‘Š| 𝑂 βˆ— 𝑂| π‘Š π‘ž π‘ž π‘ž π‘ž βˆ— 𝑂| π‘Š βˆ— 𝑂| π‘Š π‘ž π‘ž Expected βˆ— 𝑂| start Emission Counts π‘ž βˆ— 𝑂| 𝑂 βˆ— 𝑂| 𝑂 βˆ— 𝑂| 𝑂 π‘ž π‘ž π‘ž z 1 = z 2 = z 3 = z 4 = w 1 w 2 W 3 w 4 N N N N N βˆ— π‘₯ 3 |π‘Š βˆ— π‘₯ 1 |π‘Š βˆ— π‘₯ 2 |π‘Š V π‘ž π‘ž π‘ž βˆ— π‘₯ 4 |𝑂 βˆ— π‘₯ 1 |𝑂 βˆ— π‘₯ 2 |𝑂 βˆ— π‘₯ 3 |𝑂 βˆ— π‘₯ 4 |π‘Š end emission not shown π‘ž π‘ž π‘ž π‘ž π‘ž w 1 w 2 w 3 w 4

  20. Estimating Parameters from Unobserved Data Expected all of these p* arcs are Transition Counts specific to a time-step N V end βˆ— π‘Š| start βˆ— π‘Š| π‘Š βˆ— π‘Š| π‘Š βˆ— π‘Š| π‘Š π‘ž start π‘ž π‘ž π‘ž z 1 = z 2 = z 3 = z 4 = N V V V V βˆ— π‘Š| 𝑂 βˆ— π‘Š| 𝑂 V βˆ— π‘Š| 𝑂 βˆ— 𝑂| π‘Š π‘ž π‘ž π‘ž π‘ž βˆ— 𝑂| π‘Š βˆ— 𝑂| π‘Š π‘ž π‘ž Expected βˆ— 𝑂| start Emission Counts π‘ž βˆ— 𝑂| 𝑂 βˆ— 𝑂| 𝑂 βˆ— 𝑂| 𝑂 π‘ž π‘ž π‘ž z 1 = z 2 = z 3 = z 4 = w 1 w 2 W 3 w 4 N N N N N βˆ— π‘₯ 3 |π‘Š βˆ— π‘₯ 1 |π‘Š βˆ— π‘₯ 2 |π‘Š V π‘ž π‘ž π‘ž βˆ— π‘₯ 4 |𝑂 βˆ— π‘₯ 1 |𝑂 βˆ— π‘₯ 2 |𝑂 βˆ— π‘₯ 3 |𝑂 βˆ— π‘₯ 4 |π‘Š end emission not shown π‘ž π‘ž π‘ž π‘ž π‘ž w 1 w 2 w 3 w 4

  21. Estimating Parameters from Unobserved Data all of these p* arcs are Expected specific to a time-step Transition Counts N V end βˆ— π‘Š| start βˆ— π‘Š| π‘Š βˆ— π‘Š| π‘Š βˆ— π‘Š| π‘Š π‘ž start π‘ž π‘ž π‘ž z 1 = z 2 = z 3 = z 4 = =.5 =.3 =.3 N V V V V βˆ— π‘Š| 𝑂 βˆ— π‘Š| 𝑂 V βˆ— π‘Š| 𝑂 βˆ— 𝑂| π‘Š π‘ž π‘ž π‘ž π‘ž βˆ— 𝑂| π‘Š βˆ— 𝑂| π‘Š π‘ž π‘ž Expected βˆ— 𝑂| start Emission Counts π‘ž βˆ— 𝑂| 𝑂 βˆ— 𝑂| 𝑂 βˆ— 𝑂| 𝑂 π‘ž π‘ž π‘ž z 1 = z 2 = z 3 = z 4 = w 1 w 2 W 3 w 4 =.4 =.6 =.5 N N N N N βˆ— π‘₯ 3 |π‘Š βˆ— π‘₯ 1 |π‘Š βˆ— π‘₯ 2 |π‘Š V π‘ž π‘ž π‘ž βˆ— π‘₯ 4 |𝑂 βˆ— π‘₯ 1 |𝑂 βˆ— π‘₯ 2 |𝑂 βˆ— π‘₯ 3 |𝑂 βˆ— π‘₯ 4 |π‘Š end emission not shown π‘ž π‘ž π‘ž π‘ž π‘ž w 1 w 2 w 3 w 4

  22. Estimating Parameters from Unobserved Data all of these p* arcs are Expected specific to a time-step Transition Counts N V end βˆ— π‘Š| start βˆ— π‘Š| π‘Š βˆ— π‘Š| π‘Š βˆ— π‘Š| π‘Š βˆ— π‘Š| π‘Š βˆ— π‘Š| π‘Š βˆ— π‘Š| π‘Š π‘ž start π‘ž π‘ž π‘ž π‘ž π‘ž π‘ž z 1 = z 2 = z 3 = z 4 = =.5 =.3 =.3 N 1.5 V V V V βˆ— π‘Š| 𝑂 βˆ— π‘Š| 𝑂 V 1.1 βˆ— π‘Š| 𝑂 βˆ— 𝑂| π‘Š π‘ž π‘ž π‘ž π‘ž βˆ— 𝑂| π‘Š βˆ— 𝑂| π‘Š π‘ž π‘ž Expected βˆ— 𝑂| start Emission Counts π‘ž βˆ— 𝑂| 𝑂 βˆ— 𝑂| 𝑂 βˆ— 𝑂| 𝑂 βˆ— 𝑂| 𝑂 βˆ— 𝑂| 𝑂 βˆ— 𝑂| 𝑂 π‘ž π‘ž π‘ž π‘ž π‘ž π‘ž z 1 = z 2 = z 3 = z 4 = w 1 w 2 W 3 w 4 =.4 =.6 =.5 N N N N N βˆ— π‘₯ 3 |π‘Š βˆ— π‘₯ 1 |π‘Š βˆ— π‘₯ 2 |π‘Š V π‘ž π‘ž π‘ž βˆ— π‘₯ 4 |𝑂 βˆ— π‘₯ 1 |𝑂 βˆ— π‘₯ 2 |𝑂 βˆ— π‘₯ 3 |𝑂 βˆ— π‘₯ 4 |π‘Š end emission not shown π‘ž π‘ž π‘ž π‘ž π‘ž w 1 w 2 w 3 w 4

  23. Estimating Parameters from Unobserved Data Expected Transition Counts N V end βˆ— π‘Š| start βˆ— π‘Š| π‘Š βˆ— π‘Š| π‘Š βˆ— π‘Š| π‘Š π‘ž start 1.8 .1 .1 π‘ž π‘ž π‘ž z 1 = z 2 = z 3 = z 4 = N 1.5 .8 .1 V V V V βˆ— π‘Š| 𝑂 βˆ— π‘Š| 𝑂 V 1.4 1.1 .4 βˆ— π‘Š| 𝑂 βˆ— 𝑂| π‘Š π‘ž π‘ž π‘ž π‘ž βˆ— 𝑂| π‘Š βˆ— 𝑂| π‘Š π‘ž π‘ž Expected βˆ— 𝑂| start Emission Counts π‘ž βˆ— 𝑂| 𝑂 βˆ— 𝑂| 𝑂 βˆ— 𝑂| 𝑂 π‘ž π‘ž π‘ž z 1 = z 2 = z 3 = z 4 = w 1 w 2 W 3 w 4 N N N N N .4 .3 .2 .2 βˆ— π‘₯ 3 |π‘Š βˆ— π‘₯ 1 |π‘Š βˆ— π‘₯ 2 |π‘Š V .1 .6 .3 .3 π‘ž π‘ž π‘ž βˆ— π‘₯ 4 |𝑂 βˆ— π‘₯ 1 |𝑂 βˆ— π‘₯ 2 |𝑂 βˆ— π‘₯ 3 |𝑂 βˆ— π‘₯ 4 |π‘Š end emission not shown π‘ž π‘ž π‘ž π‘ž π‘ž (these numbers are made up) w 1 w 2 w 3 w 4

  24. Estimating Parameters from Unobserved Data Expected Transition MLE N V end start 1.8/2 .1/2 .1/2 N 1.5/ .8/ .1/ βˆ— π‘Š| start βˆ— π‘Š| π‘Š βˆ— π‘Š| π‘Š βˆ— π‘Š| π‘Š π‘ž π‘ž π‘ž π‘ž 2.4 2.4 2.4 z 1 = z 2 = z 3 = z 4 = V 1.4/2.9 1.1/ .4/ V V V V 2.9 2.9 βˆ— π‘Š| 𝑂 βˆ— π‘Š| 𝑂 βˆ— π‘Š| 𝑂 βˆ— 𝑂| π‘Š π‘ž π‘ž π‘ž π‘ž βˆ— 𝑂| π‘Š βˆ— 𝑂| π‘Š π‘ž π‘ž Expected βˆ— 𝑂| start Emission MLE π‘ž βˆ— 𝑂| 𝑂 βˆ— 𝑂| 𝑂 βˆ— 𝑂| 𝑂 π‘ž π‘ž π‘ž z 1 = z 2 = z 3 = z 4 = w 1 w 2 W 3 w 4 N N N N N .4/ .3/ .2/ .2/ 1.1 1.1 1.1 1.1 βˆ— π‘₯ 3 |π‘Š βˆ— π‘₯ 1 |π‘Š βˆ— π‘₯ 2 |π‘Š π‘ž π‘ž π‘ž V .1/ .6/ .3/ .3/ βˆ— π‘₯ 4 |𝑂 βˆ— π‘₯ 1 |𝑂 βˆ— π‘₯ 2 |𝑂 βˆ— π‘₯ 3 |𝑂 1.3 1.3 1.3 1.3 π‘ž π‘ž π‘ž π‘ž βˆ— π‘₯ 4 |π‘Š π‘ž end emission not shown (these numbers are made up) w 1 w 2 w 3 w 4

  25. Semi-Supervised Parameter Estimation Transition Counts Emission Counts N V end w 1 w 2 W 3 w 4 start 2 0 0 N 2 0 1 2 N 1 2 2 V 0 2 1 0 V 2 1 0

  26. Semi-Supervised Parameter Estimation Transition Counts Emission Counts N V end w 1 w 2 W 3 w 4 start 2 0 0 N 2 0 1 2 N 1 2 2 V 0 2 1 0 V 2 1 0 Expected Transition Counts Expected Emission Counts N V end w 1 w 2 W 3 w 4 start 1.8 .1 .1 N .4 .3 .2 .2 N 1.5 .8 .1 V .1 .6 .3 .3 V 1.4 1.1 .4

  27. Semi-Supervised Parameter Estimation Transition Counts Emission Counts N V end w 1 w 2 W 3 w 4 start 2 0 0 N 2 0 1 2 N 1 2 2 V 0 2 1 0 V 2 1 0 Expected Transition Counts Expected Emission Counts N V end w 1 w 2 W 3 w 4 start 1.8 .1 .1 N .4 .3 .2 .2 N 1.5 .8 .1 V .1 .6 .3 .3 V 1.4 1.1 .4

  28. Semi-Supervised Parameter Estimation Transition Counts Emission Counts N V end w 1 w 2 W 3 w 4 start 2 0 0 N 2 0 1 2 N 1 2 2 V 0 2 1 0 V 2 1 0 Mixed Transition Counts Mixed Emission Counts N V end w 1 w 2 W 3 w 4 start 3.8 .1 .1 N 2.4 .3 1.2 2.2 N 2.5 2.8 2.1 V .1 2.6 1.3 .3 V 3.4 2.1 .4 Expected Transition Counts Expected Emission Counts N V end w 1 w 2 W 3 w 4 start 1.8 .1 .1 N .4 .3 .2 .2 N 1.5 .8 .1 V .1 .6 .3 .3 V 1.4 1.1 .4

  29. Agenda HMM Detailed Definition HMM Parameter Estimation EM for HMMs General Approach Expectation Calculation

  30. EM Math maximize the average log-likelihood of our complete data (z, w), averaged across all z and according to how likely our current model thinks z is max 𝔽 𝑨 ~ π‘ž πœ„ (𝑒) (β‹…|π‘₯) log π‘ž πœ„ (𝑨, π‘₯) current parameters πœ„ new parameters posterior distribution new parameters

  31. EM Math maximize the average log-likelihood of our complete data (z, w), averaged across all z and according to how likely our current model thinks z is max 𝔽 𝑨 ~ π‘ž πœ„ (𝑒) (β‹…|π‘₯) log π‘ž πœ„ (𝑨, π‘₯) current parameters πœ„ new parameters posterior distribution new parameters 𝑨 ∈ 𝑑 1 , … , 𝑑 𝐿 𝑂 ෍ log π‘ž πœ„ (𝑨 𝑗 |𝑨 π‘—βˆ’1 ) + log π‘ž πœ„ (π‘₯ 𝑗 |𝑨 𝑗 ) 𝑗

  32. Estimating Parameters from Unobserved Data Expected Transition MLE N V end start 1.8/2 .1/2 .1/2 N 1.5/ .8/ .1/ βˆ— π‘Š| start βˆ— π‘Š| π‘Š βˆ— π‘Š| π‘Š βˆ— π‘Š| π‘Š π‘ž π‘ž π‘ž π‘ž 2.4 2.4 2.4 z 1 = z 2 = z 3 = z 4 = V 1.4/2.9 1.1/ .4/ V V V V 2.9 2.9 βˆ— π‘Š| 𝑂 βˆ— π‘Š| 𝑂 βˆ— π‘Š| 𝑂 βˆ— 𝑂| π‘Š π‘ž π‘ž π‘ž π‘ž βˆ— 𝑂| π‘Š βˆ— 𝑂| π‘Š π‘ž π‘ž Expected βˆ— 𝑂| start Emission MLE π‘ž βˆ— 𝑂| 𝑂 βˆ— 𝑂| 𝑂 βˆ— 𝑂| 𝑂 π‘ž π‘ž π‘ž z 1 = z 2 = z 3 = z 4 = w 1 w 2 W 3 w 4 N N N N N .4/ .3/ .2/ .2/ 1.1 1.1 1.1 1.1 βˆ— π‘₯ 3 |π‘Š βˆ— π‘₯ 1 |π‘Š βˆ— π‘₯ 2 |π‘Š π‘ž π‘ž π‘ž V .1/ .6/ .3/ .3/ βˆ— π‘₯ 4 |𝑂 βˆ— π‘₯ 1 |𝑂 βˆ— π‘₯ 2 |𝑂 βˆ— π‘₯ 3 |𝑂 1.3 1.3 1.3 1.3 π‘ž π‘ž π‘ž π‘ž βˆ— π‘₯ 4 |π‘Š π‘ž end emission not shown (these numbers are made up) w 1 w 2 w 3 w 4

  33. EM For HMMs (Baum-Welch Algorithm) L = π‘ž(π‘₯ 1 , β‹― , π‘₯ 𝑂 ) for(i = 1; i ≀ N; ++ i) { for(state = 0; state < K*; ++state) { π‘ž(𝑨 𝑗 = state ,π‘₯ 1 ,β‹―,π‘₯ 𝑂 ) c obs (obs i | state) += 𝑀 for(prev = 0; prev < K*; ++prev) { π‘ž(𝑨 𝑗 = state ,𝑨 𝑗+1 = next ,π‘₯ 1 ,β‹―,π‘₯ 𝑂 ) c trans (state | prev) += 𝑀 } } }

  34. EM For HMMs (Baum-Welch L = π‘ž(π‘₯ 1 , β‹― , π‘₯ 𝑂 ) Algorithm) for(i = 1; i ≀ N; ++i) { for(state = 0; state < K*; ++state) { c obs (obs i | state) += π‘ž 𝑨 𝑗 = state ,π‘₯ 1 ,…,π‘₯ 𝑗 = obs i π‘ž π‘₯ 𝑗+1:𝑂 𝑨 𝑗 = state ) 𝑀 for(prev = 0; prev < K*; ++prev) { u = p obs (obs i | state) * p trans (state | prev) c trans (state | prev) += π‘ž 𝑨 π‘—βˆ’1 = prev ,π‘₯ 1:π‘—βˆ’1 βˆ—π‘£βˆ—π‘ž π‘₯ 𝑗+1:𝑂 𝑨 𝑗 = state ) 𝑀 } } }

  35. EM For HMMs L = π‘ž(π‘₯ 1 , β‹― , π‘₯ 𝑂 ) (Baum-Welch for(i = 1; i ≀ N; ++i) { for(state = 0; state < K*; ++state) { Algorithm) c obs (obs i | state) += 𝛽( state , 𝑗) 𝛾( state , 𝑗) π‘ž 𝑨 𝑗 = state ,π‘₯ 1 ,…,π‘₯ 𝑗 = obs i π‘ž π‘₯ 𝑗+1:𝑂 𝑨 𝑗 = state ) 𝑀 for(prev = 0; prev < K*; ++prev) { u = p obs (obs i | state) * p trans (state | prev) c trans (state | prev) += 𝛽( prev , 𝑗 βˆ’ 1) 𝛾( state , 𝑗) π‘ž 𝑨 π‘—βˆ’1 = prev ,π‘₯ 1:π‘—βˆ’1 βˆ—π‘£βˆ—π‘ž π‘₯ 𝑗+1:𝑂 𝑨 𝑗 = state ) 𝑀 } } }

  36. Why Do We Need Backward Values? z i-1 z i+1 z i = = A = A A z i-1 z i+1 z i = = B = B B z i-1 z i+1 z i = = C = C C Ξ²( i, s ) is the total probability of all paths: Ξ±( i, s ) is the total probability of all paths: 1. that start at step i at state s 1. that start from the beginning 2. that terminate at the end 2. that end (currently) in s at step i 3. (that emit the observation obs at i+1) 3. that emit the observation obs at i

  37. Why Do We Need Backward Values? z i-1 z i+1 z i = = A = A A z i-1 z i+1 z i = = B = B B z i-1 z i+1 z i = = C = C C Ξ±( i, B ) Ξ²( i, B ) Ξ²( i, s ) is the total probability of all paths: Ξ±( i, s ) is the total probability of all paths: 1. that start at step i at state s 1. that start from the beginning 2. that terminate at the end 2. that end (currently) in s at step i 3. (that emit the observation obs at i+1) 3. that emit the observation obs at i

  38. Why Do We Need Backward Values? z i-1 z i+1 z i = = A = A A z i-1 z i+1 z i = = B = B B z i-1 z i+1 z i = = C = C C Ξ±( i, B ) Ξ²( i, B ) Ξ±( i, B ) * Ξ²( i, B ) = total probability of paths through state B at step i Ξ±( i, s ) is the total probability of all paths: Ξ²( i, s ) is the total probability of all paths: 1. that start from the beginning 1. that start at step i at state s 2. that end (currently) in s at step i 2. that terminate at the end 3. (that emit the observation obs at i+1) 3. that emit the observation obs at i

  39. Why Do We Need Backward Values? z i-1 z i+1 z i = = A = A A z i-1 z i+1 z i = = B = B B we can compute posterior state z i-1 z i+1 z i = probabilities = C = C C (normalize by marginal likelihood) Ξ±( i, B ) Ξ²( i, B ) Ξ±( i, s ) * Ξ²( i, s ) = total probability of paths through state s at step i Ξ±( i, s ) is the total probability of all paths: Ξ²( i, s ) is the total probability of all paths: 1. that start from the beginning 1. that start at step i at state s 2. that end (currently) in s at step i 2. that terminate at the end 3. (that emit the observation obs at i+1) 3. that emit the observation obs at i

  40. Why Do We Need Backward Values? z i-1 z i+1 z i = = A = A A z i-1 z i+1 z i = = B = B B z i-1 z i+1 z i = = C = C C Ξ±( i, B ) Ξ²( i+1, s ) Ξ²( i, s ) is the total probability of all paths: Ξ±( i, s ) is the total probability of all paths: 1. that start at step i at state s 1. that start from the beginning 2. that terminate at the end 2. that end (currently) in s at step i 3. (that emit the observation obs at i+1) 3. that emit the observation obs at i

  41. Why Do We Need Backward Values? z i-1 z i+1 z i = = A = A A z i-1 z i+1 z i = = B = B B z i-1 z i+1 z i = = C = C C Ξ±( i, B ) Ξ²( i+1, s’ ) Ξ±( i, B ) * p( s’ | B) * p(obs at i+1 | s’) * Ξ²( i+1, s’ ) = total probability of paths through the B β†’ s’ arc (at time i) Ξ±( i, s ) is the total probability of all paths: Ξ²( i, s ) is the total probability of all paths: 1. that start from the beginning 1. that start at step i at state s 2. that end (currently) in s at step i 2. that terminate at the end 3. (that emit the observation obs at i+1) 3. that emit the observation obs at i

  42. Why Do We Need Backward Values? z i-1 z i+1 z i = = A = A A we can compute z i-1 z i+1 z i = posterior transition = B = B B probabilities (normalize by z i-1 z i+1 z i = marginal likelihood) = C = C C Ξ±( i, B ) Ξ²( i+1, s’ ) Ξ±( i, B ) * p( s’ | B) * p(obs at i+1 | s’) * Ξ²( i+1, s’ ) = total probability of paths through the B β†’ s’ arc (at time i) Ξ±( i, s ) is the total probability of all paths: Ξ²( i, s ) is the total probability of all paths: 1. that start from the beginning 1. that start at step i at state s 2. that end (currently) in s at step i 2. that terminate at the end 3. (that emit the observation obs at i+1) 3. that emit the observation obs at i

  43. With Both Forward and Backward Values Ξ±( i, s ) * Ξ²( i, s) = total probability of paths through state s at step i Ξ±( i, s) * p( s’ | B) * p(obs at i+1 | s’) * Ξ²( i+1, s’ ) = total probability of paths through the s β†’ s ’ arc (at time i)

  44. With Both Forward and Backward Values Ξ±( i, s ) * Ξ²( i, s) = total probability of paths through state s at step i π‘ž 𝑨 𝑗 = 𝑑 π‘₯ 1 , β‹― , π‘₯ 𝑂 ) = 𝛽 𝑗, 𝑑 βˆ— 𝛾(𝑗, 𝑑) 𝛽(𝑂 + 1, END ) Ξ±( i, s) * p( s’ | B) * p(obs at i+1 | s’) * Ξ²( i+1, s’ ) = total probability of paths through the s β†’ s ’ arc (at time i)

  45. With Both Forward and Backward Values Ξ±( i, s ) * Ξ²( i, s) = total probability of paths through state s at step i π‘ž 𝑨 𝑗 = 𝑑 π‘₯ 1 , β‹― , π‘₯ 𝑂 ) = 𝛽 𝑗, 𝑑 βˆ— 𝛾(𝑗, 𝑑) 𝛽(𝑂 + 1, END ) Ξ±( i, s) * p( s’ | B) * p(obs at i+1 | s’) * Ξ²( i+1, s’ ) = total probability of paths through the s β†’ s ’ arc (at time i) π‘ž 𝑨 𝑗 = 𝑑, 𝑨 𝑗+1 = 𝑑 β€² π‘₯ 1 , β‹― , π‘₯ 𝑂 ) = 𝛽 𝑗, 𝑑 βˆ— π‘ž 𝑑 β€² 𝑑 βˆ— π‘ž obs 𝑗+1 𝑑 β€² βˆ— 𝛾(𝑗 + 1, 𝑑′) 𝛽(𝑂 + 1, END )

  46. Agenda HMM Detailed Definition HMM Parameter Estimation EM for HMMs General Approach Expectation Calculation

  47. HMM Expectation Calculation π‘ž 𝑨 1 , π‘₯ 1 , 𝑨 2 , π‘₯ 2 , … , 𝑨 𝑂 , π‘₯ 𝑂 = π‘ž 𝑨 1 | 𝑨 0 π‘ž π‘₯ 1 |𝑨 1 β‹― π‘ž 𝑨 𝑂 | 𝑨 π‘‚βˆ’1 π‘ž π‘₯ 𝑂 |𝑨 𝑂 emission transition = ΰ·‘ π‘ž π‘₯ 𝑗 |𝑨 𝑗 π‘ž 𝑨 𝑗 | 𝑨 π‘—βˆ’1 probabilities/parameters probabilities/parameters 𝑗 Calculate the forward (log) likelihood of an observed (sub-)sequence w 1 , …, w J Calculate the backward (log) likelihood of an observed (sub-)sequence w J+1 , …, w N

  48. HMM Likelihood Task Marginalize over all latent sequence joint likelihoods π‘ž π‘₯ 1 , π‘₯ 2 , … , π‘₯ 𝑂 = ෍ π‘ž 𝑨 1 , π‘₯ 1 , 𝑨 2 , π‘₯ 2 , … , 𝑨 𝑂 , π‘₯ 𝑂 𝑨 1 ,β‹―,𝑨 𝑂 Q: In a K-state HMM for a length N observation sequence, how many summands (different latent sequences) are there?

  49. HMM Likelihood Task Marginalize over all latent sequence joint likelihoods π‘ž π‘₯ 1 , π‘₯ 2 , … , π‘₯ 𝑂 = ෍ π‘ž 𝑨 1 , π‘₯ 1 , 𝑨 2 , π‘₯ 2 , … , 𝑨 𝑂 , π‘₯ 𝑂 𝑨 1 ,β‹―,𝑨 𝑂 Q: In a K-state HMM for a length N observation sequence, how many summands (different latent sequences) are there? A: K N

  50. HMM Likelihood Task Marginalize over all latent sequence joint likelihoods π‘ž π‘₯ 1 , π‘₯ 2 , … , π‘₯ 𝑂 = ෍ π‘ž 𝑨 1 , π‘₯ 1 , 𝑨 2 , π‘₯ 2 , … , 𝑨 𝑂 , π‘₯ 𝑂 𝑨 1 ,β‹―,𝑨 𝑂 Q: In a K-state HMM for a length N observation sequence, how many summands (different latent sequences) are there? A: K N Goal: Find a way to compute this exponential sum efficiently (in polynomial time)

  51. 2 State HMM Likelihood π‘ž π‘Š| π‘Š z 1 = z 2 = z 3 = z 4 = π‘ž 𝑂| π‘Š V V V V π‘ž 𝑂| start π‘ž π‘Š| 𝑂 z 1 = z 2 = z 3 = z 4 = N N N N π‘ž π‘₯ 1 |𝑂 π‘ž π‘₯ 4 |𝑂 π‘ž π‘₯ 3 |π‘Š π‘ž π‘₯ 2 |π‘Š w 1 w 2 w 3 w 4 z 1 = z 2 = z 3 = z 4 = V V V V π‘ž π‘Š| 𝑂 π‘ž 𝑂| π‘Š π‘ž 𝑂| start π‘ž 𝑂| 𝑂 z 1 = z 2 = z 3 = z 4 = N N N N π‘ž π‘₯ 2 |π‘Š π‘ž π‘₯ 4 |𝑂 π‘ž π‘₯ 1 |𝑂 π‘ž π‘₯ 3 |𝑂 w 1 w 2 w 3 w 4

  52. 2 State HMM Likelihood π‘ž π‘Š| π‘Š z 1 = z 2 = z 3 = z 4 = π‘ž 𝑂| π‘Š V V V V π‘ž 𝑂| start π‘ž π‘Š| 𝑂 z 1 = z 2 = z 3 = z 4 = N N N N π‘ž π‘₯ 1 |𝑂 π‘ž π‘₯ 4 |𝑂 π‘ž π‘₯ 3 |π‘Š π‘ž π‘₯ 2 |π‘Š w 1 w 2 w 3 w 4 Up until here, all the computation was the same z 1 = z 2 = z 3 = z 4 = V V V V π‘ž π‘Š| 𝑂 π‘ž 𝑂| π‘Š π‘ž 𝑂| start π‘ž 𝑂| 𝑂 z 1 = z 2 = z 3 = z 4 = N N N N π‘ž π‘₯ 2 |π‘Š π‘ž π‘₯ 4 |𝑂 π‘ž π‘₯ 1 |𝑂 π‘ž π‘₯ 3 |𝑂 w 1 w 2 w 3 w 4

  53. 2 State HMM Likelihood π‘ž π‘Š| π‘Š z 1 = z 2 = z 3 = z 4 = π‘ž 𝑂| π‘Š V V V V π‘ž 𝑂| start π‘ž π‘Š| 𝑂 z 1 = z 2 = z 3 = z 4 = N N N N π‘ž π‘₯ 1 |𝑂 π‘ž π‘₯ 4 |𝑂 π‘ž π‘₯ 3 |π‘Š π‘ž π‘₯ 2 |π‘Š w 1 w 2 w 3 w 4 Up until here, all the computation was the same Let’s reuse what computations we can z 1 = z 2 = z 3 = z 4 = V V V V π‘ž π‘Š| 𝑂 π‘ž 𝑂| π‘Š π‘ž 𝑂| start π‘ž 𝑂| 𝑂 z 1 = z 2 = z 3 = z 4 = N N N N π‘ž π‘₯ 2 |π‘Š π‘ž π‘₯ 4 |𝑂 π‘ž π‘₯ 1 |𝑂 π‘ž π‘₯ 3 |𝑂 w 1 w 2 w 3 w 4

  54. 2 State HMM Likelihood π‘ž π‘Š| π‘Š z 1 = z 2 = z 3 = z 4 = π‘ž 𝑂| π‘Š V V V V π‘ž 𝑂| start π‘ž π‘Š| 𝑂 z 1 = z 2 = z 3 = z 4 = N N N N Solution: pass information β€œforward” in π‘ž π‘₯ 1 |𝑂 π‘ž π‘₯ 4 |𝑂 π‘ž π‘₯ 3 |π‘Š the graph, e.g., from time step 2 to 3… π‘ž π‘₯ 2 |π‘Š w 1 w 2 w 3 w 4 z 1 = z 2 = z 3 = z 4 = V V V V π‘ž π‘Š| 𝑂 π‘ž 𝑂| π‘Š π‘ž 𝑂| start π‘ž 𝑂| 𝑂 z 1 = z 2 = z 3 = z 4 = N N N N π‘ž π‘₯ 2 |π‘Š π‘ž π‘₯ 4 |𝑂 π‘ž π‘₯ 1 |𝑂 π‘ž π‘₯ 3 |𝑂 w 1 w 2 w 3 w 4

  55. 2 State HMM Likelihood π‘ž π‘Š| π‘Š z 1 = z 2 = z 3 = z 4 = π‘ž 𝑂| π‘Š V V V V π‘ž 𝑂| start π‘ž π‘Š| 𝑂 z 1 = z 2 = z 3 = z 4 = N N N N Solution: pass information β€œforward” in π‘ž π‘₯ 1 |𝑂 π‘ž π‘₯ 4 |𝑂 π‘ž π‘₯ 3 |π‘Š the graph, e.g., from time step 2 to 3… π‘ž π‘₯ 2 |π‘Š w 1 w 2 w 3 w 4 Issue: these highlighted paths are only z 1 = z 2 = z 3 = z 4 = 2 of the 16 possible paths through the V V V V trellis π‘ž π‘Š| 𝑂 π‘ž 𝑂| π‘Š π‘ž 𝑂| start π‘ž 𝑂| 𝑂 z 1 = z 2 = z 3 = z 4 = N N N N π‘ž π‘₯ 2 |π‘Š π‘ž π‘₯ 4 |𝑂 π‘ž π‘₯ 1 |𝑂 π‘ž π‘₯ 3 |𝑂 w 1 w 2 w 3 w 4

  56. 2 State HMM Likelihood π‘ž π‘Š| π‘Š z 1 = z 2 = z 3 = z 4 = π‘ž 𝑂| π‘Š V V V V π‘ž 𝑂| start π‘ž π‘Š| 𝑂 z 1 = z 2 = z 3 = z 4 = N N N N Solution: pass information β€œforward” in π‘ž π‘₯ 1 |𝑂 π‘ž π‘₯ 4 |𝑂 π‘ž π‘₯ 3 |π‘Š the graph, e.g., from time step 2 to 3… π‘ž π‘₯ 2 |π‘Š w 1 w 2 w 3 w 4 Issue: these highlighted paths are only z 1 = z 2 = z 3 = z 4 = 2 of the 16 possible paths through the V V V V trellis π‘ž π‘Š| 𝑂 π‘ž 𝑂| π‘Š π‘ž 𝑂| start π‘ž 𝑂| 𝑂 z 1 = z 2 = z 3 = z 4 = Solution: marginalize out all N N N N information from previous timesteps π‘ž π‘₯ 2 |π‘Š π‘ž π‘₯ 4 |𝑂 π‘ž π‘₯ 1 |𝑂 π‘ž π‘₯ 3 |𝑂 w 1 w 2 w 3 w 4

  57. Reusing Computation z i-2 z i-1 z i = = A = A A z i-2 z i-1 z i = = B = B B z i-2 z i-1 z i = = C = C C let’s first consider β€œ any shared path ending with B (AB, BB, or CB) β†’ B”

  58. Reusing Computation 𝛽(𝑗 βˆ’ 1, 𝐡) z i-2 z i-1 z i = = A = A A 𝛽(𝑗 βˆ’ 1, 𝐢) z i-2 z i-1 z i = = B = B B 𝛽(𝑗 βˆ’ 1, 𝐷) z i-2 z i-1 z i = = C = C C let’s first consider β€œ any shared path ending with B (AB, BB, or CB) β†’ B” Assume that all necessary information has been computed and stored in 𝛽(𝑗 βˆ’ 1, 𝐡) , 𝛽(𝑗 βˆ’ 1, 𝐢) , 𝛽(𝑗 βˆ’ 1, 𝐷)

  59. Reusing Computation 𝛽(𝑗 βˆ’ 1, 𝐡) z i-2 z i-1 z i = = A = A A 𝛽(𝑗 βˆ’ 1, 𝐢) 𝛽(𝑗, 𝐢) z i-2 z i-1 z i = = B = B B 𝛽(𝑗 βˆ’ 1, 𝐷) z i-2 z i-1 z i = = C = C C let’s first consider β€œ any shared path ending with B (AB, BB, or CB) β†’ B” Assume that all necessary information has been computed and stored in 𝛽(𝑗 βˆ’ 1, 𝐡) , 𝛽(𝑗 βˆ’ 1, 𝐢) , 𝛽(𝑗 βˆ’ 1, 𝐷) Marginalize (sum) across the previous timestep’s possible states

  60. Reusing Computation 𝛽(𝑗 βˆ’ 1, 𝐡) z i-2 z i-1 z i = = A = A A 𝛽(𝑗 βˆ’ 1, 𝐢) 𝛽(𝑗, 𝐢) z i-2 z i-1 z i = = B = B B 𝛽(𝑗 βˆ’ 1, 𝐷) z i-2 z i-1 z i = = C = C C let’s first consider β€œ any shared path ending with B (AB, BB, or CB) β†’ B” marginalize across the previous hidden state values 𝛽 𝑗, 𝐢 = ෍ 𝛽 𝑗 βˆ’ 1, 𝑑 βˆ— π‘ž 𝐢 𝑑) βˆ— π‘ž(obs at 𝑗 | 𝐢) 𝑑

  61. Reusing Computation 𝛽(𝑗 βˆ’ 1, 𝐡) z i-2 z i-1 z i = = A = A A 𝛽(𝑗 βˆ’ 1, 𝐢) 𝛽(𝑗, 𝐢) z i-2 z i-1 z i = = B = B B 𝛽(𝑗 βˆ’ 1, 𝐷) z i-2 z i-1 z i = = C = C C let’s first consider β€œ any shared path ending with B (AB, BB, or CB) β†’ B” marginalize across the previous hidden state values 𝛽 𝑗, 𝐢 = ෍ 𝛽 𝑗 βˆ’ 1, 𝑑 βˆ— π‘ž 𝐢 𝑑) βˆ— π‘ž(obs at 𝑗 | 𝐢) 𝑑 computing Ξ± at time i-1 will correctly incorporate paths through time i-2 : we correctly obey the Markov property

  62. Forward Probability z i-2 z i-1 z i = = A = A A z i-2 z i-1 z i = = B = B B z i-2 z i-1 z i = = C = C C let’s first consider β€œ any shared path ending with B (AB, BB, or CB) β†’ B” marginalize across the previous hidden state values Ξ±(i, B) is the total probability of all 𝛽 𝑗 βˆ’ 1, 𝑑 β€² βˆ— π‘ž 𝐢 𝑑 β€² ) βˆ— π‘ž(obs at 𝑗 | 𝐢) 𝛽 𝑗, 𝐢 = ෍ paths to that state B from the 𝑑 β€² beginning computing Ξ± at time i-1 will correctly incorporate paths through time i-2 : we correctly obey the Markov property

  63. Forward Probability 𝛽 𝑗 βˆ’ 1, 𝑑 β€² βˆ— π‘ž 𝑑 𝑑 β€² ) βˆ— π‘ž(obs at 𝑗 | 𝑑) 𝛽 𝑗, 𝑑 = ෍ 𝑑 β€² Ξ±(i, s ) is the total probability of all paths: 1. that start from the beginning 2. that end (currently) in s at step i 3. that emit the observation obs at i

  64. Forward Probability 𝛽 𝑗 βˆ’ 1, 𝑑 β€² βˆ— π‘ž 𝑑 𝑑 β€² ) βˆ— π‘ž(obs at 𝑗 | 𝑑) 𝛽 𝑗, 𝑑 = ෍ 𝑑 β€² what are the what’s the total probability how likely is it to get immediate ways to up until now? into state s this way? get into state s ? Ξ±(i, s ) is the total probability of all paths: 1. that start from the beginning 2. that end (currently) in s at step i 3. that emit the observation obs at i

  65. Forward Algorithm Ξ± : a 2D table, N+2 x K* N+2: number of observations (+2 for the BOS & EOS symbols) K*: number of states Use dynamic programming to build the Ξ± left-to- right

  66. Forward Algorithm Ξ± = double[N+2][K*] Ξ± [0][*] = 0.0 Ξ± [0][START] = 1.0 for(i = 1; i ≀ N+1; ++ i) { }

  67. Forward Algorithm Ξ± = double[N+2][K*] Ξ± [0][*] = 0.0 Ξ± [0][START] = 1.0 for(i = 1; i ≀ N+1; ++ i) { for(state = 0; state < K*; ++state) { } }

  68. Forward Algorithm Ξ± = double[N+2][K*] Ξ± [0][*] = 0.0 Ξ± [0][START] = 1.0 for(i = 1; i ≀ N+1; ++ i) { for(state = 0; state < K*; ++state) { p obs = p emission (obs i | state) } }

  69. Forward Algorithm Ξ± = double[N+2][K*] Ξ± [0][*] = 0.0 Ξ± [0][START] = 1.0 for(i = 1; i ≀ N+1; ++ i) { for(state = 0; state < K*; ++state) { p obs = p emission (obs i | state) for(old = 0; old < K*; ++old) { p move = p transition (state | old) Ξ± [i][state] += Ξ± [i-1][old] * p obs * p move } } }

  70. Forward Algorithm Ξ± = double[N+2][K*] Ξ± [0][*] = 0.0 we still need to learn these Ξ± [0][START] = 1.0 (EM if not observed) for(i = 1; i ≀ N+1; ++ i) { for(state = 0; state < K*; ++state) { p obs = p emission (obs i | state) for(old = 0; old < K*; ++old) { p move = p transition (state | old) Ξ± [i][state] += Ξ± [i-1][old] * p obs * p move } } }

  71. Forward Algorithm Ξ± = double[N+2][K*] Ξ± [0][*] = 0.0 Ξ± [0][START] = 1.0 Q: What do we return? (How do we return the likelihood of the sequence?) for(i = 1; i ≀ N+1; ++ i) { for(state = 0; state < K*; ++state) { p obs = p emission (obs i | state) for(old = 0; old < K*; ++old) { p move = p transition (state | old) Ξ± [i][state] += Ξ± [i-1][old] * p obs * p move } } }

  72. Forward Algorithm Ξ± = double[N+2][K*] Ξ± [0][*] = 0.0 Ξ± [0][START] = 1.0 Q: What do we return? (How do we return the likelihood of the sequence?) for(i = 1; i ≀ N+1; ++ i) { for(state = 0; state < K*; ++state) { p obs = p emission (obs i | state) A: Ξ± [N+1][end] for(old = 0; old < K*; ++old) { p move = p transition (state | old) Ξ± [i][state] += Ξ± [i-1][old] * p obs * p move } } }

  73. Interactive HMM Example https://goo.gl/rbHEoc (Jason Eisner, 2002) Original: http://www.cs.jhu.edu/~jason/465/PowerPoint/lect24-hmm.xls

  74. Forward Algorithm in Ξ± = double[N+2][K*] Log-Space Ξ± [0][*] = - ∞ Ξ± [0][*] = 0.0 for(i = 1; i ≀ N+1; ++ i) { for(state = 0; state < K*; ++state) { p obs = log p emission (obs i | state) for(old = 0; old < K*; ++old) { p move = log p transition (state | old) Ξ± [i][state] = logadd( Ξ± [i][state], Ξ± [i-1][old] + p obs + p move ) } } }

  75. Forward Algorithm in Ξ± = double[N+2][K*] Log-Space Ξ± [0][*] = - ∞ Ξ± [0][*] = 0.0 for(i = 1; i ≀ N+1; ++ i) { for(state = 0; state < K*; ++state) { p obs = log p emission (obs i | state) for(old = 0; old < K*; ++old) { p move = log p transition (state | old) Ξ± [i][state] = logadd( Ξ± [i][state], Ξ± [i-1][old] + p obs + p move ) scipy.misc.logsumexp } } logadd π‘šπ‘ž, π‘šπ‘Ÿ = α‰Š π‘šπ‘ž + log 1 + exp π‘šπ‘Ÿ βˆ’ π‘šπ‘ž , π‘šπ‘ž β‰₯ π‘šπ‘Ÿ } π‘šπ‘Ÿ + log 1 + exp π‘šπ‘ž βˆ’ π‘šπ‘Ÿ , π‘šπ‘Ÿ > π‘šπ‘ž

Recommend

1 X 1 X 2 X 3 Ghostbusters HMM Chain Rule and HMMs E 1 E 2 E 3 P(X 1 ) = uniform 1/9 1/9

1 X 1 X 2 X 3 Ghostbusters HMM Chain Rule and HMMs E 1 E 2 E 3 P(X 1 ) = uniform 1/9 1/9

Hidden Markov Models 1 Hidden Markov Models Markov chains not so useful for most agents Need observations to update your beliefs Hidden Markov models (HMMs) Underlying Markov chain over states X You observe outputs (effects) at

617 views β€’ 7 slides
CSE P 590 A Markov Models and Hidden Markov Models

CSE P 590 A Markov Models and Hidden Markov Models

CSE P 590 A Markov Models and Hidden Markov Models http://upload.wikimedia.org/wikipedia/commons/b/ba/Calico_cat Dosage Compensation Reminder: Proteins Read DNA and X-Inactivation E.g.: 2 copies (mom/dad) of each

1.28k views β€’ 21 slides
Hidden Markov Models Pratik Lahiri Introduction A hidden Markov model (HMM) is a

Hidden Markov Models Pratik Lahiri Introduction A hidden Markov model (HMM) is a

Hidden Markov Models Pratik Lahiri Introduction A hidden Markov model (HMM) is a statistical Markov model in which the system being modeled is assumed to be a Markov process with unobserved (hidden) states. We call the observed event

741 views β€’ 13 slides
CSE 527 &quot; Markov Models and Hidden Markov Models !

CSE 527 &quot; Markov Models and Hidden Markov Models !

CSE 527 &quot; Markov Models and Hidden Markov Models ! http://upload.wikimedia.org/wikipedia/commons/b/ba/Calico_cat ! Dosage Compensation Reminder: Proteins Read DNA ! and X-Inactivation ! E.g.: ! 2 copies (mom/dad) of each chromosome

770 views β€’ 17 slides
CSE 427 Markov Models and Hidden Markov Models 2

CSE 427 Markov Models and Hidden Markov Models 2

CSE 427 Markov Models and Hidden Markov Models 2 http://upload.wikimedia.org/wikipedia/commons/b/ba/Calico_cat Dosage Compensation and X-Inactivation 2 copies (mom/dad) of each chromosome 1-23 Mostly, both copies of each gene are

1.07k views β€’ 60 slides
CSE P 527 Markov Models and Hidden Markov Models 1 2

CSE P 527 Markov Models and Hidden Markov Models 1 2

CSE P 527 Markov Models and Hidden Markov Models 1 2 http://upload.wikimedia.org/wikipedia/commons/b/ba/Calico_cat Dosage Compensation and X-Inactivation 2 copies (mom/dad) of each chromosome 1-23 Mostly, both copies of each gene are

970 views β€’ 66 slides
1 Real HMM Examples Real HMM Examples Speech recognition HMMs: Machine translation HMMs:

1 Real HMM Examples Real HMM Examples Speech recognition HMMs: Machine translation HMMs:

Hidden Markov Models CSE 473: Artificial Intelligence Markov chains not so useful for most agents Hidden Markov Models Eventually you dont know anything anymore Need observations to update your beliefs Hidden Markov models

567 views β€’ 9 slides
Markov chains and Hidden Markov Models 9000 Markov chains and HMMs We will discuss: Markov

Markov chains and Hidden Markov Models 9000 Markov chains and HMMs We will discuss: Markov

Markov chains and Hidden Markov Models 9000 Markov chains and HMMs We will discuss: Markov chains Hidden Markov Models (HMMs) Algorithms: Viterbi, forward, backward, posterior decoding Profile HMMs Baum-Welch algorithm 9001

1.16k views β€’ 87 slides
1 Real HMM Examples Real HMM Examples Speech recognition HMMs: Machine translation HMMs:

1 Real HMM Examples Real HMM Examples Speech recognition HMMs: Machine translation HMMs:

Hidden Markov Models CSE 473: Artificial Intelligence Hidden Markov Models Markov chains not so useful for most agents Eventually you dont know anything anymore Need observations to update your beliefs Hidden Markov models (HMMs)

253 views β€’ 7 slides
CSE 527 Lectures 12-13 Markov Models and Hidden Markov Models DNA Methylation CH 3 CpG - 2

CSE 527 Lectures 12-13 Markov Models and Hidden Markov Models DNA Methylation CH 3 CpG - 2

CSE 527 Lectures 12-13 Markov Models and Hidden Markov Models DNA Methylation CH 3 CpG - 2 adjacent nts, same strand (not Watson-Crick pair; p mnemonic for the phosphodiester bond of the DNA backbone) C of CpG is often (70-80%)

690 views β€’ 55 slides
CSE 527 Lectures 11-12 Markov Models and Hidden Markov Models DNA Methylation CH 3 CpG - 2

CSE 527 Lectures 11-12 Markov Models and Hidden Markov Models DNA Methylation CH 3 CpG - 2

CSE 527 Lectures 11-12 Markov Models and Hidden Markov Models DNA Methylation CH 3 CpG - 2 adjacent nts, same strand (not Watson-Crick pair; p mnemonic for the phosphodiester bond of the DNA backbone) C of CpG is often (70-80%)

780 views β€’ 55 slides
Markov Models Kunsch, H.R., State Space and Hidden Markov Models . ETH- Zurich, Zurich;

Markov Models Kunsch, H.R., State Space and Hidden Markov Models . ETH- Zurich, Zurich;

State Space and Hidden Markov Models Kunsch, H.R., State Space and Hidden Markov Models . ETH- Zurich, Zurich; Aliaksandr Hubin Oslo 2014 Contents 1. Introduction 2. Markov Chains 3. Hidden Markov and State Space Model 4. Filtering and

853 views β€’ 37 slides
Markov Chains and Hidden Markov Models COMP 571 Luay Nakhleh, Rice University Markov Chains and

Markov Chains and Hidden Markov Models COMP 571 Luay Nakhleh, Rice University Markov Chains and

Markov Chains and Hidden Markov Models COMP 571 Luay Nakhleh, Rice University Markov Chains and Hidden Markov Models Modeling the statistical properties of biological sequences and distinguishing regions based on these models For the

1.33k views β€’ 96 slides
Markov Chains and Hidden Markov Models COMP 571 Luay Nakhleh, Rice University 2 Markov Chains

Markov Chains and Hidden Markov Models COMP 571 Luay Nakhleh, Rice University 2 Markov Chains

1 Markov Chains and Hidden Markov Models COMP 571 Luay Nakhleh, Rice University 2 Markov Chains and Hidden Markov Models Modeling the statistical properties of biological sequences and distinguishing regions based on these models For the

437 views β€’ 32 slides
A C Standard model has 1-1 correspondence between symbols and states, P(A | T) thus P ( x i

A C Standard model has 1-1 correspondence between symbols and states, P(A | T) thus P ( x i

Outline Markov chains CSCE 471/871 Lecture 3: Markov Chains and Hidden Markov Models Hidden Markov models (HMMs) Formal definition Finding most probable state path (Viterbi algorithm) Stephen D. Scott Forward and backward

640 views β€’ 5 slides
HMM Review Lecture Outline 1. Markov models 2. Hidden Markov

HMM Review Lecture Outline 1. Markov models 2. Hidden Markov

HMM Review Lecture Outline 1. Markov models 2. Hidden Markov models 3. Viterbi algorithm MARKOV MODELS One View of Text Sequence of symbols (bytes,

489 views β€’ 46 slides
Introduction to Hidden Markov Models Antonio Art es-Rodr guez Unviersidad Carlos III de

Introduction to Hidden Markov Models Antonio Art es-Rodr guez Unviersidad Carlos III de

Introduction to Hidden Markov Models Antonio Art es-Rodr guez Unviersidad Carlos III de Madrid 2nd MLPM SS, September 17, 2014 1/33 Outline Markov and Hidden Markov Models Markov processes Definition of a HMM Applications of HMMs

940 views β€’ 37 slides
A spectral algorithm for learning hidden Markov models . . . h 3 h 2 h 1 x 3 x 2 x 1 Daniel Hsu

A spectral algorithm for learning hidden Markov models . . . h 3 h 2 h 1 x 3 x 2 x 1 Daniel Hsu

A spectral algorithm for learning hidden Markov models . . . h 3 h 2 h 1 x 3 x 2 x 1 Daniel Hsu Sham M. Kakade Tong Zhang UCSD TTI-C Rutgers A spectral algorithm for learning hidden Markov models Nr. 1 Motivation Hidden Markov Models

715 views β€’ 25 slides
CSE 427 Markov Models and Hidden Markov Models How Proteins Read DNA E.g.:

CSE 427 Markov Models and Hidden Markov Models How Proteins Read DNA E.g.:

CSE 427 Markov Models and Hidden Markov Models How Proteins Read DNA E.g.: Helix-Turn-Helix Motif Leucine Zipper Motif Down in the Groove Different patterns of hydrophobic methyls, potential H bonds, etc. at edges of different

877 views β€’ 58 slides
Markov Chains and Hidden Markov Models COMP 571 - Spring 2015 Luay Nakhleh, Rice University

Markov Chains and Hidden Markov Models COMP 571 - Spring 2015 Luay Nakhleh, Rice University

Markov Chains and Hidden Markov Models COMP 571 - Spring 2015 Luay Nakhleh, Rice University Markov Chains and Hidden Markov Models Modeling the statistical properties of biological sequences and distinguishing regions based on these models

873 views β€’ 58 slides
Outline depmixS4: an R-package for hidden Markov models Hidden Markov Models Ingmar Visser 1

Outline depmixS4: an R-package for hidden Markov models Hidden Markov Models Ingmar Visser 1

Hidden Markov Models Hidden Markov Models DepmixS4 DepmixS4 Examples Examples Conclusions Conclusions Outline depmixS4: an R-package for hidden Markov models Hidden Markov Models Ingmar Visser 1 &amp; Maarten Speekenbrink 2 DepmixS4 1

121 views β€’ 10 slides
Markov Models and Hidden Markov Models Robert Platt Northeastern University Some images and

Markov Models and Hidden Markov Models Robert Platt Northeastern University Some images and

Markov Models and Hidden Markov Models Robert Platt Northeastern University Some images and slides are used from: 1. CS188 UC Berkeley 2. RN, AIMA Markov Models We have already seen that an MDP provides a useful framework for modeling

1k views β€’ 69 slides
Markov Models and Hidden Markov Models Robert Platt Northeastern University Some images and

Markov Models and Hidden Markov Models Robert Platt Northeastern University Some images and

Markov Models and Hidden Markov Models Robert Platt Northeastern University Some images and slides are used from: 1. CS188 UC Berkeley 2. RN, AIMA Markov Models We have already seen that an MDP provides a useful framework for modeling

846 views β€’ 62 slides
Hidden Markov Models Markov Model (Finite State Machine with Probs) Modeling a sequence of

Hidden Markov Models Markov Model (Finite State Machine with Probs) Modeling a sequence of

Hidden Markov Models Markov Model (Finite State Machine with Probs) Modeling a sequence of weather observations Hidden Markov Models Assume the states in the machine are not observed and we can observe some output at certain states. Hidden

692 views β€’ 26 slides

More recommend