Markov Models Kunsch, H.R., State Space and Hidden Markov Models . - PowerPoint PPT Presentation

State Space and Hidden Markov Models Kunsch, H.R., State Space and Hidden Markov Models . ETH- Zurich, Zurich; Aliaksandr Hubin Oslo 2014

Contents 1. Introduction 2. Markov Chains 3. Hidden Markov and State Space Model 4. Filtering and Smoothing of State Densities 5. Estimation of Parameters 6. Spatial-Temporal Model 2 11/10/2014 ALIAKSANDR HUBIN. STK 9200

Introduction HMM  Broadly speaking we will address the models where observations are noisy and incomplete functions of some underlying unobservable process, called the state process, which is assumed to have simple markovian dynamics. 3 11/10/2014 ALIAKSANDR HUBIN. STK 9200

Markov chains    { , , , }, 1 , 2 ,... x m m m i • Set of states: 1 2 i N   , , , , x x x • Process moves from one state to another generating a sequence of states : 1 2 k • Markov chain property: probability of each subsequent state depends only on what was the   previous state: ( | , , , ) ( | ) P x x x x P x x   1 2 1 1 k k k k • To define Markov chain, the following probabilities have to be specified:   a  ( | ) ( ) P m m P m transition probabilities matrix and initial probabilities ij i j i i • The output of the process is the set of states at each instant of time     ( , , , ) ( | , , , ) ( , , , ) • Joint probability of all states sequence: P x x x P x x x x P x x x   1 2 1 2 1 1 2 1 k k k k     ( | ) ( , , , ) P x x P x x x   1 1 2 1 k k k   ( | ) ( | ) ( | ) ( ) P x x P x x P x x P x    1 1 2 2 1 1 k k k k 4 11/10/2014 ALIAKSANDR HUBIN. STK 9200

Simple Example of Markov Model • Epigenetic state process is considered • Two states : ‘Methylated’ and ‘Non - methylated’. • Initial probabilities: P( ‘Methylated’ ) =0.4 , P( ‘Non - methylated’ ) =0.6 • Transition probabilities described in the graph -> • Inference example: Suppose we want to calculate a probability of a sequence of states in our example, {‘Methylated’,’ Methylated’,’ Non- methylated ’, Non- methylated’}. This then corresponds to 0.4*0.3*0.7*0.8 = 6.72% 5 11/10/2014 ALIAKSANDR HUBIN. STK 9200

Hidden Markov Models • The observations are represented by a probabilistic function (discrete or continuous) of a state instead of an one-to-one correspondence of a state • The following components describe a Hidden Markov Model in the simplest case: a  ( | ) 1. Distribution of transition probabilities ; P m m ij i j   2. Distribution of initial state probabilities ; ( ) P m i i Distribution of observation probabilities 𝜘 𝑗 = 𝑄 𝑧 𝑗 𝑦 𝑗 . 3.  𝑦 𝑗 ∈ 𝑆 𝑂 - are usually addressed as state space models ;  𝑦 𝑗 ∈ {𝑛 1 , … , 𝑛 𝑂 } – are usually addressed as hidden markov chains ;  Parameters listed above estimation might be very challenging for complicated models! 6 11/10/2014 ALIAKSANDR HUBIN. STK 9200

Hidden Markov Models The following components describe a Hidden Markov Model in the general case: 𝑌 0 ~𝑏 0 𝑦 𝑒𝜈(𝑦) ; 1. Distribution of initial state probabilities Distribution of transition probabilities 𝑌 𝑢 | 𝑌 𝑢−1 = 𝑦 𝑢−1 ~𝑏 𝑢 𝑦 𝑢−1 , 𝑦 𝑒𝜈(𝑦) ; 2. Distribution of observation probabilities 𝑍 𝑢 (𝑌 𝑢 = 𝑦 𝑢 ~𝑐 𝑢 𝑦 𝑢 , 𝑧 𝑒𝑤 𝑧 ; 3. 4. The joint density of the process and observations then looks as follows: 𝑈 𝑈 = 𝑏 0 (𝑦 0 ) 𝑢=1 𝑄 𝑌 0 , … , 𝑌 𝑈 , 𝑍 1 , … , 𝑍 𝑏 𝑢 𝑦 𝑢−1 , 𝑦 𝑢 𝑐 𝑢 (𝑦 𝑢 , 𝑧 𝑢 ) . 7 11/10/2014 ALIAKSANDR HUBIN. STK 9200

Simple Example of HMM • Whether state process is considered •Two discrete states : ‘Methylated’ and ‘Non - methylated’. • Initial probabilities: P( ‘Methylated’ ) =0.4 , P( ‘Non - methylated’ ) =0.6 • Transition probabilities described in the graph -> • Locations associated are observed with respect to their state-dependent probabilities • Corresponding observation probabilities are represented in the graph -> 8 11/10/2014 ALIAKSANDR HUBIN. STK 9200

Graphical illustration of the Hidden Markov Model 9 11/10/2014 ALIAKSANDR HUBIN. STK 9200

Project application Let further: • 𝐽 𝑢 be a binary variable indication whether location t is methylated • 𝐹 𝑢 be a binary varibale indicating whether the gene, to which location 𝑢 belongs is expressed • 𝑜 𝑢 be an amount of reads for location t • 𝑧 𝑢 be a binomial variable indicating the number of methylated reads at location t • 𝑨 𝑢 be a quantitative measure of some phenotypic response for the expressed genes at location t 10 11/10/2014 ALIAKSANDR HUBIN. STK 9200

Project application 𝑨 𝑈 𝑨 1 𝑨 𝑢 𝑨 𝑢+1 … … Extended … … 𝐹 1 𝐹 𝑢 𝐹 𝑈 𝐹 𝑢+1 … … 𝐽 1 𝐽 𝑢 𝐽 𝑈 𝐽 𝑢+1 Initial … 𝑧 1 𝑧 𝑢 𝑧 𝑈 … 𝑧 𝑢+1 11 11/10/2014 ALIAKSANDR HUBIN. STK 9200

Project application • 𝐽 𝑢 = 1, 0, − location 𝑢 is not methylated ~ 𝑄(𝐽 𝑢 ) = 𝑄 − location 𝑢 is methylated 1 , − location 𝑢 is methylated 𝑄 2 , −location 𝑢 is not methylated • 𝑄 𝑗𝑘 = 𝑄 𝐽 𝑢 = 𝑘 𝐽 𝑢−1 = 𝑗 − define dynamic structures in the given neighborhood • 𝑧 𝑢 |𝐽 𝑢 , 𝑜 𝑢 − number of metylated observations in given reads and methylation status • 𝑄(𝑧 𝑢 𝐽 𝑢 , 𝑜 𝑢 = 𝑄 𝐶𝑗𝑜𝑝𝑛 𝑜 𝑢 ,𝑄 𝐽 𝑢 𝑧 𝑢 − it has binomial distribution 12 11/10/2014 ALIAKSANDR HUBIN. STK 9200

Extensions of the model 𝑞 𝑢−1 𝐶𝑓𝑢𝑏(𝛾 1−𝑞 𝑢−1 )  Let 𝑞 𝑢 be continuous: define stochastic process 𝑞(𝐽 𝑢 )= ) , giving similarities when the state do 𝑟 𝐽𝑢 𝐶𝑓𝑢𝑏(𝛾 1−𝑟 𝐽𝑢 not change but a renewal in cases where the state changes.  Link state transition probabilities to underlying genomic structure  Look at more global structures than transition probabilities 𝑄 𝑗𝑘 : -address more complex models describing more global structures (ARIMA, n-markov chains, etc.) -use simple markov chain model above as a first step, but then extract more global features of {𝐽 𝑢 }  Consider more complex spatial structures than the one-directional approach above.  Simultaneous modelling of several tissues and/or individuals at the same time. 13 11/10/2014 ALIAKSANDR HUBIN. STK 9200

Parameters’ estimation & Inference 1. Viterbi algorithm for fitting the joint distribution of the most probable sequence of states 𝑏𝑠𝑕𝑛𝑏𝑦 𝑦 1:𝑈 {𝑄(𝑦 1:𝑈 |𝑒 1:𝑈 )} Forward algorithm for fitting filtered probability of a given state, given data up to 𝑄(𝑦 𝑢 |𝑒 1 , … , 𝑒 𝑢 ) 2. 3. Forward – backward algorithm for fitting the smoothed probability of a given state, given all data 𝑄(𝑦 𝑙 |𝑒 1 , … , 𝑒 𝑈 ) Maximal likelihood maximization or Bayesian methods for parameters’ vector 𝜄 estimation 4.  Where 𝑒 𝑢 is data at point t;  Note that these algorithms are linear in the number of time points. 14 11/10/2014 ALIAKSANDR HUBIN. STK 9200

Inference. Filtering. 1. Recursion in k for prediction density of the states: 𝑢 = 𝑏 𝑢+𝑙 𝑦, 𝑦 𝑢+𝑙 𝑔 𝑢 𝑒𝜈(𝑦) 𝑔 𝑢+𝑙|𝑢 𝑦 𝑢+𝑙 𝑧 1 𝑢+𝑙−1|𝑢 𝑦 𝑧 1 2. Prediction densities for the observations given the corresponding prediction densities of the states: 𝑢 = 𝑐 𝑢+𝑙 𝑦, 𝑧 𝑢+𝑙 𝑔 𝑢 𝑒𝜈(𝑦) 𝑞 𝑧 𝑢+𝑙 𝑧 1 𝑢+𝑙 𝑦 𝑧 1 3. Thus, filtering densities of the states can be computed according to the following forward recursion in t (starting with 𝑔 0|0 ): 𝑢 𝑐 𝑢+1 𝑦 𝑢+1 ,𝑧 𝑢+1 𝑔 𝑢+1|𝑢 𝑦 𝑢+1 𝑧 1 𝑢+1 = 𝑔 𝑢+1|𝑢+1 𝑦 𝑢+1 𝑧 1 𝑢 𝑞 𝑧 𝑢+𝑙 𝑧 1 15 11/10/2014 ALIAKSANDR HUBIN. STK 9200

Interface. Smoothing. 1. Conditional on all observations sequence of states is the markov chain with the following transition densities: , where 2. The backward transition densities then become: 16 11/10/2014 ALIAKSANDR HUBIN. STK 9200

Interface. Smoothing. The smoothing densities are then computed with respect to the recursions in t: and , where 5. Note that the joint density of the subsequence of states, which might be required at ML stage: 17 11/10/2014 ALIAKSANDR HUBIN. STK 9200

Recursion order 18 11/10/2014 ALIAKSANDR HUBIN. STK 9200

Inference on a function of states. 6. It can also shown that having the smoothing distribution of the sequence of states it is easy to obtain the conditional expectation of some function assigned to the sequence, which can be recursively updated when a new observation becomes available ( t>s ): 19 11/10/2014 ALIAKSANDR HUBIN. STK 9200

Posterior mode estimation We want to get the posterior mean of the states, namely: Maximization of the posterior joint distribution of its states is invariant to being logorithmed: 20 11/10/2014 ALIAKSANDR HUBIN. STK 9200