Identification of weak lumpability in Markov chains General - PowerPoint PPT Presentation

Identification of weak lumpability in Markov chains General criteria for weak lumpability found, and the structure of the corresponding projection operator is derived Martin Nilsson Jacobi

Projections and the Markov property e e P P ˜ ˜ Macro level: ˜ s t +1 s t +2 s t π π π P P Micro level: s t +1 s t +2 s t p (˜ s t +2 | ˜ s t +1 ˜ s t ˜ s t − 1 · · · ) independent of ˜ s t ˜ s t − 1 · · · .

markov lumping (aggregation) e P π π P   0 1 1 0 aggregates Aggregation of state 2 and 3 π = 1 0 0 0   at the micro level into one 0 0 0 1 state at the macro level variables/states

e P e P X X ρ i e P P KL = P j ← i π π m ∈ L ρ m i ∈ L j ∈ K P e P = π P π + π D 1 / 2 ⌘ † D 1 / 2 ⇣ π + = � − 1 A T A T A A † � = = D ii ρ i ρ i

e P N e e P P ˜ ˜ ˜ s t +1 s t +2 s t π π π P P s t +1 s t +2 s t p (˜ s t +2 | ˜ s t +1 ˜ s t ˜ s t − 1 · · · ) independent of ˜ s t ˜ s t − 1 · · · . ⇢ π P N π + P N = π P π + � N e π P N π + = � ∀ N ( π P π + ) N

π P π + π P π + = π PP π + π P π + π = π P π + π P π + = P π + e P e P π π π π P P P π + = π + e e P π = π P P strong lumpability simple weak lumpability Either of these eq. are sufficient but not necessary for weak lumpability

Invariance conditions Interpretation of the conditions Strong lumping Weak lumping lumping π + e e P = P π + P π = π P Column space of π + invariant under P Row space of π invariant under P T Invariance typically means eigenvactors π P π + � N π P N π + = � These commutation relations also solve

strong lumping x t +1 = Px t y = Π x aggregates   0 1 1 0 Row-space spanned � Π = 1 0 0 0 by eigenvectors of P T   0 0 0 1 variables/states � ⇥ Linear combination of the rows: a b b c Search for eigenvectors with constant level structure! #levels = #aggregates = #eigenvectors with that level structure

Weak lumping   1 0 0 ρ 2 0 0 π + = Column-space spanned �   ρ 2 + ρ 3   ρ 3 0 0 by eigenvectors of P   ρ 2 + ρ 3 0 0 1 Take right eigenvectors of P , say u . Look for level structure in the vector v i = u i / ρ i . #levels = #aggregates = #eigenvectors with that level structure

Example   0 . 25 0 . 0 . 875 P = 0 . 25 0 . 166667 0 . 125   0 . 5 0 . 833333 0 . / ρ i Right � (1 ., 1 ., 1 . ) ( − 0 . 721995 , − 0 . 309426 , − 0 . 618853) (1 . 11051 , − 0 . 863731 , − 0 . 863731) ( − 0 . 801784 , 0 . 267261 , 0 . 534522) eig vec ( − 1 . 12706 , 1 . 12706 , 0 . 751375) (0 . 813733 , − 0 . 348743 , − 0 . 464991) Weak lumping: {{1},{2,3}} Left � ( − 0 . 57735 , − 0 . 57735 , − 0 . 57735) No strong lumping ( − 0 . 0733017 , − 0 . 855186 , 0 . 513112) eig vec (0 . 0000 , − 0 . 894427 , 0 . 447214)

The general case π P π + � N π P N π + = � ∀ N Assume a projection to n lumps. π gives a weak lumping i ff 1. the row-space of π is spanned by left eigenvectors with index i 1 , i 2 , . . . , i k 2. the column-space of π + is spanned by right eigenvectors with index j 1 , j 2 , . . . , j m 3. the number of indices in common between π and π + , |{ i 1 , i 2 , . . . , i k } ∩ { j 1 , j 2 , . . . , j m }| , is exactly equal to n .

The good and the bad So we have derived the general structure of a projection that gives weak lumpability in the most general form. (Good.) � � The general criterion is not constructive in the same sense as the simple ones. We need to find linear combinations of a large set of eigenvectors that show level structure. This is (probably) a hard problem. (Bad.)