Optimizing F • Apply Lagrange multipliers just like before • In this case, we have simply replaced z , x , and θ with vectors • The math is exactly the same • But we need to find the expectations we skipped before – Plug in the Dirichlet and multinomial distributions 84
Optimal q(z, θ ) • Borrowed from the Mean Field Example • See slides 62-63 • All we need to do is apply the particulars of the mixture model 85
Optimal q θ ( θ ) • Factorize q θ ( θ ) To get 86
Optimal q φ ( φ ) : The Expectation 87
Optimal q φ ( φ ) : The Expectation 88
Dirichlet Distribution • Notable for being the conjugate prior of the multinomial 89
Optimal q φ ( φ ) : The Numerator 90
Optimal q φ ( φ ) : The Numerator Def. of Dirichlet 91
Optimal q φ ( φ ) : The Numerator Previous slide 92
Optimal q φ ( φ ) : The Numerator 93
Optimal q φ ( φ ) : The Numerator 94
Optimal q φ ( φ ) : The Numerator 95
Optimal q φ ( φ ): The Normalization 96
Optimal q φ ( φ ): The Normalization from previous slide 97
Optimal q φ ( φ ): The Normalization 98
Optimal q φ ( φ ): The Normalization Def. of Dirichlet 99
Optimal q φ ( φ ): The Normalization 100
Optimal q φ ( φ ) : Conjugacy Helps 101
Recommend
More recommend
