CS786 Lecture 15: May 21, 2012 MAP inference [KF Chapter 13] CS786 P. - PDF document

26/06/2012 CS786 Lecture 15: May 21, 2012 MAP inference [KF Chapter 13] CS786 P. Poupart 2012 1 MAP Queries • Recall: MAP stands for maximum a posteriori • MAP queries: – find best assignment of all non ‐ evidence variables – ������ � Pr ��|�� • Marginal MAP queries – Find best assignment of a subset � of non ‐ evidence variables � – ������ � Pr � � � ������ � ∑ Pr ��, �|�� � CS786 P. Poupart 2012 2 1

26/06/2012 Applications • Speech recognition – Find best sequence � of words given audio signal – ������ � Pr ��|����� ��������� • Image processing – Find best pixel labeling � given image (pixel intensities) – ������ � Pr ��|������ • Medical diagnosis – Find best diagnosis � given symptoms – ������ � Pr ��|��������� CS786 P. Poupart 2012 3 MAP Inference Techniques • Variable elimination • Message passing • Optimization • Graph cut CS786 P. Poupart 2012 4 2

26/06/2012 Comparison • Inference queries: Pr � � � ∑ Pr �, � � � ∑ ∏ � � ��, �� � � � Operations: sum ‐ product • MAP queries: max � Pr � � � max ∏ � � ��� � � Operations: max ‐ product • Idea: use same algorithms, but eliminate variables by maximization instead of summation CS786 P. Poupart 2012 5 Max ‐ product Variable Elimination • Same as variable elimination, but variables are eliminated by maximization 1. Restrict factors based on evidence For each non ‐ evidence variable � 2. � do Multiply factors that contain � a. � : � � � ∏ � � ��� � � � ∈����� � � Maxout � a. � : ���\�� � �� � max � � ���� Notes: • – no need for normalization – All non ‐ evidence variables are eliminated CS786 P. Poupart 2012 6 3

26/06/2012 Maxout Operation • Let ���, �� be a potential with var. � ( � is a set) • We “maxout” � from � to produce a new potential h which is defined: ���� � max �∈��� � ���, �� f(A,B) h(B)=max A f(A,B) ab 0.9 b 0.9 a~b 0.1 ~b 0.6 ~ab 0.4 ~a~b 0.6 7 CS 786 Lecture Slides (c) 2012, P. Poupart Maximizing values • Since most MAP queries are not really interested in the maximum probability, but rather the assignment of values to variables that maximize the probability, store the maximizing values in the factors f(A,B) h(B)=max A f(A,B) ab 0.9 b 0.9 (A=a) a~b 0.1 ~b 0.6 (A=~a) ~ab 0.4 ~a~b 0.6 8 CS 786 Lecture Slides (c) 2012, P. Poupart 4

26/06/2012 Max ‐ Product VE Example CS786 P. Poupart 2012 9 Marginal MAP Queries • MAP queries: max � Pr � � � max ∏ � � ��� � � Operations: max ‐ product Complexity: NP • Marginal MAP queries: ∑ ∏ � max � Pr � � � max � ��, �� � � � Operations: max ‐ sum ‐ product Complexity: NP PP CS786 P. Poupart 2012 10 5

26/06/2012 Variable Elimination for Marginal MAP • Eliminate all variables – Eliminate the query variables by maximization – Eliminate the hidden variables by summation • Elimination order: – Exact inference: eliminate hidden variables before query variables – Approximate inference: eliminate variables in any order CS786 P. Poupart 2012 11 6