Approximating likelihood ratios with calibrated classifiers Gilles - PowerPoint PPT Presentation

Approximating likelihood ratios with calibrated classifiers Gilles Louppe DIANA meeting February 22, 2016

Likelihood ratio We want to evaluate the likelihood-ratio statistic p X ( x | θ 0 ) � λ ( D ; θ 0 , θ 1 ) = (1) p X ( x | θ 1 ) x ∈D in the likelihood-free setting, i.e. when p X ( x | θ 0 ) cannot be evaluated but samples x can be drawn from p θ 0 (resp. for θ 1 ). Issue. The input space X may be high dimensional, making it very difficult to build an approximate of p X ( x | θ 0 ). 2 / 7

Equivalent statistic Theorem. r ( x ; θ 0 , θ 1 ) = p X ( x | θ 0 ) p X ( x | θ 1 ) = p U ( u = s ( x ) | θ 0 ) (2) p U ( u = s ( x ) | θ 1 ) provided the change of variable U = s ( X ) is monotonic with r ( x ; θ 0 , θ 1 ). Idea. s ( x ) projects x into a 1D space in which only the informative content of r ( x ; θ 0 , θ 1 ) is preserved. Building an approximate of p U ( u = s ( x ) | θ 0 ) is now easy! 3 / 7

Approximating likelihood ratios with classifiers A classifier trained to distinguish samples x ∼ p θ 0 from samples x ∼ p θ 1 eventually models p X ( x | θ 1 ) s ∗ ( x ) = p X ( x | θ 0 ) + p X ( x | θ 1 ) , (3) which satisfies conditions of the theorem. Idea. For building an equivalent likelihood-ratio statistic, combine supervised learning for learning s ( x ) with calibration for learning p ( s ( x )). 4 / 7

Likelihood-free inference ˆ θ = arg max p ( D| θ ) θ p ( x | θ ) � = arg max p ( x | θ 1 ) θ x ∈D p ( s ( x ; θ, θ 1 ) | θ ) � = arg max (4) p ( s ( x ; θ, θ 1 ) | θ 1 ) , θ x ∈D where, for computational efficiency, s ( x ; θ, θ 1 ) can be a single classifier parameterized by θ, θ 1 . Note. This can then be used for computing profile likelihood ratio tests, taking into account nuisance parameters. 5 / 7

For more details... New version of http://arxiv.org/abs/1506.02169 , in preparation for submission to JASA. 6 / 7

Carl, a likelihood-free inference toolbox for Python • Approximation of likelihood ratios with classifiers Supervised learning via Scikit-Learn ✓ Calibration (histograms, KDE, isotonic regression) ✓ Automatic decomposition of mixtures ✓ Parameterized approximated ratios for inference (in progress) See toy example • Canonical inference examples (in progress) • (Minimal) Composition and fitting of PDFs, ` a la RooFit ✓ See API 7 / 7