UQ bounds for dynamics Accelerated Sensitivity Screening UQ bounds and sensitivity screening UQ Information Inequalities, variational inference and accelerated sensitivity screening Markos Katsoulakis Mathematics & Statistics University of Massachusetts, Amherst Turing Gateway to Mathematics - December 2015 Markos Katsoulakis Mathematics & Statistics University of Massachusetts, Amherst UQ Information Inequalities, variational inference and accelerated sensitivit
UQ bounds for dynamics Accelerated Sensitivity Screening UQ bounds and sensitivity screening Outline: ◮ Uncertainty Quantification Information Inequalities: ◮ What do Information metrics and Variational Inference imply for Quantities of Interest (observables)? ◮ Methods that scale appropriately for: ◮ High dimensional state/parameter space systems ◮ Long times in the case of dynamics ◮ Inequalities provide a tool to address UQ, inference and transferability questions in CG. ◮ Demonstrate methods in accelerated sensitivity screening for complex chemical reaction networks and Langevin molecular dynamics. Markos Katsoulakis Mathematics & Statistics University of Massachusetts, Amherst UQ Information Inequalities, variational inference and accelerated sensitivit
UQ bounds for dynamics Accelerated Sensitivity Screening UQ bounds and sensitivity screening Acknowledgments ◮ Paul Dupuis (Brown University) ◮ Petr Plechac (University of Delaware) ◮ Luc Rey-Bellet (University of Massachusetts, Amherst) ◮ Dion Vlachos (Chem. Engineering & EFRC, University of Delaware) ◮ Georgios Arampatzis (ETH, Zurich) ◮ Yannis Pantazis (University of Massachusetts, Amherst) ◮ Kostis Gourgoulias (University of Massachusetts, Amherst) ◮ Jie Wang (University of Massachusetts, Amherst) Markos Katsoulakis Mathematics & Statistics University of Massachusetts, Amherst UQ Information Inequalities, variational inference and accelerated sensitivit
UQ bounds for dynamics Accelerated Sensitivity Screening UQ bounds and sensitivity screening Why information-based methods? ◮ Pseudo-distance between probability measures P , Q : � dP � � R KL ( P | Q ) := log dP dQ ◮ Properties: (i) R KL ( P | Q ) ≥ 0 and (ii) R KL ( P | Q ) = 0 iff P = Q a.e. ◮ Other probability metrics and divergences: Total Variation, Hellinger, χ 2 , F-divergence, etc. ◮ Is relative entropy special? see later. ◮ Drawbacks: need absolute continuity, i.e. some probability models cannot be compared. Need other methods (stochastic coupling, Malliavin calculus, etc) Markos Katsoulakis Mathematics & Statistics University of Massachusetts, Amherst UQ Information Inequalities, variational inference and accelerated sensitivit
UQ bounds for dynamics Accelerated Sensitivity Screening UQ bounds and sensitivity screening Information metrics in inference, model selection and UQ ◮ Variational inference methods in machine learning ◮ Variational inference for building coarse-grained models in materials ◮ Information metrics for UQ and sensitivity analysis of stochastic models ◮ Information metrics for quantifying predictive skill in model selection/reduction Markos Katsoulakis Mathematics & Statistics University of Massachusetts, Amherst UQ Information Inequalities, variational inference and accelerated sensitivit
UQ bounds for dynamics Accelerated Sensitivity Screening UQ bounds and sensitivity screening Variational inference and coarse-graining Loss of Information in Coarse-Graining, CG error, model fidelity : K., Vlachos J. Chem. Phys. (2003), Majda, Abramov (2006), K., Plechac, Rey-Bellet, Tsagkarogiannis (2014), Chen, Tong, Majda (2014) ... R KL ( P | Q ) = N × O ( ǫ p ) , N = system size , ǫ = tolerance CG Parametrizations via Variational Inference: Shell (2008, 2012), Noid et al (2011), Espanol, Zuninga (2011), Bilionis, Koutsourelakis (2012), Bilionis, Zabaras (2013), K., Plechac (2013) ... min θ R KL ( P | Q ( θ )) . Machine/Statistical Learning via Variational inference: Amari (1998), Jordan et al (1999), Bottou (2003), Wainright, Jordan (2008), Hoffman, Blei, Wang, Paisley (2013)... Markos Katsoulakis Mathematics & Statistics University of Massachusetts, Amherst UQ Information Inequalities, variational inference and accelerated sensitivit
UQ bounds for dynamics Accelerated Sensitivity Screening UQ bounds and sensitivity screening Coarse-graining in molecular simulations ◮ Build parametrized approx. of Gibbs states: µ app ∼ e − β ¯ H app ( η ; θ ) ¯ ◮ Find optimal values of parameters θ ∗ , such that � µ app [ φ i ] | 2 min | E µ [ φ i ] − E ¯ θ i for selected observables φ i , e.g. F. Muller-Plathe, Chem. Phys. Chem. (’02) ◮ Parametrization depends on specific observable(s) φ . ◮ Special case: Force-matching methods, [G. Voth and collaborators] Markos Katsoulakis Mathematics & Statistics University of Massachusetts, Amherst UQ Information Inequalities, variational inference and accelerated sensitivit
UQ bounds for dynamics Accelerated Sensitivity Screening UQ bounds and sensitivity screening Coarse Graining-Parameterization-Transferability ◮ Need to ensure accurate simulation of other observables, which are not part of the parameterization, i.e. ◮ Can we improve the ”transferability”/predictability of the parametrizations? ◮ Recall the CKP inequality: for any observable φ , � | E P [ φ ] − E Q [ φ ] | ≤ || φ || ∞ 2 R KL ( P | Q ) R KL ( P | Q ): Loss of Information during Coarse Graining Markos Katsoulakis Mathematics & Statistics University of Massachusetts, Amherst UQ Information Inequalities, variational inference and accelerated sensitivit
UQ bounds for dynamics Accelerated Sensitivity Screening UQ bounds and sensitivity screening Sensitivity analysis - Information Theory Methods ◮ Consider parametrized probabilistic models P θ ( x ) dx . ◮ Info Theory concepts: Relative entropy, Fisher Information matrix, Mutual information, etc. P θ | P θ + ǫ � R KL � = Loss of Information due to perturbation by ǫ H. Liu, W. Chen, and A. Sudjianto, J. Mech. Des. (2006). N. Ludtke et al., J. Royal Soc., Interface (2008). A. J. Majda and B. Gershgorin, Proc. Natl. Acad. Sci. (2010). M. Komorowski et al, Proc. Natl. Acad. Sci. (2011). ◮ The PDF is assumed known, e.g. Gibbs µ ∼ Z − 1 e − H ( σ ) . Allows for explicit calculations of R : ∼ E µ θ [ H θ + ǫ − H θ ] + log Z θ + ǫ µ θ | µ θ + ǫ � � R KL Z θ Markos Katsoulakis Mathematics & Statistics University of Massachusetts, Amherst UQ Information Inequalities, variational inference and accelerated sensitivit
UQ bounds for dynamics Accelerated Sensitivity Screening UQ bounds and sensitivity screening Sensitivity analysis - Information Theory Methods ◮ Consider parametrized probabilistic models P θ ( x ) dx . ◮ Info Theory concepts: Relative entropy, Fisher Information matrix, Mutual information, etc. P θ | P θ + ǫ � R KL � = Loss of Information due to perturbation by ǫ H. Liu, W. Chen, and A. Sudjianto, J. Mech. Des. (2006). N. Ludtke et al., J. Royal Soc., Interface (2008). A. J. Majda and B. Gershgorin, Proc. Natl. Acad. Sci. (2010). M. Komorowski et al, Proc. Natl. Acad. Sci. (2011). ◮ The PDF is assumed known, e.g. Gibbs µ ∼ Z − 1 e − H ( σ ) . Allows for explicit calculations of R : ∼ E µ θ [ H θ + ǫ − H θ ] + log Z θ + ǫ µ θ | µ θ + ǫ � � R KL Z θ ◮ However, typically this is not the case in dynamics, non-equilibrium systems, non-gaussian fluctuations, etc. Information metrics in path space. KL divergence per unit time. Markos Katsoulakis Mathematics & Statistics University of Massachusetts, Amherst UQ Information Inequalities, variational inference and accelerated sensitivit
UQ bounds for dynamics Accelerated Sensitivity Screening UQ bounds and sensitivity screening Complex Reaction Networks " 10 6 Intermediates Reactions Number of parameters Number of Calculations 10 5 10 4 10 3 10 2 10 Sugar size from)Ref. 3 ) Right:)Example)of)reaction)network)of)a)small)oxygenate)(ethanol)on)platinum)catalyst).) Left: Number of parameters in metal-catalyzed upgrade of small biomass derivatives for the production of renewable fuels and chemicals vs. molecular size. The reaction network even of typical sugars, such as glucose, entails nearly a million of parameters. Right: Example of reaction network of a small oxygenate (ethanol on platinum catalyst); the thickness of lines indicates the reaction flux. J. E. Sutton and D. G. Vlachos. Chem. Engin. Sci. 2015. Markos Katsoulakis Mathematics & Statistics University of Massachusetts, Amherst UQ Information Inequalities, variational inference and accelerated sensitivit
UQ bounds for dynamics Accelerated Sensitivity Screening UQ bounds and sensitivity screening Deterministic Sensitivity Analysis - Background ◮ System of ODEs: y (0) = y 0 ∈ R N y = f ( y ; θ ) ˙ , • Goal: Perform SA on the model parameters θ ∈ R K . ◮ Define sensitivity indices: s k = ∂ y ∂θ k ◮ A new system of ODEs is derived and augmented to the previous: s k = ∂ f ∂ y s k + ∂ f ˙ , k = 1 , ..., K ∂θ k • need to solve K × N additional equations. Markos Katsoulakis Mathematics & Statistics University of Massachusetts, Amherst UQ Information Inequalities, variational inference and accelerated sensitivit
Recommend
More recommend
