Quantifying and reducing uncertainties on sets under Gaussian - PowerPoint PPT Presentation

Introduction to Sequential Uncertainty Reduction Towards conservative excursion set estimation Quantifying and reducing uncertainties on sets under Gaussian Process priors David Ginsbourger 1,2 Acknowledgements: a number of co-authors, notably appearing via citations! 1 Idiap Research Institute, UQOD group, Martigny, Switzerland, and 2 Department of Mathematics and Statistics, IMSV, University of Bern Gaussian Process and Uncertainty Quantification Summer School September 2018, University of Sheffield david@idiap.ch; ginsbourger@stat.unibe.ch Quantif. & reducing uncertainty on sets with GPs 1 / 35

Introduction to Sequential Uncertainty Reduction Towards conservative excursion set estimation Preamble Set up: estimate a deterministic function f : x ∈ E �→ f ( x ) ∈ F and/or quantities relying on it based on a limited number of evaluations of f . david@idiap.ch; ginsbourger@stat.unibe.ch Quantif. & reducing uncertainty on sets with GPs 2 / 35

Introduction to Sequential Uncertainty Reduction Towards conservative excursion set estimation Preamble Set up: estimate a deterministic function f : x ∈ E �→ f ( x ) ∈ F and/or quantities relying on it based on a limited number of evaluations of f . Two typical examples where f stems from numerical simulations Safety engineering: x is a vector parametrizing some system and f returns an indicator of dangerousness. It is then crucial to understand which x ’s lead to “high” values of f ( x ) . david@idiap.ch; ginsbourger@stat.unibe.ch Quantif. & reducing uncertainty on sets with GPs 2 / 35

Introduction to Sequential Uncertainty Reduction Towards conservative excursion set estimation Preamble Set up: estimate a deterministic function f : x ∈ E �→ f ( x ) ∈ F and/or quantities relying on it based on a limited number of evaluations of f . Two typical examples where f stems from numerical simulations Safety engineering: x is a vector parametrizing some system and f returns an indicator of dangerousness. It is then crucial to understand which x ’s lead to “high” values of f ( x ) . Flow simulation: x stands e.g. for the medium, boundary conditions, etc. and f returns the evolution of a fluid and/or a measure of discrepancy between simulation results and given observation results. david@idiap.ch; ginsbourger@stat.unibe.ch Quantif. & reducing uncertainty on sets with GPs 2 / 35

Introduction to Sequential Uncertainty Reduction Towards conservative excursion set estimation Preamble: Bayesian approach with GP models Typical situation : f was evaluated at a set of “points” x 1 , . . . , x n ∈ D ⊂ E and one wishes to estimate a quantity relying on f and/or run new evaluations in order to improve this estimation. david@idiap.ch; ginsbourger@stat.unibe.ch Quantif. & reducing uncertainty on sets with GPs 3 / 35

Introduction to Sequential Uncertainty Reduction Towards conservative excursion set estimation Preamble: Bayesian approach with GP models Typical situation : f was evaluated at a set of “points” x 1 , . . . , x n ∈ D ⊂ E and one wishes to estimate a quantity relying on f and/or run new evaluations in order to improve this estimation. ⇒ legitimate to rely on some approximation(s) of f knowing f ( x i ) + ǫ i ( 1 ≤ i ≤ n ) . A number of approaches do exist. . . david@idiap.ch; ginsbourger@stat.unibe.ch Quantif. & reducing uncertainty on sets with GPs 3 / 35

Introduction to Sequential Uncertainty Reduction Towards conservative excursion set estimation Preamble: Bayesian approach with GP models Typical situation : f was evaluated at a set of “points” x 1 , . . . , x n ∈ D ⊂ E and one wishes to estimate a quantity relying on f and/or run new evaluations in order to improve this estimation. ⇒ legitimate to rely on some approximation(s) of f knowing f ( x i ) + ǫ i ( 1 ≤ i ≤ n ) . A number of approaches do exist. . . Principles of the Gaussian Process approach (GP): suppose that, a priori , f is a realization of a GP ( Z x ) x ∈ D and approximate f and/or the quantities of interest via the conditional distribution of Z knowing Z x i + ε i = f ( x i ) + ǫ i . david@idiap.ch; ginsbourger@stat.unibe.ch Quantif. & reducing uncertainty on sets with GPs 3 / 35

Introduction to Sequential Uncertainty Reduction Towards conservative excursion set estimation Preamble: Bayesian approach with GP models Typical situation : f was evaluated at a set of “points” x 1 , . . . , x n ∈ D ⊂ E and one wishes to estimate a quantity relying on f and/or run new evaluations in order to improve this estimation. ⇒ legitimate to rely on some approximation(s) of f knowing f ( x i ) + ǫ i ( 1 ≤ i ≤ n ) . A number of approaches do exist. . . Principles of the Gaussian Process approach (GP): suppose that, a priori , f is a realization of a GP ( Z x ) x ∈ D and approximate f and/or the quantities of interest via the conditional distribution of Z knowing Z x i + ε i = f ( x i ) + ǫ i . ⇒ very practical for sequential design of experiments. david@idiap.ch; ginsbourger@stat.unibe.ch Quantif. & reducing uncertainty on sets with GPs 3 / 35

Introduction to Sequential Uncertainty Reduction Towards conservative excursion set estimation Preamble: example inverse problem in hydrogeology david@idiap.ch; ginsbourger@stat.unibe.ch Quantif. & reducing uncertainty on sets with GPs 4 / 35

Introduction to Sequential Uncertainty Reduction Towards conservative excursion set estimation Preamble: a costly full factorial experimental design! david@idiap.ch; ginsbourger@stat.unibe.ch Quantif. & reducing uncertainty on sets with GPs 5 / 35

Introduction to Sequential Uncertainty Reduction Towards conservative excursion set estimation Preamble: a costly full factorial experimental design! david@idiap.ch; ginsbourger@stat.unibe.ch Quantif. & reducing uncertainty on sets with GPs 6 / 35

Introduction to Sequential Uncertainty Reduction Towards conservative excursion set estimation Preamble: an application of Bayesian optimization Misfit (objective function) GP mean prediction 0.0022 0.002 0.0026 0.008 0.002 ● 0.0022 0.002 0 . 0 0 2 1 0 . 0.0027 0.0028 2 9 0.0075 0 0 0 . 0 0 0.0056 0.004 2 3 0.0085 0.003 0.0081 ● 0.0085 0.0037 0.0094 0.0032 ● 0.0034 0.0042 0.0059 0.005 0.0022 0.0061 0.003 0.0041 0.0025 0.01 0.0075 0 0.0088 . 0 0 1 0.0029 ● 5 2 2 0.0028 0.002 0.002 0 0 0.007 0 . 0.0045 0.0026 0.0025 0 . 0 0 3 2 0 0.0065 0 . 0 3 0.0014 0.0015 0.0018 0.0014 5e−04 1 0 0 1 7 0 . 4 − 0 0.003 4 5e−04 2 ● e 0 ● 0 . 0 0 5 5 5 0 0 4 . 0 0 0 . 0 0 9 e − 0 . 6 0 4e−04 1 0.0037 0.007 2 0 0 . 0 5 5 3 0 6e−04 . 0.0078 0 0 4e−04 0 . 0 0.0013 0.0011 1e−04 0 0 0 . 0 . ● 7 0 0 0 5 5 1 0 4 2 0 0 . * 0 0.0025 0 0 0.0012 2e−04 0 4 0.001 . 0 0 4 5 . − 0.0015 0 e 0 8 . 0.0024 0 0.0015 0 . 0 0 0 0 . 9e−04 1 1 0 0 4 1 4 0 ● 2 0.003 7 0 0.0014 e − . 6 0 0 0 1 0.002 0.0025 . 0 0 4 0 0.0035 1 8 0 ● . 0 0 . 0 0 1 9 Expected Improvement GP standard deviation 0.0021 8e−05 0.0019 8e−04 0.00024 6e−05 1 0.0015 2e−05 2 0.00018 0.0021 0.0011 0 2e−05 0 0.002 . 0 0.0015 8e−04 0.002 2e−05 0.002 0 1e−04 0 . 0.0011 0.00026 6e−05 6e−04 0 0.0016 3 1 0.0014 5 1 0.0018 0 0.00012 0 0 . 0.0016 0.00028 . 0.0017 0 0 0 6e−05 4e−05 1 0.0017 0.00018 9 8 e − 0 0.00032 0 0.0014 0.0014 0 4 . 0 2 . 2e−04 0 0.0016 0 1 6 0 0 0 0.0019 0.00014 . 1 2e−05 0 2 0.0013 4e−04 0 6e−04 6e−04 8e−05 0.0018 . 0 0.0019 0.00014 0 6e−05 1 0.00022 6 3e−04 0.0015 0.0019 0 5 4 . 7e−04 1 0.00036 4 0 8e−05 0 0 0 0 0.002 2e−05 0 . 0 0.0016 0 0 0 0.0018 . 0 4 0.00014 1 0 0 0 . 6 9 . 0.0015 4 0.00016 0 0 0 ● 0.0015 1 8 − 2e−05 e 0 2 0.0016 . 8e−04 0.002 0 0.0017 0.0022 0 0.002 6 8 4e−05 1 1 0.00034 0.00042 3 1e−04 5 2 0 0 6e−05 4e−05 0.0021 0 . 1 0.0019 0 0 4 0 0 0 0.0023 0 0 . 0 − 2 1 0 0 . . 0 8e−05 e 0 8 david@idiap.ch; ginsbourger@stat.unibe.ch Quantif. & reducing uncertainty on sets with GPs 7 / 35

Introduction to Sequential Uncertainty Reduction Towards conservative excursion set estimation The previous example was produced in the framework of an ongoing collaboration with G. Pirot (University of Lausanne), T. Krityakierne (now at Mahidol University, Bangkok) and P . Renard (University of Neuchˆ atel). ⇒ See ongoing Hydrol. Earth Syst. Sci. Discuss. paper (2017+). david@idiap.ch; ginsbourger@stat.unibe.ch Quantif. & reducing uncertainty on sets with GPs 8 / 35

Introduction to Sequential Uncertainty Reduction Towards conservative excursion set estimation The previous example was produced in the framework of an ongoing collaboration with G. Pirot (University of Lausanne), T. Krityakierne (now at Mahidol University, Bangkok) and P . Renard (University of Neuchˆ atel). ⇒ See ongoing Hydrol. Earth Syst. Sci. Discuss. paper (2017+). Main focus today In a related set-up, how to estimate excursion sets of f using such models and dedicated sequential design strategies? david@idiap.ch; ginsbourger@stat.unibe.ch Quantif. & reducing uncertainty on sets with GPs 8 / 35