Quantification and Density Estimation Gadi Fibich, Tel Aviv - PowerPoint PPT Presentation

A Spline-based approach to Uncertainty Quantification and Density Estimation Gadi Fibich, Tel Aviv University  Adi Ditkowski  Amir Sagiv 1

Motivation 2

Input pulse characteristics  Elliptic typical shot average over 1000 shots 3

Input pulse characteristics  Elliptic typical shot average over 1000 shots  Varies from shot to shot  Always the case 4

After 5 meters in air Average over 1000 shots typical 5

After 5 meters in air Average over 1000 shots typical 6

After 5 meters in air Want to predict Average over 1000  Average intensity after shots 5m  Probability for 0,1,2, or 3 filaments typical 7

Mathematical model NLS initial condition output 𝜔 0 (𝑦, 𝑧) 𝜔 𝑨, 𝑦, 𝑧 8

Shot to shot variation NLS random initial condition random output 𝜔 0 (𝑦, 𝑧; 𝛽) 𝜔 𝑨, 𝑦, 𝑧; 𝛽 𝛽 - noise parameter 9

Shot to shot variation NLS random initial condition random output 𝜔 0 (𝑦, 𝑧; 𝛽) 𝜔 𝑨, 𝑦, 𝑧; 𝛽 𝛽 - noise parameter Computational goals Moment estimation Density estimation e.g., average intensity 𝐅 𝜷 𝜔 2 e.g., probability for 2 filaments 10

General setting Nonlinear initial value problem ቊ 𝑣 𝑢 𝑢, 𝒚 = 𝑅 𝒚, 𝑣 𝑣 𝑣 𝑢 = 0, 𝒚 = 𝑣 0 (𝒚)  “ Quantity of interest ” (model output) 𝑔 = 𝑔[𝑣]  e.g., 𝑔 = arg(𝑣 𝑢 𝑗 , 𝑦 𝑗 ), f = ∫ u 2 𝑒𝑦, …  𝑣, 𝑔[𝑣] not given explicitly, but can be evaluated numerically 11

General setting with randomness Add randomness (in 𝑣 0 and/or 𝑅 ) ቊ 𝑣 𝑢 𝑢, 𝒚; 𝜷 = 𝑅 𝒚, 𝑣; 𝜷 𝑣 𝑣 𝑢 = 0, 𝒚; 𝜷 = 𝑣 0 (𝒚; 𝜷)  𝜷 distributed according to a known measure  “ Quantity of interest ” (model output) 𝑔 𝜷 ≔ 𝑔[𝑣 𝑢, 𝒚; 𝜷 ]  𝑔 𝜷 is a random variable 12

General setting with randomness Add randomness (in 𝑣 0 and/or 𝑅 ) ቊ 𝑣 𝑢 𝑢, 𝒚; 𝜷 = 𝑅 𝒚, 𝑣; 𝜷 𝑣 𝑣 𝑢 = 0, 𝒚; 𝜷 = 𝑣 0 (𝒚; 𝜷)  𝜷 distributed according to a known measure  “ Quantity of interest ” (model output) 𝑔 𝜷 ≔ 𝑔[𝑣 𝑢, 𝒚; 𝜷 ]  𝑔 𝜷 is a random variable Computational goals : • Moment estimation 𝐅 𝜷 𝑔 • Density estimation Probability Density Function (PDF) of 𝑔 𝜷 13

Standard statistical methods Step I – draw samples {𝛽 1 , … , 𝛽 𝑂 } Step II – compute 𝑔 1 , … , 𝑔 𝑂 , 𝑔 𝑜 := 𝑔 𝛽 𝑜 Moment estimation Density (PDF) estimation • Histogram method 1 𝑂 • 𝑂 σ 𝑜=1 Monte-Carlo 𝐅 𝛽 𝑔 ≈ 𝑔 𝑜 • Kernel density estimators (KDE) • … • … 14

Standard statistical methods Step I – draw samples {𝛽 1 , … , 𝛽 𝑂 } Step II – compute 𝑔 1 , … , 𝑔 𝑂 , 𝑔 𝑜 := 𝑔 𝛽 𝑜 Moment estimation Density (PDF) estimation • Histogram method 1 𝑂 • 𝑂 σ 𝑜=1 Monte-Carlo 𝐅 𝛽 𝑔 ≈ 𝑔 𝑜 • Kernel density estimators (KDE) • … • … Constraint : Computation of 𝑔 𝜷 𝒌 is expensive ( e.g., solving the (3+1)D NLS ) • • Can only use a small samples {𝑔 𝛽 1 , … , 𝑔 𝛽 𝑂 } 15

Standard statistical methods Step I – draw samples {𝛽 1 , … , 𝛽 𝑂 } Step II – compute 𝑔 1 , … , 𝑔 𝑂 , 𝑔 𝑜 := 𝑔 𝛽 𝑜 Moment estimation Density (PDF) estimation • Histogram method 1 𝑂 • 𝑂 σ 𝑜=1 Monte-Carlo 𝐅 𝛽 𝑔 ≈ 𝑔 𝑜 • Kernel density estimators (KDE) • … • … • Poor approximations for small N e.g. Histogram method with N=10 samples 16

Standard statistical methods Step I – draw samples {𝛽 1 , … , 𝛽 𝑂 } Step II – compute 𝑔 1 , … , 𝑔 𝑂 , 𝑔 𝑜 := 𝑔 𝛽 𝑜 Moment estimation Density (PDF) estimation • Histogram method 1 𝑂 • 𝑂 σ 𝑜=1 Monte-Carlo 𝐅 𝛽 𝑔 ≈ 𝑔 𝑜 • Kernel density estimators (KDE) • … • … • Poor approximations for small N e.g. Histogram method with N=10 samples Exact PDF 17 17

Standard statistical methods Given a sample {𝑔 𝑂 } of 𝑔 𝛽 1 , … , 𝑔 Moment estimation Density (PDF) estimation • Histogram method 1 𝑂 • Monte-Carlo 𝐅 𝛽 𝑔 ≈ 𝑂 σ 𝑜=1 𝑔 𝑜 • Kernel density estimators (KDE) • … • … How to improve? • Above methods only use {𝑔 1 , … , 𝑔 𝑂 } • Uncertainty Quantification (UQ) approach: Utilize The relation 𝑔 𝛽 1. 2. Smoothness of 𝑔 𝛽 18

Approximation-based estimation • 𝑞 is the PDF of 𝑔 moment, PDF 𝐅 𝛽 𝑔 , 𝑞 𝑔 𝛽 approximation 𝑜 → ∞ moment, PDF 𝑔 𝑂 𝛽 𝐅 𝛽 𝑔 𝑂 , 𝑞 𝑂 can take a large sample, since computation of 𝑔 𝑂 𝛽 is cheap • 𝑞 and 𝑞 𝑂 are the PDFs of 𝑔 and 𝑞 𝑂 Questions  How accurate are E 𝛽 𝑔 − E 𝛽 𝑔 𝑂 , ||𝑞 − 𝑞 𝑂 || ?  Which approximation method should be used? 19

Approximation-based estimation • 𝑞 is the PDF of 𝑔 moment, PDF 𝐅 𝛽 𝑔 , 𝑞 𝑔 𝛽 cannot take a large sample approximation 𝑜 → ∞ moment, PDF 𝑔 𝑂 𝛽 𝐅 𝛽 𝑔 𝑂 , 𝑞 𝑂 can take a large sample, since computation of 𝑔 𝑂 𝛽 is cheap • 𝑞 and 𝑞 𝑂 are the PDFs of 𝑔 and 𝑞 𝑂 Questions  How accurate are E 𝛽 𝑔 − E 𝛽 𝑔 𝑂 , ||𝑞 − 𝑞 𝑂 || ?  Which approximation method should be used? 20

Approximation-based estimation • 𝑞 is the PDF of 𝑔 moment, PDF 𝐅 𝛽 𝑔 , 𝑞 𝑔 𝛽 cannot take a large sample approximation 𝑂 → ∞ moment, PDF 𝑔 𝑂 𝛽 𝐅 𝛽 𝑔 𝑂 , 𝑞 𝑂 can take a large sample, since computation of 𝑔 𝑂 𝛽 is cheap • 𝑞 𝑂 is the PDF of 𝑔 𝑂 Questions  Which approximation should be used?  How small are E 𝛽 𝑔 − E 𝛽 𝑔 and ||𝑞 − 𝑞 𝑂 || ? 𝑂 21

Noise dimension  One-dimensional noise 𝛽 ∈ 𝑆 − 𝑠 2  Random input power 𝜔 0 = 1 + 𝛽 𝑓  Random temperature  …  Multi-dimensional noise 𝛽 ∈ 𝑆 𝑒  Random input power and incidence angle  … 22

Noise dimension  One-dimensional noise 𝛽 ∈ 𝑆 − 𝑠 2  Random input power 𝜔 0 = 1 + 𝛽 𝑓  Random temperature  …  Multi-dimensional noise 𝛽 ∈ 𝑆 𝑒  Random input power and incidence angle  … 23

Generalized Polynomial Chaos (gPC) Standard uncertainty quantification approach:  Approximate 𝑔 using orthogonal polynomials 𝑟 𝑜 (𝛽) 𝑂−1 𝑔 𝑂 𝛽 = ෍ 𝑟 𝑜 , 𝑔 𝑟 𝑜 𝛽 𝑜=0  Spectral accuracy for moments 𝑂 = 𝑃 𝑓 −𝛿𝑂 , 𝐅 𝛽 𝑔 − 𝐅 𝛽 𝑔 𝑂 ≫ 1 if f is analytic 24

Generalized Polynomial Chaos (gPC) Standard uncertainty quantification approach:  Approximate 𝑔 using orthogonal polynomials 𝑟 𝑜 (𝛽) 𝑂−1 𝑔 𝑂 𝛽 = ෍ 𝑟 𝑜 , 𝑔 𝑟 𝑜 𝛽 𝑜=0  Spectral accuracy for moments 𝑂 = 𝑃 𝑓 −𝛿𝑂 , 𝐅 𝛽 𝑔 − 𝐅 𝛽 𝑔 𝑂 ≫ 1 if f is analytic  Problem solved 25

Generalized Polynomial Chaos (gPC) Standard uncertainty quantification approach:  Approximate 𝑔 using orthogonal polynomials 𝑟 𝑜 (𝛽) 𝑂−1 𝑔 𝑂 𝛽 = ෍ 𝑟 𝑜 , 𝑔 𝑟 𝑜 𝛽 𝑜=0  Spectral accuracy for moments 𝑂 = 𝑃 𝑓 −𝛿𝑂 , 𝐅 𝛽 𝑔 − 𝐅 𝛽 𝑔 𝑂 ≫ 1 if f is analytic  Problem solved But, Moment estimation Density estimation Spectral accuracy reached only for No theory for 𝑞 − 𝑞 𝑂 large N Will it work in practice? How to achieve ``good ’’ accuracy with e.g. 𝑶 = 𝟐𝟏 samples? 26

Example: Density estimation with gPC 𝛽 𝑔 = tanh 9𝛽 + 2 , 𝛽 ∼ Uniform [−1, 1] 𝛽 27

Example: Density estimation with gPC 𝛽 𝑔 = tanh 9𝛽 + 2 , 𝛽 ∼ Uniform [−1, 1] PDF approximation, 𝑂 = 12 𝛽 28

Example: Density estimation with gPC 𝛽 𝑔 = tanh 9𝛽 + 2 , 𝛽 ∼ Uniform [−1, 1] PDF approximation, 𝑂 = 12 𝛽 Lemma 1 𝑞 𝑧 = ෍ 𝑔 ′ 𝛽 𝑔 𝛽 =𝑧 29

Example: Density estimation with gPC 𝛽 𝑔 = tanh 9𝛽 + 2 , 𝛽 ∼ Uniform [−1, 1] PDF approximation, 𝑂 = 12 𝛽 Lemma 1 𝑞 𝑧 = ෍ 𝑔 ′ 𝛽 f′(α) 𝑔 𝛽 =𝑧 𝛽 30

Example: Density estimation with gPC 𝛽 𝑔 = tanh 9𝛽 + 2 , 𝛽 ∼ Uniform [−1, 1] PDF approximation, 𝑂 = 12 𝛽 Lemma 1 𝑞 𝑧 = ෍ 𝑔 ′ 𝛽 f′(α) 𝑔 𝛽 =𝑧 𝛽 31

Example: Density estimation with gPC 𝛽 𝑔 = tanh 9𝛽 + 2 , 𝛽 ∼ Uniform [−1, 1] PDF approximation, 𝑂 = 12 𝛽 Lemma 1 𝑞 𝑧 = ෍ 𝑔 ′ 𝛽 f′(α) 𝑔 𝛽 =𝑧 Conclusion Although gPC is spectrally accurate in 𝑀 2 , it produces “ artificial ” zero derivatives which 𝛽 ``contaminate ’’ the PDF 32