Polynomial Chaos Acceleration for the Bayesian Inference of Random - PowerPoint PPT Presentation

Bayesian Inference Coordinate transformation for Uncertain Correlation Function PC surrogate model Inference Example Polynomial Chaos Acceleration for the Bayesian Inference of Random Fields with Gaussian Priors and Uncertain Covariance Hyper-Parameters Ihab Sraj 1 , Olivier Le Maître 2 , 1 , 3 , Omar Knio 1 , 3 and Ibrahim Hoteit 1 1KAUST, CEMSE and PSE 3Duke University 2LIMSI-CNRS SRI-UQ Center Mechanical Engineering and Materials Ihab.Sraj@kaust.edu.sa UPR-3251, Orsay, France Science Ibrahim.Hoteit@kaust.edu.sa olm@limsi.fr omar.knio@duke.edu Omar.Knio@kaust.edu.sa www.limsi.fr/Individu/olm olivier.le.maitre@duke.edu Olivier.LeMaitre@kaust.edu.sa KAUST - MCS 350 - UQ course O.P . Le Maître et.al Change of Coordinates & PC Acceleration for Bayesian Inference

Bayesian Inference Coordinate transformation for Uncertain Correlation Function PC surrogate model Inference Example 1 Bayesian Inference 2 Coordinate transformation for Uncertain Correlation Function 3 PC surrogate model 4 Inference Example O.P . Le Maître et.al Change of Coordinates & PC Acceleration for Bayesian Inference

Bayesian Inference Coordinate transformation for Uncertain Correlation Function PC surrogate model Inference Example Inference of parameter field We want to infer a parameter field M ∈ L 2 (Ω) , from a set of observations d ∈ R m of a given process, a model u ( M ) ∈ R m that predicts the observation, the Bayesian rule to update our knowledge of M . p ( M , σ 2 o | d ) ∝ p ( d | M , σ 2 o ) p M ( M ) p o ( σ 2 o ) p ( d | M , σ 2 o ) is the likelihood of the observations, p M ( M ) is the Gaussian field’s prior, σ 2 o is an error model hyper-parameter with prior of p o ( σ 2 o ) . Classical choices are i.i.d. model errors with Gaussian distribution N ( 0 , σ 2 o ) leading to m � � − x 2 1 o ) . p ( d | M , σ 2 � p ǫ ( d i − u i ( M ) , σ 2 p ǫ ( x , σ 2 o ) = o ) , = exp 2 σ 2 � 2 πσ 2 o i = 1 o with uninformative Jeffrey’s prior for σ o . O.P . Le Maître et.al Change of Coordinates & PC Acceleration for Bayesian Inference

Bayesian Inference Coordinate transformation for Uncertain Correlation Function PC surrogate model Inference Example Gaussian field’s prior We shall consider prior M that are centered Gaussian processes with covariance function C ( x , x ′ ) . The prior M ( x ) can then be decomposed in Principal Orthogonal Components (KL decomposition), ∞ ∞ � � C ( x , x ′ ) = λ k φ k ( x ) φ k ( x ′ ) , � M ( x ) = λ k Φ k ( x ) η k , k = 1 k = 1 where the η k ’s are iid standard Gaussian random variables. Upon truncation of the expansion of M to its K dominant terms, K � � M ( x ) ≈ M K ( x ) = λ k Φ k ( x ) η k , k = 1 Inference problem for the stochastic coordinates η k ’s : p ( η , σ 2 o | d ) ∝ p ( d | η , σ 2 o ) p η ( η ) p o ( σ 2 o ) , with m 1 � � −� η � 2 / 2 p ( d | η , σ 2 � p ǫ ( d i − u i ( η ) , σ 2 p η ( η ) = ( 2 π ) K / 2 exp , o ) = o ) . i = 1 O.P . Le Maître et.al Change of Coordinates & PC Acceleration for Bayesian Inference

Bayesian Inference Coordinate transformation for Uncertain Correlation Function PC surrogate model Inference Example Uncertainty in the covariance function The selection of the covariance function affects the inference procedure and C is in general uncertain. ⇒ family of covariance functions C ( q ) with hyper-parameters q having prior p q ( q ) (also inferred). Following this approach, we write K � � M ( x , q ) ≈ M K ( x , q ) = λ k ( q )Φ k ( x , q ) η k , k = 1 where the η k ’s are still i.i.d. standard Gaussian random variables and ( λ k ( q ) , Φ k ( q )) are the dominant proper elements of C ( x , x ′ , q ) . p ( η , q , σ 2 o | d ) ∝ p ( d | η , q , σ 2 o ) p η ( η ) p q ( q ) p o ( σ 2 o ) . KL decomp many KL decomposition q ( λ k , φ k ) k =1 ,K many model solves Model solve Likelihood Posterior K η p ( d | η , q , σ 2 p ( η , q , σ 2 X p U ( η , q ) 0 ) 0 | d ) M k = λ k φ k η k k =1 change of coordinate σ 2 Use of PC surrogate 0 O.P . Le Maître et.al Change of Coordinates & PC Acceleration for Bayesian Inference

Bayesian Inference Coordinate transformation for Uncertain Correlation Function PC surrogate model Inference Example Reference Basis For any covariance parameters q , the elements of the KL expansion are solution of ˆ C ( x , x ′ , q )Φ k ( x ′ , q ) d x ′ = λ k ( q )Φ k ( x , q ) , (Φ k , Φ k ) X = 1 . Ω We observe that { Φ k ( q ) } is a CONS of L 2 (Ω) . It suggests the introduction of a reference orthonormal basis { ¯ Φ k } , defined for a prescribed reference covariance function C , and to project M k ( q ) onto this reference subspace. Φ k ( q ) = √ λ k ( q )Φ k ( q ) , it comes Let ˜ � ∞ K K � � ˜ � � b k , k ′ ( q )¯ b k , k ′ ( q ) = (˜ M k ( q ) = Φ k ( q ) η k = Φ k η k , Φ k ( q ) , Φ k ′ ) X . k = 1 k = 1 k ′ = 1 For a finite dimensional reference basis (with K modes for simplicity), it comes K K � ˜ � ¯ M k ( q ) = Φ k ( q ) η k ≈ M K = Φ k ¯ η k ( q ) , η ( q ) = B ( q ) η . ¯ k = 1 k = 1 O.P . Le Maître et.al Change of Coordinates & PC Acceleration for Bayesian Inference

Bayesian Inference Coordinate transformation for Uncertain Correlation Function PC surrogate model Inference Example Change of coordinates The effect of q are reflected by a linear change of stochastic coordinates η �→ ¯ η ( q ) , such that � � η ( q ) = B ( q ) η ⇒ Σ 2 ( q ) = E η ( q ) η t ( q ) = B ( q ) B t ( q ) . ¯ We note that ¯ η ( q ) is Gaussian with conditional density � � − η t (Σ 2 ( q )) − 1 η 1 p η ( η | q ) = exp , � 2 2 π | Σ 2 ( q ) | where | Σ 2 ( q ) | is the determinant of Σ 2 ( q ) . We shall assume Σ 2 ( q ) non-singular a.s. Regarding the selection of the reference basis : select of particular hyper-parameter value : C = C (¯ q ) use the q -averaged covariance function, ˆ ¯ C = �C� = C ( q ) p q ( q ) d q . The latter choice is optimal in terms of representation error (averaged over q ). O.P . Le Maître et.al Change of Coordinates & PC Acceleration for Bayesian Inference

Bayesian Inference Coordinate transformation for Uncertain Correlation Function PC surrogate model Inference Example Example of Gaussian covariance function Consider Ω = [ 0 , 1 ] and a Gaussian covariance function with uncertain correlation length : � � − ( x − x ′ ) 2 C ( x , x ′ , q = { l } ) = σ 2 f exp , l ∼ U [ 0 . 1 , 1 ] . 2 l 2 0 0.8 10 l = 0 . 1 l = 0 . 2 0.7 l = 0 . 3 l = 0 . 4 −5 0.6 10 l = 0 . 5 l = 1 . 0 0.5 �C� −10 λ K C 0.4 10 l = 0 . 1 0.3 l = 0 . 2 l = 0 . 3 −15 0.2 10 l = 0 . 4 l = 0 . 5 0.1 l = 1 . 0 �C� −20 0 10 −1 −0.5 0 0.5 1 1 5 9 13 17 21 25 x − x ′ K F IGURE : (Left) Reference covariance functions C ( l ) for different values of l , as indicated. Also plotted is the q -averaged covariance �C� and (Right) Spectra of the covariance functions shown in the left plot. O.P . Le Maître et.al Change of Coordinates & PC Acceleration for Bayesian Inference

Bayesian Inference Coordinate transformation for Uncertain Correlation Function PC surrogate model Inference Example Example of Gaussian covariance function We define the approximation errors : ǫ M ( K , q ) = � M ( q ) − M K ( q ) � L 2 ˆ E 2 ǫ 2 , M ( K ) = M ( K , q ) p q ( q ) d q . � M ( q ) � L 2 l = 0 . 1 0 0 10 10 l = 0 . 2 l = 0 . 3 l = 0 . 4 l = 0 . 5 −2 −2 10 10 l = 1 . 0 �C� E M ( K ) ǫ M ( K, l ) −4 −4 10 10 l = 0 . 1 l = 0 . 2 l = 0 . 3 −6 −6 l = 0 . 4 10 10 l = 0 . 5 l = 1 . 0 �C� −8 −8 10 10 1 5 9 13 17 21 25 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 K l F IGURE : (Left) Error E M ( K ) in approximating the Gaussian Process M by M K for different reference covariance functions based on selected correlation lengths l as indicated. Also plotted are results obtained with �C� . (Right) Relative error ǫ M ( K = 15 , l ) for the same cases as in the left plot. O.P . Le Maître et.al Change of Coordinates & PC Acceleration for Bayesian Inference

Bayesian Inference Coordinate transformation for Uncertain Correlation Function PC surrogate model Inference Example Example of Gaussian covariance function K = 1 K = 4 K = 7 1 1 1 0.5 0.5 0.5 0 0 0 0.2 0.2 0.2 0.4 0.4 0.4 0.6 0 0.6 0 0.6 0 0.8 1 x 0.8 1 x 0.8 1 x l l l K = 10 K = 13 K = 19 1 1 1 0.5 0.5 0.5 0 0 0 0.2 0.2 0.2 0.4 0.4 0.4 0.6 0 0.6 0 0.6 0 0.8 1 x 0.8 1 x 0.8 1 x l l l F IGURE : Dependence of eigen-functions φ k ( q ) with the length-scale hyper-parameter l and selected k as indicated. O.P . Le Maître et.al Change of Coordinates & PC Acceleration for Bayesian Inference