A Randomized Likelihood Method for Data Reduction in Large-scale - PowerPoint PPT Presentation

A Randomized Likelihood Method for Data Reduction in Large-scale Inverse Problems Ellen B. Le Tan Bui-Thanh Aaron Myers Institute for Computational Engineering and Sciences (ICES) The University of Texas at Austin SIAM CSE 15 Salt Lake City π -day, 2015 Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 1 / 31

Big data, big (inverse) problems An inverse problem: find parameters of a model given real observations. Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 2 / 31

Big data, big (inverse) problems An inverse problem: find parameters of a model given real observations. y obs is our (noisy) data vector, Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 2 / 31

Big data, big (inverse) problems An inverse problem: find parameters of a model given real observations. y obs is our (noisy) data vector, ex. temperature measurements on a thermal fin ǫ ∼ N ( 0 , Γ) y obs := w ( x j ) + ǫ j , j = 1 , . . . N j Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 2 / 31

Big data, big (inverse) problems An inverse problem: find parameters of a model given real observations. y obs is our (noisy) data vector, ex. temperature measurements on a thermal fin ǫ ∼ N ( 0 , Γ) y obs := w ( x j ) + ǫ j , j = 1 , . . . N j u is the parameter we want Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 2 / 31

Big data, big (inverse) problems An inverse problem: find parameters of a model given real observations. y obs is our (noisy) data vector, ex. temperature measurements on a thermal fin ǫ ∼ N ( 0 , Γ) y obs := w ( x j ) + ǫ j , j = 1 , . . . N j u is the parameter we want Physics model: −∇ · ( e u ∇ w ) = 0 in Ω − e u ∇ w · n = Bi w on ∂ Ω \ Γ R , − e u ∇ w · n = − 1 on Γ R , Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 2 / 31

Big data, big (inverse) problems An inverse problem: find parameters of a model given real observations. y obs is our (noisy) data vector, ex. temperature measurements on a thermal fin ǫ ∼ N ( 0 , Γ) y obs := w ( x j ) + ǫ j , j = 1 , . . . N j u is the parameter we want Physics model:  −∇ · ( e u ∇ w ) = 0 in Ω   − e u ∇ w · n = Bi w on ∂ Ω \ Γ R , ⇒ G ( u ) = w , Our forward map  − e u ∇ w · n = − 1  on Γ R , Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 2 / 31

Big data, big (inverse/optimization) problems Minimize the cost. Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 3 / 31

Big data, big (inverse/optimization) problems Minimize the cost. Cost is � � u ; y obs , u 0 J s.t. −∇ · ( e u ∇ w ) = 0 in Ω − e u ∇ w · n = Bi w on ∂ Ω \ Γ R , − e u ∇ w · n = − 1 on Γ R , Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 3 / 31

Big data, big (inverse/optimization) problems Minimize the cost. Cost is � � � � := 1 2 � y obs − G ( u ) u ; y obs , u 0 � � J � 2 Γ � �� � data misfit s.t. −∇ · ( e u ∇ w ) = 0 in Ω − e u ∇ w · n = Bi w on ∂ Ω \ Γ R , − e u ∇ w · n = − 1 on Γ R , Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 3 / 31

Big data, big (inverse/optimization) problems Minimize the cost. Cost is � � � � := 1 1 2 � y obs − G ( u ) u ; y obs , u 0 2 � u − u 0 � 2 � � J + � C 2 Γ � �� � � �� � data misfit some regularization s.t. −∇ · ( e u ∇ w ) = 0 in Ω − e u ∇ w · n = Bi w on ∂ Ω \ Γ R , − e u ∇ w · n = − 1 on Γ R , Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 3 / 31

More data, higher numerical rank −∇ · ( e u ∇ w ) = 0 in Ω − e u ∇ w · n = Bi w on ∂ Ω \ Γ R , − e u ∇ w · n = − 1 on Γ R , Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 4 / 31

More data, higher numerical rank −∇ · ( e u ∇ w ) = 0 in Ω − e u ∇ w · n = Bi w on ∂ Ω \ Γ R , − e u ∇ w · n = − 1 on Γ R , 6 5 4 3 2 1 0 0.2 0.4 0.6 0.8 1 observation locations Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 4 / 31

More data, higher numerical rank −∇ · ( e u ∇ w ) = 0 in Ω − e u ∇ w · n = Bi w on ∂ Ω \ Γ R , − e u ∇ w · n = − 1 on Γ R , 2 6 10 2001 data 993 data Misfit Hessian singular values 497 data 5 249 data 0 125 data 10 63 data 4 3 −2 10 2 −4 1 10 0 0.2 0.4 0.6 0.8 1 0 10 20 30 40 50 60 observation locations index Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 4 / 31

More data, higher numerical rank −∇ · ( e u ∇ w ) = 0 in Ω − e u ∇ w · n = Bi w on ∂ Ω \ Γ R , − e u ∇ w · n = − 1 on Γ R , 6 5 4 3 2 1 0 0.2 0.4 0.6 0.8 1 observation locations Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 5 / 31

More data, higher numerical rank −∇ · ( e u ∇ w ) = 0 in Ω − e u ∇ w · n = Bi w on ∂ Ω \ Γ R , − e u ∇ w · n = − 1 on Γ R , 2 6 10 Misfit Hessian singular values 5 2001 data 1 10 1001 data 501 data 4 251 data 0 10 126 data 63 data 3 −1 10 2 −2 1 10 0 0.2 0.4 0.6 0.8 1 0 10 20 30 40 50 60 observation locations index Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 5 / 31

Big data issues in large-scale inverse problems Big data issues More data 1 Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 6 / 31

Big data issues in large-scale inverse problems Big data issues More data → higher rank 1 Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 6 / 31

Big data issues in large-scale inverse problems Big data issues More data → higher rank → more PDE solves 1 Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 6 / 31

Big data issues in large-scale inverse problems Big data issues More data → higher rank → more PDE solves → more $$$ 1 Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 6 / 31

Big data issues in large-scale inverse problems Big data issues More data → higher rank → more PDE solves → more $$$ 1 There’s a lot of redundancy in big data 2 Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 6 / 31

Big data issues in large-scale inverse problems Big data issues More data → higher rank → more PDE solves → more $$$ 1 There’s a lot of redundancy in big data 2 Furthermore, carrying big data along (I/O, data moving, etc) 3 large-scale inversion is costly Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 6 / 31

Big data issues in large-scale inverse problems Big data issues More data → higher rank → more PDE solves → more $$$ 1 There’s a lot of redundancy in big data 2 Furthermore, carrying big data along (I/O, data moving, etc) 3 large-scale inversion is costly Challenge How to reduce the cost for big-data-meets-big-inverse-problems? Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 6 / 31

Big data issues in large-scale inverse problems Big data issues More data → higher rank → more PDE solves → more $$$ 1 There’s a lot of redundancy in big data 2 Furthermore, carrying big data along (I/O, data moving, etc) 3 large-scale inversion is costly Challenge How to reduce the cost for big-data-meets-big-inverse-problems? An answer: reduce the amount of data. Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 6 / 31

Big data issues in large-scale inverse problems Big data issues More data → higher rank → more PDE solves → more $$$ 1 There’s a lot of redundancy in big data 2 Furthermore, carrying big data along (I/O, data moving, etc) 3 large-scale inversion is costly Challenge How to reduce the cost for big-data-meets-big-inverse-problems? An answer: reduce the amount of data. BUT HOW? Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 6 / 31

A Randomized Likelihood Method for Big Data � � �� misfit = 1 2 y obs − G ( u ) � Γ − 1 � � 2 � 2 Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 7 / 31

A Randomized Likelihood Method for Big Data � � �� misfit = 1 2 y obs − G ( u ) � Γ − 1 � � 2 � 2 � �� 1 � εε T � � 2 y obs − G ( u ) Γ − 1 � � , e.g. ε ∼ N ( 0 , I ) � E ε 2 � 2 Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 7 / 31

A Randomized Likelihood Method for Big Data � � �� misfit = 1 2 y obs − G ( u ) � Γ − 1 � � 2 � 2 � �� = 1 � εε T � � 2 y obs − G ( u ) Γ − 1 � � , e.g. ε ∼ N ( 0 , I ) � E ε 2 � 2 Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 7 / 31

A Randomized Likelihood Method for Big Data � � �� misfit = 1 2 y obs − G ( u ) � Γ − 1 � � 2 � 2 � �� = 1 � εε T � � 2 y obs − G ( u ) Γ − 1 � � , e.g. ε ∼ N ( 0 , I ) � E ε 2 � 2 �� �� 2 � � = 1 y obs − G ( u ) ε T Γ − 1 2 E ε 2 Bui-Thanh, Le, Myers (ICES, UT Austin) Big data meets big inversions 7 / 31