Learning interacting kernels of mean-field equations of particle - PowerPoint PPT Presentation

Learning interacting kernels of mean-field equations of particle systems Fei Lu Department of Mathematics, Johns Hopkins University Joint work with Quanjun Lang Related work with: Mauro Maggioni, Sui Tang, Ming Zhong, Zhongyang Li and Cheng Zhang October 28, 2020 Junior Colloquium, JHU FL acknowledges supports from JHU, NSF

An inverse problem Nonparametric Learning Numerical examples Outline Motivation and problem statement 1 Nonparametric Learning 2 Numerical examples 3 Ongoing work and open problems 4 2 / 24

An inverse problem Motivation Nonparametric Learning Previous work Numerical examples An inverse problem Consider the mean-field equation x ∈ R d , t > 0 , ∂ t u = ν ∆ u + ∇ · [ u ( K φ ∗ u )] , where K φ ( x ) = ∇ (Φ( | x | )) = φ ( | x | ) x | x | . Question: identify φ from data { u ( x m , t l ) } M , L m , l = 1 ? Goal: An algorithm → � φ identifiability: function space of learning convergence rate when ∆ x = M − 1 / d → 0 3 / 24

An inverse problem Motivation Nonparametric Learning Previous work Numerical examples Motivation ∂ t u = ν ∆ u + ∇ · [ u ( K φ ∗ u )] Interacting particles/agents: � N t | ) X j √ t − X i d t = 1 φ ( | X j dt X i t − X i t 2 ν dB i + t , i = 1 , . . . , N | X j N t − X i t | i ′ = 1 X i t : the i-th particle’s position; B i t : Brownian motion � N i = 1 δ ( X i u ( x , t ) = lim N →∞ t − x ) Propagation of chaos 1st- and 2nd-order models Application in many disciplines: Statistical physics, quantum mechanics Biology [Keller-Segal1970, Cucker-Smale2000] Social science [Motsch-Tadmor2014] Monte Carlo sampling [Del Moral13] Epidemiology (Agent-based model for COVID19 at Imperial) 4 / 24

An inverse problem Motivation Nonparametric Learning Previous work Numerical examples Previous work: finite N ∂ t u = ν ∆ u + ∇ · [ u ( K φ ∗ u )] Interacting particles/agents: N t | ) X j � √ t − X i d t = 1 φ ( | X j dt X i t − X i t 2 ν dB i + t , i = 1 , . . . , N | X j N t − X i t | i ′ = 1 Maggioni JHU team: [M., L., Tang, Zhong, Miller, Li, Zhang: PNAS19, SPA20, etc] Data: many trajectories { X ( m ) [ 0 , T ] } M m = 1 , ν = 0 ; ν > 0, finite N Function space of learning: φ ∈ L 2 ( ρ T ) with ρ T ← | X j t − X i t | Nonparametric estimation ( Ac = b ) )) Opinion Dynamics Lennard-Jones Prey-Predator 5 / 24

An inverse problem Motivation Nonparametric Learning Previous work Numerical examples Previous work: finite N ∂ t u = ν ∆ u + ∇ · [ u ( K φ ∗ u )] Interacting particles/agents: N � t | ) X j √ t − X i d t = 1 dt X i φ ( | X j t − X i t 2 ν dB i + t , i = 1 , . . . , N | X j N t − X i t | i ′ = 1 Maggioni JHU team: [M., L., Tang, Zhong, Miller, Li, Zhang: PNAS19, SPA20, etc] Identifiability: a coercivity condition for L 2 ( ρ T ) Optimal convergence rate: s E µ 0 [ � � 2 s + 1 . φ T , M , H n ∗ − φ true � L 2 ( ρ T ) ] ≤ C ((log M ) / M ) Learning rate -5 -0.6 errors Rel Err Slope=-0.34 slope -0.36 Optimal decay -6 -0.8 log 2 (error) optimal decay log 10 (Rel Err) -7 -1 -8 -1.2 -9 -1.4 -10 -1.6 12 13 14 15 16 17 18 19 20 21 2.5 3 3.5 4 log 10 (M) log 2 (M) Opinion Dynamics Lennard-Jones Prey-Predator 6 / 24

An inverse problem Motivation Nonparametric Learning Previous work Numerical examples What if N → ∞ ? Data: many trajectories { X ( m ) [ 0 , T ] } M m = 1 ; � N i = 1 δ ( X i Data: density u ( x , t ) = lim N →∞ t − x ) { u ( x m , t l ) } M , L m , l = 1 ∂ t u = ν ∆ u + ∇ · [ u ( K φ ∗ u )] 7 / 24

An inverse problem Motivation Nonparametric Learning Previous work Numerical examples What if N → ∞ ? Data: many trajectories { X ( m ) [ 0 , T ] } M m = 1 ; � N i = 1 δ ( X i Data: density u ( x , t ) = lim N →∞ t − x ) { u ( x m , t l ) } M , L m , l = 1 ∂ t u = ν ∆ u + ∇ · [ u ( K φ ∗ u )] How to estimate φ from data? � T � � � � 2 dx dt ? � ∇ . ( u ( K ψ ∗ u )) − g Minimize E 0 ( ψ ) = 0 R d (with g = ∂ t u − ν ∆ u ) Derivatives not available from data. 8 / 24

An inverse problem A probabilistic error functional Nonparametric Learning Identifiability Numerical examples Convergence rate Outline Motivation and problem statement 1 Nonparametric learning 2 ◮ A probabilistic error functional ◮ Identifiability: function spaces of learning ◮ Rate of convergence Numerical examples 3 Ongoing work and open problems 4 9 / 24

An inverse problem A probabilistic error functional Nonparametric Learning Identifiability Numerical examples Convergence rate A probabilistic error functional � T � �� � � E ( ψ ) := 1 � 2 u − 2 ν u ( ∇ · K ψ ∗ u ) + 2 ∂ t u (Ψ ∗ u ) � K ψ ∗ u dx dt T 0 R d = � � ψ, ψ � � G T − 2 � � ψ, φ � � G T Expectation of the negative log-likelihood of the process � √ dX t = − K φ ∗ u ( X t , t ) dt + 2 ν dB t , L ( X t ) = u ( · , t ) , Derivative-in-space free! G T is a reproducing kernel for a RKHS � T 1 � � � � � φ, ψ � � GT := R d � ( K φ ∗ u ) , ( K ψ ∗ u ) � u ( x , t ) dx dt = R + φ ( r ) ψ ( s ) G T ( r , s ) dr ds T R + 0 ψ = � n i = 1 c i φ i ⇒ E ( ψ ) = c ⊤ Ac − 2 b ⊤ c with A ij = � � φ i , φ j � � G T n � � c = A − 1 b � � ⇒ Estimator: φ n = c i φ i , i = 1 10 / 24

An inverse problem A probabilistic error functional Nonparametric Learning Identifiability Numerical examples Convergence rate Discrete data From data { u ( x m , t l ) } M , L m , l = 1 : H n = span { φ i } n i = 1 , n � � c i c n , M , L = A − 1 � with � φ n , M , L = n , M , L φ i , n , M , L b n , M , L . i = 1 Inverse problem: well-posed/ identifiable, A − 1 ? Choice of H n : { φ i } and n ? Convergence rate when ∆ x = M − 1 / d → 0? → hypothesis testing and model selection 11 / 24

An inverse problem A probabilistic error functional Nonparametric Learning Identifiability Numerical examples Convergence rate Invertibility of A and function space Recall that H = span { φ i } n i = 1 , � � � � A ij = φ i , φ j G T , with integral kernel G T → RKHS H G T . if { φ i } orthonormal in H G T : A = I n if { φ i } orthonormal in L 2 ( ρ T ) : minimal eigenvalue of A = c H , T = ψ ∈H , � ψ � L 2 ( ρ T ) = 1 � inf � ψ, ψ � � G T > 0 ( Coercivity condition ) ′ ◮ measure ρ T ← | X t − X t | (“pairwise distance”) 12 / 24

An inverse problem A probabilistic error functional Nonparametric Learning Identifiability Numerical examples Convergence rate Error bounds H = L 2 ( ρ T ) or RKHS H G T . Theorem (Lang-Lu20) i = 1 and � Let H = span { φ i } n φ n the projection of φ on H ⊂ H . Assume regularity conditions. Then C b √ − 1 � � � � φ n , M , L − � n + C A n � φ � H φ n � H ≤ 2 c H , T (∆ x + ∆ t ) , If if H = L 2 ( ρ T ) : assume coercivity condition on H with c H , T > 0, if H = RKHS, set c H , T = 1 ∆ x + ∆ t comes from numerical integrator (Riemann sum) Dominating order: n ∆ x (if ∆ t = 0) 13 / 24

An inverse problem A probabilistic error functional Nonparametric Learning Identifiability Numerical examples Convergence rate Optimal dimension and rate of convergence Total error: trade-off � � φ n , M , ∞ − φ � H ≤ � � φ n , M , ∞ − � � � φ n � H + φ n − φ � H � �� � � �� � approximation error inference error Theorem (Lang-Lu20) φ n � H � n (∆ x ) α and � � Assume � � φ n , M , ∞ − � φ n − φ � H � n − s . Then, with optimal dimension n ≈ (∆ x ) − α/ ( s + 1 ) : � � φ n , M , ∞ − φ � H � (∆ x ) α s / ( s + 1 ) 14 / 24

An inverse problem Smooth kernel Nonparametric Learning Non-smooth kernel Numerical examples Singular kernel Outline Motivation and problem statement 1 Nonparametric learning 2 Numerical examples 3 ◮ Granular media: smooth kernel φ ( r ) = 3 r 2 ◮ Opinion dynamics: piecewise linear φ ◮ Repulsion-attraction: singular φ Ongoing work and open problems 4 15 / 24

An inverse problem Smooth kernel Nonparametric Learning Non-smooth kernel Numerical examples Singular kernel Numerical example 1: granular media x ∈ R d , t > 0 , ∂ t u = ν ∆ u + ∇ · [ u ( K φ ∗ u )] , where K φ ( x ) = ∇ (Φ( | x | )) = φ ( | x | ) x | x | . φ ( r ) = 3 r 2 0.025 Original initial New initial Wasserstein distance 0.02 0.015 0.01 0.005 0 0 0.5 1 Time t Wasserstein W 2 ( u , � The solution u ( x , t ) Estimators of φ u ) 16 / 24

An inverse problem Smooth kernel Nonparametric Learning Non-smooth kernel Numerical examples Singular kernel Numerical example 1: granular media The solution u ( x , t ) Estimators of φ -6.58 + 10 0 -6.58 + 10 -1 Error functional E 10 0 -6.58 + 10 -2 -6.58 + 10 -3 Test point Test point -6.58 + 10 -4 Slope = 1.64 Slope = 4.05 10 -1 Optimal = 2.00 Optimal = 4.00 -6.58 + 10 -5 10 -1 10 0 10 -1 10 0 x x Convergence rate of L 2 ( ρ T ) error Convergence rate of E M , L almost optimal almost optimal 17 / 24