Stochastic algorithm for optimal transport Statistical aspects of stochastic algorithms for entropic optimal transportation between probability measures J´ er´ emie Bigot Institut de Math´ ematiques de Bordeaux Equipe Image, Optimisation et Probabilit´ es (IOP) Universit´ e de Bordeaux Joint work with Bernard Bercu (IMB, Bordeaux) Statistical modeling for shapes and imaging The Mathematics of Imaging, IHP , March 2019
Stochastic algorithm for optimal transport Motivations from of a ressource allocation problem Motivations from of a ressource allocation problem 1 Wassertein optimal transport 2 Regularized optimal transport and stochastic optimisation 3 Data-driven choice of the regularization parameter ? 4
Stochastic algorithm for optimal transport Motivations from of a ressource allocation problem An example of a ressource allocation problem Data at hand 1 : locations of Police stations in Chicago spatial locations of reported incidents of crime (with the exception of murders) in Chicago in 2014 Questions (of interest ?) : given the location of a crime, which Police station should intervene ? how updating the answer in an “online fashion” along the year ? 1. Open Data from Chicago : https://data.cityofchicago.org
Stochastic algorithm for optimal transport Motivations from of a ressource allocation problem An example of a ressource allocation problem Locations y 1 , . . . , y J of Police stations in Chicago
Stochastic algorithm for optimal transport Motivations from of a ressource allocation problem An example of a ressource allocation problem Spatial location X 1 of the first reported incident of crime in Chicago in the year 2014
Stochastic algorithm for optimal transport Motivations from of a ressource allocation problem An example of a ressource allocation problem Spatial locations X 1 , X 2 of reported incidents of crime in Chicago in chronological order
Stochastic algorithm for optimal transport Motivations from of a ressource allocation problem An example of a ressource allocation problem Spatial locations X 1 , X 2 , X 3 of reported incidents of crime in Chicago in chronological order
Stochastic algorithm for optimal transport Motivations from of a ressource allocation problem An example of a ressource allocation problem Spatial locations X 1 , . . . , X 4 of reported incidents of crime in Chicago in chronological order
Stochastic algorithm for optimal transport Motivations from of a ressource allocation problem An example of a ressource allocation problem Spatial locations X 1 , . . . , X 5 of reported incidents of crime in Chicago in chronological order
Stochastic algorithm for optimal transport Motivations from of a ressource allocation problem An example of a ressource allocation problem Spatial locations of reported incidents of crime in Chicago in chronological order (first 100)
Stochastic algorithm for optimal transport Motivations from of a ressource allocation problem An example of a ressource allocation problem Spatial locations of reported incidents of crime in Chicago in chronological order (first 1000)
Stochastic algorithm for optimal transport Motivations from of a ressource allocation problem An example of a ressource allocation problem Spatial locations X 1 , . . . , X N of reported incidents of crime in Chicago in chronological order (total N = 16104 )
Stochastic algorithm for optimal transport Motivations from of a ressource allocation problem An example of a ressource allocation problem Heat map (kernel density estimation) of spatial locations of reported incidents of crime in Chicago in 2014
Stochastic algorithm for optimal transport Wassertein optimal transport Motivations from of a ressource allocation problem 1 Wassertein optimal transport 2 Regularized optimal transport and stochastic optimisation 3 Data-driven choice of the regularization parameter ? 4
Stochastic algorithm for optimal transport Wassertein optimal transport Statistical approach to ressource allocation Modeling assumptions : spatial locations of reported incidents of crime : a sequence of iid random variables X 1 , . . . , X n sampled from an unknown probability measure µ with support X ⊂ R 2 locations of Police station : a known and discrete probability measure � J ν = ν j δ y j j = 1 where y j ∈ R 2 represent the spatial location of the j -th Police station ν j is a positive weight representing the “capacity” of each Police station (we took ν j = 1 / J that is uniform weights)
Stochastic algorithm for optimal transport Wassertein optimal transport Statistical approach to ressource allocation Point of view in this talk : ressource allocation can be solved by finding an optimal transportation map T : X → { y 1 , . . . , y J } which pushes forward µ onto ν = � J j = 1 ν j δ y j (notation : T # µ = ν ), with respect to a given cost function, e.g. a distance on X � d � 1 / p � x , y ∈ R d (here d = 2 ) ( x k − y k ) p c ( x , y ) = � x − y � ℓ p = , k = 1 Question : how doing on-line estimation of such a map using the observations X 1 , . . . , X n ∼ iid µ ?
Stochastic algorithm for optimal transport Wassertein optimal transport Optimal transport between probability measures Let T : X → { y 1 , . . . , y J } such that T # µ = ν Let Π( µ, ν ) be the set of probability measures on X × X with marginals µ and ν Definition The optimal transport problem between µ and ν is � W 0 ( µ, ν ) = c ( x , T ( x )) d µ ( x ) , (Monge’s formulation) min T : T # µ = ν X or � W 0 ( µ, ν ) = min c ( x , y ) d π ( x , y ) , (Kantorovich’s formulation) π ∈ Π( µ,ν ) X×X where c ( x , y ) is the cost function of moving mass from x to y .
Stochastic algorithm for optimal transport Wassertein optimal transport An example of semi-discrete optimal transport Optimal transport of an absolutely continuous measure µ onto a discrete measure ν (black dots)
Stochastic algorithm for optimal transport Wassertein optimal transport An example of semi-discrete optimal transport Optimal transport of µ onto the discrete measure ν (black dots) - Optimal map T for the Euclidean cost c ( x , y ) = � x − y � ℓ 2
Stochastic algorithm for optimal transport Wassertein optimal transport Semi-discrete optimal transport Unicity of an optimal mapping T : supp ( µ ) → { y 1 , . . . , y J } such that T # µ = ν given, for all 1 ≤ j ≤ J , by 1 � � T − 1 ( y j ) = x ∈ supp ( µ ) : c ( x , y j ) − v ∗ j , 0 ≤ c ( x , y k ) − v ∗ k , 0 for all 1 ≤ k ≤ J 0 ∈ R J is any maximizer of the un-regularized semi-dual where v ∗ problem of the Kantorovich’s formulation of OT. The sets { T − 1 ( y j ) } are the so-called Laguerre cells (important concept from computational geometry). 1. M´ erigot (2018), Cuturi and Peyr´ e (2017)
Stochastic algorithm for optimal transport Regularized optimal transport and stochastic optimisation Motivations from of a ressource allocation problem 1 Wassertein optimal transport 2 Regularized optimal transport and stochastic optimisation 3 Data-driven choice of the regularization parameter ? 4
Stochastic algorithm for optimal transport Regularized optimal transport and stochastic optimisation Optimal transport between probability measures Problem : computational cost of optimal transport for data analysis 1 Case of discrete measures : if � K � K µ = µ i δ x i and ν = ν j δ y j i = 1 j = 1 then the cost to evaluate W 0 ( µ, ν ) (linear program) is generally O ( K 3 log K ) 1. See the recent book by Cuturi & Peyr´ e (2018)
Stochastic algorithm for optimal transport Regularized optimal transport and stochastic optimisation Regularized optimal transport Definition (Cuturi (2013)) Let µ and ν be any probability measures supported on X . Then, the regularized optimal transport problem between µ and ν is � W ε ( µ, ν ) = c ( x , y ) d π ( x , y ) + ε KL ( π | µ ⊗ ν ) , min π ∈ Π( µ,ν ) X×X where ǫ > 0 (regularization parameter) and � � � d π � � KL ( π | ξ ) = log d ξ ( x , y ) − 1 d π ( x , y ) , with ξ = µ ⊗ ν. X×X Case of discrete measures : for ǫ > 0 Sinkhorn algorithm (iterative scheme) to compute W ε ( µ, ν ) computational cost of O ( K 2 ) at each iteration
Stochastic algorithm for optimal transport Regularized optimal transport and stochastic optimisation Stochastic optimal transport Proposition (Genevay, Cuturi, Peyr´ e and Bach (2016)) Let µ be any probability measure and ν = � J j = 1 ν j δ y j . For ε > 0 , solve the smooth concave maximization problem W ε ( µ, ν ) = max v ∈ R J H ε ( v ) , where H ε ( v ) := E [ h ε ( X , v )] � �� � Stochastic optimization where X is a random variable with distribution µ , and for x ∈ X and v ∈ R J , � J � v j − c ( x , y j ) � � � J � h ε ( x , v ) = v j ν j − ε log exp ν j − ε. ε j = 1 j = 1
Recommend
More recommend
