Variance Reduction for Matrix Games Yair Carmon Yujia Jin - PowerPoint PPT Presentation

Poster #212 Variance Reduction for Matrix Games Yair Carmon Yujia Jin Aaron Sidford Kevin Tian ( presenting )

Poster #212 Zero-sum games min x ∈𝒴 max y ∈𝒵 f ( x , y ) Super useful! ๏ Constraints: checks feasibility (e.g. GAN’s) y ๏ Robustness: represents uncertainty (e.g. adversarial training) y

Poster #212 Zero-sum games min x ∈𝒴 max y ∈𝒵 f ( x , y ) Super useful! ๏ Constraints: checks feasibility (e.g. GAN’s) y ๏ Robustness: represents uncertainty (e.g. adversarial training) y Ideal (approximate) solution -Nash equilibrium ϵ player is happy player is happy x y f ( x , y ) ≥ max y ′ � ∈𝒵 f ( x , y ′ � ) − ϵ f ( x , y ) ≤ min x ′ � ∈𝒴 f ( x ′ � , y ) + ϵ

Poster #212 Zero-sum games min x ∈𝒴 max y ∈𝒵 f ( x , y ) Super useful! ๏ Constraints: checks feasibility (e.g. GAN’s) y ๏ Robustness: represents uncertainty (e.g. adversarial training) y Ideal (approximate) solution -Nash equilibrium ϵ player is happy player is happy x y f ( x , y ) ≥ max y ′ � ∈𝒵 f ( x , y ′ � ) − ϵ f ( x , y ) ≤ min x ′ � ∈𝒴 f ( x ′ � , y ) + ϵ We assume is convex-concave Nash equilibrium exists f ⟹

Poster #212 Our contributions

Poster #212 Our contributions 1. Variance reduction framework 
 for general (convex-concave) min x ∈𝒴 max y ∈𝒵 f ( x , y )

Poster #212 Our contributions 1. Variance reduction framework 
 for general (convex-concave) min x ∈𝒴 max y ∈𝒵 f ( x , y ) Centered Fast algorithm gradient estimator

Poster #212 Our contributions Geometry 1. Variance reduction framework 
 matters for general (convex-concave) min x ∈𝒴 max y ∈𝒵 f ( x , y ) Centered Fast algorithm gradient estimator

Poster #212 Our contributions Geometry 1. Variance reduction framework 
 matters for general (convex-concave) min x ∈𝒴 max y ∈𝒵 f ( x , y ) Centered Fast algorithm gradient estimator 2. Concrete centered gradient estimators 
 f ( x ) = y ⊤ Ax for “Sampling from the difference”

Poster #212 Our contributions “Sampling from the difference” } Geometry 1. Variance reduction framework 
 matters for general (convex-concave) min x ∈𝒴 max y ∈𝒵 f ( x , y ) Centered New runtimes for Fast algorithm gradient estimator y ∈𝒵 y ⊤ Ax min x ∈𝒴 max 2. Concrete centered gradient estimators 
 f ( x ) = y ⊤ Ax for

Poster #212 Bilinear games y ∈𝒵 y ⊤ Ax , A ∈ ℝ m × n min x ∈𝒴 max ๏ Simplest case ๏ Local model for smooth zero-sum game ๏ Important by themselves

Poster #212 Bilinear games y ∈𝒵 y ⊤ Ax , A ∈ ℝ m × n min x ∈𝒴 max ๏ Simplest case Geometry ๏ Local model for smooth zero-sum game matters ๏ Important by themselves

Poster #212 Bilinear games y ∈𝒵 y ⊤ Ax , A ∈ ℝ m × n min x ∈𝒴 max ๏ Simplest case Geometry ๏ Local model for smooth zero-sum game matters ๏ Important by themselves simplex 𝒴 = 𝒵 = Matrix games / LP

Poster #212 Bilinear games y ∈𝒵 y ⊤ Ax , A ∈ ℝ m × n min x ∈𝒴 max ๏ Simplest case Geometry ๏ Local model for smooth zero-sum game matters ๏ Important by themselves simplex Euclidean, simplex 𝒴 = 𝒵 = 𝒴 = 𝒵 = Matrix games / LP Hard margin SVM

Poster #212 Bilinear games y ∈𝒵 y ⊤ Ax , A ∈ ℝ m × n min x ∈𝒴 max ๏ Simplest case Geometry ๏ Local model for smooth zero-sum game matters ๏ Important by themselves simplex Euclidean, simplex Euclidean 𝒴 = 𝒵 = 𝒴 = 𝒵 = 𝒴 = 𝒵 = Matrix games / LP Hard margin SVM Linear regression

Poster #212 Bilinear games y ∈𝒵 y ⊤ Ax , A ∈ ℝ m × n min x ∈𝒴 max ๏ Simplest case Geometry ๏ Local model for smooth zero-sum game matters ๏ Important by themselves Balamurugan & Bach `16 simplex Euclidean, simplex Euclidean 𝒴 = 𝒵 = 𝒴 = 𝒵 = 𝒴 = 𝒵 = Matrix games / LP Hard margin SVM Linear regression

Poster #212 Bilinear games y ∈𝒵 y ⊤ Ax , A ∈ ℝ m × n min x ∈𝒴 max ๏ Simplest case Geometry ๏ Local model for smooth zero-sum game matters ๏ Important by themselves Balamurugan & Bach `16 simplex Euclidean, simplex Euclidean 𝒴 = 𝒵 = 𝒴 = 𝒵 = 𝒴 = 𝒵 = Matrix games / LP Hard margin SVM Linear regression Our work

Poster #212 Algorithms and rates A ∈ ℝ m × n y ∈𝒵 y ⊤ Ax min x ∈𝒴 max

Poster #212 Algorithms and rates for A ∈ ℝ m × n y ∈𝒵 y ⊤ Ax simplicity min x ∈𝒴 max m ≍ n x ↦ Ax takes n 2 time

Poster #212 Algorithms and rates for A ∈ ℝ m × n y ∈𝒵 y ⊤ Ax simplicity min x ∈𝒴 max m ≍ n x ↦ Ax takes n 2 time Exact gradient (Nemirovski `04, Nesterov `07) n 2 ⋅ L ϵ

Poster #212 Algorithms and rates for L = { A ∈ ℝ m × n y ∈𝒵 y ⊤ Ax simplicity simplex-simplex min x ∈𝒴 max max ij | A ij | m ≍ n simplex-ball x ↦ Ax takes n 2 time max i ∥ A i : ∥ 2 Exact gradient (Nemirovski `04, Nesterov `07) n 2 ⋅ L ϵ

Poster #212 Algorithms and rates for L = { A ∈ ℝ m × n y ∈𝒵 y ⊤ Ax simplicity simplex-simplex min x ∈𝒴 max max ij | A ij | m ≍ n Geometry simplex-ball x ↦ Ax takes n 2 time max i ∥ A i : ∥ 2 matters Exact gradient (Nemirovski `04, Nesterov `07) n 2 ⋅ L ϵ

Poster #212 Algorithms and rates for L = { A ∈ ℝ m × n y ∈𝒵 y ⊤ Ax simplicity simplex-simplex min x ∈𝒴 max max ij | A ij | m ≍ n Geometry simplex-ball x ↦ Ax takes n 2 time max i ∥ A i : ∥ 2 matters Exact gradient Stochastic gradient (Nemirovski `04, Nesterov `07) (GK95, NJLS09, CHW10) n ⋅ L 2 n 2 ⋅ L ϵ 2 ϵ

Poster #212 Algorithms and rates for L = { A ∈ ℝ m × n y ∈𝒵 y ⊤ Ax simplicity simplex-simplex min x ∈𝒴 max max ij | A ij | m ≍ n Geometry simplex-ball x ↦ Ax takes n 2 time max i ∥ A i : ∥ 2 matters Exact gradient Stochastic gradient Variance reduction (Nemirovski `04, Nesterov `07) (GK95, NJLS09, CHW10) (our approach) n ⋅ L 2 n 2 ⋅ L n 2 + n 3/2 ⋅ L ϵ 2 ϵ ϵ

Poster #212 Algorithms and rates for L = { A ∈ ℝ m × n y ∈𝒵 y ⊤ Ax simplicity simplex-simplex min x ∈𝒴 max max ij | A ij | m ≍ n Geometry simplex-ball x ↦ Ax takes n 2 time max i ∥ A i : ∥ 2 matters Exact gradient Stochastic gradient Variance reduction (Nemirovski `04, Nesterov `07) (GK95, NJLS09, CHW10) (our approach) n ⋅ L 2 n 2 ⋅ L n 2 + n 3/2 ⋅ L ϵ 2 ϵ ϵ Image credit: Chawit Waewsawangwong

Poster #212 Algorithms and rates for L = { A ∈ ℝ m × n y ∈𝒵 y ⊤ Ax simplicity simplex-simplex min x ∈𝒴 max max ij | A ij | m ≍ n Geometry simplex-ball x ↦ Ax takes n 2 time max i ∥ A i : ∥ 2 matters Exact gradient Stochastic gradient Variance reduction (Nemirovski `04, Nesterov `07) (GK95, NJLS09, CHW10) (our approach) n ⋅ L 2 n 2 ⋅ L n 2 + n 3/2 ⋅ L ϵ 2 ϵ ϵ Image credit: Chawit Waewsawangwong

Poster #212 Algorithms and rates for L = { A ∈ ℝ m × n y ∈𝒵 y ⊤ Ax simplicity simplex-simplex min x ∈𝒴 max max ij | A ij | m ≍ n Geometry simplex-ball x ↦ Ax takes n 2 time max i ∥ A i : ∥ 2 matters Exact gradient Stochastic gradient Variance reduction (Nemirovski `04, Nesterov `07) (GK95, NJLS09, CHW10) (our approach) n ⋅ L 2 n 2 ⋅ L n 2 + n 3/2 ⋅ L ϵ 2 ϵ ϵ Image credit: Chawit Waewsawangwong

Poster #212 Algorithms and rates for L = { A ∈ ℝ m × n y ∈𝒵 y ⊤ Ax simplicity simplex-simplex min x ∈𝒴 max max ij | A ij | m ≍ n Geometry simplex-ball x ↦ Ax takes n 2 time max i ∥ A i : ∥ 2 matters Exact gradient Stochastic gradient Variance reduction (Nemirovski `04, Nesterov `07) (GK95, NJLS09, CHW10) (our approach) n ⋅ L 2 n 2 ⋅ L n 2 + n 3/2 ⋅ L ϵ 2 VR always ϵ ϵ better Geometry matters Image credit: Chawit Waewsawangwong

Poster #212 Algorithms and rates for L = { A ∈ ℝ m × n y ∈𝒵 y ⊤ Ax simplicity simplex-simplex min x ∈𝒴 max max ij | A ij | m ≍ n Geometry simplex-ball x ↦ Ax takes n 2 time max i ∥ A i : ∥ 2 matters Exact gradient Stochastic gradient Variance reduction (Nemirovski `04, Nesterov `07) (GK95, NJLS09, CHW10) (our approach) n ⋅ L 2 n 2 ⋅ L n 2 + n 3/2 ⋅ L ϵ 2 VR always VR better ϵ ϵ better for passes Ω (1) over data Geometry matters Image credit: Chawit Waewsawangwong

Poster #212 It’s all in the gradient estimator Reference point x 0

Poster #212 It’s all in the gradient estimator Centered gradient estimator g x 0 ( ⋅ ) 𝔽∥ g x 0 ( x ) − ∇ f ( x ) ∥ 2 * ≤ L 2 ∥ x − x 0 ∥ 2 and 𝔽 g x 0 ( x ) = ∇ f ( x ) Reference point x 0 Also using this concept in the Euclidean setting: VR for non-convex optimization (AH`16, RHSPS`16, FLLZ`18, ZXG`18) & bilinear saddle-point problems (BB`16)

Poster #212 It’s all in the gradient estimator Centered gradient estimator g x 0 ( ⋅ ) 𝔽∥ g x 0 ( x ) − ∇ f ( x ) ∥ 2 * ≤ L 2 ∥ x − x 0 ∥ 2 and 𝔽 g x 0 ( x ) = ∇ f ( x ) variance Reference point x 0 Also using this concept in the Euclidean setting: VR for non-convex optimization (AH`16, RHSPS`16, FLLZ`18, ZXG`18) & bilinear saddle-point problems (BB`16)