Cutting Convex Sets With Margin Shay Moran (Google AI and Technion) - PowerPoint PPT Presentation

Cutting Convex Sets With Margin Shay Moran (Google AI and Technion)

Background • A geometric problem that arises in Density Estimation • Density Estimation = Distribution Learning w.r.t Total Variation • Progress on problem ⟹ improved sample complexity bounds for optimal density estimators • Based on joint works with Olivier Bousquet, Mark Braverman, Klim Efremenko, Daniel Kane, and Gillat Kol

The Game Fix a norm ! on ℝ ! and 𝜗 > 0 • 𝑦, 𝑠) = ball of radius 𝑠 around ⃗ 𝐶( ⃗ 𝑦 • Cutting Game Player versus Adversary • 𝐶 0 = 𝐶(0, 1) • At round 𝑢 = 0,1, … • Player picks ⃗ 𝑦 " ∈ 𝐶 " • Adversary picks halfspace 𝐼 " disjoint from 𝐶 ⃗ 𝑦 " , 𝜗 • Update 𝐶 "#$ = 𝐶 " ∩ 𝐼 " • Player wants to reach 𝐶 " = ∅ as fast as possible • 𝜗

Example: ℓ ! in 2 dimensions Cu#ng Game Player versus Adversary • 𝑪 𝟏 = 𝑪(𝟏, 𝟐) • At round 𝑢 = 0,1, … • Player picks ⃗ 𝑦 ! ∈ 𝐶 ! • Adversary picks halfspace 𝐼 ! disjoint from 𝐶 ⃗ 𝑦 ! , 𝜗 • Update 𝐶 !"# = 𝐶 ! ∩ 𝐼 ! • Player wants to reach 𝐶 ! = ∅ as fast as possible •

Example: ℓ ! in 2 dimensions Example: ℓ ! in 2 dimensions Cutting Game Cutting Game Round Player versus Adversary Player versus Adversary • • 1 𝐶 0 = 𝐶(0, 1) ! 0 = !(0, 1) • • At round 𝑢 = 0,1, … At round ( = 0,1, … • • Player picks 𝒚 𝒖 ∈ 𝑪 𝒖 Player picks * ! ∈ , ! • • Adversary picks halfspace - " disjoint from ! ⃗ Adversary picks halfspace 𝐼 ! disjoint from 𝐶 ⃗ 𝑦 ! , 𝜗 / " , 0 • • Update 𝐶 !"# = 𝐶 ! ∩ 𝐼 ! Update ! "#$ = ! " ∩ - " • • Player wants to reach ! " = ∅ as fast as possible Player wants to reach 𝐶 ! = ∅ as fast as possible • •

Example: ℓ ! in 2 dimensions Cutting Game Round Player versus Adversary • 1 𝐶 0 = 𝐶(0, 1) • At round 𝑢 = 0,1, … • Player picks ⃗ 𝑦 ! ∈ 𝐶 ! • Adversary picks halfspace 𝑰 𝒖 disjoint from 𝑪 𝒚 𝒖 , 𝝑 • Update 𝐶 !"# = 𝐶 ! ∩ 𝐼 ! • Player wants to reach 𝐶 ! = ∅ as fast as possible •

Example: ℓ ! in 2 dimensions Cu#ng Game Round Player versus Adversary • 1 𝐶 0 = 𝐶(0, 1) • At round 𝑢 = 0,1, … • Player picks ⃗ 𝑦 ! ∈ 𝐶 ! • Adversary picks halfspace 𝐼 ! disjoint from 𝐶 ⃗ 𝑦 ! , 𝜗 • Update 𝑪 𝒖"𝟐 = 𝑪 𝒖 ∩ 𝑰 𝒖 • Player wants to reach 𝐶 ! = ∅ as fast as possible •

Example: ℓ ! in 2 dimensions Cutting Game Round Player versus Adversary • 2 𝐶 0 = 𝐶(0, 1) • At round 𝑢 = 0,1, … • Player picks 𝒚 𝒖 ∈ 𝑪 𝒖 • Adversary picks halfspace 𝐼 ! disjoint from 𝐶 ⃗ 𝑦 ! , 𝜗 • Update 𝐶 !"# = 𝐶 ! ∩ 𝐼 ! • Player wants to reach 𝐶 ! = ∅ as fast as possible •

Example: ℓ ! in 2 dimensions Cu#ng Game Round Player versus Adversary • 2 𝐶 0 = 𝐶(0, 1) • At round 𝑢 = 0,1, … • Player picks ⃗ 𝑦 ! ∈ 𝐶 ! • Adversary picks halfspace 𝑰 𝒖 disjoint from 𝑪 𝒚 𝒖 , 𝝑 • Update 𝐶 !"# = 𝐶 ! ∩ 𝐼 ! • Player wants to reach 𝐶 ! = ∅ as fast as possible •

Example: ℓ ! in 2 dimensions Cutting Game Round Player versus Adversary • 2 𝐶 0 = 𝐶(0, 1) • At round 𝑢 = 0,1, … • Player picks ⃗ 𝑦 ! ∈ 𝐶 ! • Adversary picks halfspace 𝐼 ! disjoint from 𝐶 ⃗ 𝑦 ! , 𝜗 • Update 𝑪 𝒖"𝟐 = 𝑪 𝒖 ∩ 𝑰 𝒖 • Player wants to reach 𝐶 ! = ∅ as fast as possible •

Example: ℓ ! in 2 dimensions Cu#ng Game Round Player versus Adversary • 3 𝐶 0 = 𝐶(0, 1) • At round 𝑢 = 0,1, … • Player picks 𝒚 𝒖 ∈ 𝑪 𝒖 • Adversary picks halfspace 𝐼 ! disjoint from 𝐶 ⃗ 𝑦 ! , 𝜗 • Update 𝐶 !"# = 𝐶 ! ∩ 𝐼 ! • Player wants to reach 𝐶 ! = ∅ as fast as possible •

Example: ℓ ! in 2 dimensions Cutting Game Round Player versus Adversary • 3 𝐶 0 = 𝐶(0, 1) • At round 𝑢 = 0,1, … • Player picks ⃗ 𝑦 ! ∈ 𝐶 ! • Adversary picks halfspace 𝑰 𝒖 disjoint from 𝑪 𝒚 𝒖 , 𝝑 • Update 𝐶 !"# = 𝐶 ! ∩ 𝐼 ! • Player wants to reach 𝐶 ! = ∅ as fast as possible •

Example: ℓ ! in 2 dimensions Cutting Game Round Player versus Adversary • 3 𝐶 0 = 𝐶(0, 1) • At round 𝑢 = 0,1, … • Player picks ⃗ 𝑦 ! ∈ 𝐶 ! • Adversary picks halfspace 𝐼 ! disjoint from 𝐶 ⃗ 𝑦 ! , 𝜗 • Update 𝑪 𝒖"𝟐 = 𝑪 𝒖 ∩ 𝑰 𝒖 • Player wants to reach 𝐶 ! = ∅ as fast as possible •

Example: ℓ ! in 2 dimensions Cutting Game Round Player versus Adversary • 4 𝐶 0 = 𝐶(0, 1) • At round 𝑢 = 0,1, … • Player picks ⃗ 𝑦 ! ∈ 𝐶 ! • Adversary picks halfspace 𝐼 ! disjoint from 𝐶 ⃗ 𝑦 ! , 𝜗 • Update 𝐶 !"# = 𝐶 ! ∩ 𝐼 ! • Player wants to reach 𝐶 ! = ∅ as fast as possible •

Example: ℓ ! in 2 dimensions Cu#ng Game Round Player versus Adversary • 4 𝐶 0 = 𝐶(0, 1) • At round 𝑢 = 0,1, … • Player picks ⃗ 𝑦 ! ∈ 𝐶 ! • Adversary picks halfspace 𝑰 𝒖 disjoint from 𝑪 𝒚 𝒖 , 𝝑 • Update 𝐶 !"# = 𝐶 ! ∩ 𝐼 ! • Player wants to reach 𝐶 ! = ∅ as fast as possible •

Example: ℓ ! in 2 dimensions Cutting Game Round Player versus Adversary • 4 𝐶 0 = 𝐶(0, 1) • At round 𝑢 = 0,1, … • Player picks ⃗ 𝑦 ! ∈ 𝐶 ! • Adversary picks halfspace 𝐼 ! disjoint from 𝐶 ⃗ 𝑦 ! , 𝜗 • Update 𝑪 𝒖"𝟐 = 𝑪 𝒖 ∩ 𝑰 𝒖 • Player wants to reach 𝐶 ! = ∅ as fast as possible •

Example: ℓ ! in 2 dimensions Cu#ng Game Round Player versus Adversary • 5 𝐶 0 = 𝐶(0, 1) • At round 𝑢 = 0,1, … • Player picks 𝒚 𝒖 ∈ 𝑪 𝒖 • Adversary picks halfspace 𝐼 ! disjoint from 𝐶 ⃗ 𝑦 ! , 𝜗 • Update 𝐶 !"# = 𝐶 ! ∩ 𝐼 ! • Player wants to reach 𝐶 ! = ∅ as fast as possible •

Example: ℓ ! in 2 dimensions Cutting Game Round Player versus Adversary • 5 𝐶 0 = 𝐶(0, 1) • At round 𝑢 = 0,1, … • Player picks ⃗ 𝑦 ! ∈ 𝐶 ! • Adversary picks halfspace 𝑰 𝒖 disjoint from 𝑪 𝒚 𝒖 , 𝝑 • Update 𝐶 !"# = 𝐶 ! ∩ 𝐼 ! • Player wants to reach 𝐶 ! = ∅ as fast as possible •

Example: ℓ ! in 2 dimensions Cu#ng Game Round Player versus Adversary • 5 𝐶 0 = 𝐶(0, 1) • At round 𝑢 = 0,1, … • Player picks ⃗ 𝑦 ! ∈ 𝐶 ! • Adversary picks halfspace 𝐼 ! disjoint from 𝐶 ⃗ 𝑦 ! , 𝜗 • Update 𝑪 𝒖"𝟐 = 𝑪 𝒖 ∩ 𝑰 𝒖 • Player wants to reach 𝐶 ! = ∅ as fast as possible •

Example: ℓ ! in 2 dimensions Cutting Game Round Player versus Adversary • 6 𝐶 0 = 𝐶(0, 1) • At round 𝑢 = 0,1, … • Player picks ⃗ 𝑦 ! ∈ 𝐶 ! • Adversary picks halfspace 𝐼 ! disjoint from 𝐶 ⃗ 𝑦 ! , 𝜗 • Update 𝐶 !"# = 𝐶 ! ∩ 𝐼 ! • Player wants to reach 𝑪 𝒖 = ∅ as fast as possible • Player wins at round 𝑢 = 6

The Problem 𝑈 % , 𝑒, 𝜗 : = min number of rounds 𝑈 s.t. player has a strategy that guarantees 𝐶 # = ∅ against any adversary Goal. Provide tight bounds on 𝑈 " , 𝑒, 𝜗 • Arbitrary norms? \\can be further extended to convex sets • Norm = ℓ " ?

Known Bounds 𝑈 % , 𝑒, 𝜗 : = min number of rounds 𝑈 s.t. player has a strategy that guarantees 𝐶 # = ∅ against any adversary Goal. Provide tight bounds on 𝑈 " , 𝑒, 𝜗 • ( ∀ norm % ): 𝑈 ≤ 𝑃(𝑒 log 1/𝜗 ) • ( ∃ norm % ): 𝑈 ≥ Ω(𝑒 log 1/𝜗 ) \\ ! = ℓ "

Known Bounds 𝑈 % , 𝑒, 𝜗 : = min number of rounds 𝑈 s.t. player has a strategy that guarantees 𝐶 # = ∅ against any adversary Goal. Provide tight bounds on 𝑈 " , 𝑒, 𝜗 • ℓ $ : \\related to optimal density estimator %&' ( • 𝑈 ≤ 𝑃 ) ! %&' ( # • 𝑈 ≥ Ω \\ 𝑒 ≥ & Ω $ & ) $ • ℓ " for 𝑞 ∈ 1,2 : 𝑈 ≤ 𝑃 " \\indepenent of 𝑒 ) ! "#! $ ( • ℓ " for 𝑞 ∈ (2, ∞): 𝑈 ≤ 𝑃 ) !