Approximating Submodular Functions Everywhere Nick Harvey February - PowerPoint PPT Presentation

Approximating Submodular Functions Everywhere Nick Harvey February 16, 2008 Joint work with M. Goemans, S. Iwata and V. Mirrokni Nick Harvey Approximating Submodular Functions Everywhere

Submodular Functions ◮ Definition f : 2 [ n ] → R is submodular if, for all A , B ⊆ [ n ]: f ( A ) + f ( B ) ≥ f ( A ∪ B ) + f ( A ∩ B ) Equivalent definition f is submodular if, for all A ⊆ B and i / ∈ B : f ( A ∪ { i } ) − f ( A ) ≥ f ( B ∪ { i } ) − f ( B ) ◮ Discrete analogue of convex functions [Lov´ asz ’83] ◮ Arise in combinatorial optimization, probability, economics (diminishing returns), geometry, etc. ◮ Fundamental Examples Rank function of a matroid, cut function of a graph, ... Nick Harvey Approximating Submodular Functions Everywhere

Optimizing Submodular Functions (Given Oracle Access) Minimization ◮ Can solve min S f ( S ) with polynomially many oracle calls [GLS], [Schrijver ’01], [Iwata, Fleischer, Fujishige ’01], ... Example: Given matroids M 1 = ( E , I 1 ) and M 2 = ( E , I 2 ) max {| I | : I ∈ I 1 ∩ I 2 } = min { r 1 ( S ) + r 2 ( E \ S ) : S ⊆ E } Maximization ◮ Can approximate max S f ( S ) to within 2 / 5, assuming f ≥ 0. [Feige, Mirrokni, Vondr´ ak ’07] Nick Harvey Approximating Submodular Functions Everywhere

Approximating Submodular Functions Everywhere Definition f : 2 [ n ] → R is monotone if, for all A ⊆ B ⊆ [ n ]: f ( A ) ≤ f ( B ) Problem Given a monotone, submodular f , construct using poly( n ) oracle queries a function ˆ f such that: ˆ f ( S ) ≤ f ( S ) ≤ α ( n ) · ˆ f ( S ) ∀ S ⊆ [ n ] Nick Harvey Approximating Submodular Functions Everywhere

Approximating Submodular Functions Everywhere Definition f : 2 [ n ] → R is monotone if, for all A ⊆ B ⊆ [ n ]: f ( A ) ≤ f ( B ) Problem Given a monotone, submodular f , construct using poly( n ) oracle queries a function ˆ f such that: ˆ f ( S ) ≤ f ( S ) ≤ α ( n ) · ˆ f ( S ) ∀ S ⊆ [ n ] Approximation Quality ◮ How small can we make α ( n )? ◮ α ( n ) = n is trivial Nick Harvey Approximating Submodular Functions Everywhere

Approximating Submodular Functions Everywhere Positive Result Problem Given a monotone, submodular f , construct using poly( n ) oracle queries a function ˆ f such that: ˆ f ( S ) ≤ f ( S ) ≤ α ( n ) · ˆ f ( S ) ∀ S ⊆ [ n ] Our Positive Result �� A deterministic algorithm that constructs ˆ f ( S ) = c i with i ∈ S ◮ α ( n ) = √ n + 1 for matroid rank functions f , or ◮ α ( n ) = O ( √ n log n ) for general monotone submodular f Also, ˆ f is submodular. Nick Harvey Approximating Submodular Functions Everywhere

Approximating Submodular Functions Everywhere Almost Tight Our Positive Result �� A deterministic algorithm that constructs ˆ f ( S ) = c i with i ∈ S ◮ α ( n ) = √ n + 1 for matroid rank functions f , or ◮ α ( n ) = O ( √ n log n ) for general monotone submodular f Our Negative Result With polynomially many oracle calls, α ( n ) = Ω( √ n / log n ) (even for randomized algs) Nick Harvey Approximating Submodular Functions Everywhere

Application Submodular Load Balacing Problem (Svitkina and Fleischer ’08) Given submodular functions f i : 2 V → R for i ∈ [ k ] , partition V into V 1 , · · · , V k to min max f i ( V i ) V 1 ,..., V k i For f i ( S ) = � j ∈ S c i , j , this is scheduling on unrelated machines. [Lenstra, Shmoys, Tardos ’90] Nick Harvey Approximating Submodular Functions Everywhere

Application Submodular Load Balacing Problem (Svitkina and Fleischer ’08) Given submodular functions f i : 2 V → R for i ∈ [ k ] , partition V into V 1 , · · · , V k to min max f i ( V i ) V 1 ,..., V k i For f i ( S ) = � j ∈ S c i , j , this is scheduling on unrelated machines. [Lenstra, Shmoys, Tardos ’90] Our solution �� Approximate f i by ˆ f i ( S ) = j ∈ S c i , j for each i . Then solve 2 ( V i ) ˆ min max f i V 1 ,..., V k i using Lenstra, Shmoys, Tardos. Get O ( √ n log n )-approx solution. Nick Harvey Approximating Submodular Functions Everywhere

Application Submodular Max-Min Fair Allocation Problem (Golovin ’05, Khot and Ponnuswami ’07) Given submodular functions f i : 2 V → R for i ∈ [ k ] , partition V into V 1 , · · · , V k to max min f i ( V i ) V 1 ,..., V k i For f i ( S ) = � j ∈ S c i , j , this is Santa Claus problem. √ There is a ˜ O ( k )-approximation algorithm [Asadpour-Saberi ’07]. O ( √ n k 1 / 4 )-approximate solution. Immediately get ˜ Nick Harvey Approximating Submodular Functions Everywhere

Polymatroid Definition Given submodular f , polymatroid � � x ∈ R n � P f = + : x i ≤ f ( S ) for all S ⊆ [ n ] i ∈ S A few properties [Edmonds ’70]: ◮ Can optimize over P f with greedy algorithm ◮ Separation problem for P f is submodular fctn minimization ◮ For monotone f , can reconstruct f : f ( S ) = max � 1 S , x � x ∈ P f Nick Harvey Approximating Submodular Functions Everywhere

Our Approach: Geometric Relaxation We know: f ( S ) = max � 1 S , x � x ∈ P f Suppose that: Q ⊆ P f ⊆ λ Q Then: ˆ f ( S ) ≤ f ( S ) ≤ λ ˆ f ( S ) where ˆ f ( S ) = max x ∈ Q � 1 S , x � λ Q P f Q Nick Harvey Approximating Submodular Functions Everywhere

John’s Theorem [1948] Maximum Volume Ellipsoids Definition A convex body K is centrally symmetric if x ∈ K ⇐ ⇒ − x ∈ K . Nick Harvey Approximating Submodular Functions Everywhere

John’s Theorem [1948] Maximum Volume Ellipsoids Definition A convex body K is centrally symmetric if x ∈ K ⇐ ⇒ − x ∈ K . Definition An ellipsoid E is an α -ellipsoidal approximation of K if E ⊆ K ⊆ α · E . Nick Harvey Approximating Submodular Functions Everywhere

John’s Theorem [1948] Maximum Volume Ellipsoids Definition A convex body K is centrally symmetric if x ∈ K ⇐ ⇒ − x ∈ K . Definition An ellipsoid E is an α -ellipsoidal approximation of K if E ⊆ K ⊆ α · E . Theorem Let K be a centrally symmetric convex body in R n . Let E max (or John ellipsoid) be maximum volume ellipsoid contained in K. Then K ⊆ √ n · E max . Nick Harvey Approximating Submodular Functions Everywhere

John’s Theorem [1948] Maximum Volume Ellipsoids Definition A convex body K is centrally symmetric if x ∈ K ⇐ ⇒ − x ∈ K . Definition An ellipsoid E is an α -ellipsoidal approximation of K if E ⊆ K ⊆ α · E . Theorem Let K be a centrally symmetric convex body in R n . Let E max (or John ellipsoid) be maximum volume ellipsoid contained in K. Then K ⊆ √ n · E max . Algorithmically? Nick Harvey Approximating Submodular Functions Everywhere

Ellipsoids Basics Definition ◮ An ellipsoid is E ( A ) = { x ∈ R n : x T Ax ≤ 1 } where A ≻ 0 is positive definite matrix. Handy notation √ ◮ Write � x � A = x T Ax . Then E ( A ) = { x ∈ R n : � x � A ≤ 1 } Optimizing over ellipsoids ◮ max x ∈ E ( A ) � c , x � = � c � A − 1 Nick Harvey Approximating Submodular Functions Everywhere

Algorithms for Ellipsoidal Approximations Explicitly Given Polytopes ◮ Can find E max in P-time (up to ǫ ) if explicitly given as K = { x : Ax ≤ b } [Gr¨ otschel, Lov´ asz and Schrijver ’88], [Nesterov, Nemirovski ’89], [Khachiyan, Todd ’93], ... Polytopes given by Separation Oracle ◮ only n + 1-ellipsoidal approximation for convex bodies given by weak separation oracle [Gr¨ otschel, Lov´ asz and Schrijver ’88] ◮ No (randomized) n 1 − ǫ -ellipsoidal approximation [J. Soto ’08] Nick Harvey Approximating Submodular Functions Everywhere

Finding Larger and Larger Inscribed Ellipsoids Informal Statement ◮ We have A ≻ 0 such that E ( A ) ⊆ K . ◮ Suppose we find z ∈ K but z far outside of E ( A ). ◮ Then should be able to find A ′ ≻ 0 such that ◮ E ( A ′ ) ⊆ K ◮ vol E ( A ′ ) > vol E ( A ) Nick Harvey Approximating Submodular Functions Everywhere

Finding Larger and Larger Inscribed Ellipsoids Informal Statement ◮ We have A ≻ 0 such that E ( A ) ⊆ K . ◮ Suppose we find z ∈ K but z far outside of E ( A ). ◮ Then should be able to find A ′ ≻ 0 such that ◮ E ( A ′ ) ⊆ K ◮ vol E ( A ′ ) > vol E ( A ) Nick Harvey Approximating Submodular Functions Everywhere

Finding Larger and Larger Inscribed Ellipsoids Formal Statement Theorem If A ≻ 0 and z ∈ R n with d = � z � 2 A ≥ n then E ( A ′ ) is max volume ellipsoid inscribed in conv { E ( A ) , z , − z } where A ′ = n d − 1 n − 1 A + n � 1 − d − 1 � Azz T A d 2 d n − 1 Moreover, vol E ( A ′ ) = k n ( d ) · vol E ( A ) where �� d � n − 1 � n � n − 1 k n ( d ) = n d − 1 z E(A) −z Nick Harvey Approximating Submodular Functions Everywhere

Finding Larger and Larger Inscribed Ellipsoids Formal Statement Theorem If A ≻ 0 and z ∈ R n with d = � z � 2 A ≥ n then E ( A ′ ) is max volume ellipsoid inscribed in conv { E ( A ) , z , − z } where A ′ = n d − 1 n − 1 A + n � 1 − d − 1 � Azz T A d 2 d n − 1 Moreover, vol E ( A ′ ) = k n ( d ) · vol E ( A ) where �� d � n − 1 � n � n − 1 k n ( d ) = n d − 1 z −z Nick Harvey Approximating Submodular Functions Everywhere