Submodular Maximization Seffi Naor Lecture 2 4th Cargese Workshop - PowerPoint PPT Presentation

Submodular Maximization Seffi Naor Lecture 2 4th Cargese Workshop on Combinatorial Optimization Seffi Naor Submodular Maximization

Submodular Maximization Constrained Submodular Maximization Family of allowed subsets M ⊆ 2 N . f ( S ) max S ∈ M s . t . Seffi Naor Submodular Maximization

Constrained Maximization - Problem I Seffi Naor Submodular Maximization

Constrained Maximization - Problem I Seffi Naor Submodular Maximization

Constrained Maximization - Problem I (Cont.) Problem I - Submodular Welfare Input: Collection Q of unsplittable items. 1 f i : 2 Q → R + monotone submodular utility, 1 � i � k . 2 Goal: Assign all items to maximize social welfare: ∑ k i = 1 f i ( Q i ) . Arises in the context of combinatorial auctions . [Lehman-Lehman-Nisan-01] Seffi Naor Submodular Maximization

Constrained Maximization - Problem II Problem II - Submodular Maximization Over a Matroid Input: Matroid M = ( N , I ) and submodular f : 2 N → R + . Goal: Find S ∈ I maximizing f ( S ) . Case of monotone f captures: Submodular Welfare , Max- k -Coverage , Generalized-Assignment . . . Seffi Naor Submodular Maximization

Constrained Maximization - Problem II Problem II - Submodular Maximization Over a Matroid Input: Matroid M = ( N , I ) and submodular f : 2 N → R + . Goal: Find S ∈ I maximizing f ( S ) . Case of monotone f captures: Submodular Welfare , Max- k -Coverage , Generalized-Assignment . . . Combinatorial Approach: Greedy and local search techniques. For some cases provides best-known/tight approximations: Knapsack constraint [Sviridenko-04] intersection of k matroids [Lee-Sviridenko-Vondr ´ ak-09],[Ward-12] k -exchange systems [Feldman-Naor-S-Ward-11] Seffi Naor Submodular Maximization

The Greedy Approach [Nemhauer-Wolsey-Fisher-78] Greedy is a ( 1 / 2 ) -approximation for maximizing a monotone submodular f over a matroid. Seffi Naor Submodular Maximization

The Greedy Approach [Nemhauer-Wolsey-Fisher-78] Greedy is a ( 1 / 2 ) -approximation for maximizing a monotone submodular f over a matroid. Uniform Matroid: � � 1 − 1 Greedy is a -approximation [Nemhauser-Wolsey-Fisher-78] . e Captures Max- k -Coverage . Tight for coverage functions [Feige-98] . Seffi Naor Submodular Maximization

The Greedy Approach [Nemhauer-Wolsey-Fisher-78] Greedy is a ( 1 / 2 ) -approximation for maximizing a monotone submodular f over a matroid. Uniform Matroid: � � 1 − 1 Greedy is a -approximation [Nemhauser-Wolsey-Fisher-78] . e Captures Max- k -Coverage . Tight for coverage functions [Feige-98] . Non-monotone f over a matroid: ≈ 0.309 -approximation (fractional local search). [Vondr´ ak-09] ≈ 0.325 -approximation (simulated annealing). [Gharan-Vondr´ ak-11] ≈ 0.478 -hard absolute! [Gharan-Vondr´ ak-11] Seffi Naor Submodular Maximization

Uniform Matroid Notation: f S ( u ) = f ( S ∪ u ) − f ( S ) Greedy Algorithm S 0 ← ∅ . 1 for i = 1 to k do : 2 u i ← argmax u / ∈ S i − 1 { f S i − 1 ( u ) } . Return S k . 3 Seffi Naor Submodular Maximization

Uniform Matroid Notation: f S ( u ) = f ( S ∪ u ) − f ( S ) Greedy Algorithm S 0 ← ∅ . 1 for i = 1 to k do : 2 u i ← argmax u / ∈ S i − 1 { f S i − 1 ( u ) } . Return S k . 3 Theorem [Nemhauer-Wolsey-Fisher-78] For monotone submodular f , � � k � � 1 − 1 � 1 − 1 � 1 − · f ( OPT ) � · f ( OPT ) f ( S k ) � k e Seffi Naor Submodular Maximization

Uniform Matroid Notation: f S ( u ) = f ( S ∪ u ) − f ( S ) Greedy Algorithm S 0 ← ∅ . 1 for i = 1 to k do : 2 u i ← argmax u / ∈ S i − 1 { f S i − 1 ( u ) } . Return S k . 3 Theorem [Nemhauer-Wolsey-Fisher-78] For monotone submodular f , � � k � � 1 − 1 � 1 − 1 � 1 − · f ( OPT ) � · f ( OPT ) f ( S k ) � k e Non-Monotone Submodular Functions 1 / e is best factor (continuous approach via multilinear extension) Seffi Naor Submodular Maximization

Uniform Matroid Randomized Greedy Algorithm S 0 ← ∅ . 1 for i = 1 to k do : 2 u i ← uniformly choose in random an element from M i . S i ← S i − 1 ∪ u i . Return S k . 3 Seffi Naor Submodular Maximization

Uniform Matroid Randomized Greedy Algorithm S 0 ← ∅ . 1 for i = 1 to k do : 2 u i ← uniformly choose in random an element from M i . S i ← S i − 1 ∪ u i . Return S k . 3 How is M i defined? M i ⊆ N \ S i − 1 : ∑ f S i − 1 ( u ) s . t . | M i | = k . max u ∈ M i Assumptions: (w.l.o.g. by adding dummy elements) |N \ S i − 1 | � k ∀ u ∈ N \ S i − 1 , f S i − 1 ( u ) � 0 comment: “empty” iteration if a dummy element is chosen. Seffi Naor Submodular Maximization

Performance of Randomized Greedy Theorem [Buchbinder-Feldman-N-Schwartz-14] For monotone submodular f , � � k � � � � 1 − 1 1 − 1 E [ f ( S k )] � 1 − · f ( OPT ) � · f ( OPT ) k e Seffi Naor Submodular Maximization

Performance of Randomized Greedy Theorem [Buchbinder-Feldman-N-Schwartz-14] For monotone submodular f , � � k � � � � 1 − 1 1 − 1 E [ f ( S k )] � 1 − · f ( OPT ) � · f ( OPT ) k e Theorem [Buchbinder-Feldman-N-Schwartz-14] For non-monotone submodular f , � 1 � k � 1 − 1 � · f ( OPT ) � · f ( OPT ) E [ f ( S k )] � k e Seffi Naor Submodular Maximization

Monotone Submodular Functions condition on first i − 1 steps: expected gain at i th step: E [ f S i − 1 ( u i )] = 1 f S i − 1 ( u ) � 1 k · ∑ k · ∑ f S i − 1 ( u ) u ∈ M i u ∈ OPT \ S i − 1 f ( OPT ∪ S i − 1 ) − f ( S i − 1 ) f ( OPT ) − f ( S i − 1 ) � � k k Seffi Naor Submodular Maximization

Monotone Submodular Functions condition on first i − 1 steps: expected gain at i th step: E [ f S i − 1 ( u i )] = 1 f S i − 1 ( u ) � 1 k · ∑ k · ∑ f S i − 1 ( u ) u ∈ M i u ∈ OPT \ S i − 1 f ( OPT ∪ S i − 1 ) − f ( S i − 1 ) f ( OPT ) − f ( S i − 1 ) � � k k taking expectations over all outcomes: E [ f S i − 1 ( u i )] � f ( OPT ) − E [ f ( S i − 1 )] k Seffi Naor Submodular Maximization

Monotone Submodular Functions condition on first i − 1 steps: expected gain at i th step: E [ f S i − 1 ( u i )] = 1 f S i − 1 ( u ) � 1 k · ∑ k · ∑ f S i − 1 ( u ) u ∈ M i u ∈ OPT \ S i − 1 f ( OPT ∪ S i − 1 ) − f ( S i − 1 ) f ( OPT ) − f ( S i − 1 ) � � k k taking expectations over all outcomes: E [ f S i − 1 ( u i )] � f ( OPT ) − E [ f ( S i − 1 )] k rearranging: ( E [ f ( S i )] = E [ f ( S i − 1 )] + E [ f S i − 1 ( u i )] ) � � 1 − 1 f ( OPT ) − E [ f ( S i )] � · [ f ( OPT ) − E [ f ( S i − 1 )]] k Seffi Naor Submodular Maximization

Monotone Submodular Functions implying: � i � 1 − 1 f ( OPT ) − E [ f ( S i )] � · [ f ( OPT ) − E [ f ( S 0 )]] k � i � 1 − 1 · f ( OPT ) � k Seffi Naor Submodular Maximization

Monotone Submodular Functions implying: � i � 1 − 1 f ( OPT ) − E [ f ( S i )] � · [ f ( OPT ) − E [ f ( S 0 )]] k � i � 1 − 1 · f ( OPT ) � k thus: � � k � � � � 1 − 1 1 − 1 E [ f ( S k )] � 1 − · f ( OPT ) � · f ( OPT ) k e completing the proof. Seffi Naor Submodular Maximization

Non-Monotone Submodular Functions condition on first i − 1 steps: expected gain at i th step: E [ f S i − 1 ( u i )] = 1 f S i − 1 ( u ) � 1 k · ∑ ∑ k · f S i − 1 ( u ) u ∈ M i u ∈ OPT \ S i − 1 f ( OPT ∪ S i − 1 ) − f ( S i − 1 ) � k Seffi Naor Submodular Maximization

Non-Monotone Submodular Functions condition on first i − 1 steps: expected gain at i th step: E [ f S i − 1 ( u i )] = 1 f S i − 1 ( u ) � 1 k · ∑ ∑ k · f S i − 1 ( u ) u ∈ M i u ∈ OPT \ S i − 1 f ( OPT ∪ S i − 1 ) − f ( S i − 1 ) � k but what is f ( OPT ∪ S i − 1 ) for non-monotone f ? Seffi Naor Submodular Maximization

Non-Monotone Submodular Functions condition on first i − 1 steps: expected gain at i th step: E [ f S i − 1 ( u i )] = 1 f S i − 1 ( u ) � 1 k · ∑ ∑ k · f S i − 1 ( u ) u ∈ M i u ∈ OPT \ S i − 1 f ( OPT ∪ S i − 1 ) − f ( S i − 1 ) � k but what is f ( OPT ∪ S i − 1 ) for non-monotone f ? Lemma For all 0 � i � k , � i � 1 − 1 E [ f ( OPT ∪ S i )] � · f ( OPT ) k proof deferred for now ... Seffi Naor Submodular Maximization

Non-Monotone Submodular Functions taking expectations over all outcomes: � f ( OPT ∪ S i − 1 ) − f ( S i − 1 ) � E [ f S i − 1 ( u i )] � E k � i − 1 � 1 − 1 · f ( OPT ) − E [ f ( S i − 1 )] k � k Seffi Naor Submodular Maximization

Non-Monotone Submodular Functions taking expectations over all outcomes: � f ( OPT ∪ S i − 1 ) − f ( S i − 1 ) � E [ f S i − 1 ( u i )] � E k � i − 1 � 1 − 1 · f ( OPT ) − E [ f ( S i − 1 )] k � k it can be proved by induction that: � i − 1 E [ f ( S i )] � i � 1 − 1 k · · f ( OPT ) k Seffi Naor Submodular Maximization