Deriving greedy algorithms and Lagrangian-relaxation algorithms - PowerPoint PPT Presentation

Deriving greedy algorithms and Lagrangian-relaxation algorithms Neal E. Young February 16, 2007

set cover standard randomized rounding existence proof method of conditional probabilities algorithm iterated sampling existence proof method of conditional probabilities algorithm vertex cover (duality) existence proof method of conditional probabilities algorithm implicit primal-dual algorithm multicommodity flow existence proof algorithm for integer solution algorithm for fractional solution lower bound on iterations fast algorithm for explicitly given problems two open questions

set cover input: collection s 1 , s 2 , . . . , s m of sets over universe U m � minimize x i subject to i =1 � ( ∀ e ∈ U ) x i ≥ 1 s i ∋ e ( ∀ i ) x i ∈ { 0 , 1 } ◮ Value of optimal fractional solution x ∗ is a lower bound on optimal integer solution.

a fractional set cover x ∗ .3 .7 .7 .3 a,b,c a,c,d b,d,e c,e sets elements a b c d e 1 1 1.3 1.4 1

standard randomized rounding Let x ∗ be an optimal fractional set cover. Let λ = ln 2 n . For each set s i ∈ S independently do: choose s i with probability p i . = min { λ x ∗ i , 1 } . Theorem With positive probability, chosen sets form a cover of size at most 2 ln(2 n ) � i x ∗ i .

coverage Let x ∗ be an optimal fractional set cover. Let λ = ln 2 n . For each set s i ∈ S independently do: . = min { λ x ∗ choose s i with probability p i i , 1 } . Probability element e not covered: � � exp( − λ x ∗ 1 − p i < i ) s i ∋ e s i ∋ e � � � x ∗ = exp − λ i s i ∋ e ≤ exp( − λ ) = 1 / 2 n Pr[ exists uncovered element ] < 1 / 2

cost Let x ∗ be an optimal fractional set cover. Let λ = ln 2 n . For each set s i ∈ S independently do: . = min { λ x ∗ choose s i with probability p i i , 1 } . Expected number of sets chosen is � � x ∗ p i ≤ ln(2 n ) i . i i i x ∗ Pr[ more than 2 ln(2 n ) � i sets chosen ] ≤ 1 / 2

proof of theorem Let x ∗ be an optimal fractional set cover. Let λ = ln 2 n . For each set s i ∈ S independently do: . = min { λ x ∗ choose s i with probability p i i , 1 } . Theorem With positive probability, chosen sets form a cover of size at most i x ∗ 2 ln(2 n ) � i . Proof. Pr[ exists uncovered element ] < 1 / 2 Pr[ more than 2 ln(2 n ) � i x ∗ i sets chosen ] ≤ 1 / 2 i x ∗ Pr[ chosen sets form cover of size ≤ 2 ln(2 n ) � i ] > 0

method of conditional probabilities Let x ∗ be an optimal fractional set cover. converts existence proof into an efficient algorithm Let λ = ln 2 n . For each set s i ∈ S independently do: . = min { λ x ∗ choose s i with probability p i i , 1 } . Random experiment is random walk starting here. Method of conditional Each node labeled with probabilities chooses expected number of bad events next node so labels 0.93 if random walk was to start there. don't increase... 0.33 2.1 0.33 0.67 0 0.33 2.3 2.7 1.3 0 0 1 0 2 0 0 0 0 0 1 0 0 3 4 4 4 0 1 3 0 0 0 0 0 0 0 0 0 3 Leaf label is number of ... thus guarantees an outcome bad events in that outcome. with no bad events.

algorithm (incomplete) Let x ∗ be an optimal fractional set cover. Let λ = ln 2 n . For each set s i ∈ S independently do: . = min { λ x ∗ choose s i with probability p i i , 1 } . � method of conditional probabilities ⇓ Let x ∗ be an optimal fractional set cover. Let λ = ln 2 n . For i = 1 , 2 , . . . , m sequentially do: include or exclude s i — whichever keeps conditional probability of failure below 1.

conditional probability of failure Let x ∗ be an optimal fractional set cover. — coverage Let λ = ln 2 n . For i = 1 , 2 , . . . , m sequentially do: include or exclude s i — whichever keeps conditional probability of failure below 1. Given first t choices, probability that elt e won’t be covered is zero if e is already covered, and otherwise � 1 − p i . s i ∋ e , i > t Conditional probability that chosen sets will fail to cover is at most � � 1 − p i . e not yet s i ∋ e , i ≥ t covered

conditional probability of failure Let x ∗ be an optimal fractional set cover. — cost Let λ = ln 2 n . For i = 1 , 2 , . . . , m sequentially do: include or exclude s i — whichever keeps conditional probability of failure below 1. Given first t choices, expected number of chosen sets is � # first t sets chosen + p i . i > t Given first t choices, probability that too many sets will be chosen is at most # first t sets chosen + � i > t p i . 2 ln 2 n � i x ∗ i

pessimistic estimator Φ t Let x ∗ be an optimal fractional set cover. Let λ = ln 2 n . For i = 1 , 2 , . . . , m sequentially do: include or exclude s i — whichever keeps conditional probability of failure below 1. Given first t choices, probability of failure is at most . � � Φ t = 1 − p i e not yet s i ∋ e , i ≥ t covered + # first t sets chosen + � i > t p i . i x ∗ 2 ln 2 n � i

pessimistic estimator Φ t Let x ∗ be an optimal fractional set cover. Let λ = ln 2 n . For i = 1 , 2 , . . . , m sequentially do: include or exclude s i — whichever keeps conditional probability of failure below 1. Given first t choices, probability of failure is at most . � � Φ t = 1 − p i e not yet s i ∋ e , i ≥ t covered + # first t sets chosen + � i > t p i . i x ∗ 2 ln 2 n � i ◮ Φ 0 < 1 ◮ E [Φ t +1 | Φ t ] ≤ Φ t ◮ If Φ m < 1, then outcome is successful.

algorithm Let x ∗ be an optimal fractional set cover. Let λ = ln 2 n . For i = 1 , 2 , . . . , m sequentially do: include or exclude s i — whichever makes Φ i < 1. + # first t sets chosen + � i > t p i . � � � � Φ t = 1 − p i . i x ∗ 2 ln 2 n � i e not yet s i ∋ e , i ≥ t covered Corollary Algorithm returns a cover of size at most 2 ln(2 n ) × opt .

sample and increment x* randomized rounding via iterated sampling x ˆ 7 8 9 Let x ∗ ≥ 0 be a fractional solution. 3 4 5 6 1 2 Let | x ∗ | denote � i x ∗ i . t = 0 Define distribution p by p i . = x ∗ i ′ x ∗ i / � i ′ . Let ˆ x ← 0 . For t = 1 , 2 , 3 , . . . do: Sample random index i according to p . Increment ˆ x i . x ( t ) denote ˆ Let ˆ x after t samples. ... like weighted balls in bins.

illustration of sampling distribution fractional set cover x*: .3 .7 .7 .3 a,b,c a,c,d b,d,e c,e sets elements a b c d e 1 1 1.3 1.4 1 probability distribution p on sets: .15 .35 .35 .15 a,b,c a,c,d b,d,e c,e a b c d e .5 .5 .65 .7 .5

sample and increment x* — for set cover x ˆ 7 8 9 3 4 5 6 Let x ∗ ≥ 0 be a fractional solution. 1 2 Let | x ∗ | denote � i x ∗ i . t = 0 Define distribution p by p i . = x ∗ i / | x ∗ | . Let ˆ x ← 0 . For t = 1 , 2 , 3 , . . . do: Sample random index i according to p . Increment ˆ x i — add s i to the cover. x ( t ) denote ˆ Let ˆ x after t samples.

sample and increment x* — for set cover x ˆ 7 8 9 3 4 5 6 Let x ∗ ≥ 0 be a fractional solution. 1 2 Let | x ∗ | denote � i x ∗ i . t = 0 Define distribution p by p i . = x ∗ i / | x ∗ | . Let ˆ x ← 0 . For t = 1 , 2 , 3 , . . . do: Sample random index i according to p . Increment ˆ x i — add s i to the cover. x ( t ) denote ˆ Let ˆ x after t samples. ◮ For any element e , with each sample, s i ∋ e x ∗ i / | x ∗ | ≥ 1 / | x ∗ | . Pr[ e is covered] = �

existence proof for set cover x* x ˆ 7 8 9 Theorem 3 4 5 6 With positive probability, 1 2 after T = ⌈ ln( n ) | x ∗ |⌉ samples, x ( T ) is a set cover. t = 0 ˆ Proof. For any element e : ◮ With each sample, Pr[ e is covered] = � s i ∋ e x ∗ i / | x ∗ | ≥ 1 / | x ∗ | . ◮ After T samples, Pr[ e is not covered] ≤ (1 − 1 / | x ∗ | ) T < 1 / n . So, expected number of uncovered elements is less than 1. Corollary There exists a set cover of size at most ⌈ ln( n ) | x ∗ |⌉ .

method of conditional probabilities x* x ˆ Let x ∗ ≥ 0 be a fractional solution. 7 8 9 3 4 5 6 Let ˆ x ← 0 . 1 2 For t = 1 , 2 , 3 , . . . , T do: t = 0 Increment ˆ x i , where i is chosen to keep expected number of not-covered elements below 1. x ( T ) . Return ˆ Given first t samples, expected number of not-covered elements is at most . � (1 − 1 / | x ∗ | ) T − t . Φ t = e not yet covered

algorithm x* the greedy set-cover algorithm x ˆ 7 8 9 3 4 5 6 Let ˆ x ← 0 . 1 2 t = 0 For t = 1 , 2 , 3 , . . . , T do: Increment ˆ x i , where i is chosen to minimize the number of not-yet-covered elements. x ( T ) . Return ˆ Corollary The greedy algorithm returns a cover of size at most ⌈ ln( n ) min x ∗ | x ∗ |⌉ .

algorithm x* the greedy set-cover algorithm x ˆ 7 8 9 3 4 5 6 Let ˆ x ← 0 . 1 2 t = 0 For t = 1 , 2 , 3 , . . . , T do: Increment ˆ x i , where i is chosen to minimize the number of not-yet-covered elements. x ( T ) . Return ˆ Corollary The greedy algorithm returns a cover of size at most ⌈ ln( n ) min x ∗ | x ∗ |⌉ . Can also derive Chvatal’s weighted set cover algorithm and show H (max s | s | )-approximation.