Advanced Algorithms (VIII) Shanghai Jiao Tong University Chihao - PowerPoint PPT Presentation

Advanced Algorithms (VIII) Shanghai Jiao Tong University Chihao Zhang April 26, 2020

The Probabilistic Method

The Probabilistic Method Design a probability space Ω

The Probabilistic Method Design a probability space Ω Show that Pr[the object exists] > 0

The Probabilistic Method Design a probability space Ω Show that Pr[the object exists] > 0 Bad events , each happens w.p. A 1 , A 2 , …, A m p i

The Probabilistic Method Design a probability space Ω Show that Pr[the object exists] > 0 Bad events , each happens w.p. A 1 , A 2 , …, A m p i Pr[ ¯ A 1 ∧ ¯ A 2 … ∧ ¯ Is ? A m ] > 0

We can apply the union bound

We can apply the union bound ≥ 1 − ∑ ⋃ ⋂ ¯ = 1 − Pr Pr A i A i p i i ∈ [ m ] i ∈ [ m ] i ∈ [ m ]

We can apply the union bound ≥ 1 − ∑ ⋃ ⋂ ¯ = 1 − Pr Pr A i A i p i i ∈ [ m ] i ∈ [ m ] i ∈ [ m ] ⋂ ∑ ¯ So if Pr A i > 0 p i < 1 i ∈ [ m ] i ∈ [ m ]

We can apply the union bound ≥ 1 − ∑ ⋃ ⋂ ¯ = 1 − Pr Pr A i A i p i i ∈ [ m ] i ∈ [ m ] i ∈ [ m ] ⋂ ∑ ¯ So if Pr A i > 0 p i < 1 i ∈ [ m ] i ∈ [ m ] The union bound is tight when bad events are disjoint

On the other hand, if the bad events are mutually independent…

On the other hand, if the bad events are mutually independent… = ∏ ⋂ ¯ Pr A i (1 − p i ) i ∈ [ m ] i ∈ [ m ]

On the other hand, if the bad events are mutually independent… = ∏ ⋂ ¯ Pr A i (1 − p i ) i ∈ [ m ] i ∈ [ m ] ⋂ ¯ So as long as none of Pr A i > 0 p i = 1 i ∈ [ m ]

On the other hand, if the bad events are mutually independent… = ∏ ⋂ ¯ Pr A i (1 − p i ) i ∈ [ m ] i ∈ [ m ] ⋂ ¯ So as long as none of Pr A i > 0 p i = 1 i ∈ [ m ] The two cases correspond to two extremes of the dependency

Lovász Local Lemma

Lovász Local Lemma The Lovász local lemma (LLL) captures partial dependency between bad events

Lovász Local Lemma The Lovász local lemma (LLL) captures partial dependency between bad events Erd ő s and Lovász, Infinite and Finite Sets , 1975

The Dependency Graph

The Dependency Graph We describe the dependency of bad events in a graph

The Dependency Graph We describe the dependency of bad events in a graph A 2 A 1 A 4 A 3

The Dependency Graph We describe the dependency of bad events in a graph V = { A 1 , …, A n } A 2 A 1 A 4 A 3

The Dependency Graph We describe the dependency of bad events in a graph V = { A 1 , …, A n } A 2 A 1 N ( A i ) = { A j ∣ A i ∼ A j } A 4 A 3

The Dependency Graph We describe the dependency of bad events in a graph V = { A 1 , …, A n } A 2 A 1 N ( A i ) = { A j ∣ A i ∼ A j } A 4 A 3 Δ = max i ∈ [ m ] | N ( A i ) |

The Dependency Graph We describe the dependency of bad events in a graph V = { A 1 , …, A n } A 2 A 1 N ( A i ) = { A j ∣ A i ∼ A j } A 4 A 3 Δ = max i ∈ [ m ] | N ( A i ) | 4 Δ p ≤ 1 A i ⊥ { A j } j ∉ N ( A i ) Pr[ A i ] ≤ p

The Dependency Graph We describe the dependency of bad events in a graph V = { A 1 , …, A n } A 2 A 1 N ( A i ) = { A j ∣ A i ∼ A j } A 4 A 3 Δ = max i ∈ [ m ] | N ( A i ) | 4 Δ p ≤ 1 ⟹ A i ⊥ { A j } j ∉ N ( A i ) Pr[ A i ] ≤ p

The Dependency Graph We describe the dependency of bad events in a graph V = { A 1 , …, A n } A 2 A 1 N ( A i ) = { A j ∣ A i ∼ A j } A 4 A 3 Δ = max i ∈ [ m ] | N ( A i ) | 4 Δ p ≤ 1 ⋂ ¯ Pr A i > 0 ⟹ A i ⊥ { A j } j ∉ N ( A i ) i ∈ [ m ] Pr[ A i ] ≤ p

Proof of (Symmetric) LLL

For , we prove by induction on that S ⊆ [ m ] | S |

For , we prove by induction on that S ⊆ [ m ] | S | A i ∣ ⋂ ¯ ∀ i ∉ S , Pr A j ≤ 2 p j ∈ S

For , we prove by induction on that S ⊆ [ m ] | S | A i ∣ ⋂ ¯ ∀ i ∉ S , Pr A j ≤ 2 p j ∈ S Assume and the statement holds for | S | = s smaller S

For , we prove by induction on that S ⊆ [ m ] | S | A i ∣ ⋂ ¯ ∀ i ∉ S , Pr A j ≤ 2 p j ∈ S Assume and the statement holds for | S | = s smaller S For every , we use to denote the event T ⊆ [ m ] F T ⋂ ¯ A i i ∈ T

T ∈ ( ≤ s ) [ m ] It is clear that for every , Pr[ F T ] ≥ (1 − 2 p ) s > 0

T ∈ ( ≤ s ) [ m ] It is clear that for every , Pr[ F T ] ≥ (1 − 2 p ) s > 0 We partition into where S = S 1 ∪ S 2 S 1 = { j ∣ j ∼ i } S

T ∈ ( ≤ s ) [ m ] It is clear that for every , Pr[ F T ] ≥ (1 − 2 p ) s > 0 We partition into where S = S 1 ∪ S 2 S 1 = { j ∣ j ∼ i } S If , then | S 2 | = s Pr[ A i ∣ S ] = Pr[ A i ∣ S 2 ] ≤ p

T ∈ ( ≤ s ) [ m ] It is clear that for every , Pr[ F T ] ≥ (1 − 2 p ) s > 0 We partition into where S = S 1 ∪ S 2 S 1 = { j ∣ j ∼ i } S If , then | S 2 | = s Pr[ A i ∣ S ] = Pr[ A i ∣ S 2 ] ≤ p Otherwise, Pr[ A i ∩ F S 1 ∩ F S 2 ] Pr[ A i ∣ F S ] = Pr[ A i ∣ F S 1 ∩ F S 2 ] = Pr[ F S 1 ∩ F S 2 ]

Pr[ A i ∩ F S 1 ∩ F S 2 ] Pr[ A i ∩ F S 1 ∣ F S 2 ] Pr[ A i ∣ F S ] = = Pr[ F S 1 ∩ F S 2 ] Pr[ F S 1 ∣ F S 2 ]

Pr[ A i ∩ F S 1 ∩ F S 2 ] Pr[ A i ∩ F S 1 ∣ F S 2 ] Pr[ A i ∣ F S ] = = Pr[ F S 1 ∩ F S 2 ] Pr[ F S 1 ∣ F S 2 ] Pr[ A i ∩ F S 1 ∣ F S 2 ] ≤ Pr[ A i ∣ F S 2 ] ≤ p

Pr[ A i ∩ F S 1 ∩ F S 2 ] Pr[ A i ∩ F S 1 ∣ F S 2 ] Pr[ A i ∣ F S ] = = Pr[ F S 1 ∩ F S 2 ] Pr[ F S 1 ∣ F S 2 ] Pr[ A i ∩ F S 1 ∣ F S 2 ] ≤ Pr[ A i ∣ F S 2 ] ≤ p ≥ 1 − 2 dp ≥ 1 ⋃ Pr[ F S 1 ∣ F S 2 ] = 1 − Pr A j ∣ F S 2 2 j ∈ S 1

Pr[ A i ∩ F S 1 ∩ F S 2 ] Pr[ A i ∩ F S 1 ∣ F S 2 ] Pr[ A i ∣ F S ] = = Pr[ F S 1 ∩ F S 2 ] Pr[ F S 1 ∣ F S 2 ] Pr[ A i ∩ F S 1 ∣ F S 2 ] ≤ Pr[ A i ∣ F S 2 ] ≤ p } ≥ 1 − 2 dp ≥ 1 ⋃ Pr[ F S 1 ∣ F S 2 ] = 1 − Pr A j ∣ F S 2 2 j ∈ S 1

Pr[ A i ∩ F S 1 ∩ F S 2 ] Pr[ A i ∩ F S 1 ∣ F S 2 ] Pr[ A i ∣ F S ] = = Pr[ F S 1 ∩ F S 2 ] Pr[ F S 1 ∣ F S 2 ] Pr[ A i ∩ F S 1 ∣ F S 2 ] ≤ Pr[ A i ∣ F S 2 ] ≤ p } ≥ 1 − 2 dp ≥ 1 ⋃ Pr[ F S 1 ∣ F S 2 ] = 1 − Pr A j ∣ F S 2 2 j ∈ S 1 ⟹

Pr[ A i ∩ F S 1 ∩ F S 2 ] Pr[ A i ∩ F S 1 ∣ F S 2 ] Pr[ A i ∣ F S ] = = Pr[ F S 1 ∩ F S 2 ] Pr[ F S 1 ∣ F S 2 ] Pr[ A i ∩ F S 1 ∣ F S 2 ] ≤ Pr[ A i ∣ F S 2 ] ≤ p } ≥ 1 − 2 dp ≥ 1 ⋃ Pr[ F S 1 ∣ F S 2 ] = 1 − Pr A j ∣ F S 2 2 j ∈ S 1 ⟹ Pr[ A i ∣ F S ] ≤ 2 p

Applications of LLL

Edge-Disjoint Paths

Edge-Disjoint Paths pairs of users, each has a collection of paths n m F i connecting them

Edge-Disjoint Paths pairs of users, each has a collection of paths n m F i connecting them Each path in shares edges with no more F i than paths in for any j ≠ i k F j

Edge-Disjoint Paths pairs of users, each has a collection of paths n m F i connecting them Each path in shares edges with no more F i than paths in for any j ≠ i k F j If , then there is a way to choose 8 nk ≤ m edge-disjoint paths connecting pairs n n

Define the probability space as

Define the probability space as “Each pair of users chooses a path from its collection uniformly at random”

Define the probability space as “Each pair of users chooses a path from its collection uniformly at random” For every , define the bad event as i ≠ j E ij

Define the probability space as “Each pair of users chooses a path from its collection uniformly at random” For every , define the bad event as i ≠ j E ij “the path chosen in overlaps with the path F i chosen in ” F j

Define the probability space as “Each pair of users chooses a path from its collection uniformly at random” For every , define the bad event as i ≠ j E ij “the path chosen in overlaps with the path F i chosen in ” F j ⋂ ¯ So we only need to show Pr E ij > 0 { i , j } ∈ ( n 2 )

{ i , j } ∈ ( 2 ) n Pr[ E ij ] ≤ k For each , we have m

{ i , j } ∈ ( 2 ) n Pr[ E ij ] ≤ k For each , we have m and are dependent only when E ij E i ′ j ′ { i , j } ∩ { i ′ , j ′ } ≠ ∅

{ i , j } ∈ ( 2 ) n Pr[ E ij ] ≤ k For each , we have m and are dependent only when E ij E i ′ j ′ { i , j } ∩ { i ′ , j ′ } ≠ ∅ So the maximum degree of the dependency graph is at most 2 n

{ i , j } ∈ ( 2 ) n Pr[ E ij ] ≤ k For each , we have m and are dependent only when E ij E i ′ j ′ { i , j } ∩ { i ′ , j ′ } ≠ ∅ So the maximum degree of the dependency graph is at most 2 n

{ i , j } ∈ ( 2 ) n Pr[ E ij ] ≤ k For each , we have m and are dependent only when E ij E i ′ j ′ { i , j } ∩ { i ′ , j ′ } ≠ ∅ So the maximum degree of the dependency graph is at most 2 n The LLL condition is then 8 nk ≤ m

Satisfiability

Satisfiability Recall that -SAT problem is -hard for k NP k ≥ 3

Satisfiability Recall that -SAT problem is -hard for k NP k ≥ 3 On the other hand, if the formula is sparse, then it is always satisfiable