Randomized Algorithms I Probability Contention Resolution Minimum - PowerPoint PPT Presentation

Randomized Algorithms I • Probability • Contention Resolution • Minimum Cut Philip Bille

Randomized Algorithms I • Probability • Contention Resolution • Minimum Cut

Probability • Probability spaces. • Set of possible outcomes Ω . ∑ • Each element has probability and . 𝗃 ∈ Ω 𝗊 ( 𝗃 ) ≥ 𝟣 𝗊 ( 𝗃 ) = 𝟤 𝗃∈Ω Pr( 𝖥 ) = ∑ • Event is a subset of and probability of is . 𝖥 Ω 𝗊 ( 𝗃 ) E 𝗃∈𝖥 • The complementary event is 𝖥 Ω − 𝖰 and Pr( 𝖥 ) = 𝟤 − Pr( 𝖥 ) . • Example. Flip two fair coins. Ω 𝖥 • Ω = { 𝖨𝖨 , 𝖨𝖴 , 𝖴𝖨 , 𝖴𝖴 } . 𝖨𝖨 𝖴𝖴 for each outcome i. • 𝗊 ( 𝗃 ) = 𝟤 / 𝟧 𝖥 • Event = "the coins are the same" 𝖥 𝖴𝖨 𝖨𝖴 . • Pr( 𝖥 ) = 𝟤 / 𝟥

Probability 𝖥 𝖦 • Conditional probability. • What is the probability that event occurs given that event occurred? 𝖥 𝖦 • The conditional probability of given : 𝖥 𝖦 Pr( 𝖥 ∣ 𝖦 ) = Pr( 𝖥 ∩ 𝖦 ) Pr( 𝖦 ) • Example. Pr( 𝖥 ∣ 𝖦 ) = Pr( 𝖥 ∩ 𝖦 ) = 2/8 5/8 = 2 • Pr( 𝖦 ) 5

Probability • Independence. • Events and are independent if information about does not a ff ect outcome 𝖥 𝖦 𝖥 of and vice versa. 𝖦 Pr( 𝖥 ∣ 𝖦 ) = Pr( 𝖥 ) Pr( 𝖦 ∣ 𝖥 ) = Pr( 𝖦 ) • Same as Pr( 𝖥 ∩ 𝖦 ) = Pr( 𝖥 ) ⋅ Pr( 𝖦 )

Probability • Union bound. • What is the probability that any of event E 1 , ..., E k will happen, i.e., what is ? Pr( 𝖥 𝟤 ∪ 𝖥 𝟥 ∪ ⋯ ∪ 𝖥 𝗅 ) 𝖥 𝟤 𝖥 𝟤 𝖥 𝟥 𝖥 𝟦 𝖥 𝟥 𝖥 𝟦 • If events are disjoint, Pr( 𝖥 𝟤 ∪ ⋯ ∪ 𝖥 𝗅 ) = Pr( 𝖥 𝟤 ) + ⋯ + Pr( 𝖥 𝗅 ) . • If events overlap, . Pr( 𝖥 𝟤 ∪ ⋯ ∪ 𝖥 𝗅 ) < Pr( 𝖥 𝟤 ) + ⋯ + Pr( 𝖥 𝗅 ) • In both cases, the union bound holds: Pr( 𝖥 𝟤 ∪ ⋯ ∪ 𝖥 𝗅 ) ≤ Pr( 𝖥 𝟤 ) + ⋯ + Pr( 𝖥 𝗅 )

Randomized Algorithms I • Probability • Contention Resolution • Minimum Cut

Contention Resolution • Contention resolution. Consider n processes 𝖰 𝟤 , …, 𝖰 𝗈 trying to access a shared database: • If two or more processes access database at the same time, all processes are locked out. • Processes cannot communicate. • Goal. Come up with a protocol to ensure all processes will access database. • Challenge. Need symmetry breaking paradigm. 𝖰 𝟤 𝖰 𝟥 database 𝖰 𝗈

Contention Resolution • Applications. • Distributed communication and interference. • Illustrates simplicity and power of randomized algorithms.

Contention Resolution • Protocol. Each process accesses the database at time t with probability p = 1/n. 𝖰 𝟤 𝖰 𝟥 database 𝖰 𝗈

Contention Resolution • Analysis. How do we analyze the protocol? 𝖰 𝟤 𝖰 𝟥 database 𝖰 𝗈

Contention Resolution • Success for a single process in a single round. 𝖳 𝗃 , 𝗎 = event that 𝖰 𝗃 successfully accesses database at time . 𝗎 • 𝗈−𝟤 𝗈 ( 𝟤 − 𝟤 𝗈 ) Pr ( 𝖳 𝗃 , 𝗎 ) = 𝗊 ( 𝟤 − 𝗊 ) 𝗈−𝟤 = 𝟤 ≥ 𝟤 𝖿𝗈 probability that process i requests access. 𝗈−𝟤 ( 𝟤 − 𝟤 𝗈 ) probability that no other converges to 𝟤 / 𝖿 from above. process requests access.

Contention Resolution • Failure for a single process in rounds 𝟤 , …, 𝗎 . event that fails to access database in any of rounds . 𝖦 𝗃 , 𝗎 = 𝖰 𝗃 𝟤 , …, 𝗎 • Pr ( 𝖦 𝗃 , 𝗎 ) = Pr ( Pr ( 𝖳 𝗃 , 𝗌 ) = ( 𝟤 − 𝟤 𝗎 ≤ ( 𝟤 − 𝟤 𝖳 𝗃 , 𝗌 ) = 𝗈−𝟤 ) 𝗎 𝗎 𝗎 𝗈 ( 𝟤 − 𝟤 𝗈 ) 𝖿𝗈 ) ⋂ ∏ 𝗌 = 𝟤 𝗌 = 𝟤 independence. probability that 𝖰 𝗃 does not Pr ( 𝖳 𝗃 , 𝗎 ) ≥ 𝟤 succeed in round 1 and 𝖿𝗈 round 2 and ... and round t. 𝗎 = ⌈𝖿𝗈⌉ ⇒ Pr ( 𝖦 𝗃 , 𝗎 ) ≤ ( 𝟤 − 𝟤 ≤ ( 𝟤 − 𝟤 ⌈𝖿𝗈⌉ 𝖿𝗈 𝖿𝗈 ) 𝖿𝗈 ) ≤ 𝟤 • 𝖿 𝗎 = ⌈𝖿𝗈⌉ ( 𝖽 ln 𝗈 ) ⇒ Pr ( 𝖦 𝗃 , 𝗎 ) ≤ ( 𝖽 ln 𝗈 𝟤 𝖿 ) = 𝟤 • 𝗈 𝗈 𝖽 ( 𝟤 − 𝟤 𝗈 ) converges to 𝟤 / 𝖿 from below.

Contention Resolution • Failure for at least one process in rounds 𝟤 , …, 𝗎 . event that at least one of n processes fails to access database in any of • 𝖦 𝗎 = rounds . 𝟤 , …, 𝗎 Pr ( 𝖦 𝗎 ) = Pr ( Pr ( 𝖦 𝗃 , 𝗎 ) ≤ 𝗈 ( 𝟤 − 𝟤 𝖦 𝗃 , 𝗎 ) ≤ 𝗎 𝗈 𝗈 𝖿𝗈 ) ⋃ ∑ 𝗃 = 𝟤 𝗃 = 𝟤 Pr ( 𝖦 𝗃 , 𝗎 ) ≤ ( 𝟤 − 𝟤 𝗎 𝖿𝗈 ) union bound probability that any one of 𝖰 𝟤 , …, 𝖰 𝗈 fails in rounds 𝟤 , …, 𝗎 𝗎 = ⌈𝖿𝗈⌉𝟥 ln 𝗈 ⇒ Pr ( 𝖦 𝗎 ) ≤ 𝗈 ( 𝟤 − 𝟤 ≤ 𝗈 ( ⌈𝖿𝗈⌉𝟥 ln 𝗈 𝟥 ln 𝗈 𝖿𝗈 ) 𝟤 𝖿 ) = 𝗈 𝗈 𝟥 = 𝟤 • . 𝗈 • ⇒ Probability that all processes successfully access the database after rounds is at least . ⌈𝖿𝗈⌉𝟥 ln 𝗈 𝟤 − 𝟤 / 𝗈

Contention Resolution • Conclusion. After ⌈𝖿𝗈⌉𝟥 ln 𝗈 rounds all processes have accessed database with probability at least 𝟤 − 𝟤 / 𝗈 . • Success probability. • For large probability is very close to 1. 𝗈 • More rounds will further increase probability of success. • Simplicity. • Very simple and e ff ective protocol. • Di ffi cult to solve deterministically.

Randomized Algorithms I • Probability • Contention Resolution • Minimum Cut

Minimum Cut • Graphs. Consider undirected, connected graph G = (V,E). • Cuts. • A cut (A,B) is a partition of V into two non-empty disjoint sets A and B. • The size of a cut (A,B) is the number of edges crossing the cut. • A minimum cut is a cut of minimum size. A B

Minimum Cut • Applications. • Network fault tolerance. • Image segmentation. • Parallel computation • Social network analysis. • ...

Minimum Cut • Which solutions do we know?

Minimum Cut b {a,b} a c c d d • Contraction algorithm. • Pick edge e = (u,v) uniformly at random. • Contract e. • Replace e by single vertex w. • Preserve edges, updating endpoints of u and v to w. • Preserve parallel edges, but remove self-loops. • Repeat until two vertices a and b left. • Return cut (all vertices contracted into a, all vertices contracted into b).

b {a,b} {a,b,c} a c c d d d cut is ({a,b,c}, {d}) of size 2

Minimum Cut • Analysis. • Consider minimum cut (A,B) with crossing edges F . • What is the probability that the contraction algorithm returns (A,B)? A B F

Minimum Cut • Round 1. • What is the probability that we contract an edge from F in round 1? ⇒ ∑ • Each vertex has deg ≥ | 𝖦 | (otherwise smaller cut exists) 𝖾𝖿𝗁 ( 𝗐 ) ≥ | 𝖦 | 𝗈 . 𝗐∈𝖶 ∑ 𝗐∈𝖶 𝖾𝖿𝗁 ( 𝗐 ) ≥ | 𝖦 | 𝗈 ∑ 𝖾𝖿𝗁 ( 𝗐 ) = 𝟥𝗇 ⇒ 𝗇 = . • 𝟥 𝟥 𝗐∈𝖶 = | 𝖦 | | 𝖦 | 𝗈 / 𝟥 = 𝟥 | 𝖦 | • Probability we contract edge from F is ≤ . 𝗇 𝗈 A B F

Minimum Cut • Round j+1. • What is the probability that we contract an edge in round 𝗄 + 𝟤 from , given that 𝖦 no edge from was contracted in rounds 𝖦 𝟤 , …, 𝗄 ? • 𝖧′ is graph after rounds with 𝗄 𝗈 − 𝗄 nodes and no edges from was contracted 𝖦 in rounds 𝟤 , …, 𝗄 . • Every cut in 𝖧′ is a cut in 𝖧 ⇒ at least | 𝖦 | edges incident to every node in 𝖧′ | 𝖦 | ( 𝗈 − 𝗄 ) ≤ | 𝖦 | 𝟥 • ⇒ 𝖧′ contains at least edges ⇒ probability is . = 𝟥 𝗇 𝗈 − 𝗄 A B F

Minimum Cut • Success after all rounds. 𝖥 𝗄 = event that an edge from F is not contracted in round j. • Pr ( 𝖥 𝗈−𝟥 ∩ ⋯ ∩ 𝖥 𝟤 ) • The probability that we return the correct minimum cut is . • We know: Pr ( 𝖥 𝟤 ) ≥ 𝟤 − 𝟥 • . 𝗈 Pr ( 𝖥 𝗄 + 𝟤 ∣ 𝖥 𝟤 ∩ ⋯ ∩ 𝖥 𝗄 ) ≥ 𝟤 − 𝟥 • . 𝗈 − 𝗄 ⇒ Pr ( 𝖥 𝟤 ∩ ⋯ ∩ 𝖥 𝗄 + 𝟤 ) ≥ 𝟥 • Conditional probability definition + algebra . 𝗈 𝟥

Minimum Cut • Conclusion. ≥ 𝟥 / 𝗈 𝟥 • We return the correct minimum cut with probability in polynomial time. • Probability amplification. • Correct solution only with very small probability • Run contraction algorithm many times and return smallest cut. 𝗈 𝟥 ln 𝗈 • With runs with independent random choices the probability of failure to ≤ ( 𝟤 − 𝟥 ≤ ( 𝗈 𝟥 ln 𝗈 𝟥 ln 𝗈 𝗈 𝟥 ) 𝖿 ) 𝟤 = 𝟤 find minimum cut is . 𝗈 𝟥 • Time. • ϴ (n 2 log n) iterations that take Ω (m) time each. • More techniques and tricks ⇒ m log O(1) n time solution. [Karger 2000]

Minimum Cut • Monte Carlo algorithm. • Randomized algorithm. • Guarantee on running time, likely to find correct answer. • Las Vegas algorithm. • Randomized algorithm. • Guaranteed to find the correct answer, likely to be fast.

Randomized Algorithms I • Probability • Contention Resolution • Minimum Cut