Randomness in Computing L ECTURE 1 Randomness in Computing • Course information • Verifying polynomial identities • Probability Amplification Probability Review Sofya Raskhodnikova 1/21/2020 Sofya Raskhodnikova;Randomness in Computing
Course information 1. Course staff 6. Syllabus 2. Course website(s) 7. Homework logistics 3. Piazza bonus 8. Collaboration policy 4. Prerequisites 9. Exams and grading 5. Textbook(s) 1/21/2020 Sofya Raskhodnikova; Randomness in Computing Sofya Raskhodnikova; Randomness in Computing
Tips for the course • Concepts in this course take some time to sink in: be careful not to fall behind. • Do the assigned reading on each topic before the corresponding lecture. • Attend the lectures: most of the material will be presented on the blackboard (and some of it is not in the book). • Attend the discussions: practice problem solving. • Take advantage of office hours. • Be active in lectures and on piazza. • Allocate lots of time for the course: comparable to a project course, but spread more evenly. Sofya Raskhodnikova; Randomness in Computing 1/21/2020
Tips for the course: HW • Start working on HW early. • Spread your HW time over multiple days. • You can work in groups (up to 4 people), but spend 1-2 hours thinking about it on your own before your group meeting. Sofya Raskhodnikova; Randomness in Computing 1/21/2020
Tips: learning problem solving To learn problem solving, you have to do it: • Try to think how you would solve any presented problem before you read/hear the answer. • Do exercises in addition to HW. Sofya Raskhodnikova; Randomness in Computing 1/21/2020
Tips: how to read a math text • Not like reading a mystery novel. • The goal is not to get the answers, but to learn the techniques. • Always try to foresee what is coming next. • Always think how you would approach a problem before reading the solution. • This applies to things that are not explicitly labeled as problems. Sofya Raskhodnikova; Randomness in Computing 1/21/2020
Skills we will work on • Mathematical reasoning • Expressing your ideas – abstractly (suppress inessential details) – precisely (rigorously) • Probabilistic thinking • Algorithmic thinking • Problem solving • Having FUN with all of the above!!! Sofya Raskhodnikova; Randomness in Computing 1/21/2020
Uses of Randomness in Computing • To speed up algorithms. • To enable new applications: – Symmetry breaking in distributed algorithms, cryptography, privacy, online games and gambling. • To simulate real world events in physical systems: model them as happening randomly. • To analyze algorithms when data is generated from some distribution: – learning theory, data compression. • To analyze algorithms when errors happen randomly – error-correcting codes. • Analyzing statistics from sampling. Sofya Raskhodnikova; Randomness in Computing 1/21/2020
§ 1.1 (MU) Verifying Polynomial Identities 𝑦 − 6 ≡ ? 𝑦 6 − 7𝑦 + 37 • 𝑦 + 1 𝑦 − 2 𝑦 + 3 𝑦 − 4 𝑦 + 5 Task: Given two polynomials 𝐺 𝑦 and 𝐻 𝑦 , verify if 𝐺 𝑦 ≡ 𝐻 𝑦 . Idea 1 (deterministic): Convert both polynomials to canonical form 𝑑 𝑗 𝑦 𝑗 . 𝑒 𝑗=0 Faster with Fourier Transform 𝑒 • If 𝐺 𝑦 is given as Π 𝑗=1 𝑦 − 𝑏 𝑗 , conversion by consecutively multiplying monomials requires Θ(𝑒 2 ) multiplications of coefficients. Sofya Raskhodnikova; Randomness in Computing 1/21/2020
§ 1.1 (MU) Verifying Polynomial Identities Task: Given two polynomials 𝐺 𝑦 and 𝐻 𝑦 , verify if 𝐺 𝑦 ≡ 𝐻 𝑦 . Idea 2 (randomized): Evaluate the polynomials on random integers. Let 𝑒 = max degree of 𝐺(𝑦) and 𝐻(𝑦) Pick 𝑠 uniformly from 1, … , 100𝑒 . 1. Compute 𝐺(𝑠) and 𝐻(𝑠) . 2. O(d) ops for product form 3. 3. 𝐬𝐟𝐤𝐟𝐝𝐮 if 𝐺 𝑠 ≠ 𝐻 𝑠 ; o. w. 𝐛𝐝𝐝𝐟𝐪𝐮. Error Analysis: Probability of accepting incorrectly 1-sided error Fundamental Theorem of Algebra A polynomial of degree d has at most d roots . 𝑒 1 Pr error = Pr 𝑠 is a root of 𝐺 𝑦 − 𝐻 𝑦 ≤ 100𝑒 = 100 Sofya Raskhodnikova; Randomness in Computing 1/21/2020
Review: Axioms of Probability Probability space has three components • Sample space Ω • Family of allowable events 𝐹 ⊆ Ω • A probability function Pr that maps events E to ℝ such that Pr 𝐹 ∈ [0,1] for any event 𝐹; Pr Ω = 1; For any finite or countable sequence of pairwise disjoint events 𝐹 1 , 𝐹 2 , … , Pr 𝐹 𝑗 = Pr 𝐹 𝑗 . 𝑗≥1 𝑗≥1 Sofya Raskhodnikova; Randomness in Computing 1/21/2020
Review: Inclusion-Exclusion Principle For any two events 𝐹 1 and 𝐹 2 , Pr 𝐹 1 ∪ 𝐹 2 = Pr(𝐹 1 ) + Pr 𝐹 2 − Pr 𝐹 1 ∩ 𝐹 2 . For any three events 𝐹 1 , 𝐹 2 and 𝐹 3 , Pr 𝐹 1 ∪ 𝐹 2 ∪ 𝐹 3 = Pr(𝐹 1 ) + Pr 𝐹 2 + Pr 𝐹 3 − Pr 𝐹 1 ∩ 𝐹 2 − Pr 𝐹 1 ∩ 𝐹 3 − Pr 𝐹 2 ∩ 𝐹 3 + Pr 𝐹 1 ∩ 𝐹 2 ∩ 𝐹 3 For any 𝑜 events 𝐹 1 , 𝐹 2 , … , 𝐹 𝑜 , Pr 𝐹 𝑗 = Pr 𝐹 𝑗 − Pr 𝐹 𝑗 ∩ 𝐹 𝑘 𝑗∈[𝑜] 𝑗∈[𝑜] 𝑗<𝑘 𝑘 ∩ 𝐹 𝑙 − ⋯ + −1 𝑜+1 Pr + Pr 𝐹 𝑗 ∩ 𝐹 𝐹 𝑗 . 𝑗<𝑘<𝑙 𝑗∈ 𝑜 Sofya Raskhodnikova; Randomness in Computing 1/21/2020
Review: Union Bound Union Bound For finite or countable sequence of events 𝐹 1 , 𝐹 2 , … , Pr 𝐹 𝑗 ≤ Pr 𝐹 𝑗 . 𝑗≥1 𝑗≥1 Sofya Raskhodnikova; Randomness in Computing 1/21/2020
Probability Amplification • Our algorithm for verifying polynomial identities accepts 1 incorrectly with probability ≤ 100 Idea: Repeat the algorithm and accept if all iterations accept. Pr error in all 𝑙 iterations 𝑙 1 ≤ 100 Sofya Raskhodnikova; Randomness in Computing 1/21/2020
Review: Independence Independent events • Two events 𝐹 1 and 𝐹 2 are independent if Pr 𝐹 1 ∩ 𝐹 2 = Pr 𝐹 1 ⋅ Pr 𝐹 2 . • Events 𝐹 1 , … , 𝐹 𝑜 are mutually independent if, for every subset 𝐽 ∈ 𝑜 , [𝒐] denotes {𝟐, 𝟑, … , 𝒐} Pr 𝐹 𝑗 = Pr 𝐹 𝑗 . 𝑗∈𝐽 𝑗∈𝐽 • If all pairs of events among 𝐹 1 , … , 𝐹 𝑜 are independent then 𝐹 1 , … , 𝐹 𝑜 are pairwise independent. • Pairwise independence does not necessarily imply mutual independence! Sofya Raskhodnikova; Randomness in Computing 1/21/2020
Independence: Example • We toss a fair coin twice. • Let A be the event that the 1 st flip is HEADS. • Let B be the event that the 2 nd flip is HEADS. • Let C be the event that both flips are the same. Are events A,B,C pairwise independent? Are they mutually independent? Sofya Raskhodnikova; Randomness in Computing 1/21/2020
Review: Conditional Probability Conditional Probability The conditional probability of event 𝐹 given event 𝐺 is = Pr(E ∩ 𝐺) Pr 𝐹 𝐺 ⋅ Pr(𝐺) Well defined only if Pr 𝑮 ≠ 𝟏 • When 𝐹 and 𝐺 are independent, = Pr(E ∩ 𝐺) = Pr E ⋅ Pr(𝐺) Pr 𝐹 𝐺 = Pr 𝐹 . Pr(𝐺) Pr(𝐺) Sofya Raskhodnikova; Randomness in Computing 1/21/2020
Review question • Event E: the numbers on two dice sum to 8. • Event F: the numbers on two dice are both even. • What is the probability of E given that F occurred, Pr(𝐹|𝐺) ? A. Less than 1/3 B. 1/3 C. Greater than 1/3, but less than 2/3 D. 2/3 E. Greater than 2/3 Sofya Raskhodnikova; Randomness in Computing 1/21/2020
Review question Card dealing We deal two cards. What is the probability that the second card is an ace, given that the first is an ace? A. 3/52 B. 3/51 C. 4/52 D. 5/52 E. None of the answers above are correct. Sofya Raskhodnikova; Randomness in Computing 1/21/2020
Product rule For any two events 𝐹 1 and 𝐹 2 , Pr 𝐹 1 ∩ 𝐹 2 = Pr(𝐹 1 ) ⋅ Pr 𝐹 2 |𝐹 1 . For all events 𝐹 1 , … , 𝐹 𝑜 , 𝑜−1 𝐹 𝑗 ) 𝑜 Pr(∩ 𝑗=1 𝐹 𝑗 ) = Pr 𝐹 1 ⋅ Pr 𝐹 2 𝐹 1 ⋅ … ⋅ (𝐹 𝑜 | ∩ 𝑗=1 Sofya Raskhodnikova; Randomness in Computing 1/21/2020
Sampling without replacement • Let 𝐹 𝑗 be the event that we choose a root in iteration 𝑗 Pr error in all 𝑙 iterations = Pr 𝐹 1 ∩ ⋯ ∩ 𝐹 𝑙 = Pr 𝐹 1 ⋅ Pr 𝐹 2 𝐹 1 ⋅ … ⋅ Pr[𝐹 𝑙 |𝐹 1 ∩ ⋯ ∩ 𝐹 𝑙−1 ] • It is 0 if 𝑙 > 𝑒. 𝑒 − (𝑘 − 1) • If 𝑙 ≤ 𝑒 , then Pr 𝐹 𝑘 𝐹 1 ∩ ⋯ ∩ 𝐹 𝑘−1 = 100𝑒 − (𝑘 − 1) 𝑙 1 Pr error in all 𝑙 iterations ≤ 100 Sofya Raskhodnikova; Randomness in Computing 1/21/2020
Recommend
More recommend
