Honors Combinatorics CMSC-27410 = Math-28410 CMSC-37200 Instructor: - PowerPoint PPT Presentation

Honors Combinatorics CMSC-27410 = Math-28410 ∼ CMSC-37200 Instructor: Laszlo Babai University of Chicago Week 8, Thursday, May 28, 2020 CMSC-27410=Math-28410 ∼ CMSC-3720 Honors Combinatorics

Variance of sum of random variables Var ( X ) = E (( X − E ( X )) 2 ) = E ( X 2 ) − ( E ( X )) 2 variance � Var ( X ) ≥ 0 standard deviation σ ( X ) = Var ( X ) CMSC-27410=Math-28410 ∼ CMSC-3720 Honors Combinatorics

Variance of sum of random variables Var ( X ) = E (( X − E ( X )) 2 ) = E ( X 2 ) − ( E ( X )) 2 variance � Var ( X ) ≥ 0 standard deviation σ ( X ) = Var ( X ) Cov ( X , Y ) = E ( XY ) − E ( X ) E ( Y ) covariance If X , Y independent then Cov ( X , Y ) = CMSC-27410=Math-28410 ∼ CMSC-3720 Honors Combinatorics

Variance of sum of random variables Var ( X ) = E (( X − E ( X )) 2 ) = E ( X 2 ) − ( E ( X )) 2 variance � Var ( X ) ≥ 0 standard deviation σ ( X ) = Var ( X ) Cov ( X , Y ) = E ( XY ) − E ( X ) E ( Y ) covariance If X , Y independent then Cov ( X , Y ) = 0 CMSC-27410=Math-28410 ∼ CMSC-3720 Honors Combinatorics

Variance of sum of random variables Var ( X ) = E (( X − E ( X )) 2 ) = E ( X 2 ) − ( E ( X )) 2 variance � Var ( X ) ≥ 0 standard deviation σ ( X ) = Var ( X ) Cov ( X , Y ) = E ( XY ) − E ( X ) E ( Y ) covariance If X , Y independent then Cov ( X , Y ) = 0 Cov ( X , X ) = CMSC-27410=Math-28410 ∼ CMSC-3720 Honors Combinatorics

Variance of sum of random variables Var ( X ) = E (( X − E ( X )) 2 ) = E ( X 2 ) − ( E ( X )) 2 variance � Var ( X ) ≥ 0 standard deviation σ ( X ) = Var ( X ) Cov ( X , Y ) = E ( XY ) − E ( X ) E ( Y ) covariance If X , Y independent then Cov ( X , Y ) = 0 Cov ( X , X ) = Var ( X ) CMSC-27410=Math-28410 ∼ CMSC-3720 Honors Combinatorics

Variance of sum of random variables Var ( X ) = E (( X − E ( X )) 2 ) = E ( X 2 ) − ( E ( X )) 2 variance � Var ( X ) ≥ 0 standard deviation σ ( X ) = Var ( X ) Cov ( X , Y ) = E ( XY ) − E ( X ) E ( Y ) covariance If X , Y independent then Cov ( X , Y ) = 0 Cov ( X , X ) = Var ( X ) Y = � n i = 1 X i Var ( Y ) = CMSC-27410=Math-28410 ∼ CMSC-3720 Honors Combinatorics

Variance of sum of random variables Var ( X ) = E (( X − E ( X )) 2 ) = E ( X 2 ) − ( E ( X )) 2 variance � Var ( X ) ≥ 0 standard deviation σ ( X ) = Var ( X ) Cov ( X , Y ) = E ( XY ) − E ( X ) E ( Y ) covariance If X , Y independent then Cov ( X , Y ) = 0 Cov ( X , X ) = Var ( X ) Y = � n i = 1 X i Var ( Y ) = � � j Cov ( X i , X j ) = � i Var ( X i ) + � i � j Cov ( X i , X j ) i If the X i are independent then Var ( Y ) = CMSC-27410=Math-28410 ∼ CMSC-3720 Honors Combinatorics

Variance of sum of random variables Var ( X ) = E (( X − E ( X )) 2 ) = E ( X 2 ) − ( E ( X )) 2 variance � Var ( X ) ≥ 0 standard deviation σ ( X ) = Var ( X ) Cov ( X , Y ) = E ( XY ) − E ( X ) E ( Y ) covariance If X , Y independent then Cov ( X , Y ) = 0 Cov ( X , X ) = Var ( X ) Y = � n i = 1 X i Var ( Y ) = � � j Cov ( X i , X j ) = � i Var ( X i ) + � i � j Cov ( X i , X j ) i If the X i are independent then Var ( Y ) = � i Var ( X i ) CMSC-27410=Math-28410 ∼ CMSC-3720 Honors Combinatorics

Variance of sum of random variables Var ( X ) = E (( X − E ( X )) 2 ) = E ( X 2 ) − ( E ( X )) 2 variance � Var ( X ) ≥ 0 standard deviation σ ( X ) = Var ( X ) Cov ( X , Y ) = E ( XY ) − E ( X ) E ( Y ) covariance If X , Y independent then Cov ( X , Y ) = 0 Cov ( X , X ) = Var ( X ) Y = � n i = 1 X i Var ( Y ) = � � j Cov ( X i , X j ) = � i Var ( X i ) + � i � j Cov ( X i , X j ) i If the X i are pairwise independent then Var ( Y ) = � i Var ( X i ) CMSC-27410=Math-28410 ∼ CMSC-3720 Honors Combinatorics

Variance of sum of random variables X = ± 1 with prob ( 1 / 2 , 1 / 2 ) E ( X ) = CMSC-27410=Math-28410 ∼ CMSC-3720 Honors Combinatorics

Variance of sum of random variables X = ± 1 with prob ( 1 / 2 , 1 / 2 ) E ( X ) = 0 Var ( X ) = CMSC-27410=Math-28410 ∼ CMSC-3720 Honors Combinatorics

Variance of sum of random variables X = ± 1 with prob ( 1 / 2 , 1 / 2 ) E ( X ) = 0 Var ( X ) = E ( X 2 ) − ( E ( X )) 2 = 1 − 0 = 1 CMSC-27410=Math-28410 ∼ CMSC-3720 Honors Combinatorics

Variance of sum of random variables X = ± 1 with prob ( 1 / 2 , 1 / 2 ) E ( X ) = 0 Var ( X ) = E ( X 2 ) − ( E ( X )) 2 = 1 − 0 = 1 X i = ± 1 with prob ( 1 / 2 , 1 / 2 ) independent ( i ∈ [ n ]) Y := � n i = 1 X i CMSC-27410=Math-28410 ∼ CMSC-3720 Honors Combinatorics

Variance of sum of random variables X = ± 1 with prob ( 1 / 2 , 1 / 2 ) E ( X ) = 0 Var ( X ) = E ( X 2 ) − ( E ( X )) 2 = 1 − 0 = 1 X i = ± 1 with prob ( 1 / 2 , 1 / 2 ) independent ( i ∈ [ n ]) Y := � n i = 1 X i Var ( Y ) = CMSC-27410=Math-28410 ∼ CMSC-3720 Honors Combinatorics

Variance of sum of random variables X = ± 1 with prob ( 1 / 2 , 1 / 2 ) E ( X ) = 0 Var ( X ) = E ( X 2 ) − ( E ( X )) 2 = 1 − 0 = 1 X i = ± 1 with prob ( 1 / 2 , 1 / 2 ) independent ( i ∈ [ n ]) Y := � n Var ( Y ) = � n i = 1 X i i = 1 Var ( X i ) = n CMSC-27410=Math-28410 ∼ CMSC-3720 Honors Combinatorics

Variance of sum of random variables X = ± 1 with prob ( 1 / 2 , 1 / 2 ) E ( X ) = 0 Var ( X ) = E ( X 2 ) − ( E ( X )) 2 = 1 − 0 = 1 X i = ± 1 with prob ( 1 / 2 , 1 / 2 ) independent ( i ∈ [ n ]) Y := � n Var ( Y ) = � n i = 1 X i i = 1 Var ( X i ) = n standard deviation of Y : CMSC-27410=Math-28410 ∼ CMSC-3720 Honors Combinatorics

Variance of sum of random variables X = ± 1 with prob ( 1 / 2 , 1 / 2 ) E ( X ) = 0 Var ( X ) = E ( X 2 ) − ( E ( X )) 2 = 1 − 0 = 1 X i = ± 1 with prob ( 1 / 2 , 1 / 2 ) independent ( i ∈ [ n ]) Y := � n Var ( Y ) = � n i = 1 X i i = 1 Var ( X i ) = n √ √ standard deviation of Y : Var Y = n CMSC-27410=Math-28410 ∼ CMSC-3720 Honors Combinatorics

Variance of sum of random variables X = ± 1 with prob ( 1 / 2 , 1 / 2 ) E ( X ) = 0 Var ( X ) = E ( X 2 ) − ( E ( X )) 2 = 1 − 0 = 1 X i = ± 1 with prob ( 1 / 2 , 1 / 2 ) independent ( i ∈ [ n ]) Y := � n Var ( Y ) = � n i = 1 X i i = 1 Var ( X i ) = n √ √ standard deviation of Y : Var Y = n √ If k = O ( n ) and k ≡ n ( mod 2 ) then P ( Y = k ) = Θ( CMSC-27410=Math-28410 ∼ CMSC-3720 Honors Combinatorics

Variance of sum of random variables X = ± 1 with prob ( 1 / 2 , 1 / 2 ) E ( X ) = 0 Var ( X ) = E ( X 2 ) − ( E ( X )) 2 = 1 − 0 = 1 X i = ± 1 with prob ( 1 / 2 , 1 / 2 ) independent ( i ∈ [ n ]) Y := � n Var ( Y ) = � n i = 1 X i i = 1 Var ( X i ) = n √ √ standard deviation of Y : Var Y = n √ If k = O ( n ) and k ≡ n ( mod 2 ) then √ P ( Y = k ) = Θ( 1 / n ) CMSC-27410=Math-28410 ∼ CMSC-3720 Honors Combinatorics

Variance of sum of random variables X = ± 1 with prob ( 1 / 2 , 1 / 2 ) E ( X ) = 0 Var ( X ) = E ( X 2 ) − ( E ( X )) 2 = 1 − 0 = 1 X i = ± 1 with prob ( 1 / 2 , 1 / 2 ) independent ( i ∈ [ n ]) Y := � n Var ( Y ) = � n i = 1 X i i = 1 Var ( X i ) = n √ √ standard deviation of Y : Var Y = n √ If k = O ( n ) and k ≡ n ( mod 2 ) then √ P ( Y = k ) = Θ( 1 / n ) √ If k = Θ( n ) then P ( | Y | ≤ k ) = CMSC-27410=Math-28410 ∼ CMSC-3720 Honors Combinatorics

Variance of sum of random variables X = ± 1 with prob ( 1 / 2 , 1 / 2 ) E ( X ) = 0 Var ( X ) = E ( X 2 ) − ( E ( X )) 2 = 1 − 0 = 1 X i = ± 1 with prob ( 1 / 2 , 1 / 2 ) independent ( i ∈ [ n ]) Y := � n Var ( Y ) = � n i = 1 X i i = 1 Var ( X i ) = n √ √ standard deviation of Y : Var Y = n √ If k = O ( n ) and k ≡ n ( mod 2 ) then √ P ( Y = k ) = Θ( 1 / n ) √ If k = Θ( n ) then P ( | Y | ≤ k ) = Θ( 1 ) CMSC-27410=Math-28410 ∼ CMSC-3720 Honors Combinatorics

Variance of sum of random variables X = ± 1 with prob ( 1 / 2 , 1 / 2 ) E ( X ) = 0 Var ( X ) = E ( X 2 ) − ( E ( X )) 2 = 1 − 0 = 1 X i = ± 1 with prob ( 1 / 2 , 1 / 2 ) independent ( i ∈ [ n ]) Y := � n Var ( Y ) = � n i = 1 X i i = 1 Var ( X i ) = n √ √ standard deviation of Y : Var Y = n √ If k = O ( n ) and k ≡ n ( mod 2 ) then √ P ( Y = k ) = Θ( 1 / n ) √ If k = Θ( n ) then P ( | Y | ≤ k ) = Θ( 1 ) √ Central Limit Theorem: for k = O ( n ) � k 1 e − t 2 / ( 2 n ) dt P ( | Y | ≤ k ) ≈ √ 2 π n − k CMSC-27410=Math-28410 ∼ CMSC-3720 Honors Combinatorics

Strong concentration of sum of independent random variables √ Central Limit Theorem: for k = O ( n ) � k 1 e − t 2 / ( 2 n ) dt P ( | Y | ≤ k ) ≈ √ 2 π n − k tail hard to estimate – too small against the error we may conjecture P ( | Y | ≥ k ) ≈ e − k 2 / ( 2 n ) CMSC-27410=Math-28410 ∼ CMSC-3720 Honors Combinatorics

Strong concentration of sum of independent random variables √ Central Limit Theorem: for k = O ( n ) � k 1 e − t 2 / ( 2 n ) dt P ( | Y | ≤ k ) ≈ √ 2 π n − k tail hard to estimate – too small against the error we may conjecture P ( | Y | ≥ k ) ≈ e − k 2 / ( 2 n ) Theorem (Chernoff bound) X 1 , . . . , X n independent random variables, | X i | ≤ 1 Y = � n i = 1 X i � �     � � �   � �   < 2 e − a 2 / ( 2 n )  = ⇒ ( ∀ a ∈ R )   P  X i ≥ a   � �       � �        � �  i � � CMSC-27410=Math-28410 ∼ CMSC-3720 Honors Combinatorics