INSTITUTO POLITÉCNICO NACIONAL CENTRO DE INVESTIGACION EN COMPUTACION Laboratorio de Ciberseguridad Probability, Random Processes and Inference Dr. Ponciano Jorge Escamilla Ambrosio pescamilla@cic.ipn.mx http://www.cic.ipn.mx/~pescamilla/
CIC Inequalities Inequalities and Limit and Limit Theorems Theorems ❑ What should I do if I can’t calculate a probability or expectation exactly? ➢ Simulations using Monte Carlo: “Monte Carlo” just means that the simulations use random numbers (the term originated from the Monte Carlo Casino in Monaco). ➢ Bounds using inequalities: A bound on a probability gives a provable guarantee that the probability is in a certain range. ➢ Approximations using limit theorems: Limit theorems let us make approximations which are likely to work well when we have a large number of data points. 2
CIC Markov Markov Inequality Inequality ❑ If a random variable X can only take nonnegative values ( X ≥ 0), then: 𝑄 𝑌 ≥ 𝑏 ≤ 𝐹 𝑌 𝑏 for all constant a > 0 ❑ The Markov inequality asserts that if a nonnegative random variable has a small mean, then the probability that it takes a large value must also be small. 3
CIC Markov Markov Inequality Inequality 4
CIC Chebyshev Chebyshev Inequality Inequality ❑ If X is a random variable with mean μ and variance 2 . Then for any a > 0: ❑ The Chebyshev inequality asserts that if a random variable has small variance, then the probability that it takes a value far from its mean is also small. 5
CIC Chebyshev Chebyshev Inequality Inequality ❑ Proof . By Markov’s inequality , ❑ Substituting c for a , for c > 0, we have the following equivalent form of Chebyshev’s inequality: ➢ This gives us an upper bound on the probability of an r.v. being more than c standard deviations away from its mean, e.g., there can’t be more than a 25 % chance of being 2 or more standard deviations from the mean. 6
CIC Law of Large Numbers Law of Large Numbers ❑ We started the course by saying that, in the long term, about half of the flips of a fair coin yield tail. ❑ This is our intuitive understanding of probability. ❑ The law of large number explains that our model of uncertain events conforms to that property. ❑ The central limit theorem tells us how fast this convergence happens. 7
CIC Weak Law Weak Law of Large Numbers of Large Numbers ❑ Let X be a random variable for which the mean, E[X] = is unknown. Let X 1 ,…, X n denote n independent, repeated measurements of X; that is, the X j ’s are independent, identically distributed (i.i.d.) random variables with the same PDF as X. The sample mean of the sequence is used to estimate E[X]: 8
CIC Weak Law Weak Law of Large Numbers of Large Numbers ❑ We have: ❑ Using independence we have: 9
CIC Weak Law Weak Law of Large Numbers of Large Numbers ❑ Applying Chebyshev inequality: ❑ For any fixed 𝜗 > 0, the right-hand side of this inequality goes to zero as n increases. 10
CIC Weak Law Weak Law of Large Numbers of Large Numbers 11
CIC Weak Law Weak Law of Large Numbers of Large Numbers The weak law of large numbers asserts that the sample mean of a large number of i.i.d. random variables is very close to the true mean, with high probability. 12
CIC Weak Law Weak Law of Large Numbers of Large Numbers 13
CIC The The Centr Central al Limit Limit Theorem Theorem 14
CIC The The Centr Central al Limit Limit Theorem Theorem 15
CIC The The Centr Central al Limit Limit Theorem Theorem 16
CIC The The Centr Central al Limit Limit Theorem Theorem ❑ The PDF of the standardized sum of a large number of continuous IID random variables will converge to a Gaussian PDF. ❑ In many practical situations one can model a random variable as having arisen from the contributions of many small and similar physical effects. 17
CIC The The Centr Central al Limit Limit Theorem Theorem ❑ Example - PDF for sum of lID U (-1/2 , 1/2) random variables. Consider the sum: 18
CIC The The Centr Central al Limit Limit Theorem Theorem 19
CIC The The Centr Central al Limit Limit Theorem Theorem 20
CIC The The Centr Central al Limit Limit Theorem Theorem PDF of sum of N i.i.d. U (0 , 1) random variables 21
CIC The The Centr Central al Limit Limit Theorem Theorem PDF of sum of N i.i.d. U (0 , 1) random variables 22
CIC The The Str Strong Law of ong Law of Large Large Numbers umbers 23
CIC The The Str Strong Law of ong Law of Large Large Numbers umbers 24
CIC The The Str Strong Law of ong Law of Large Large Numbers umbers 25
Recommend
More recommend
