MATH 20: PROBABILITY Fundamental Theorems of Probability Theory - - PowerPoint PPT Presentation

β–Ά
math 20 probability
SMART_READER_LITE
LIVE PREVIEW

MATH 20: PROBABILITY Fundamental Theorems of Probability Theory - - PowerPoint PPT Presentation

Jan 25, 2024 484 likes β€’913 views

MATH 20: PROBABILITY Fundamental Theorems of Probability Theory Xingru Chen xingru.chen.gr@dartmouth.edu XC 2020 Fundamental Theorems of Probability Theory 4 5 Central Li Ce Limit 8 Th Theo eorem em 3 2 6 7 1 Law of La La

slide-1
SLIDE 1

MATH 20: PROBABILITY

Fundamental Theorems

  • f

Probability Theory Xingru Chen xingru.chen.gr@dartmouth.edu

XC 2020

slide-2
SLIDE 2

Fundamental Theorems of Probability Theory

4 3 2 1 5 6 7 8

La Law of La Large ge Nu Numbers Ce Central Li Limit Th Theo eorem em

XC 2020

slide-3
SLIDE 3

La Law of La Large ge Number bers

Β§ frequency as interpretation

  • f

probability Β§ convergence

  • f

the sa sample m mea ean

XC 2020

slide-4
SLIDE 4

LAW W OF OF LARGE GE NU NUMBE MBERS FOR OR DISCRETE RAND NDOM OM VARIABL BLES

discrete random variables

XC 2020

slide-5
SLIDE 5

𝐹(π‘Œ) 𝜁

𝑄(π‘Œ β‰₯ 𝜁)

in inequ quality ity

Ma Markov

𝑸(𝒀 β‰₯ 𝜻) ≀ 𝑭(𝒀) 𝜻

This is a sample text. Insert your desired text here. This is a sample text. Insert your desired text here. Let π‘Œ be a nonnegative discrete random variable with expected value 𝐹(π‘Œ), and let 𝜁 > 0 be any positive

  • number. Then

π‘Œ: : nonnegative

XC 2020

slide-6
SLIDE 6

Let π‘Œ be a po posi sitive discrete random variable with expected value 𝐹(π‘Œ), and let 𝜁 > 0 be any positive number.

Proof

𝑸(𝒀 β‰₯ 𝜻) ≀ 𝑭(𝒀) 𝜻

&

π’š

π’šπ’(π’š) = 𝑭(𝒀)

𝑄 π‘Œ β‰₯ 𝜁 = +

!"#

𝑛(𝑦) +

π’š

𝒏(π’š) = 𝟐 𝑭 𝒀 = +

π’š

π’šπ’(π’š) β‰₯ +

π’š"𝜻

π’šπ’ π’š β‰₯ 𝜻 +

π’š"𝜻

𝒏 π’š = πœ»π‘„ π‘Œ β‰₯ 𝜁

XC 2020

slide-7
SLIDE 7

π‰πŸ‘ πœ»πŸ‘

𝑸(|𝒀 βˆ’ 𝝂| β‰₯ 𝜻)

in inequalit ity

Ch Chebyshev

𝑸(|𝒀 βˆ’ 𝝂| β‰₯ 𝜻) ≀ π‰πŸ‘ πœ»πŸ‘

This is a sample text. Insert your desired text here. This is a sample text. Insert your desired text here. Let π‘Œ be a discrete random variable with expected value 𝜈 = 𝐹(π‘Œ) and variance 𝜏' = π‘Š(π‘Œ), and let 𝜁 > 0 be any positive

  • number. Then

π‘Œ: : not t necessarily no nonne nnegative ve

XC 2020

slide-8
SLIDE 8

Let X be a discrete random variable with expected value 𝜈 = 𝐹(π‘Œ) and variance 𝜏' = π‘Š(π‘Œ), and let 𝜁 > 0 be any positive number.

Proof

𝑸(|𝒀 βˆ’ 𝝂| β‰₯ 𝜻) ≀ π‰πŸ‘ πœ»πŸ‘

&

π’š

(π’š βˆ’ 𝝂)πŸ‘π’(π’š) = 𝑾(𝒀)

𝑄(|𝒀 βˆ’ 𝝂| β‰₯ 𝜻) = +

|π’š)𝝂|"𝜻

𝑛(𝑦) +

π’š

𝒏(π’š) = 𝟐 𝜏' = π‘Š π‘Œ = +

!

(𝑦 βˆ’ 𝜈)'𝑛(𝑦) β‰₯ +

!)+ "#

𝑦 βˆ’ 𝜈 '𝑛 𝑦 β‰₯ 𝜁' +

!)+ "#

𝑛 𝑦 = 𝜁'𝑄(|π‘Œ βˆ’ 𝜈| β‰₯ 𝜁)

XC 2020

slide-9
SLIDE 9

in inequalit ity

Ch Chebyshev

𝑸(|𝒀 βˆ’ 𝝂| β‰₯ 𝜻) ≀ π‰πŸ‘ πœ»πŸ‘

in inequ quality ity

Ma Markov

𝑸(𝒀 β‰₯ 𝜻) ≀ 𝑭(𝒀) 𝜻

π‘Œ: : nonnegative 𝑭(𝒀): kn known π‘Œ: : not t necessarily no nonne nnegative ve 𝑭(𝒀): kn known 𝑾(𝒀): kn known

XC 2020

slide-10
SLIDE 10

Example 1

Β§ Let π‘Œ be any random variable which takes

  • n

values 0, 1, 2, β‹― , π‘œ and has 𝐹(π‘Œ) = π‘Š (π‘Œ) = 1. Show that for any positive integer 𝑙, 𝑄(π‘Œ β‰₯ 𝑙 + 1) ≀ "

##.

Ch Chebys byshev Inequality 𝑄(|π‘Œ βˆ’ 𝜈| β‰₯ 𝜁) ≀ 𝜏$ 𝜁$ Ma Markov kov Inequality 𝑄(π‘Œ β‰₯ 𝜁) ≀ 𝐹(π‘Œ) 𝜁

XC 2020

slide-11
SLIDE 11

Example 1

Β§ Let π‘Œ be any random variable which takes

  • n

values 0, 1, 2, β‹― , π‘œ and has 𝐹(π‘Œ) = π‘Š (π‘Œ) = 1. Show that for any positive integer 𝑙, 𝑄(π‘Œ β‰₯ 𝑙 + 1) ≀ "

##.

Ch Chebyshev Inequ quality 𝑄(|π‘Œ βˆ’ 𝜈| β‰₯ 𝜁) ≀ 𝜏' 𝜁' 𝜈 = 𝜏' = 1, 𝜁 = 𝑙 𝑄 π‘Œ βˆ’ 1 β‰₯ 𝑙 = 𝑄(π‘Œ β‰₯ 𝑙 + 1) ≀ 1 𝑙'

XC 2020

slide-12
SLIDE 12

Example 2

Β§ Choose π‘Œ with distribution 𝑛 𝑦 = βˆ’πœ 𝜁

" $ " $

.

Ch Chebyshev Inequ quality 𝑄(|π‘Œ βˆ’ 𝜈| β‰₯ 𝜁) ≀ 𝜏' 𝜁' 𝜈 = 0, 𝜏' = 𝜁' 𝑄 π‘Œ β‰₯ 𝜁 ≀ 𝜁' 𝜁' = 1

𝑄 π‘Œ β‰₯ 𝜁 = 1

XC 2020

slide-13
SLIDE 13

Law of Large Numbers

Β§ Let π‘Œ", π‘Œ$, β‹―, π‘Œ% be an independent trials process, with same finite expected value 𝜈 = 𝐹(π‘Œ

&) and

finite variance 𝜏$ = π‘Š(π‘Œ

&).

Β§ Let 𝑇% = π‘Œ" + π‘Œ$ + β‹― + π‘Œ%. Then for any 𝜁 > 0, 𝑄(| '$

% βˆ’ 𝜈| β‰₯ 𝜁) β†’ 0,

as π‘œ β†’ +∞. Β§ Equivalently, lim

%β†’)𝑄( '$ % βˆ’ 𝜈 < 𝜁) = 1.

!

As

,!

  • is

an average

  • f

the individual

  • utcomes,

the LLN is

  • ften

referred to as the la law

  • f

averages.

Large Numbers

XC 2020

slide-14
SLIDE 14

Law of Large Numbers

Β§ Let π‘Œ", π‘Œ$, β‹― , π‘Œ% be an independent trials process with 𝐹 π‘Œ

& = 𝜈 and

π‘Š π‘Œ

& = 𝜏$.

Β§ Let 𝑇% = π‘Œ" + π‘Œ$ + β‹― + π‘Œ% be the sum, and 𝐡% = '$

% be

the

  • average. Then

=

𝐹 𝐡% = 𝜈

=

π‘Š 𝐡% = *#

% ,

𝐸 𝐡% = *

%

π‘œ β†’ +∞ π‘Š 𝐡% β†’ 0 𝐸 𝐡% β†’ 0

XC 2020

slide-15
SLIDE 15

Let π‘Œ., π‘Œ', β‹―, π‘Œ- be an independent trials process, with same finite expected value 𝜈 = 𝐹(π‘Œ

/) and

finite variance 𝜏' = π‘Š(π‘Œ

/).

Proof

𝑄(| 𝑇% π‘œ βˆ’ 𝜈| β‰₯ 𝜁) β†’ 0, as π‘œ β†’ +∞.

𝑾 𝑻𝒐 𝒐 = π‰πŸ‘ 𝒐

𝑄 𝑇- π‘œ βˆ’ 𝜈 β‰₯ 𝜁 ≀ 𝜏' π‘œπœ' 𝑭 𝑻𝒐 𝒐 = 𝝂 π‘œ β†’ +∞ 𝑸(|𝒀 βˆ’ 𝝂| β‰₯ 𝜻) ≀ π‰πŸ‘ πœ»πŸ‘ 𝜏' π‘œπœ' β†’ 0

XC 2020

slide-16
SLIDE 16

Law of Large Numbers

(Strong) Law

  • f

Large Numbers (Weak) Law

  • f

Large Numbers

lim

%β†’)𝑄( 𝑇%

π‘œ βˆ’ 𝜈 < 𝜁) = 1 𝑄 lim

%β†’)

𝑇% π‘œ = 𝜈 = 1 con convergence ce of

  • f the

the sa sample me mean

XC 2020

slide-17
SLIDE 17

Example 3

Β§ Consider the general Bernoulli trial process. Β§ As usual, we let π‘Œ = 1 if the

  • utcome

is a success and 0 if it is a failure. Expect cted value 𝑭(𝒀) I

+∈-

𝑦𝑛(𝑦) = 1Γ—π‘ž + 0Γ— 1 βˆ’ π‘ž = π‘ž Ber Bernoulli t tria ial 𝑛 𝑦 = L π‘ž, π‘Œ = 1 1 βˆ’ π‘ž, π‘Œ = 0 Variance ce π‘Š π‘Œ 𝐹 π‘Œ$ βˆ’ 𝜈$ = π‘ž βˆ’ π‘ž$ = π‘ž(1 βˆ’ π‘ž)

XC 2020

slide-18
SLIDE 18

Example 3

Β§ Now consider π‘œ Bernoulli trials. Β§ Then 𝑇% = βˆ‘./"

%

π‘Œ. is the number

  • f

successes in π‘œ trials and 𝜈 = 𝐹

'$ %

= 𝐹 π‘Œ. = π‘ž.

La Law

  • f

La Large N Numbers lim

  • β†’2 𝑄( 𝑇-

π‘œ βˆ’ 𝜈 < 𝜁) = 1 𝜈 = π‘ž lim

  • β†’2 𝑄( 𝑇-

π‘œ βˆ’ π‘ž < 𝜁) = 1

XC 2020

slide-19
SLIDE 19

Coin Tossing

π‘œ β†’ +∞ 𝜈 = 1 2 π‘Œ" + π‘Œ$ + β‹― + π‘Œ% π‘œ

XC 2020

slide-20
SLIDE 20

Dice Rolling

π‘œ β†’ +∞ 𝜈 = 7 2 π‘Œ" + π‘Œ$ + β‹― + π‘Œ% π‘œ

XC 2020

slide-21
SLIDE 21

Bernoulli Trials

We can start with a random experiment about which little can be predicted and, by taking averages,

  • btain

an experiment in which the

  • utcome

can be predicted with a high degree

  • f

certainty.

XC 2020

slide-22
SLIDE 22

La Law of La Large ge Number bers

Β§ frequency as interpretation

  • f

probability Β§ convergence

  • f

the sa sample m mea ean 𝑇% = π‘Œ" + π‘Œ$ + β‹― + π‘Œ%, 𝐡% = '$

%

𝐹 𝐡% = 𝜈, π‘Š 𝐡% = *#

%

𝑄 𝑇% π‘œ βˆ’ 𝜈 β‰₯ 𝜁 ≀ 𝜏$ π‘œπœ$

XC 2020

slide-23
SLIDE 23

LAW W OF OF LARGE GE NU NUMBE MBERS FOR OR CONT ONTINU NUOU OUS RAND NDOM OM VARIABL BLES

continuous random variables

XC 2020

slide-24
SLIDE 24

π‰πŸ‘ πœ»πŸ‘

𝑸(|𝒀 βˆ’ 𝝂| β‰₯ 𝜻)

in inequalit ity

Ch Chebyshev

𝑸(|𝒀 βˆ’ 𝝂| β‰₯ 𝜻) ≀ π‰πŸ‘ πœ»πŸ‘

This is a sample text. Insert your desired text here. This is a sample text. Insert your desired text here. Let π‘Œ be a continuous random variable with density function 𝑔(𝑦). Suppose π‘Œ has a finite expected value 𝜈 = 𝐹(π‘Œ) and fintite variance 𝜏' = π‘Š(π‘Œ). Then for any positive 𝜁 > 0 we have fi finite te exp xpecte ted va value fi finite te vari riance

XC 2020

slide-25
SLIDE 25

Law of Large Numbers

Β§ Let π‘Œ", π‘Œ$, β‹―, π‘Œ% be an independent trials process with a continuous density function 𝑔, fini nite te expected value 𝜈 and fini nite te variance 𝜏$. Β§ Let 𝑇% = π‘Œ" + π‘Œ$ + β‹― + π‘Œ%. Then for any 𝜁 > 0, 𝑄(| '$

% βˆ’ 𝜈| β‰₯ 𝜁) β†’ 0,

as π‘œ β†’ +∞. Β§ Equivalently, lim

%β†’)𝑄( '$ % βˆ’ 𝜈 < 𝜁) = 1.

!

As

,!

  • is

an average

  • f

the individual

  • utcomes,

the LLN is

  • ften

referred to as the la law

  • f

averages.

XC 2020

slide-26
SLIDE 26

Example 4

Β§ Suppose we choose at random π‘œ numbers from the interval [0, 1] with uniform distribution. Β§ Let π‘Œ. describes the 𝑗th choice. Expect cted value 𝑭(𝒀) 𝐹 π‘Œ = 1 2 𝑏 + 𝑐 = 1 2 0 + 1 = 1 2 Un Unifor

  • rm

distribu bution

  • n

𝑔 𝑦 = 1 𝑐 βˆ’ 𝑏 , 𝑏 ≀ 𝑦 ≀ 𝑐 𝑔 𝑦 = 1, 0 ≀ 𝑦 ≀ 1 Variance ce π‘Š π‘Œ π‘Š π‘Œ = 𝐹 π‘Œ$ βˆ’ 𝜈$ = 1 12 (𝑐 βˆ’ 𝑏)$= 1 12 (1 βˆ’ 0)$= 1 12

XC 2020

slide-27
SLIDE 27

Example 4

Β§ Suppose we choose at random π‘œ numbers from the interval [0, 1] with uniform distribution. Β§ Let π‘Œ. describes the 𝑗th choice and 𝑇% = βˆ‘./"

%

π‘Œ..

Ch Chebyshev Inequ quality

𝑄(|π‘Œ βˆ’ 𝜈| β‰₯ 𝜁) ≀ 𝜏$ 𝜁$ 𝜈 = "

$, 𝜏$ = " "$

𝑄 𝑇% π‘œ βˆ’ 1 2 β‰₯ 𝜁 ≀ 1 12π‘œπœ$ Expect cted value 𝑭(𝑇% π‘œ ) = 1 2 Variance ce π‘Š 𝑇% π‘œ = 1 12π‘œ

XC 2020

slide-28
SLIDE 28

XC 2020

slide-29
SLIDE 29

Example 5

Β§ Suppose we choose π‘œ real numbers at random, using a normal distribution with mean and variance

  • 1. Then

Β§ 𝜈 = 𝐹 π‘Œ. = 0 Β§ 𝜏$ = π‘Š π‘Œ. = 1

Ch Chebyshev Inequ quality

𝑄(|π‘Œ βˆ’ 𝜈| β‰₯ 𝜁) ≀ 𝜏$ 𝜁$ 𝜈 = 0, 𝜏$ = "

%

𝑄 𝑇% π‘œ β‰₯ 𝜁 ≀ 1 π‘œπœ$ Expect cted value 𝑭(𝑇% π‘œ ) = 0 Variance ce π‘Š 𝑇% π‘œ = 1 π‘œ

XC 2020

slide-30
SLIDE 30

XC 2020

slide-31
SLIDE 31

Example 6

Β§ Suppose we choose n numbers from (βˆ’βˆž, +∞) with a Cauchy density with parameter 𝑏 = 1. Β§ We know that for the Cauchy density the expected value and variance are undefined. Expect cted value 𝑭(𝒀) undefined Cauch chy distribu bution

  • n

𝑔 𝑦 = 1 πœŒπ‘(1 + 𝑦 𝑏)$ 𝑔 𝑦 = 1 𝜌(1 + 𝑦)$ Variance ce π‘Š π‘Œ undefined

XC 2020

slide-32
SLIDE 32

Example 6

§ In this case, the density function for 𝐡% = '$

% is

given by Cauch chy distribu bution

  • n

𝑔 𝑦 = 1 𝜌(1 + 𝑦)$

!

The density function for 𝐡% is the same for all π‘œ.

!

The Law

  • f

Large Numbers does not hold. Why?

XC 2020

slide-33
SLIDE 33

Example 6

§ In this case, the density function for 𝐡% = '$

% is

given by Cauch chy distribu bution

  • n

𝑔 𝑦 = 1 𝜌(1 + 𝑦)$

!

The density function for 𝐡% is the same for all π‘œ.

!

The Law

  • f

Large Numbers does not hold. Why? finite expected value and finite variance!

XC 2020

slide-34
SLIDE 34

MONTE CARLO METHOD

estimate the area

  • f

the region

XC 2020

slide-35
SLIDE 35

Monte Carlo method

Β§ Let 𝑕(𝑦) be continuous function defined for 𝑦 ∈ [0, 1] with values in [0, 1]. Β§ How to estimate the area

  • f

the region under the graph

  • f

𝑕(𝑦)?

XC 2020

slide-36
SLIDE 36

Monte Carlo method: naive

Β§ Let 𝑕(𝑦) be continuous function defined for 𝑦 ∈ [0, 1] with values in [0, 1]. Β§ How to estimate the area

  • f

the region under the graph

  • f

𝑕(𝑦)?

Choose a large number

  • f

random values for 𝑦 and 𝑧 with uniform distribution and seeing what fraction

  • f

the points 𝑄(𝑦, 𝑧) fall inside the region under the graph.

XC 2020

slide-37
SLIDE 37

Monte Carlo method: advanced

Β§ Let 𝑕(𝑦) be continuous function defined for 𝑦 ∈ [0, 1] with values in [0, 1]. Β§ How to estimate the area

  • f

the region under the graph

  • f

𝑕(𝑦)?

Choose a large number

  • f

independent values π‘Œ- at random from [0, 1] with uniform density. Set 𝑍

  • = 𝑕(π‘Œ-).

Area: 𝐡 U

3 .

𝑕 𝑦 𝑒𝑦 Expected value: 𝜈 𝐹 𝑍

  • = 𝐹 𝑕 π‘Œ-

= U

3 .

𝑕 𝑦 𝑔 𝑦 𝑒𝑦 = U

3 .

𝑕 𝑦 𝑒𝑦

XC 2020

slide-38
SLIDE 38

Monte Carlo method: advanced

Β§ Let 𝑕(𝑦) be continuous function defined for 𝑦 ∈ [0, 1] with values in [0, 1]. Β§ How to estimate the area

  • f

the region under the graph

  • f

𝑕(𝑦)?

Choose a large number

  • f

independent values π‘Œ- at random from [0, 1] with uniform density. Set 𝑍

  • = 𝑕(π‘Œ-).

Area: 𝐡 U

3 .

𝑕 𝑦 𝑒𝑦 Expected value: 𝜈 𝐹 𝑍

  • =

= U

3 .

𝑕 𝑦 𝑒𝑦 𝐡 = 𝜈 β‰ˆ 1 π‘œ +

45.

  • 𝑍
  • = 1

π‘œ +

45.

  • 𝑕(π‘Œ-)

XC 2020

slide-39
SLIDE 39

Monte Carlo method: advanced

Β§ Let 𝑕(𝑦) be continuous function defined for 𝑦 ∈ [0, 1] with values in [0, 1]. Β§ How to estimate the area

  • f

the region under the graph

  • f

𝑕(𝑦)?

Area: 𝐡 U

3 .

𝑕 𝑦 𝑒𝑦 Expected value: 𝜈 𝐹 𝑍

  • =

= U

3 .

𝑕 𝑦 𝑒𝑦 𝐡 = 𝜈 β‰ˆ 1 π‘œ +

45.

  • 𝑍
  • = 1

π‘œ +

45.

  • 𝑕 π‘Œ- = 𝐡-

𝜏' = 𝐹 𝑍

  • βˆ’ 𝜈 '

= U

3 .

(𝑕 𝑦 βˆ’ 𝜈)'𝑒𝑦 ≀ 1 𝑄(|𝐡- βˆ’ 𝐡| β‰₯ 𝜁) ≀ 𝜏' π‘œπœ' ≀ 1 π‘œπœ'

XC 2020

slide-40
SLIDE 40

v

Qu Quiz 12 Qu

Question 3

A student’s score

  • n

a particular probability final is a random variable with values

  • f

[0, 100], mean 70, and variance 25. Using Chebyshev’s Inequality, find a lower bound for the probability that the student’s score will fall between 65 and 75.

𝑄 65 ≀ π‘Œ ≀ 75 = 𝑄 π‘Œ βˆ’ 70 ≀ 5 = 1 βˆ’ 𝑄 π‘Œ βˆ’ 70 β‰₯ 5 β‰₯ 1 βˆ’ 25 5' = 1 βˆ’ 1 = 0 𝑸(|𝒀 βˆ’ 𝝂| β‰₯ 𝜻) ≀ π‰πŸ‘ πœ»πŸ‘ no not int nteresting ng

XC 2020

slide-41
SLIDE 41

v

Qu Quiz 12 Qu

Question 4

A student’s score

  • n

a particular probability final is a random variable with values

  • f

[0, 100], mean 70, and variance 25. If 10 students take the final, find a lower bound for the probability that the class average will fall between 65 and 75.

𝑄 65 ≀ 𝐡- ≀ 75 = 𝑄 𝐡- βˆ’ 70 ≀ 5 = 1 βˆ’ 𝑄 𝐡- βˆ’ 70 β‰₯ 5 β‰₯ 1 βˆ’ 25/10 5' = 1 βˆ’ 1 10 = 9 10 𝑸(|𝒀 βˆ’ 𝝂| β‰₯ 𝜻) ≀ π‰πŸ‘ πœ»πŸ‘

XC 2020