1 EE361: SIGNALS AND SYSTEMS II CH1: PROBABILITY http://www.ee.unlv.edu/~b1morris/ee361
2 SAMPLE SPACE AND EVENTS CHAPTER 1.1-1.2
3 INTRODUCTION Very important mathematical framework that is used in every ECE discipline (and science and engineering in general) The world is not perfect and some variability occurs in everything we do professionally The recorded sound of a voice saying a number multiple times The pixel values of successive images
4 (RANDOM) EXPERIMENT Process of observation (Random if outcome cannot be determined with certainty) Outcome – the result of an observation of an experiment Will use 𝜊 (xi) for an outcome Think of Schrödinger's cat paradox from quantum mechanics an experiment has many possible outcomes but only after observation can it be known
5 EXAMPLE RANDOM EXPERIMENTS Roll of a dice Toss of a coin Drawing of a card from a deck
6 SAMPLE SPACE The universal set of sample space 𝑇 – the set of all possible outcomes of a random experiment Sample point – an element in set 𝑇 (a possible outcome) Cardinality – the number of elements in a set
7 EXAMPLES Single coin toss 𝑇 = 𝐼, 𝑈 = {Heads, Tails} 𝑇 = 2 for two possible outcomes Two coin tosses 𝑇 = {𝐼𝐼, 𝐼𝑈, 𝑈𝐼, 𝑈𝑈} 𝑇 = 4
8 SAMPLE SPACE Discrete 𝑇 – finite number of sample points (outcomes) or countably infinite Countable set – elements can be placed in one-to-one correspondence with positive integers Continuous 𝑇 – sample points constitute a continuum Example: transistor lifetime in hours Transistor can live until it dies at time 𝜐 𝑇 = {𝜐: 0 ≤ 𝜐 < ∞}
9 EVENTS Set notations Event – subset of sample space 𝑇 𝜊 ∈ 𝑌 - 𝜊 is an element of Even roll of die 𝐵 = {2, 4, 6} (belongs to) set 𝑇 Sum of rolls greater than 6 𝜊 ∉ 𝑇 - 𝜊 is not an element of 𝑇 Elementary event – sample 𝐵 ⊂ 𝐶 – 𝐵 contained in 𝐶 point of 𝑇 (single outcome of 𝑇 ) 𝐵 is a subset of 𝐶 if every element Certain event – 𝑇 ⊂ 𝑇 of 𝐵 is contained in 𝐶
10 ALGEBRA OF SETS CHAPTER 1.3
ҧ ҧ 11 SET ALGEBRA I 1 Equality – 𝐵 = 𝐶 if 𝐵 ⊂ 𝐶 and 𝐶 ⊂ 𝐵 Venn Diagram 2 Complement – suppose 𝐵 ⊂ 𝑇 , 𝐵 = {𝜊: 𝜊 ∈ 𝑇 and 𝜊 ∉ 𝐵} All elements in 𝑇 but not in 𝐵 𝐵 is event 𝐵 did not occur 3 Union – set containing all elements in either 𝐵 or 𝐶 or both 𝐵 ∪ 𝐶 = {𝜊: 𝜊 ∈ 𝐵 or 𝜊 ∈ 𝐶} Event either 𝐵 of 𝐶 occurred 4 Intersection – set containing all elements in both 𝐵 and 𝐶 𝐵 ∩ 𝐶 = {𝜊: 𝜊 ∈ 𝐵 and 𝜊 ∈ 𝐶} Both event 𝐵 and 𝐶 occurred https://www.onlinemathlearning.com/venn-diagram.html
12 SET ALGEBRA II Null set – set containing no elements 𝑇 = 𝜚 or 𝑇 = {𝜚} Disjoint sets (mutually exclusive) Sets 𝐵 and 𝐶 have no common elements 𝐵 ∩ 𝐶 = 𝜚 Partition of 𝑇 – a way to divide space 𝑇 up completely 𝑙 If 𝐵 𝑗 ∩ 𝐵 𝑘 = 𝜚 (𝑗 ≠ 𝑘) and ڂ 𝑗=1 𝐵 𝑗 = 𝑇 Then {𝐵 𝑗 : 1 ≤ 𝑗 ≤ 𝑙} is a partition
13 SET ALGEBRA III (Cartesian) product of sets Size (cardinality) of set |𝐵| - number of elements in 𝐵 (when 𝐷 = 𝐵 × 𝐶 = 𝑏, 𝑐 : 𝑏 ∈ 𝐵, 𝑐 ∈ 𝐶 countable) Set of ordered pairs of elements Properties Example If 𝐵 ∩ 𝐶 = 𝜚 , then 𝐵 ∪ 𝐶 = 𝐵 + 𝐶 𝐵 = 𝑏 1 , 𝑏 2 , 𝑏 3 , 𝐶 = {𝑐 1 , 𝑐 2 } 𝜚 = 0 𝐷 = 𝐵 × 𝐶 = If 𝐵 ⊂ 𝐶 , then 𝐵 ≤ |𝐶| { 𝑏 1 , 𝑐 1 , 𝑏 1 , 𝑐 2 , 𝑏 2 , 𝑐 1 , 𝐵 ∪ 𝐶 + 𝐵 ∩ 𝐶 = 𝐵 + |𝐶| 𝑏 2 , 𝑐 2 , 𝑏 3 , 𝑐 1 , 𝑏 3 , 𝑐 2 } 𝐸 = 𝐶 × 𝐵 = { 𝑐 1 , 𝑏 1 , 𝑐 1 , 𝑏 2 , 𝑐 1 , 𝑏 3 , 𝑐 2 , 𝑏 1 , 𝑐 2 , 𝑏 2 , 𝑐 2 , 𝑏 3 }
ҧ ҧ 14 LAWS OF SETS Commutative 𝐵 ∪ 𝐶 = 𝐶 ∪ 𝐵 𝐵 ∩ 𝐶 = 𝐶 ∩ 𝐵 Associativity 𝐵 ∪ 𝐶 ∪ 𝐷 = 𝐵 ∪ 𝐶 ∪ 𝐷 𝐵 ∩ 𝐶 ∩ 𝐷 = 𝐵 ∩ 𝐶 ∩ 𝐷 Distributivity 𝐵 ∩ 𝐶 ∪ 𝐷 = 𝐵 ∩ 𝐶 ∪ (𝐵 ∩ 𝐷) 𝐵 ∪ 𝐶 ∩ 𝐷 = 𝐵 ∪ 𝐶 ∩ (𝐵 ∪ 𝐷) De Morgan’s 𝐵 ∩ ത 𝐵 ∪ ത 𝐵 ∪ 𝐶 = 𝐶 𝐵 ∩ 𝐶 = 𝐶
15 EVENT SPACE Collection 𝐺 of subsets of sample space 𝑇 Also known as a sigma field A set of subsets 𝐵 such that 𝑇 ∈ 𝐺 If 𝐵 ∈ 𝐺 , then ҧ 𝐵 ∈ 𝐺 ∞ 𝐵 𝑗 ∈ 𝐺 If 𝐵 𝑗 ∈ 𝐺 for 𝑗 ≥ 1 , then ڂ 𝑗=1 Example coin toss – 𝑇 = {𝐼, 𝑈} 1 = {𝑇, 𝜚} , 𝐺 𝐺 2 = {𝑇, 𝜚, 𝐼, 𝑈} 𝐺 3 = {𝑇, 𝜚, 𝐼} – not event space since ഥ 𝐼 = 𝑈 not included
16 PROBABILITY SPACE/EQUALLY LIKELY EVENTS CHAPTER 1.4-1.5
17 PROBABILITY SPACE Assigns a real number to events in event space 𝐺 Known as probability measure Given a random experiment with sample space 𝑇 𝐵 is an event defined in 𝐺 Probability of event 𝐵 𝑄(𝐵) Probability space is defined over an event space Triple: (𝑇, 𝐺, 𝑄) = (sample space, event space, probability measure)
18 PROBABILITY MEASURE Two methods to define Classical definition Relative frequency definition
ҧ ҧ 19 CLASSICAL (FINITE OUTCOMES) DEFINITION Defined a priori, without need for experimentation Only possible to use for simple problems with finite and equally likely outcomes 𝐵 - probability of event 𝐵 𝑄 𝐵 = 𝑇 𝐵 𝑇 − 𝐵 𝐵 𝑄 𝐵 = 𝑇 = = 1 − 𝑇 = 1 − 𝑄 𝐵 |𝑡| Given disjoint sets 𝐵 ∩ 𝐶 = 𝜚 𝑄 𝐵 ∪ 𝐶 = 𝑄 𝐵 + 𝑄(𝐶)
20 EXAMPLE: DICE ROLL 𝑇 = {1, 2, 3, 4, 5, 6} Define events A: Roll (outcome) is even 𝐵 = {2,4,6} B: Roll odd 𝐶 = {1,3,5} C: Roll prime 𝐷 = {1,2,3,5} Probability of events 𝑄 𝐵 = 3 6 = 1 𝑄 𝐶 = 1 𝑄 𝐷 = 4 6 = 2 2 2 3
21 RELATIVE FREQUENCY DEFINITION Repeated experiment definition Random experiment repeated 𝑜 times 𝑜(𝐵) relative frequency of event A 𝑄 𝐵 = lim 𝑜→∞ 𝑜 𝑜(𝐵) – number of times event A occurs 𝑜 𝐵 0 ≤ ≤ 1 𝑜 0 – never occurs 1 – occurs every time Like classical, 𝐵 ∩ 𝐶 = 𝜚 ⇒ 𝑄 𝐵 ∪ 𝐶 = 𝑄 𝐵 + 𝑄(𝐶)
22 AXIOMATIC DEFINITION Given probability space (𝑇, 𝐺, 𝑄) and event 𝐵 ∈ 𝐺 Axiom 1: 𝑄 𝐵 ≥ 0 Axiom 2: 𝑄 𝑇 = 1 One outcome certainly happens Axiom 3: 𝑄 𝐵 ∪ 𝐶 = 𝑄 𝐵 + 𝑄(𝐶) , if 𝐵 ∩ 𝐶 = 𝜚
ҧ 23 ELEMENTARY PROPERTIES OF PROB. 1 𝑄 𝐵 = 1 − 𝑄 𝐵 2 𝑄 𝜚 = 0 3 𝑄 𝐵 ≤ 𝑄 𝐶 if 𝐵 ⊂ 𝐶 4 𝑄 𝐵 ≤ 1 ⇒ 0 ≤ 𝑄 𝐵 ≤ 1 5 𝑄 𝐵 ∪ 𝐶 = 𝑄 𝐵 + 𝑄 𝐶 − 𝑄 𝐵 ∩ 𝐶 ≤ 𝑄 𝐵 + 𝑄(𝐶) 8 Given 𝐵 1 , 𝐵 2 , … , 𝐵 𝑜 finite sequence of mutually exclusive events (𝐵 𝑗 ∩ 𝐵 𝑘 = 𝜚, 𝑗 ≠ 𝑘) , 𝑜 𝑜 𝐵 𝑗 = σ 𝑗=1 𝑄ڂ 𝑗=1 𝑄(𝐵 𝑗 )
24 EQUALLY LIKELY EVENTS CHAPTER 1.5
25 EQUALLY LIKELY EVENTS Finite sample space (n-finite) 𝑇 = {𝜊 1 , 𝜊 2 , … , 𝜊 𝑜 } and 𝑄 𝜊 𝑗 = 𝑞 𝑗 𝑜 σ 𝑗=1 0 ≤ 𝑞 𝑗 ≤ 1 𝑞 𝑗 = 1 𝐵 =ڂ 𝑗∈𝐽 𝜊 𝑗 ⇒ σ 𝜊 𝑗 ∈𝐵 𝑞 𝜊 𝑗 = σ 𝑗∈𝐽 𝑞 𝑗 Equally likely events 𝑞 1 = 𝑞 2 = ⋯ = 𝑞 𝑜 𝑞 𝑗 = 1 𝑜 , 𝑗 = 1, 2, … , 𝑜 and 𝑄 𝐵 = 𝑜 𝐵 /𝑜
26 CONDITIONAL PROBABILITY CHAPTER 1.6
27 CONDITIONAL PROBABILITY Probability of an event A given event B has occurred 𝑄 𝐵∩𝐶 , 𝑄 𝐵 𝐶 = 𝑄 𝐶 > 0 𝑄 𝐶 𝑄(𝐵 ∩ 𝐶) – joint probability of A and B 𝑄 𝐶 𝐵 = 𝑄 𝐵∩𝐶 𝑄(𝐵) ⇒ 𝑄 𝐵 ∩ 𝐶 = 𝑄 𝐵 𝐶 𝑄 𝐶 = 𝑄 𝐶 𝐵 𝑄 𝐵
28 BAYES RULE Incredibly important relationship from joint probability 𝑄 𝐵 𝐶 = 𝑄 𝐶 𝐵 𝑄 𝐵 𝑄(𝐶) 𝑄 𝐵 𝐶 - posterior (what we want to estimate) 𝑄(𝐶|𝐵) – likelihood (probability of observing B given A) 𝑄(𝐵) – prior (probability of A with no extra information) 𝑄(𝐶) – marginal likelihood (total probability) Tells how to update our believes based on the arrival of new, relevant pieces of evidence
29 TOTAL PROBABILITY/INDEPENDENT EVENTS CHAPTER 1.7-1.8
30 TOTAL PROBABILITY Let B be an event in S 𝑜 𝑜 𝑄 𝐶 = σ 𝑗=1 𝑄 𝐶 ∩ 𝐵 𝑗 = σ 𝑗=1 𝑄 𝐶 𝐵 𝑗 𝑄(𝐵 𝑗 ) Exhaustive 𝑜 ڂ 𝑗=1 𝐵 𝑗 = 𝑇 Mutually exclusive 𝐵 𝑗 ∩ 𝐵 𝑘 = 𝜚 From Bayes 𝑄 𝐶 𝐵 𝑗 𝑄 𝐵 𝑗 𝑄 𝐵 𝑗 𝐶 = 𝑄 𝐶 𝐵 𝑗 𝑄(𝐵 𝑗 ) 𝑜 σ 𝑗=1
Recommend
More recommend
