Introduction Mathematics for Informatics 4a Jos´ e Figueroa-O’Farrill Topic : Probability and random processes Lectures : AT3 on Wednesday and Friday at 12:10pm Office hours : by appointment (email) at JCMB 6321 Email : j.m.figueroa@ed.ac.uk Lecture 1 18 January 2012 Jos´ e Figueroa-O’Farrill mi4a (Probability) Lecture 1 1 / 23 Jos´ e Figueroa-O’Farrill mi4a (Probability) Lecture 1 2 / 23 The future was uncertain The future is still uncertain Jos´ e Figueroa-O’Farrill mi4a (Probability) Lecture 1 3 / 23 Jos´ e Figueroa-O’Farrill mi4a (Probability) Lecture 1 4 / 23
The universe is fundamentally uncertain! There is no god of Algebra, but... there are gods of probability Fu Lu Shou Shichi Fukujin Lakshmi Tyche Fortuna Jos´ e Figueroa-O’Farrill mi4a (Probability) Lecture 1 5 / 23 Jos´ e Figueroa-O’Farrill mi4a (Probability) Lecture 1 6 / 23 The mathematical study of Probability What it is all about Some notable names Mathematical probability aims to formalise everyday sentences of the type: Gerolamo Cardano (1501-1576) Pierre de Fermat (1601-1665) “The chance of A is p ” Blaise Pascal (1623-1662) where A is some “event” and p is some “measure” of the Christiaan Huygens (1629-1695) likelihood of occurrence of that event. Jakob Bernoulli (1654-1705) Example Abraham de Moivre (1667-1754) “There is a 20% chance of snow.” Thomas Bayes (1702-1761) “There is 5% chance that the West Antarctic Ice Sheet will Pierre-Simon Laplace (1749-1827) collapse in the next 200 years.” Adrien-Marie Legendre (1752-1833) “There is a low probability of Northern Rock having a liquidity problem.” Andrei Markov (1866-1922) Andrei Kolmogorov (1903-1987) Claude Shannon (1916-2001) Jos´ e Figueroa-O’Farrill mi4a (Probability) Lecture 1 7 / 23 Jos´ e Figueroa-O’Farrill mi4a (Probability) Lecture 1 8 / 23
Trials and outcomes “Events are what we assign a probability to” Definition This requires introducing some language. An event A is a subset of Ω . We say that an event A has (not) Definition/Notation occurred if the outcome of the trial is (not) contained in A . By a trial (or an experiment ) we mean any process which has a well-defined set Ω of outcomes . Ω is called the sample Example space . Tossing a coin and getting a head: A = { H } . Tossing two coins and getting at least one head: Example A = { ( H , H ) , ( H , T ) , ( T , H ) } . Tossing a coin: Ω = { H , T } . Rolling a die and getting an even number: A = { , , } Tossing two coins: Ω = { ( H , H ) , ( H , T ) , ( T , H ) , ( T , T ) } . Rolling two dice and getting a total of 5: , ) , ( , ) , ( , ) , ( , A = { ( ) } Example (Rolling a (6-sided) die) Warning (for infinite Ω ) , , , , , } . Ω = { Not all subsets of Ω need be events! Jos´ e Figueroa-O’Farrill mi4a (Probability) Lecture 1 9 / 23 Jos´ e Figueroa-O’Farrill mi4a (Probability) Lecture 1 10 / 23 The language of sets Definition Let us consider subsets of a set Ω . The union of A and B is denoted A ∪ B : Definition The complement of A ⊂ Ω is denoted A c ⊂ Ω : or or both ω ∈ A ∪ B ⇐ ⇒ ω ∈ A ω ∈ B ω ∈ A c ⇐ ⇒ ω �∈ A Clearly ( A c ) c = A . Ω Ω A B Example (The empty set) The complement of Ω is the Remark A c empty set ∅ : A For all subsets A of Ω , A ∪ B ω �∈ ∅ ∀ ω ∈ Ω A ∪ ∅ = A and A ∪ Ω = Ω . Jos´ e Figueroa-O’Farrill mi4a (Probability) Lecture 1 11 / 23 Jos´ e Figueroa-O’Farrill mi4a (Probability) Lecture 1 12 / 23
Distributivity identities Definition The intersection of A and B is denoted A ∩ B : Union and intersection obey distributive properties. and ω ∈ A ∩ B ⇐ ⇒ ω ∈ A ω ∈ B Theorem If A ∩ B = ∅ we say A and B are disjoint . Let ( A i ) i ∈ I be a family of subsets of Ω indexed by some index set I and let B ⊂ Ω . Then Ω � � ( B ∩ A i ) = B ∩ A i A B i ∈ I i ∈ I and � � ( B ∪ A i ) = B ∪ A i i ∈ I i ∈ I Remark For all subsets A of Ω , A ∩ B A ∩ ∅ = ∅ and A ∩ Ω = A . Jos´ e Figueroa-O’Farrill mi4a (Probability) Lecture 1 13 / 23 Jos´ e Figueroa-O’Farrill mi4a (Probability) Lecture 1 14 / 23 De Morgan’s Theorem Union and intersection are “dual” under complementation. Proof. Theorem (De Morgan’s) Let ( A i ) i ∈ I be a family of subsets of Ω indexed by some index � ω ∈ ( B ∪ A i ) ⇐ ⇒ ∀ i ∈ I , ω ∈ B ∪ A i set I . Then i ∈ I � c � c ⇒ ∀ i ∈ I , or �� �� ⇐ ω ∈ B ω ∈ A i � � A c A c = and = A i A i or i i ⇐ ⇒ ω ∈ B ω ∈ A i ∀ i ∈ I i ∈ I i ∈ I i ∈ I i ∈ I � or ⇐ ⇒ ω ∈ B ω ∈ A i i ∈ I � Remark A i . ⇐ ⇒ ω ∈ B ∪ This shows that complementation together with either union or i ∈ I interesection is enough, since, e.g., The other equality is proved similarly. A ∪ B = ( A c ∩ B c ) c Jos´ e Figueroa-O’Farrill mi4a (Probability) Lecture 1 15 / 23 Jos´ e Figueroa-O’Farrill mi4a (Probability) Lecture 1 16 / 23
Definition The difference A \ B = A ∩ B c and the symmetric difference Proof. A △ B = ( A \ B ) ∪ ( B \ A ) . Ω Ω � c �� � ω ∈ A i ⇐ ⇒ ω �∈ A i i ∈ I i ∈ I A B A B ⇐ ⇒ ω �∈ A i ∀ i ∈ I ⇒ ω ∈ A c ⇐ ∀ i ∈ I i � A c i . ⇐ ⇒ ω ∈ i ∈ I A \ B A △ B The other equality if proved similarly. Remark Notice that A \ B = A \ ( A ∩ B ) . Jos´ e Figueroa-O’Farrill mi4a (Probability) Lecture 1 17 / 23 Jos´ e Figueroa-O’Farrill mi4a (Probability) Lecture 1 18 / 23 Probability/Set theory dictionary Which subsets can be events? For finite Ω , any subset can be an event. Notation Set-theoretic language Probabilistic language For infinite Ω , it is not always sensible to allow all subsets Universe Sample space Ω to be events. (Trust me!) member of Ω outcome ω ∈ Ω If A is an event, it seems reasonable that A c is also an subset of Ω some outcome in A occurs A ⊂ Ω A c complement of A no outcome in A occurs event. intersection Both A and B A ∩ B Similarly, if A and B are events, it seems reasonable that A ∪ B union Either A or B (or both) A ∪ B and A ∩ B should also be events. difference A , but not B A \ B In summary, the collection of events must be closed under symmetric difference Either A or B , but not both A △ B complementation and pairwise union and intersection. By empty set impossible event ∅ induction, it must also be closed under finite union and whole universe certain event Ω intersection: if A 1 , . . . , A N are events, so should be A 1 ∩ A 2 ∩ · · · ∩ A N and A 1 ∪ A 2 ∪ · · · ∪ A N . Jos´ e Figueroa-O’Farrill mi4a (Probability) Lecture 1 19 / 23 Jos´ e Figueroa-O’Farrill mi4a (Probability) Lecture 1 20 / 23
σ -fields The following example, shows that this is not enough. Definition Example A family F of subsets of Ω is a σ -field if Alice and Bob play a game in which they toss a coin in turn. Ω ∈ F 1 The winner is the first person to obtain H . Intuition says that the the person who plays first has an advantage. We would like to if A 1 , A 2 , . . . ∈ F , then � ∞ i = 1 A i ∈ F 2 quantify this intuition. Suppose Alice goes first. She wins if and if A ∈ F , then A c ∈ F 3 only if the first H turns out after an odd number of tosses. Let ω i be the outcome TT · · · T H . Then the event that Alice wins is Remark � �� � i − 1 It follows from De Morgan’s theorem that for a σ -field F , if A = { ω 1 , ω 3 , ω 5 , . . . } , which is a disjoint union of a countably i = 1 A i ∈ F . Also Ω ∈ F , since Ω = ∅ c . A 1 , A 2 , · · · ∈ F then � ∞ infinite number of events. In order to compute the likelihood of Finally, a σ -field is closed under (symmetric) difference. Alice winning, it had better be the case that A is an event, so one demands that the family of events be closed under Example countably infinite unions; that is, if A i , for i = 1, 2, . . . , are events, then so is � ∞ i = 1 A i . The smallest σ -field is F = { ∅ , Ω } . The largest is the power set of Ω (i.e., the collection of all subsets of Ω ). Jos´ e Figueroa-O’Farrill mi4a (Probability) Lecture 1 21 / 23 Jos´ e Figueroa-O’Farrill mi4a (Probability) Lecture 1 22 / 23 Summary With any experiment or trial we associate a pair ( Ω , F ) , where Ω , the sample space , is the set of all possible outcomes of the experiment; and F , the family of events , is a σ -field of subsets of Ω : a family of subset of Ω containing the empty set and closed under complementation and countable unions. In the next lecture we will see how to enhance the pair ( Ω , F ) to a “probability space” by assigning a measure of likelihood (i.e., a “probability”) to the events in F . There is a dictionary between the languages of set theory and of probability. In particular, we will use the set-theoretic language freely. Jos´ e Figueroa-O’Farrill mi4a (Probability) Lecture 1 23 / 23
Recommend
More recommend
