Probability & Stochastic Processes Introduction to Probability Theory Sample Spaces Event Spaces Probability Measure Probability Functions Random Variables Moments of Random Variables Introduction to Stochastic Processes Dr Conor McArdle EE414 - Probability & Stochastic Processes 1/60
Introduction to Probability Theory Probability theory is concerned with the description and calculation of the properties of random phenomena, as occur in games of chance, computer and telecommunications systems, financial markets, electronic and optical circuits and many other random systems. Although such systems are random, in the sense that it is difficult or impossible to predict exactly how the system will behave in the future, probability theory can provide characterisation of the type of randomness involved and yield useful measures, such as average values of system parameters or the likelihood of certain events occurring in the future. To develop a rigorous mathematical theory of probability, the starting point is the notion of a random experiment and an abstract probability space . A random experiment E is an experiment satisfying the following conditions: all possible distinct outcomes are known a priori the outcome is not known a priori for any particular trial of the experiment the experiment is repeatable under identical conditions Dr Conor McArdle EE414 - Probability & Stochastic Processes 2/60
Introduction to Probability Theory Many random phenomena can be modelled by the notion of a random experiment, for example: Recording the output voltage of a noise generator Observing the daily closing price of crude oil Measuring the number of packets queueing at the input port of a network router Each different random experiment E defines a its own particular sample space , event space and probability measure , which collectively form an abstract probability space for the random experiment. A probability space is the collection ( Ω , F , P ) where Ω the sample space is the set of all possible outcomes of a random experiment E F the event space is a collection of events , where each event is a subset of the sample space and the collection forms a σ -field P the probability measure is an assignment of a real number in the interval [0,1] to each event in the event space. Dr Conor McArdle EE414 - Probability & Stochastic Processes 3/60
Introduction to Probability Theory Example : Random experiment of tossing a fair coin Sample Space Ω = { H, T } , Event Space F = {{ H } , { T } , { H, T } , {}} Probability Measure P defined by P ( { H } ) = 1 2 , P ( { T } ) = 1 2 , P ( { H, T } ) = 1 , P ( {} ) = 0 Considering this example where the sample space is discrete ( countable ), it may appear unnecessary to define events to which probabilities are assigned. Why not simply assign probabilities directly to outcomes in the sample space? Consider instead an experiment where a random selection of a real number between 0 and 10 is made (an uncountable sample space), then the probability of any particular outcome must be zero since there is an infinity of such outcomes in the sample space. However, if events are defined as intervals of the real line (e.g. [0,5]), the events can have non-zero probability values (e.g. the probability of an outcome occurring within the interval [0,5] will be non-zero). Dr Conor McArdle EE414 - Probability & Stochastic Processes 4/60
Introduction to Probability Theory So that we can form a useful theory for all random experiments (particularly those with uncountable sample spaces), the probability measure is only defined on specified subsets of the sample space (the events) rather than on individual outcomes in the sample space. Note that this stipulation does not preclude us from defining events consisting of a single outcome, but we draw the distinction between an outcome ω ∈ Ω (an element of Ω ) and an event { ω } ⊂ Ω (a subset of Ω ). The definition of the event space as a σ -field further specifies which subsets of Ω can belong to the same event space. That is, there is a certain relationship between the subsets of the sample space Ω that are chosen as events in the event space. The properties of a σ -field (and so of any event space) ensure that if events A and B have probabilities defined then logical combinations of these events (e.g. the outcome is in either A or B ) are also events in the event space and so also have probabilities defined. Any subset of Ω that does not belong to the event space of a random experiment will simply not have a defined probability. We next look at the sample space, event space and probability measure in some detail. Dr Conor McArdle EE414 - Probability & Stochastic Processes 5/60
Probability & Stochastic Processes Introduction to Probability Theory Sample Spaces Event Spaces Probability Measure Probability Functions Random Variables Moments of Random Variables Introduction to Stochastic Processes Dr Conor McArdle EE414 - Probability & Stochastic Processes 6/60
Sample Spaces A sample space Ω is the non-empty set of all outcomes (also known as sample points , elementary outcomes or elementary events ) of a random experiment E . The sample space takes different forms depending on the random experiment in question. We have seen an example of a finite sample space { H, T } , in the case of the coin tossing random experiment, and also an uncountable sample space (a interval of the real line [0 , 10] ) in the case of the random number experiment. What follows are some examples of more general sample spaces: Example 1 A finite sample space Ω = { a k : k = 1 , 2 , ..., K } . Specific examples are: A binary space { 0 , 1 } A finite space of integers { 0 , 1 , 2 , ..., k − 1 } . (Also denoted Z k ). Dr Conor McArdle EE414 - Probability & Stochastic Processes 7/60
Sample Spaces Example 2 A countably infinite space Ω = { a k : k = 1 , 2 , ... } . Specific examples are: All non-negative integers { 0 , 1 , 2 , ... } , denoted Z + All integers { ..., − 2 , − 1 , 0 , 1 , 2 , ... } , denoted Z Example 3 An uncountably infinite space. Examples are the real line R or intervals of R such as ( a, b ) , [ a, b ) , ( a, b ] , [ a, ∞ ) , ( −∞ , ∞ ) . Example 4 A space consisting of k -dimensional vectors with coordinates taking values in one of the previously described spaces. The usual name for such a vector space is a product space . For example, let A denote one of the abstract spaces previously considered. Define the cartesian product A k as: A k = { ( a o , a 1 , ..., a k − 1 ) : a i ∈ A } Dr Conor McArdle EE414 - Probability & Stochastic Processes 8/60
Sample Spaces Specific examples of this type of space are: R k { 0 , 1 } k [ a, b ] k Example 5 Let A be one of the sample spaces in examples 1-3. Form a new sample space consisting of all waveforms (or functions of time) with values in A (e.g. all real valued time functions). This space is a product space of infinite dimension. For example: A t = { all waveforms { x ( t ) : t ∈ [0 , ∞ ) } : x ( t ) ∈ A, ∀ t } Exercise 1 Specify appropriate sample spaces that model the outcomes of the following ran- dom systems: (i) tossing a coin where a head is assigned a value of 1 and a tail a value of 0 (ii) rolling a die (iii) rolling three dice simultaneously (iv) choosing a random coordinate within a cube (v) an infinite random binary waveform. Dr Conor McArdle EE414 - Probability & Stochastic Processes 9/60
Probability & Stochastic Processes Introduction to Probability Theory Sample Spaces Event Spaces Probability Measure Probability Functions Random Variables Moments of Random Variables Introduction to Stochastic Processes Dr Conor McArdle EE414 - Probability & Stochastic Processes 10/60
Event Spaces The event space F of a sample space Ω is a non-empty collection of subsets of Ω , which has the following properties: 1 If F ∈ F then also F c ∈ F n � 2 If for some finite n , F i ∈ F , i = 1 , 2 , ..., n then also F i ∈ F i =1 ∞ � 3 If F i ∈ F , i = 1 , 2 , ... then also F i ∈ F i =1 These properties specify that an event space is a σ -field (or σ -algebra ) over Ω . Note that the definition of the σ -field, as above, specifies only that the collection be closed under complementation and countable unions. However, these requirements immediately yield additional closure properties. The countably infinite version of De Morgans’s ’Laws’ of elementary set theory require that if F i , i = 1 , 2 , ... are all members of a σ -field then so is: � ∞ ∞ � c � � c F i = F i i =1 i =1 Dr Conor McArdle EE414 - Probability & Stochastic Processes 11/60
Recommend
More recommend
