Utility Theory CMPUT 654: Modelling Human Strategic Behaviour - PowerPoint PPT Presentation


 Utility Theory CMPUT 654: Modelling Human Strategic Behaviour 
 S&LB §3.1

Recap: Course Essentials Course webpage: jrwright.info/bgtcourse/ Contacting me: • Discussion board: piazza.com/ualberta.ca/fall2019/cmput654/ 
 for public questions about assignments, lecture material, etc. • Email: james.wright@ualberta.ca 
 for private questions (health problems, inquiries about grades) • Office hours: After every lecture, or by appointment

Utility, informally A utility function is a real-valued function that indicates how much an agent prefers an outcome. Rational agents act to maximize their expected utility. Nontrivial claim: 1. Why should we believe that an agent's preferences can be adequately represented by a single number ? 2. Why should agents maximize expected value rather than some other criterion? Von-Neumann and Morgenstern's Theorem shows when these are true.

Outline 1. Informal statement 2. Theorem statement (von Neumann & Morgenstern) 3. Proof sketch 4. Fun game! 5. Representation theorem (Savage)

� � � Formal Setting: 
 Outcome Definition: Let � be a set of outcomes : O O = Z ∪ Δ ( O ) Not a typo! where � is some set of "actual outcomes", and Z represents the set of lotteries over finite subsets of � : Δ ( X ) X [ p 1 : x 1 , …, p k : x k ] k ∑ with � and � p j = 1 x j ∈ X ∀ 1 ≤ j ≤ k j =1

Formal Setting: Preference Relation A preference relation is a relationship between outcomes. Definition 
 For a specific preference relation � , write: ⪰ 1. � if the agent weakly prefers � to � , o 1 ⪰ o 2 o 1 o 2 2. � if the agent strictly prefers � to � , o 1 ≻ o 2 o 1 o 2 3. � if the agent is indifferent between � and � . o 1 ∼ o 2 o 1 o 2

Formal Setting Definition 
 A utility function is a function � . A utility function u : O → ℝ represents a preference relation � iff: ⪰ 1. � , and o 1 ⪰ o 2 ⟺ u ( o 1 ) ≥ u ( o 2 ) k ∑ 2. � . u ([ p 1 : o 1 , …, p k : o k ]) = p j u ( o j ) j =1

Representation Theorem Theorem: [von Neumann & Morgenstern, 1944] 
 satisfies the axioms Completeness , Suppose that a preference relation � ⪰ Transitivity , Monotonicity , Substitutability , Decomposability , and Continuity . Then there exists a function � such that u : O → ℝ 1. � , and o 1 ⪰ o 2 ⟺ u ( o 1 ) ≥ u ( o 2 ) k ∑ 2. � . u ([ p 1 : o 1 , …, p k : o k ]) = p j u ( o j ) j =1 That is, there exists a utility function that represents � . ⪰

� � Completeness and Transitivity Definition (Completeness): ∀ o 1 , o 2 : ( o 1 ≻ o 2 ) ∨ ( o 1 ≺ o 2 ) ∨ ( o 1 ∼ o 2 ) Definition (Transitivity): ∀ o 1 , o 2 : ( o 1 ⪰ o 2 ) ∧ ( o 2 ⪰ o 3 ) ⟹ o 1 ⪰ o 3

� Transitivity Justification: 
 � o 1 Money Pump ≻ ≻ 1 ¢ 1 ¢ � o 3 � o 2 1 ¢ ≻ • Suppose that � and � and � . ( o 1 ≻ o 2 ) ( o 2 ≻ o 3 ) ( o 3 ≻ o 1 ) • Starting from � , you are willing to pay 1 ¢ (say) to switch to � o 3 o 2 • But from � , you should be willing to pay 1 ¢ to switch to � o 2 o 1 • But from � , you should be willing to pay 1 ¢ to switch back to o 1 again... o 3

� Monotonicity Definition (Monotonicity): 
 If � and � , then o 1 ≻ o 2 p > q . [ p : o 1 , (1 − p ) : o 2 ] ≻ [ q : o 1 , (1 − q ) : o 2 ] You should prefer a 90% chance of getting $1000 to 
 a 50% chance of getting $1000.

� Substitutability Definition (Substitutability): 
 If � , then for all sequences � and � o 1 ∼ o 2 o 3 , …, o k p , p 3 , …, p k k ∑ with � p + p j = 1, j =3 [ p : o 1 , p 3 : o 3 , …, p k : o k ] ∼ [ p : o 2 , p 3 : o 3 , …, p k : o k ] If I like apples and bananas equally, then I should be indifferent between a 30% chance of getting an apple and a 30% chance of getting a banana.

� 
 Decomposability aka "No Fun in Gambling" Definition (Decomposability) : 
 Let � denote the probability that lottery � selects outcome � . P ℓ ( o ) ℓ o If � , then � . P ℓ 1 ( o j ) = P ℓ 2 ( o j ) ∀ o j ∈ O ℓ 1 ∼ ℓ 2 Example: 
 Let � ℓ 1 = [0.5 : [0.5 : o 1 , 0.5 : o 2 ], 0.5 : o 3 ] Let � ℓ 2 = [0.25 : o 1 , 0.25 : o 2 , 0.5 : o 3 ] Then � , because ℓ 1 ∼ ℓ 2 P ℓ 1 ( o 1 ) = 0.5 × 0.5 = 0.25 = P ℓ 2 ( o 1 ) P ℓ 1 ( o 2 ) = 0.5 × 0.5 = 0.25 = P ℓ 2 ( o 2 ) P ℓ 1 ( o 3 ) = 0.5 = P ℓ 2 ( o 3 )

� Continuity Definition (Continuity): If � , then � such that o 1 ≻ o 2 ≻ o 3 ∃ p ∈ [0,1] o 2 ∼ [ p : o 1 , (1 − p ) : o 3 ]

� Proof Sketch: 
 Construct the utility function 1. If � satisfies Completeness, Transitivity, Monotonicity, Decomposability, then for ⪰ every � , there exists some � such that: o 1 ≻ o 2 ≻ o 3 p (a) � , and o 2 ≻ [ q : o 1 , (1 − q ) : o 3 ] ∀ q < p (b) � . o 2 ≺ [ q : o 1 , (1 − q ) : o 3 ] ∀ q > p 2. If � additionally satisfies Continuity, then ⪰ . ∃ p : o 2 ∼ [ p : o 1 , (1 − p ) : o 3 ] o + ∈ O o − ∈ O o + o − Question: Are � and � 3. Choose maximal � and minimal � . guaranteed to exist? o ∼ [ p : o + , (1 − p ) : o − ] 4. Construct � such that � . u ( o ) = p

� Proof sketch: 
 Check the properties 1. � o 1 ⪰ o 2 ⟺ u ( o 1 ) ≥ u ( o 2 ) o ∼ [ p : o + , (1 − p ) : o − ] such that � . u ( o ) = p

� 
 Proof sketch: 
 Check the properties k ∑ 2. � u ([ p 1 : o 1 , …, p k : o k ]) = p j u ( o j ) j =1 (i) Let � u * = u ([ p 1 : o 1 , …, p k : o k ]) ℓ j = [ u ( o j ) : o + , (1 − u ( o j )) : o − ] (ii) Replace � with � , giving 
 o j [ p 1 : ℓ 1 , …, p k : ℓ k ] = [ p 1 : [ u ( o 1 ) : o + , (1 − u ( o 1 )) : o − ], …, p k : [ u ( o k ) : o + , (1 − u ( o k )) : o − ]] u ([ p 1 : ℓ 1 , …, p k : ℓ k ]) = u * (iii) Question: What is � ? u ([ p 1 : ℓ 1 , …, p k : ℓ k ]) k ( p j × u ( o j ) ) ∑ o + (iv) Question: What is the probability of getting � in � ? [ p 1 : ℓ 1 , …, p k : ℓ k ] j =1 k k k ( p j × u ( o j ) ) ( p j × u ( o j ) ) : o + , 1 − ( p i × u ( o j ) ) ∑ ∑ ∑ u ( ℓ *) = : o − (v) Construct � ℓ * = j =1 j =1 j =1 k ( p j × u ( o j ) ) ∑ ( why? ) 
 (vi) Observe that � [ p 1 : ℓ 1 , …, p k : ℓ k ] ∼ ℓ * u ([ p 1 : ℓ 1 , …, p k : ℓ k ]) = u * = u ( ℓ *) = ∎ j =1

� Caveats & Details Utility functions are not uniquely defined. ( Why? ) • Invariant to affine transformations (i.e., m > 0): 𝔽 [ u ( X )] ≥ 𝔽 [ u ( Y )] ⟺ X ⪰ Y ⟺ 𝔽 [ mu ( X ) + b ] ≥ 𝔽 [ mu ( Y ) + b ] ⟺ X ⪰ Y This means we're not stuck with a range of [0,1]!

� Caveats & Details The proof depended on minimal and maximal elements of � , but that is not critical. O Construction for unbounded outcomes/preferences: 1. Pick two outcomes � . Construct utility for all outcomes � : o s ≺ o e o s ⪯ o ⪯ o e u : { o ∈ O ∣ o s ⪯ o ⪯ o e } → [0,1] 2. For outcomes � outside that range, choose � . o ′ � o s ′ � ≺ o ′ � ≺ o s ≺ o e ≺ o e ′ � 3. Construct utility � . u ′ � : { o ∈ O ∣ o s ′ � ⪯ o ⪯ o e ′ � } → [0,1] 4. Find � and � such that � and � . b ∈ ℝ mu ′ � ( o s ) + b = u ( o s ) mu ′ � ( o e ) + b = u ( o e ) m > 0 5. Let � for all � . u ( o ) = mu ′ � ( o ) + b o ∈ { o ′ � ∈ O ∣ o s ′ � ⪯ o ′ � ⪯ o e ′ � }

Fun game: 
 Buying lottery tickets Write down the following numbers: 1. How much would you pay for the lottery 
 [0.3 : $5, 0.3 : $7, 0.4 : $9]? 2. How much would you pay for the lottery 
 [ p : $5, q : $7, (1 - p - q ) : $9]? 3. How much would you pay for the lottery 
 [ p : $5, q : $7, (1 - p - q ) : $9] 
 if you knew the last seven draws had been 5,5,7,5,9,9,5?

Beyond 
 von Neumann & Morgenstern • The first step of the fun game was a good match to the utility theory we just learned. • Question: If two agents have different prices for 
 [0.3 : $5, 0.3 : $7, 0.4 : $9], what does that say about their utility functions for money? • The second and third steps, not so much! • Question: If two agents have different prices for 
 [ p : $5, q : $7, (1 - p - q ) : $9], 
 what does that say about their utility functions ? • What if two people have the same prices for step 2 but different prices once they hear what the last few draws were?

Another Formal Setting • States : Set � of elements � with subsets � S s , s ′ � , … A , B , C , … • Consequences : Set � of elements � F f , g , h , … • Acts : Arbitrary functions � f : S → F • Preference relation between acts ⪰ ( f ⪰ g given B ) ⟺ • 
 f ′ � ⪰ g ′ � for every f ′ � , g ′ � that agree with f , g respectively on B and each other on B

� Another 
 Representation Theorem Theorem: [Savage, 1954] 
 Suppose that a preference relation satisfies postulates P1-P6. ⪰ Then there exists a utility function � and a probability measure � U P such that f ⪰ g ⟺ ∑ P [ B i ] U [ f i ] ≥ ∑ . P [ B i ] U [ g i ] i i