ECE 4524 Artificial Intelligence and Engineering Applications Lecture 21: Decisions and Utility Reading: AIAMA 16.1-16.3 Today’s Schedule: ◮ Introduction to Decision Theory - Maximum Expected Utility ◮ Utility Functions ◮ Examples
Introduction to Decision Theory ◮ A decision is a choice among alternatives. For stochastic environments there is a probability (belief) of that alternative. ◮ If we consider our agent model as a discrete Bayesian Network, the probability of the resulting state after applying action a given the action and evidence, e , is s ′ P (result( a ) = s ′ | a , e ) ◮ To choose an action we might just argmax P (result( a ) = s ′ | a , e ) a but that assumes all outcomes are equally desirable. ◮ Before we used a function to order the desirability of results, how do we do this including the probability?
Maximum Expected Utility ◮ Define the utility of a state s ′ as a function U ( s ′ ) ∈ R ◮ Written as a function of the action requires we weight the utility of an outcome by the probability of its occurrence U ( a | e ) = P (result( a ) = s ′ | a , e ) U ( s ′ ) ◮ To choose an action we select the action with the highest expected utility, EU ( a | e ) best action = argmax EU ( a | e ) a
Maximum Expected Utility ◮ For discrete state spaces � EU ( a | e ) = P (result( a ) = s ′ | a , e ) U ( s ′ ) s ′ ◮ For continuous state spaces � EU ( a | e ) = f (result( a ) = s ′ | a , e ) U ( s ′ ) ds ′ This is used in Optimal Control where result is the solution to a differential equation. It’s integral over time is called the ”cost to go”.
Example: Lottery tickets Do you buy lottery tickets ◮ never ◮ rarely ◮ often ◮ whenever I have the money
Another Example Recall our test for a disease P ( D | T ) = P ( T | D ) P ( D ) P ( T ) ◮ Consider the decision to take the test with no evidence (screening) ◮ What is the state? outcomes?
Why AI is hard So why is this course not just maximum expected utility? ◮ computing probabilities in general is #-Hard ◮ building models of the world is hard ◮ utility is subjective ◮ you might have to explore the state space to evaluate utility
Utility Functions What constitutes a valid utility function? ◮ Define a preference as an ordering among outcomes, denoted A ≻ B A ∼ B ◮ Define a lottery for outcomes S i with probability p i as L = [( p 1 , S 1 ); ( p 2 , S 2 ); · · · ( p n , S n ); ]
The axioms of utility theory ◮ orderability ◮ transitivity ◮ continuity ◮ substitutability ◮ monotonicity ◮ Decomposibility, aka no fun in gambling These axioms lead to a numerical relationship among utility as equivalent to preference.
Economic models of utility ◮ risk averse v/s risk seeking ◮ optimizer’s curse
Next Actions ◮ Reading on Learning (AIAMA 18.1-18.3) ◮ No warmup Note: This concludes part three of the course. Remember PS 3 is due 4/5. Quiz 3 will be Thursday 4/12.
Recommend
More recommend
