Introduction to Artificial Intelligence Planning under Uncertainty Janyl Jumadinova November 2, 2016
Goals and Preferences 2/17
Preferences ◮ Actions result in outcomes . ◮ Agents have preferences over outcomes. 3/17
Preferences ◮ Actions result in outcomes . ◮ Agents have preferences over outcomes. A rational agent will do the action that has the best outcome for them 3/17
Preferences ◮ Actions result in outcomes . ◮ Agents have preferences over outcomes. A rational agent will do the action that has the best outcome for them ◮ Sometimes agents don’t know the outcomes of the actions, but they still need to compare actions ◮ Agents have to act. (Doing nothing is (often) an action) 3/17
Preferences over Outcomes 4/17
Lotteries ◮ An agent may not know the outcomes of their actions, but only have a probability distribution of the outcomes. ◮ A lottery is a probability distribution over outcomes. [ p 1 : o 1 , p 2 : o 2 , ..., p k : o k ], where o i are outcomes and p i ≥ 0 s.t. � i p i = 1 ◮ The lottery specifies that outcome o i occurs with probability p i . 5/17
Measure of Preference ◮ We would like a measure of preference that can be combined with probabilities: value ([ p : o 1 , 1 − p : o 2 ]) = p × value ( o 1 ) + (1 − p ) × value ( o 2 ) 6/17
Measure of Preference ◮ We would like a measure of preference that can be combined with probabilities: value ([ p : o 1 , 1 − p : o 2 ]) = p × value ( o 1 ) + (1 − p ) × value ( o 2 ) ◮ Money does not act this way: $1 , 000 , 000 or [0 . 5 : $1 , 0 . 5 : 2 , 000 , 000]? 6/17
Theorem ◮ If preferences follow the preceding properties, then preferences can be measured by a function: utility : outcomes → [0 , 1] such that 7/17
Utility as a function of money 8/17
Additive Utility ◮ Suppose the outcomes can be described in terms of features X 1 , ..., X n . ◮ An additive utility is one that can be decomposed into set of factors: u ( X 1 , ..., X n ) = f 1 ( X 1 ) + ... + f n ( X n ). 9/17
Additive Utility ◮ Suppose the outcomes can be described in terms of features X 1 , ..., X n . ◮ An additive utility is one that can be decomposed into set of factors: u ( X 1 , ..., X n ) = f 1 ( X 1 ) + ... + f n ( X n ). ◮ This assumes additive independence . ◮ Strong assumption : contribution of each feature doesnt depend on other features. 9/17
Additive Utility ◮ An additive utility has a canonical representation: u ( X 1 , ..., X n ) = w 1 × u 1 ( X 1 ) + ... + w n u n ( X n ). 10/17
Additive Utility ◮ An additive utility has a canonical representation: u ( X 1 , ..., X n ) = w 1 × u 1 ( X 1 ) + ... + w n u n ( X n ). ◮ If best i is the best value of X i , u i ( X i = best i ) = 1. ◮ If worst i is the worst value of X i , u i ( X i = worst i ) = 0. 10/17
Additive Utility ◮ An additive utility has a canonical representation: u ( X 1 , ..., X n ) = w 1 × u 1 ( X 1 ) + ... + w n u n ( X n ). ◮ If best i is the best value of X i , u i ( X i = best i ) = 1. ◮ If worst i is the worst value of X i , u i ( X i = worst i ) = 0. ◮ w i are weights, � i w i = 1. ◮ The weights reflect the relative importance of features. We can determine weights by comparing outcomes. 10/17
Utility and Time ◮ Would you prefer $1000 today or $1000 next year? 11/17
Utility and Time ◮ Would you prefer $1000 today or $1000 next year? ◮ What price would you pay now to have an eternity of happiness? 11/17
Utility and Time ◮ Would you prefer $1000 today or $1000 next year? ◮ What price would you pay now to have an eternity of happiness? ◮ How can you trade off pleasures today with pleasures in the future? 11/17
Utility and Time 12/17
Rewards and Values 13/17
Rewards and Values 14/17
Framing Effects 15/17
Framing Effects 16/17
Framing Effects 17/17
Recommend
More recommend
