Introduction to Dynamic Programming For use in Population - PowerPoint PPT Presentation

Introduction to Dynamic Programming For use in Population Applications J.R. Walker Department of Economics University of University of Wisconsin–Madison Economics of Population, Spring 2012 JRW DP

Life Cycle Decision Making Human Capital Formation: Amount and type of schooling, timing Household formation. Marriage, Cohabitation, Separation, Divorce Childbearing: Number of children, timing of 1st, spacing JRW DP

Life Cycle Decision Making Cont. Health: payoffs from healthy/risky behaviors Migration: Where to move? When to move? Employment, Savings, Retirement JRW DP

INVESTMENT DECISIONS May problems are investment decisions. Thus tools learned here, useful elsewhere JRW DP

Dynamic problems Typical problem: Life cycle consumption. For a given sequence of income by age a : y ( a ) Determine optimal consumption plan from today to (known) end of life. JRW DP

Symbolic representation V ( c 0 , c 1 , . . . , c T ) = c 0 , c 1 ,..., c T U ( c 0 , c 1 , . . . , c T ) max (1) subject to: y 0 , y 1 , . . . , y T given (2) JRW DP

Feasible Set Defining the feasible consumption sequences depends on structure of capital market. For example, if can borrow or lend at same interest (all periods) Then budget constraint is: A t + 1 = ( 1 + r ) A t + y t − c t , t = 0 , 1 , . . . , T , A 0 = a 0 JRW DP

DP simplification DP breaks this one T + 1–dimensional problem into T + 1 one dimensional problems We known how to solve one dimensional problems. JRW DP

Bellman Principle Two fundamental principles of life: Golden Rule: Do unto to others . . . Bellman’s Principle: Don’t worry how you got here, do the best you can. JRW DP

Backward Induction Example: Optimal Consumption Plan We will study ”finite horizon (lifetime) problems.” Last Period, T < ∞ Period T : enumerate all feasible situations (states, denoted x ). In consumption example, the states defined by asset levels A T (and age). Determine the best course of action in period T for each state, x . (max c T u ( c T ) , s.t. c T ≤ A T = ⇒ ˆ c T ( A T ) , Remember best action ˆ c T ( x ) and payoff V ( x ) = u (ˆ c T ( x )) . JRW DP

Backward Induction Continued Period T − 1: enumerate all feasible states x T − 1 . Determine best course of action in period T − 1 for each state using Bellman’s Principle. V T − 1 = max { u ( c T − 1 ) + β V T ( A T ) } c T − 1 ∈ C T − 1 c ∈ C feasible consumption; β discount factor. Perfect capital markets: A T = ( 1 + r ) A T − 1 + y T − 1 − c T − 1 JRW DP

Recursion Notice recursion. In period T we solved for the optimal consumption so V T ( A T ) is known. The budget constraint links assets in periods T − 1 , T by the consumption level of period T − 1. Thus, we have enough information to determine the best action (optimal consumption) in period T − 1. JRW DP

Continue back to current period Title of slide says it all. Continue these steps back to period 0. For a given initial asset level, A 0 = a 0 , and sequence of � T � incomes j = 0 , we have solved the dynamic problem y j ˆ c 0 , ˆ c 1 , . . . , ˆ c T . JRW DP

Original Meaning of DP In 1958 when Bellman labeled the solution procedure Dynamic Programming there were so few people writing code, the term programmer didn’t exist. Usage at that time, programming meant planning — Dynamic Planning. Planning indeed: Solved consumption for entire life (cradle to grave). JRW DP

Example 1: Capital Budgeting Our firm has $5 million to invest in mfg plants for expansion. Each plant has a few projects for on how it intends to the money. Each project gives the cost ( c ) and the tot revenue ( r expected Each will enact one project. JRW DP

Example 2: Shortest Path Picture of Network. JRW DP

Commonalities List of commonalities of examples 1 and 2. JRW DP