Lecture: Continuous Time Models with Investment Applications Simon - PowerPoint PPT Presentation

Lecture: Continuous Time Models with Investment Applications Simon Gilchrist Boston Univerity and NBER EC 745 Fall, 2013

Brownian Motion Brownian motion (Wiener process): Continous time stochastic process with three properties: Markov process: probability distribution for all future values depends only on its current value. Independent increments: probability distribution for the change in the process is independent of any other non-overlapping time interval. Changes in the process over any finite interval are normally distributed witha variance that increases linearly with the time interval.

Formal Definition If z(t) is a wiener process then any change in z, ∆ z corresponding to a time interval ∆ t satisfies the following conditions: √ ∆ z = ε t ∆ t ε t ∼ N (0 , 1) E ( ε t ε t + s ) = 0 for t � = s Intution: Change in z ( t ) over a finite interval T . Divide T into n = T/ ∆ t : n � √ ∆ z = z ( s + T ) − z ( s ) = ε i ∆ t i =1 E (∆ z ) = 0 V (∆ z ) = n ∆ t = T

Brownian motion with drift Brownian motion with drift: dx = αdt + σdz where dz is a Wiener process. Over any finite interval ∆ t, ∆ x is distributed normal with E (∆ x ) = α ∆ t, V (∆ x ) = σ 2 ∆ t.

Random walk representation of Brownian motion: Show that dx is the limit of a discrete time random walk with drift. Suppose ∆ x = ∆ h with prob p = − ∆ h with prob q = 1 − p then E ∆ x = ( p − q )∆ h and E (∆ x 2 ) − E (∆ x ) 2 V ( x ) = (1 − ( p − q ) 2 )∆ h 2 = 4 pq ∆ h 2 =

Binomial distribution Let a time interval t have n = t/ ∆ t discrete steps, then x t − x o is a serise of n independent trials with ∆ h a success occurring with prob p and − ∆ h a failure, occurring with prob (1 − p ) = q . So x t − x o has a binomial distribution with: E ( x t − x o ) = n ( p − q )∆ h = t ( p − q )∆ h/ ∆ t and V ( x t − x o ) = n ∗ ((1 − ( p − q ) 2 )∆ h 2 = 4 pqt ∆ h 2 / ∆ t

Random walk representation of Brownian motion: Choose ∆ h, p, q so that mean and variance of x t − x o depends only on t and not on step-size ∆ t or jump ∆ h : √ ∆ h = σ ∆ t � � � � √ √ p = 1 1 + α //q = 1 1 − α ∆ t ∆ t 2 σ 2 σ then √ p − q = α ∆ t = α σ 2 ∆ h σ This implies 2 E ( x t − x o ) = t α ∆ h = αt σ 2 ∆ t and � � � α � 2 σ 2 V ( x t − x o ) = t 1 − ∆ t σ so ∆ t → 0 V ( x t − x o ) = tσ 2 . lim

Comments: Brownian motion is limit of discrete time random walk where mean and variance are independent of step-size ∆ t and jump ∆ h . This limiting process has the property that variance grows linearly per unit of time. For any finite interval, total distance travelled is infinite as ∆ t → 0 : | ∆ x | = ∆ h so E | ∆ x | = ∆ h and nE | ∆ x | = t ∆ h tσ ∆ t = √ → ∞ ∆ t Brownian motion is not differentiable in the conventional sense: � � � � � � � � ∆ x ∆ h � � � � � = � → ∞ � � ∆ t ∆ t so dx/dt does not exist and we cannot compute E ( dx/dt ) . We � 1 � can compute E ( dx ) and E ( dx ) however. dt

Ito Processes Generalize brownian motion (Ito processes): dx = a ( x, t ) dt + b ( x, t ) dz where dz = wiener process and a ( x, t ) , b ( x, t ) are non-random function of state. E ( dx ) = a ( x, t ) dt so a ( x, t ) is instantaneous rate of drift. Instantaneous variance: E ( dx 2 ) − E ( dx ) 2 V ( dx ) = a ( x, t ) 2 dt 2 + 2 E (( a ( x, t ) b ( x, t ) dtdz ) + b ( x, t ) 2 var ( dz ) = The first two terms are of order dt 2 and dt 3 / 2 so that b ( x, t ) 2 var ( dz ) V ( dx ) = b ( x, t ) 2 dt =

Example 1: Geometric brownian motion: Let dx = αxdt + σxdz If x is a geometric brownian motion then F ( x ) = ln( x ) is brownian motion with drift: dF = ( α − 0 . 5 σ 2 ) dt + σdz This implies that �� α − 0 . 5 σ 2 � � t, σ 2 t ln( x t /x o ) ∼ N Using properties of the log-normal we have x o e αt E ( x t ) = o e 2 αt ( e σ 2 t − 1) x 2 V ( x t ) =

Present values Also we have the present value expression: � ∞ � ∞ x ( t ) e − rt dt x o e − ( r − t ) dt E = o o x o = r − α The drift rate α can be interpreted as the dividend growth rate.

Example 2: Continous time AR1 (Ornstein-Uhlenbeck) Let dx = η ( µ − x ) dt + σdz Then E ( x t ) = µ + ( x o − µ ) e − ηt → µ as t → ∞ V ( x t ) = σ 2 2 η (1 − e − 2 ηt ) → σ 2 2 η as t → ∞ If η → ∞ x is a constant. Need to adjust both σ , η to vary the degree of mean reversion.

Ito’s Lemma: Ito process is continuous but not differentiable. What about functions of x, F ( x, t ) where dx = a ( x, t ) dt + b ( x, t ) dz Consider taylor-series expansion of F ( x, t ) (ignore higher order derivatives of t ): ∂ 2 F ∂ 3 F dF = ∂F ∂x dt + ∂F ∂t dt + 1 ∂x 2 dx 2 + 1 ∂x 3 dx 3 + .. 2 6 We want all terms of order dt : dx is of order dt ( dx ) 2 = b ( x, t ) 2 dt + higher order terms

Ito’s Lemma This implies ∂ 2 F ∂F ∂x dt + ∂F ∂t dt + 1 ∂x 2 dx 2 dF = 2 � ∂F � ∂F � � ∂ 2 F �� + 1 2 b ( x, t ) 2 dF = ∂t + a ( x, t ) dt ∂x 2 ∂x � ∂F � + b ( x, t ) dz ∂x Taking expectations we have: � ∂F � ∂F � � ∂ 2 F �� + 1 2 b ( x, t ) 2 E ( dF ) = ∂t + a ( x, t ) dt ∂x 2 ∂x 2 b ( x, t ) 2 � � ∂ 2 F Because of uncertainty the term 1 is of first-order. ∂x 2 I.e. owing to Jensen’s inequality, if the function is concave at x uncertainty lowers the value of dF .

Example: Geometric brownian motion Let dx = αxdt + σxdz Let F ( x ) = ln( x ) � � ∂F � � ∂ 2 F �� � ∂F � + 1 2 b ( x, t ) 2 dF = a ( x ) dt + b ( x ) dz ∂x 2 ∂x ∂x � � − 1 �� α + 1 dt + σx 1 2 σ 2 x 2 = xdz x 2 � � α − 1 2 σ 2 = dt + σdz The log of geometric brownian motion is a brownian motion with drift.

Dynamic programming in continuous time: Start with discrete time problem: � � 1 F ( x, t ) = max u { π ( x, u, t )∆ t + 1 + ρ ∆ tE F ( x ′ , t + ∆ t ) | x, u where π ( x, u, t ) is flow profit given state x and policy u. Rearrange to get � � F ( x ′ , t + ∆ t ) − F ( x, t ) | x, u ρ ∆ tF ( x, t ) = max u { π ( x, u, t )∆ t + E Divide by ∆ t and take limit as ∆ t → 0 � � u { π ( x, u, t ) + 1 F ′ ( x ′ , t ) | x, u ρF ( x, t ) = max dtE }

1 Suppose x follows an Ito process: dx = a ( x, u, t ) dt + b ( x, u, t ) dz then up to o (∆ t ) � � F ( x ′ , t + ∆ t ) − F ( x, t ) | x, u E = [ F t ( x, t ) + a ( x, u, t ) F x ( x, t ) 1 2 b 2 ( x, u, t ) F xx ( x, t )]∆ t + We now have that the return equation satisifies: ρF ( x, t ) = max u { π ( x, u, t ) + F t ( x, t ) + a ( x, u, t ) F x ( x, t ) 1 2 b 2 ( x, u, t ) F xx ( x, t ) } +

Hamilton-Jacobi-Bellman equation If there is an infinite horizon and a () and b () don’t depend explicitly on time then the value F ( x ) satisfies the ordinary differential equation: u { π ( x, u ) + a ( x, u ) F ′ ( x ) + 1 2 b 2 ( x, u ) F ′′ ( x ) } ρF ( x ) = max This is the continuous time equivalent of the Bellman equation.

Optimal stopping problem: Discrete time Let π ( x ) denote flow profit of a machine. Let Ω( x ) denote the terminal payoff. ρ Assume that π ( x ) − 1+ ρ Ω( x ) is increasing in x. Assume that Φ( x ′ | x ) , the distribution function is first-order stochastic dominant (i.e. an increase in x shifts the probability distribution of x ′ to the right) – example AR1, Random walk.

Optimal policy Value function: � �� � 1 F ( x ′ | x F ( x ) = max Ω( x ); π ( x ) + 1 + ρE Solution: stop if x < x ∗ for some value x ∗ to be determined.

Optimal stopping problem in continuous time Assume Ito process for x : dx = a ( x ) dt + b ( x ) dz Profit relative to flow value of terminal payoff π ( x ) − ρ Ω( x ) is increasing in x . Return function: � � 1 F ( x ) = max Ω( x ); π ( x ) + 1 + ρdtE [ F ( x + dx | x ] Solution: stop if x < x ∗ for some value x ∗ to be determined.

Value on continuation region If x > x ∗ the return function satisfies ρF ( x ) = π ( x ) + 1 dtE ( dF ) which implies: ρF ( x ) = π ( x ) + a ( x ) F ′ ( x ) + 1 2 b 2 ( x ) F ′′ ( x ) for x > x ∗ Because x ∗ is endogenous, we need two boundary conditions to solve this differential equation.

Optimality conditions Value matching: F ( x ∗ ) = Ω( x ∗ ) Smooth pasting: F x ( x ∗ ) = Ω x ( x ∗ ) Suppose Ω( x ) = 0 . At boundary: 0 = π ( x ∗ ) + 1 2 b 2 ( x ∗ ) F ′′ ( x ∗ ) The solution implies wait until π ( x ) ≤ π ( x ∗ ) < 0 before stopping (t is worthwile to incur some loss before closing down the machine).

Heuristic argument for smooth pasting: Suppose F x < Ω x at x ∗ . We have upward kink. Then there exists an x ∗∗ > x ∗ such that Ω( x ∗∗ ) > F ( x ∗∗ ) and we should stop at x ∗∗ . Suppose F x > Ω x at x ∗ . We have downward kink. Then payoff is convex at optimum and there is value to waiting and determining what the realized value of x will be.

Smooth pasting – slightly less heuristic Assume a = 1 , b = 1 , √ Over the interval ∆ t , x rise by ∆ h with p = 1 / 2[1 + ∆ t ] and √ falls by ∆ h with prob q = 1 / 2[1 − ∆ t ] . Consider the alternative policy of waiting until ∆ t to take action.