1 MDP Algorithms for Portfolio Optimization Problems in pure Jump Markets Nicole B¨ auerle (joint work with U. Rieder) Universit¨ at Karlsruhe (TH) Research University • founded 1825 Linz, October 2008
2 Overview • Model and optimization problem • Solution via discrete-time MDPs • Solution method and computational aspects • Model extensions Financial optimization problems October 2008 Nicole B¨ auerle
3 Model and optimization problem Financial optimization problems October 2008 Nicole B¨ auerle
4 The market model Suppose we have a financial market with one bond and d risky assets whose prices evolve as follows • Price process ( S 0 t ) of the bond: S 0 t := e rt , r ≥ 0 . • Price processes ( S k t ) of the risky assets k = 1 , . . . , d : dS k t = S k µ k dt + dC k � � t − t R are constants and C t := � N t where µ k ∈ I n =1 Y n with N = ( N t ) a Poisson process with rate λ > 0 and iid random vectors ( Y n ) with values in ( − 1 , ∞ ) d and distribution Q where I E � Y n � < ∞ . Financial optimization problems October 2008 Nicole B¨ auerle
5 The price processes of the risky assets If we denote by 0 := T 0 < T 1 < T 2 < . . . the jump time points of the Poisson process and if t ∈ [ T n , T n +1 ) , then for k = 1 , . . . , d S k t = S k � � µ k ( t − T n ) T n exp . At the time of a jump we have S k T n − S k T n − = S k T n − Y k n . In what follows we denote S t := ( S 1 t , . . . , S d t ) . ( S t ) is a so-called Piecewise Deterministic Markov Process (PDMP). Financial optimization problems October 2008 Nicole B¨ auerle
6 A typical sample path Simulated stock price in the PDMP model. Financial optimization problems October 2008 Nicole B¨ auerle
7 Portfolios and self-financing strategies Portfolio strategy: ( F t ) -predictable stochastic process ( π t ) with values in R d | u ≥ 0 , u · e ≤ 1 } U := { u ∈ I where π t = ( π 1 t , . . . , π d t ) gives the fractions of wealth invested in the risky assets at time t . 1 − π t · e is the fraction invested in the bond. The equation for the wealth process ( X π t ) is then: � � dX π t = X π r + π t · ( µ − re ) dt + π t dC t . t Financial optimization problems October 2008 Nicole B¨ auerle
8 The optimization problem Let U : (0 , ∞ ) → I R + be an increasing, concave utility function and define for a portfolio strategy π and t ∈ [0 , T ] , x > 0 : E t,x U ( X π V π ( t, x ) := I T ) . V ( t, x ) := sup π V π ( t, x ) . Obviously V π ( T, x ) := U ( x ) = V ( T, x ) . Davis (1993), Norberg (2003), Sch¨ al (2004,2005), Kirch and Runggaldier (2005), Jacobsen (2006), B. and Rieder (2008) Financial optimization problems October 2008 Nicole B¨ auerle
9 Solution via discrete-time MDPs Financial optimization problems October 2008 Nicole B¨ auerle
10 Solution via discrete-time MDPs: The model • State space: E = [0 , T ] × (0 , ∞ ) . A state ( t, x ) gives the jump time point t and the wealth x of the process directly after the jump. • Action space: A := { α : [0 , T ] → U measurable } . For α ∈ A we define the movement of the wealth between jumps by „Z t « φ α t ( x ) := x exp r + α s · ( µ − re ) ds . 0 • Transition probability: � T − t � � �� e − λs t + s, φ α � � � B | t, x, α 1 + α s · y q := λ 1 B s ( x ) Q ( dy ) ds 0 := e − λ ( T − t ) U φ α � � � � • One-stage reward function: r t, x, α T − t ( x ) . Financial optimization problems October 2008 Nicole B¨ auerle
11 Solution via discrete-time MDPs A sequence ( f n ) with f n ∈ F := { f : E → A measurable } is called Markov policy . The expected reward of such a Markov policy is given by � ∞ �� � E ( f n ) � J ( f n ) ( t, x ) := I T k , X k , f k ( T k , X k ) ( t, x ) ∈ E. r , t,x k =0 Define J ( t, x ) := sup ( f n ) J ( f n ) ( t, x ) , for ( t, x ) ∈ E. Theorem 1: We have V ( t, x ) = J ( t, x ) , for ( t, x ) ∈ E and the optimal portfolio strategies ”coincide”. Financial optimization problems October 2008 Nicole B¨ auerle
12 Some important operators We define the following operators L, T f , T which act on M := { v : E → I R + | v is measurable } : I � e − λ ( T − t ) U φ α � � ( Lv )( t, x, α ) := T − t ( x ) + v ( s, y ) q ( ds, dy | t, x, α ) . ( T f v )( t, x ) := Lv ( t, x, f ( t, x )) , ( t, x ) ∈ E, f ∈ F. ( T v )( t, x ) = sup Lv ( t, x, α ) . α ∈ A From MDP theory it follows that = n →∞ T f 0 . . . T f n − 1 0 lim J ( f n ) n →∞ T n J f := J ( f ) = lim f 0 = T f J f . J f Financial optimization problems October 2008 Nicole B¨ auerle
13 A peculiar norm Let b ( t, x ) := e β ( T − t ) (1 + x ) , ( t, x ) ∈ E for β ≥ 0 . Next we introduce the weighted supremum norm � · � b on I M by v ( t, x ) � v � b := sup and B b := { v ∈ I M | � v � b < ∞} . I b ( t, x ) ( t,x ) ∈ E Finally, we define the set M c := { v ∈ I B b | v is continuous , v ( t, x ) is concave and I increasing in x, decreasing in t } . Financial optimization problems October 2008 Nicole B¨ auerle
14 Properties of T and T f Theorem 2: It holds that a) T : I M c → I M c . b) For v, w ∈ I B b and f ∈ F we have �T f v − T f w � b ≤ c β � v − w � b �T v − T w � b ≤ c β � v − w � b . � µ ) � λ (1+¯ y ) 1 − e − T ( β + λ − ¯ Thus the operators T f , T with c β := . are β + λ − ¯ µ contracting if c β < 1 which is the case if β is large enough. Financial optimization problems October 2008 Nicole B¨ auerle
15 Characterization of the value function Theorem 3: a) The value function V is the unique fixed point of T in I M c . b) For J 0 ∈ I M c it holds that c n β � V − T n J 0 � b ≤ �T J 0 − J 0 � b . 1 − c β c) There exists an optimal stationary portfolio strategy π , i.e. there exists an f ∈ F such that π t = f ( T n , X n )( t − T n ) for t ∈ [ T n , T n +1 ) . Financial optimization problems October 2008 Nicole B¨ auerle
16 Solution method and computational aspects Financial optimization problems October 2008 Nicole B¨ auerle
17 Howard’s policy improvement algorithm Theorem 4: Howard’s policy improvement algorithm works for the portfolio problem. More precisely, let f, g ∈ F . a) If for some subset E 0 ⊂ E g ( t, x ) ∈ D ( t, x, f ) := { α ∈ A | LJ f ( t, x, α ) > J f ( t, x ) } , ( t, x ) ∈ E 0 g ( t, x ) = f ( t, x ) , ( t, x ) / ∈ E 0 then J g ≥ J f and J g ( t, x ) > J f ( t, x ) for ( t, x ) ∈ E 0 . b) If D ( t, x, f ) = ∅ for all ( t, x ) ∈ E then J f = V , i.e. the decision rule f defines the optimal stationary portfolio strategy. Application: Suppose r = µ i and U is continuously differentiable and U ′ ( x + u · Y ) Y is integrable for all x > 0 and � u � small. Then ”invest all EY ≤ 0 . the money in the bond” is optimal if and only if I Financial optimization problems October 2008 Nicole B¨ auerle
18 Approximating the utility function Let U ( n ) , U be utility functions and denote by V ( n ) , V the corresponding value functions and by A ∗ n ( t, x ) := { α ∈ A : T ( n ) V ( n ) ( t, x ) = L ( n ) V ( n ) ( t, x, α ) } A ∗ ( t, x ) := { α ∈ A : T V ( t, x ) = LV ( t, x, α ) } . Moreover, let us denote by LsA ∗ { α ∈ A ; α is an accumulation point of a sequence ( α n ) n ( t, x ) := with α n ∈ A ∗ n ( t, x ) ∀ n ∈ I N } � � A ∗ the upper limit of the set sequence n ( t, x ) . Jouini and Napp (2004), Sch¨ al (1975) Financial optimization problems October 2008 Nicole B¨ auerle
19 Approximating the utility function Theorem 5: a) If U and ˜ U are two utility functions with corresponding value functions V and ˜ V , then e T ¯ µ � V − ˜ V � b ≤ � U − ˜ U � b . 1 − c β � U ( n ) � be a sequence of utility functions with lim n →∞ � U ( n ) − U � b = 0 . b) Let Then lim n →∞ � V ( n ) − V � b = 0 and we get that ∅ � = LsA ∗ n ( t, x ) ⊂ A ∗ ( t, x ) for all ( t, x ) ∈ E , i.e. in particular, the limit f ∗ of a sequence of decision rules ( f ∗ n ) with f ∗ n ( t, x ) ∈ A ∗ n ( t, x ) for all ( t, x ) ∈ E defines an optimal portfolio strategy for the model with utility function U . Financial optimization problems October 2008 Nicole B¨ auerle
20 State space discretization Choose a grid G ⊂ E and define the grid operator T G on I M c by � T v ( t, x ) , for ( t, x ) ∈ G T G v ( t, x ) := linear interpolation , else and b G : E → I R + by a linear interpolation of b . For v ∈ I M c : v ( t, x ) � v � G := sup b G ( t, x ) ≤ � v � b . ( t,x ) ∈ E Theorem 6: Suppose that c G < 1 . Then it holds for J 0 ∈ I M c that 1 � � � V − T n c n G J 0 � G ≤ G �T G J 0 − J 0 � G + m ( h ) 1 − c G where m ( h ) := � V − T G V � G → 0 if the mesh size tends to zero. Financial optimization problems October 2008 Nicole B¨ auerle
21 Numerical example One stock ( d = 1 ): µ = r = 0 . Relative jump distribution: pλ + e − λ + y dy � , y ≥ 0 Q ( dy ) = (1 − p ) λ − ( y + 1) λ − − 1 , − 1 < y < 0 p = 0 . 5 , λ + = λ − = 1 . Utility function U ( x ) = 1 γx γ , γ = 0 . 5 . Horizon T = 1 Mesh size: h = 0 . 01 Financial optimization problems October 2008 Nicole B¨ auerle
Recommend
More recommend
