Log-optimal Investment as MDPs Log-optimal Investment in Markovian Environments Csaba Szepesv´ ari Computer and Automation Research Institute of the Hungarian Academy of Sciences Kende u. 13-17, Budapest 1111, Hungary E-mail: szcsaba@sztaki.hu Morgen Stanley Quantitative and Financial Mathematics Conference 21 October, 2005 Co-workers: Remi Munos, Andr´ as Antos Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Outline Outline Introduction 1 Markovian Decision Problems Log-optimal Investment 2 FX Markets Stock Market Solution Methods for MDPs 3 Classics Approximate Methods Does it Work? Application to Log-optimal Investment 4 5 Conclusions Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Outline Outline Introduction 1 Markovian Decision Problems Log-optimal Investment 2 FX Markets Stock Market Solution Methods for MDPs 3 Classics Approximate Methods Does it Work? Application to Log-optimal Investment 4 5 Conclusions Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Outline Outline Introduction 1 Markovian Decision Problems Log-optimal Investment 2 FX Markets Stock Market Solution Methods for MDPs 3 Classics Approximate Methods Does it Work? Application to Log-optimal Investment 4 5 Conclusions Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Outline Outline Introduction 1 Markovian Decision Problems Log-optimal Investment 2 FX Markets Stock Market Solution Methods for MDPs 3 Classics Approximate Methods Does it Work? Application to Log-optimal Investment 4 5 Conclusions Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Outline Outline Introduction 1 Markovian Decision Problems Log-optimal Investment 2 FX Markets Stock Market Solution Methods for MDPs 3 Classics Approximate Methods Does it Work? Application to Log-optimal Investment 4 5 Conclusions Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Introduction Markovian Decision Problems Outline Introduction 1 Markovian Decision Problems Log-optimal Investment 2 FX Markets Stock Market Solution Methods for MDPs 3 Classics Approximate Methods Does it Work? Application to Log-optimal Investment 4 5 Conclusions Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Introduction Markovian Decision Problems Markovian Decision Problems Definition ( X , A , P , r ) MDP: State space X ( ⊂ R d ) Action space A Transition probabilities P ( ·| x , a ) Reward function r ( x , a ) . Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Introduction Markovian Decision Problems Markovian Decision Problems Definition ( X , A , P , r ) MDP: State space X ( ⊂ R d ) Action space A Transition probabilities P ( ·| x , a ) Reward function r ( x , a ) . Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Introduction Markovian Decision Problems Markovian Decision Problems Definition ( X , A , P , r ) MDP: State space X ( ⊂ R d ) Action space A Transition probabilities P ( ·| x , a ) Reward function r ( x , a ) . Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Introduction Markovian Decision Problems Markovian Decision Problems Definition ( X , A , P , r ) MDP: State space X ( ⊂ R d ) Action space A Transition probabilities P ( ·| x , a ) Reward function r ( x , a ) . Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Introduction Markovian Decision Problems Process View X A t , R t X t +1 X t π : X → A V π ( x ) = E [ � ∞ t =0 γ t R t | X 0 = x, π ] � ∞ Q π ( x, a ) = E [ t =0 γ t R t | X 0 = x, A 0 = a, π ] 0 < γ < 1 Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Introduction Markovian Decision Problems Reinforcement Learning Goal: Finding an optimal policy .. in an unknown MDP by just observing a trajectory .. when a generative model of the MDP is given ..large MDP .. when a model of the MDP is given Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Introduction Markovian Decision Problems Reinforcement Learning Goal: Finding an optimal policy .. in an unknown MDP by just observing a trajectory .. when a generative model of the MDP is given ..large MDP .. when a model of the MDP is given Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Introduction Markovian Decision Problems Reinforcement Learning Goal: Finding an optimal policy .. in an unknown MDP by just observing a trajectory .. when a generative model of the MDP is given ..large MDP .. when a model of the MDP is given Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Introduction Markovian Decision Problems Reinforcement Learning Goal: Finding an optimal policy .. in an unknown MDP by just observing a trajectory .. when a generative model of the MDP is given ..large MDP .. when a model of the MDP is given Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Log-optimal Investment FX Markets Outline Introduction 1 Markovian Decision Problems Log-optimal Investment 2 FX Markets Stock Market Solution Methods for MDPs 3 Classics Approximate Methods Does it Work? Application to Log-optimal Investment 4 5 Conclusions Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Log-optimal Investment FX Markets Simple FX Example 2-currency exchange rates: dollar: p 12 ( t ) euro: p 21 ( t ) p 12 ( t ) – amount of dollar purchased for 1 euro W t – wealth (calc’ed in dollars) α t – relative portfolio; proportion of wealth in euros Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Log-optimal Investment FX Markets Simple FX Example 2-currency exchange rates: dollar: p 12 ( t ) euro: p 21 ( t ) p 12 ( t ) – amount of dollar purchased for 1 euro W t – wealth (calc’ed in dollars) α t – relative portfolio; proportion of wealth in euros Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Log-optimal Investment FX Markets Simple FX Example 2-currency exchange rates: dollar: p 12 ( t ) euro: p 21 ( t ) p 12 ( t ) – amount of dollar purchased for 1 euro W t – wealth (calc’ed in dollars) α t – relative portfolio; proportion of wealth in euros Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Log-optimal Investment FX Markets Simple FX Example 2-currency exchange rates: dollar: p 12 ( t ) euro: p 21 ( t ) p 12 ( t ) – amount of dollar purchased for 1 euro W t – wealth (calc’ed in dollars) α t – relative portfolio; proportion of wealth in euros Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Log-optimal Investment FX Markets Simple FX Example 2-currency exchange rates: dollar: p 12 ( t ) euro: p 21 ( t ) p 12 ( t ) – amount of dollar purchased for 1 euro W t – wealth (calc’ed in dollars) α t – relative portfolio; proportion of wealth in euros Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Log-optimal Investment FX Markets Simple FX Example 2-currency exchange rates: dollar: p 12 ( t ) euro: p 21 ( t ) p 12 ( t ) – amount of dollar purchased for 1 euro W t – wealth (calc’ed in dollars) α t – relative portfolio; proportion of wealth in euros Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Log-optimal Investment FX Markets FX: Dynamics and Bid-Ask Spread 2-currency exchange rates: dollar: p 12 ( t ) euro: p 21 ( t ) Dynamics of dollar’s exchange rate: p 12 ( t + 1 ) = ρ t + 1 p 12 ( t ) Bid-ask spread: p 12 ( t + 1 ) p 21 ( t + 1 ) = η 2 t + 1 < 1 Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Log-optimal Investment FX Markets FX: Dynamics and Bid-Ask Spread 2-currency exchange rates: dollar: p 12 ( t ) euro: p 21 ( t ) Dynamics of dollar’s exchange rate: p 12 ( t + 1 ) = ρ t + 1 p 12 ( t ) Bid-ask spread: p 12 ( t + 1 ) p 21 ( t + 1 ) = η 2 t + 1 < 1 Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Log-optimal Investment FX Markets FX: Dynamics and Bid-Ask Spread 2-currency exchange rates: dollar: p 12 ( t ) euro: p 21 ( t ) Dynamics of dollar’s exchange rate: p 12 ( t + 1 ) = ρ t + 1 p 12 ( t ) Bid-ask spread: p 12 ( t + 1 ) p 21 ( t + 1 ) = η 2 t + 1 < 1 Csaba Szepesv´ ari Log-optimal Investment as MDPs
Log-optimal Investment as MDPs Log-optimal Investment FX Markets FX: Dynamics and Bid-Ask Spread 2-currency exchange rates: dollar: p 12 ( t ) euro: p 21 ( t ) Dynamics of dollar’s exchange rate: p 12 ( t + 1 ) = ρ t + 1 p 12 ( t ) Bid-ask spread: p 12 ( t + 1 ) p 21 ( t + 1 ) = η 2 t + 1 < 1 Csaba Szepesv´ ari Log-optimal Investment as MDPs
Recommend
More recommend
