Turnpike property in finite-dimensional nonlinear optimal control elat 1 Emmanuel Tr´ 1 Univ. Paris 6 (Labo. J.-L. Lions) et Institut Universitaire de France IHP , nov. 2014 E. Tr´ elat Turnpike in optimal control
E. Tr´ elat Turnpike in optimal control
Turnpike property The solution of an optimal control problem in large time should spend most of its time near a steady-state. In infinite horizon the solution should converge to that steady-state. Historically: discovered in econometry (Von Neumann points). The first turnpike result was discovered in 1958 by Dorfman, Samuelson and Solow, in view of deriving efficient programs of capital accumulation, in the context of a Von Neumann model in which labor is treated as an intermediate product. Paul Samuelson (1915–2009) Nobel Prize in Economic Science, 1970 E. Tr´ elat Turnpike in optimal control
Turnpike property The solution of an optimal control problem in large time should spend most of its time near a steady-state. In infinite horizon the solution should converge to that steady-state. Dorfman - Samuelson - Solow, 1958 Thus in this unexpected way, we have found a real normative significance for steady growth – not steady growth in general, but maximal von Neumann growth. It is, in a sense, the single most effective way for the system to grow, so that if we are planning long-run growth, no matter where we start, and where we desire to end up, it will pay in the intermediate stages to get into a growth phase of this kind. It is exactly like a turnpike paralleled by a network of minor roads. There is a fastest route between any two points; and if the origin and destination are close together and far from the turnpike, the best route may not touch the turnpike. But if origin and destination are far enough apart, it will always pay to get on to the turnpike and cover distance at the best rate of travel, even if this means adding a little mileage at either end. The best intermediate capital configuration is one which will grow most rapidly, even if it is not the desired one, it is temporarily optimal. E. Tr´ elat Turnpike in optimal control
Turnpike property The solution of an optimal control problem in large time should spend most of its time near a steady-state. In infinite horizon the solution should converge to that steady-state. Turnpike theorems have been derived in the 60’s for discrete-time optimal control prob- lems arising in econometry (Mac Kenzie, 1963). Continous versions by Haurie for particular dynamics (economic growth models). See also Carlson Haurie Leizarowitz 1991, Zaslavski 2000. More recently, in biology: Rapaport 2005, Coron Gabriel Shang 2014; human locomo- tion: Chitour Jean Mason 2012. Linear heat and wave equations: Porretta Zuazua 2013. Rockafellar 1973, Samuelson 1972: saddle point feature of the extremal equations of optimal control. Different point of view by Anderson Kokotovic (1987), Wilde Kokotovic (1972): exponential dichotomy property → hyperbolicity phenomenon. E. Tr´ elat Turnpike in optimal control
General nonlinear optimal control problem R n × I R m → I R n f : I dynamics R n × I R n → I R k , R = ( R 1 , . . . , R k ) R : I terminal conditions f 0 : I R n × I R m → I R instantaneous cost of class C 2 . Optimal control problem ( OCP ) T For T > 0 fixed, find u T ( · ) ∈ L ∞ ( 0 , T ; I R m ) such that x ( t ) = f ( x ( t ) , u ( t )) ˙ R ( x ( 0 ) , x ( T )) = 0 Z T f 0 ( x ( t ) , u ( t )) dt min 0 Examples of terminal conditions R : point-to-point, point-to-free, periodic, ... Optimal (assumed) solution: ( x T ( · ) , u T ( · )) . E. Tr´ elat Turnpike in optimal control
General nonlinear optimal control problem Pontryagin maximum principle ⇒ ∃ ( λ T ( · ) , λ 0 T ) � = ( 0 , 0 ) such that x T ( t ) = ∂ H ∂λ ( x T ( t ) , λ T ( t ) , λ 0 ˙ T , u T ( t )) λ T ( t ) = − ∂ H ˙ ∂ x ( x T ( t ) , λ T ( t ) , λ 0 T , u T ( t )) ∂ H ∂ u ( x T ( t ) , λ T ( t ) , λ 0 T , u T ( t )) = 0 where H ( x , λ, λ 0 , u ) = � λ, f ( x , u ) � + λ 0 f 0 ( x , u ) Moreover we have transversality conditions „ − λ T ( 0 ) « k X γ i ∇ R i ( x T ( 0 ) , x T ( T )) = λ T ( T ) i = 1 (generic...) assumption made throughout: no abnormal ⇒ λ 0 T = − 1 E. Tr´ elat Turnpike in optimal control
Static optimal control problem Static optimal control problem f 0 ( x , u ) min R n × I R m ( x , u ) ∈ I f ( x , u )= 0 Optimal (assumed) solution: (¯ x , ¯ u ) . Lagrange multipliers ⇒ (¯ λ, ¯ λ 0 ) � = ( 0 , 0 ) such that ∂ H f (¯ x , ¯ u ) = 0 x , ¯ λ, ¯ λ 0 , ¯ ∂λ (¯ u ) = 0 λ 0 ∂ f 0 D E λ, ∂ f ¯ x , ¯ ¯ x , ¯ i.e. − ∂ H ∂ x (¯ λ, ¯ u ) + ∂ x (¯ λ, ¯ u ) = 0 x , ¯ λ, ¯ λ 0 , ¯ ∂ x (¯ u ) = 0 λ 0 ∂ f 0 D λ, ∂ f E ∂ H ¯ x , ¯ ¯ x , ¯ ∂ u (¯ λ, ¯ ∂ u (¯ λ, ¯ u ) + u ) = 0 x , ¯ λ, ¯ λ 0 , ¯ ∂ u (¯ u ) = 0 H ( x , λ, λ 0 , u ) = � λ, f ( x , u ) � + λ 0 f 0 ( x , u ) (generic...) assumption made throughout: no abnormal ⇒ ¯ λ 0 T = − 1 (Mangasarian-Fromowitz) E. Tr´ elat Turnpike in optimal control
( OCP ) T Static optimal control problem ˙ x ( t ) = f ( x ( t ) , u ( t )) f 0 ( x , u ) min R ( x ( 0 ) , x ( T )) = 0 R n × I R m ( x , u ) ∈ I Z T f ( x , u )= 0 f 0 ( x ( t ) , u ( t )) dt min 0 x T ( t ) = ∂ H ∂ H x , ¯ ˙ ∂λ (¯ λ, − 1 , ¯ ∂λ ( x T ( t ) , λ T ( t ) , − 1 , u T ( t )) u ) = 0 λ T ( t ) = − ∂ H − ∂ H ˙ x , ¯ ∂ x (¯ λ, − 1 , ¯ ∂ x ( x T ( t ) , λ T ( t ) , − 1 , u T ( t )) u ) = 0 ∂ H ∂ H x , ¯ ∂ u ( x T ( t ) , λ T ( t ) , − 1 , u T ( t )) = 0 ∂ u (¯ λ, − 1 , ¯ u ) = 0 H ( x , λ, λ 0 , u ) = � λ, f ( x , u ) � + λ 0 f 0 ( x , u ) x , ¯ (¯ λ, ¯ u ) : equilibrium point of the extremal equations E. Tr´ elat Turnpike in optimal control
It is expected that, in large time T , the optimal extremal solution ( x T ( · ) , λ T ( · ) , u T ( · )) of ( OCP ) T approximately consists of 3 pieces: x , ¯ short-time: ( x T ( 0 ) , λ T ( 0 ) , u T ( 0 )) → (¯ λ, ¯ u ) 1 (transient arc) x , ¯ long-time, stationary: (¯ λ, ¯ 2 u ) x , ¯ short-time: (¯ λ, ¯ u ) → ( x T ( T ) , λ T ( T ) , u T ( T )) (transient arc) 3 E. Tr´ elat Turnpike in optimal control
The main result ∂ 2 H x , ¯ H ∗ # = ∂ ∗ ∂ # (¯ λ, − 1 , ¯ u ) A = H x λ − H u λ H − 1 W = − H xx + H ux H − 1 uu H xu , B = H u λ , uu H xu . Theorem (Tr´ elat Zuazua, JDE 2014) H uu < 0 , W > 0 rank ( B , AB , . . . , A n − 1 B ) = n (Kalman condition) x , ¯ (¯ λ ) ”almost satisfies” the terminal + transversality conditions Then for T > 0 large enough: u � ≤ C 1 ( e − C 2 t + e − C 2 ( T − t ) ) x � + � λ T ( t ) − ¯ � x T ( t ) − ¯ λ � + � u T ( t ) − ¯ ∀ t ∈ [ 0 , T ] Moreover: E − A + A ∗ E − − E − BH − 1 uu B ∗ E − − W = 0 minimal solution of Riccati E + A + A ∗ E + − E + BH − 1 uu B ∗ E + − W = 0 maximal solution of Riccati C 2 = − max { Re ( µ ) | µ ∈ Spec ( A − BH − 1 uu B ∗ E − ) } > 0 . E. Tr´ elat Turnpike in optimal control
Particular case: linear quadratic ( OCP ) T Static optimal control problem ˙ x ( t ) = Ax ( t ) + Bu ( t ) , x ( 0 ) = x 0 , x ( T ) = x 1 “ 1 ( x − x d ) ∗ Q ( x − x d ) min 2 Z T R n × I R m ( x , u ) ∈ I min 1 “ ( x ( t ) − x d ) ∗ Q ( x ( t ) − x d ) Ax + Bu = 0 2 ” 0 + ( u − u d ) ∗ U ( u − u d ) ” + ( u ( t ) − u d ) ∗ U ( u ( t ) − u d ) dt λ + Bu d = 0 x T ( t ) = Ax T ( t ) + BU − 1 B ∗ λ T ( t ) + Bu d x + BU − 1 B ∗ ¯ ˙ A ¯ λ − Qx d = 0 λ T ( t ) = Qx T ( t ) − A ∗ λ T ( t ) − Qx d ˙ x − A ∗ ¯ Q ¯ E. Tr´ elat Turnpike in optimal control
Particular case: linear quadratic ( OCP ) T Static optimal control problem ˙ x ( t ) = Ax ( t ) + Bu ( t ) , x ( 0 ) = x 0 , x ( T ) = x 1 “ 1 ( x − x d ) ∗ Q ( x − x d ) min 2 Z T R n × I R m ( x , u ) ∈ I min 1 “ ( x ( t ) − x d ) ∗ Q ( x ( t ) − x d ) Ax + Bu = 0 2 ” 0 + ( u − u d ) ∗ U ( u − u d ) ” + ( u ( t ) − u d ) ∗ U ( u ( t ) − u d ) dt λ + Bu d = 0 x T ( t ) = Ax T ( t ) + BU − 1 B ∗ λ T ( t ) + Bu d x + BU − 1 B ∗ ¯ ˙ A ¯ λ − Qx d = 0 λ T ( t ) = Qx T ( t ) − A ∗ λ T ( t ) − Qx d ˙ x − A ∗ ¯ Q ¯ Theorem U > 0 , Q > 0 rank ( B , AB , . . . , A n − 1 B ) = n (Kalman condition) u � ≤ C 1 ( e − C 2 t + e − C 2 ( T − t ) ) x � + � λ T ( t ) − ¯ ⇒ � x T ( t ) − ¯ λ � + � u T ( t ) − ¯ ∀ t ∈ [ 0 , T ] E. Tr´ elat Turnpike in optimal control
Example in LQ case Example ( x ( T ) free ⇒ λ ( T ) = 0) ˙ x 1 ( t ) = x 2 ( t ) , x 1 ( 0 ) = 0 Z T min 1 “ ( x 1 ( t ) − 2 ) 2 + ( x 2 ( t ) − 7 ) 2 + u ( t ) 2 ” dt x 2 ( t ) = − x 1 ( t ) + u ( t ) , ˙ x 2 ( 0 ) = 0 2 0 Optimal solution of the static problem: ¯ x 1 = ¯ ¯ x 2 = 0 , u “ ( x 1 − 2 ) 2 + ( x 2 − 7 ) 2 + u 2 ” min x 2 = 0 x 1 = u whence ¯ ¯ ¯ x = ( 1 , 0 ) , u = 1 , λ = ( − 7 , 1 ) E. Tr´ elat Turnpike in optimal control
Example in LQ case Example ( x ( T ) free ⇒ λ ( T ) = 0) ˙ x 1 ( t ) = x 2 ( t ) , x 1 ( 0 ) = 0 Z T min 1 “ ( x 1 ( t ) − 2 ) 2 + ( x 2 ( t ) − 7 ) 2 + u ( t ) 2 ” dt x 2 ( t ) = − x 1 ( t ) + u ( t ) , ˙ x 2 ( 0 ) = 0 2 0 Oscillation of ( x 1 ( · ) , x 2 ( · )) around the steady-state ( 1 , 0 ) E. Tr´ elat Turnpike in optimal control
Recommend
More recommend
