Closed-Loop Impulse Control of Oscillating Systems A. N. Daryin and - PowerPoint PPT Presentation

Closed-Loop Impulse Control of Oscillating Systems A. N. Daryin and A. B. Kurzhanski Moscow State (Lomonosov) University Faculty of Computational Mathematics and Cybernetics Periodic Control Systems, 2007

Outline Problem 1 Dynamic Programming Approach 2 Numerical Algorithm 3 Ellipsoidal Approximation 4 Asymptotic Solution (∆ t → ∞ ) 5 Unilateral Impulses 6 Double Constraint Approach 7 Generalized Impulse Control Problem 8

Oscillating System L 1 C 1 k 1 m 1 L 2 w 1 C 2 k 2 m 2 w 2 m N − 1 w N − 1 k N VC N L N m N w N F

Oscillating System  m 1 ¨ w 1 = k 2 ( w 2 − w 1 ) − k 1 w 1      m i ¨ w i = k i +1 ( w i +1 − w i ) − k i ( w i − w i − 1 ) m ν ¨ w ν = k ν +1 ( w ν +1 − w ν ) − k ν ( w ν − w ν − 1 ) + u ( t )      m N ¨ w N = − k N ( w N − w N − 1 ) w i = w i ( t ) — displacements from the equilibrium m i — masses of the loads k i — stiffness coefficients u ( t ) = dU dt — impulse control ( U ∈ BV )

N → ∞ ρ ( ξ ) w tt ( t , ξ ) = [ Y ( ξ ) w ξ ( t , ξ )] ξ , t > t 0 , 0 < ξ < L w ( t , 0) = 0 , w ξ ( t , L ) = u ( t ) / Y ( L ) , t � t 0 w ( t 0 , ξ ) = w 0 ( ξ ) , w 0 ( ξ ) , w t ( t 0 , ξ ) = ˙ 0 � ξ � L w ( t , ξ ) — displacement from the equilibrium u ( t ) = dU dt — impulse control ρ ( ξ ) — mass density Y ( ξ ) — Young modulus

Oscillating System Normalized matrix form: dx ( t ) = Ax ( t ) dt + BdU ( t )   w 1 ( t ) � w ( t ) � .  .  x ( t ) = w ( t ) = .   w ( t ) ˙ w N ( t ) This system is completely controllable .

Impulse Control Problem Problem (1) Minimize J ( U ( · )) = Var [ t 0 , t 1 ] U ( · ) + ϕ ( x ( t 1 + 0)) over U ( · ) ∈ BV [ t 0 , t 1 ] where x ( t ) is the trajectory generated by control input u ( t ) = d U d t starting from x ( t 0 − 0) = x 0 . 2 N � u ( t ) = h i δ ( t − τ i ) i =1 Important particular case: ϕ ( x ) = I ( x | { 0 } ) — completely stop oscillations on fixed time interval [ t 0 , t 1 ].

The Value Function Definition The minimum of J ( U ( · )) with fixed initial position x ( t 0 − 0) = x 0 is called the value function: V ( t 0 , x 0 ) = V ( t 0 , x 0 ; t 1 , ϕ ( · )) . � � � � p , x 1 − e ( t 1 − t 0 ) A x 0 V ( t 0 , x 0 ) = inf ϕ ( x 1 ) + sup . � � B T e ( t 1 −· ) A T p � x 1 ∈ R n p ∈ R n � C [ t 0 , t 1 ] The value function is convex and its conjugate equals � V ∗ ( t 0 , p ) = ϕ ∗ ( e ( t 0 − t 1 ) A T p ) + I � e ( t 0 − t 1 ) A T p � � � B �·� [ t 0 , t 1] � � � B T e ( t 1 −· ) A T p � � where � p � [ t 0 , t 1 ] = C [ t 0 , t 1 ] . �

Dynamic Programming Equation The value function V ( t , x ; t 1 , ϕ ( · )) satisfies the Principle of Optimality V ( t 0 , x 0 ; t 1 , ϕ ( · )) = V ( t 0 , x 0 ; τ, V ( τ, · ; t 1 , ϕ ( · ))) , τ ∈ [ t 0 , t 1 ] The value function it is the solution to the Hamilton–Jacobi–Bellman quasi-variational inequality: min { H 1 ( t , x , V t , V x ) , H 2 ( t , x , V t , V x ) } = 0 , V ( t 1 , x ) = V ( t 1 , x ; t 1 , ϕ ( · )) . � � � B T V x � � H 1 = V t + � V x , Ax � , H 2 = min u ∈ S 1 � V x , Bu � + 1 = − � + 1 .

The Control Structure H 1 ( t , x ) = 0 ( t , x ) H 2 ( t , x ) = 0 choose jump direction wait d = − B T V x dU ( t ) = 0 choose jump amplitude min α � 0 : H 1 ( t , x + α d ) = 0 jump U ( τ ) = α · d · χ ( τ − t )

Numerical Algorithm The value function is V ( t 0 , x 0 ) = max � p , x 0 � . p ∈ R n ‚ B T e ( t 1 − t ) AT p ‚ ‚ ‚ � 1 ‚ ‚ ∀ t ∈ [ t 0 , t 1 ] � � � B T e ( t 1 − t ) A T p � � Replace � � 1 by a finite number of linear inequalities, and [ t 0 , t 1 ] with a finite number of time instants: ˆ V ( t 0 , x 0 ) = max � p , x 0 � p ∈ R n q i , B T e ( t 1 − t ) AT p D E � 1 , i =1 , M t = θ 1 ,θ 2 ,...,θ K which is a LP problem.

Numerical Algorithm Finding control for given ( t , x ) is a LP ranging problem. The error estimate is � � K − 1 �� V ( t , x ) � ˆ V ( t , x ) � V ( t , x ) 1 + O ,

Ellipsoidal Approximation X ν [ t ] — backward reach set under condition Var U � ν V ( t , x ) = min { ν | x ∈ X ν [ t ] } We look for an approximation of X ν [ t ]. Ellipsoids: � � � � � � x − q , Q − 1 ( x − q ) E ( q , Q ) = x � 1 1 ρ ( ℓ | E ( q , Q )) = � ℓ, q � + � ℓ, Q ℓ � 2 (see Kurzhanski and V´ alyi, 1997)

Ellipsoidal Approximation Ellipsoidal approximation is derived through comparison principle for Hamilton–Jacobi equations (Kurzhanski, 2006): X − ν [ t ] = E (0 , ( ν − k ( t )) Z ( t )) � ˙ � = AZ + ZA T − η ( t ) BB T Z Z ( t 1 ) = 0 ˙ = − 1 k 4 η ( t ) k ( t 1 ) = 0 Here η ( t ) � 0 is a parameter function � X − X ν [ t ] = cl ν [ t ] ν ( · )

Ellipsoidal Approximation 1 0.5 x 10 0 −0.5 −1 −1.5 −1 −0.5 0 0.5 1 1.5 x 9

Asymptotic Solution (∆ t → ∞ ) h i = − ω 2 ¨ i h i + b i u , i = 1 , N . 1 h 1 , dh 1 /dt 0 −1 −2 −3 0 5 10 15 Time t 1 h 2 , dh 2 /dt 0 −1 0 5 10 15 Time t 1 h 3 , dh 3 /dt 0 −1 0 5 10 15 Time t

Asymptotic Solution (∆ t → ∞ ) N � � cl X 1 [ t ] = C = C j t � t 1 j =1 X 1 [ t ] — backward reach set under condition Var U � 1 � � � ( h , ˙ j + ˙ � ω 2 � j h 2 h 2 j � b 2 C j = h ) — ellipsoids j � � j + ˙ � ω 2 j h 2 h 2 � j t →−∞ V = max → (*) V b 2 j =1 , N j Control strategy: � � � = 1 ⇐ � B T V x “Optimal”: jump if ⇒ h j = 0 for all maximizers j in (*). Useless after first jump. � � � B T V x � � 1 − ε : ε -optimal: jump if V Var U ( · ) � 1 − ε

Unilateral Impulses Additional constraint: dU � 0 ( dU � 0). General case: KdU � 0 ( K — matrix) or dU ∈ K ( K — cone). The minimum number of impulses is the same — 2 N . Numerical procedures apply with minor modifications. Asymptotic solution does not change. The problem may be not solvable on small time intervals.

Unilateral Impulses 7.5 7 6.5 Minimal Control Norm 6 Unilateral Impulses 5.5 5 4.5 4 3.5 Bilateral Impulses 3 1 2 10 10 ∆ t

Impulse vs Bang-Bang Controls 25 Bang−Bang Control 20 Minimal Control Norm 15 Impulse Control 10 5 10 20 30 40 50 60 70 80 90 100 ∆ t

Double Constraint Approach Problem (2) � t 1 Minimize J ( u ) = | u ( t ) | dt + ϕ ( x ( t 1 )) t 0 over controls u ( t ) satisfying | u ( t ) | � µ , where x ( t ) is the trajectory generated by control u starting from x ( t 0 ) = x 0 . Here controls are bounded functions. Optimal controls only take values − µ , 0, µ . V µ ( t , x ) is the value function for Problem 2. 0 � V µ ( t , x ) − V ( t , x ) = O ( µ − 1 ) for each ( t , x )

Double Constraint Approach mu = 5 5 4 Not Solvable 3 u = 0 2 u = − µ 1 x 2 0 −1 u = µ −2 u = 0 −3 Not Solvable −4 −5 −5 −4 −3 −2 −1 0 1 2 3 4 5 x 1

Double Constraint Approach 25 20 u = − µ u = 0 15 10 5 x 2 0 −5 −10 u = µ u = 0 −15 −20 −25 −25 −20 −15 −10 −5 0 5 10 15 20 25 x 1

Generalized Impulse Control Problem Problem (3) Minimize J ( u ) = ρ ∗ [ u ] + ϕ ( x ( t 1 + 0)) over distributions u ∈ D 1 [ α, β ] , ( α, β ) ⊇ [ t 0 , t 1 ] where x ( t ) is the trajectory generated by control u starting from x ( t 0 − 0) = x 0 . Here ρ ∗ [ u ] is the conjugate norm to the norm ρ on C 1 [ α, β ]: � � ψ ( t ) � 2 + � ψ ′ ( t ) � 2 . ρ [ ψ ] = max t ∈ [ α,β ] 2 N � h (0) δ ( t − τ i ) + h (1) δ ′ ( t − τ i ) . u ( t ) = i i i =1

Reduction to Impulse Control Problem d t + d 2 U 1 u = d U 0 u ∈ D 1 : U 0 , U 1 ∈ BV d t 2 Problem 3 reduces to a particular case of Problem 1 for the system � � x = Ax + B u , ˙ B = B AB and the control � U 0 ( t ) � u = d U d t , U ( t ) = . U 1 ( t ) Error bound for numerical algorithm: � � K − 1 + M − 2 �� V ( t , x ) � ˆ V ( t , x ) � V ( t , x ) 1 + O ,

Examples Chain of 5 springs String (10 elements)

References Bellman, R. (1957). Dynamic Programming . Princeton Univ. Press. Bensoussan, A. and J.-L. Lions (1982). Contrˆ ole impulsionnel et in´ equations quasi variationnelles . Paris. Crandall, M. G. and P.-L. Lions (1983). Viscosity solutions of Hamilton–Jacobi equations. Trans. Amer. Math. Soc. 277 , 1–41. Daryin, A. N., A. B. Kurzhanski and A. V. Seleznev (2005). A dynamic programming approach to the impulse control synthesis problem. In: Proc. Joint 44th IEEE CDC-ECC 2005 . IEEE. Seville. Demyanov, V. F. (1974). Minimax: Directional Derivates . Nauka. Moscow. Dykhta, V. A. and O. N. Sumsonuk (2003). Optimal impulsive control with applications . Fizmatlit. Moscow. Gusev, M. I. (1975). On optimal control of generalized processes under non-convex state constraints. In: Differential Games and Control Problems . UNC AN SSSR. Sverdlovsk. Kalman, R. E. (1960). On the general theory of control systems. In: Proc. 1st IFAC Congress . Vol. 1. IFAC. Butterworths. London. Krasovski, N. N. (1957). On a problem of optimal regulation. Prikl. Math. & Mech. 21 (5), 670–677.