Why dont we move slower? On the cost of time in the neural control - PowerPoint PPT Presentation

Why don’t we move slower? On the cost of time in the neural control of movement eric Jean 1 Bastien Berret 2 Fr´ ed´ 1 ENSTA ParisTech (and Team GECO, INRIA Saclay) 2 University Paris Sud IHP, November 26, 2014 F. Jean (ENSTA) Why don’t we move slower? IHP, 2014 1 / 27

Warning This is not a non-holonomic talk. . . not even a mathematical one. Application of optimal control to the modelling of human motor control. F. Jean (ENSTA) Why don’t we move slower? IHP, 2014 2 / 27

Cost of time in human motions Outline Cost of time in human motions 1 Recovering g 2 Inverse optimal control 3 Experimental results 4 From self paced to slow/fast motions 5 F. Jean (ENSTA) Why don’t we move slower? IHP, 2014 3 / 27

Cost of time in human motions What is the duration of a natural movement? (Natural = realize one task, without constraint of time, precision,. . . ) Usual theory: duration results from the minimization of a compromise between the cost of the time and the cost of the motion. a b 40 25 35 20 effort cost 30 Cost of time [a.u.] time cost Costs [a.u.] total cost (effort + time) 25 ? 15 20 10 15 linear 10 hyperbolic 5 exponential - 5 quadratic exponential + 0 0 0 0.25 0.5 0.75 1 1.25 1.5 1.75 2 0 0.25 0.5 0.75 1 1.25 1.5 1.75 2 Duration [s] Duration [s] F. Jean (ENSTA) Why don’t we move slower? IHP, 2014 4 / 27

Cost of time in human motions Cost of the motion (in fixed time): solution of an inverse optimal control problem, large literature in the case of the arm [Flash, Shadmehr, Berret, . . . ] Cost of the time: lot of different modelling, psychologists → hyperbolic costs (concave functions) economists, behaviourists → exponential costs (convex functions) Only interpretations/explanations, no quantitative results. a b 40 25 35 20 effort cost 30 Cost of time [a.u.] time cost Costs [a.u.] total cost (effort + time) 25 ? 15 20 10 15 linear 10 hyperbolic 5 exponential - 5 quadratic exponential + 0 0 0 0.25 0.5 0.75 1 1.25 1.5 1.75 2 0 0.25 0.5 0.75 1 1.25 1.5 1.75 2 Duration [s] Duration [s] F. Jean (ENSTA) Why don’t we move slower? IHP, 2014 5 / 27

Cost of time in human motions Modelling Dynamics of the motion: ˙ x = f ( x, u ) Paradigm Any registered trajectory x ( · ) from x 0 to x f is an optimal solution of � t u min ( g ( t ) + L ( x u ( t ) , u ( t ))) dt, u 0 among all u ( · ) defined on [0 , t u ] s.t. x u (0) = x 0 , x u ( t u ) = x f . � T g ( t ) dt : cost of the time T 0 � T L ( x u , u ) : cost of the motion in fixed time T 0 F. Jean (ENSTA) Why don’t we move slower? IHP, 2014 6 / 27

Cost of time in human motions Modelling Dynamics of the motion: ˙ x = f ( x, u ) Paradigm Any registered trajectory x ( · ) from x 0 to x f is an optimal solution of � t u min ( g ( t ) + L ( x u ( t ) , u ( t ))) dt, u 0 among all u ( · ) defined on [0 , t u ] s.t. x u (0) = x 0 , x u ( t u ) = x f . � T g ( t ) dt : cost of the time T ← what we are looking for! 0 � T L ( x u , u ) : cost of the motion in fixed time T 0 F. Jean (ENSTA) Why don’t we move slower? IHP, 2014 6 / 27

Recovering g Outline Cost of time in human motions 1 Recovering g 2 Inverse optimal control 3 Experimental results 4 From self paced to slow/fast motions 5 F. Jean (ENSTA) Why don’t we move slower? IHP, 2014 7 / 27

Recovering g Necessary condition Fix x f . Value function of the problem in fixed time : � t L ( x u , u ) : for x u joining x to x f in time t } V ( t, x ) = inf { 0 Set T = movement time from x 0 to x f . Then � � t g ( s ) ds + V ( t, x 0 ) � T ∈ argmin , t ∈ [0 , + ∞ ) 0 and so g ( T ) = − ∂V ∂t ( T, x 0 ) . More precisely, g ( T ) = − H 0 ( x ∗ ( T ) , p ∗ ( T ) , u ∗ ( T )) where: H 0 ( x, p, u ) = � p, f ( x, u ) � + L ( x, u ) normal Hamiltonian in fixed time, ( x ∗ ( · ) , u ∗ ( · )) optimal solution in time T with adjoint vector p ∗ ( · ) . F. Jean (ENSTA) Why don’t we move slower? IHP, 2014 8 / 27

Recovering g Remark: requires two technical assumptions on the fixed time problem: existence of minimizers no abnormal minimizers (property of the dynamics) Recovering g from experimental data: Fix x f and choose initial conditions x 0 ( a ) , a ∈ [ a 1 , a 2 ] . → T ( a ) = time of motion from x 0 ( a ) to x f . Experiments − Assume a �→ T ( a ) is invertible and set a ∗ ( t ) = T − 1 ( t ) [Ex: a amplitude ⇒ T ր ] g ( t ) = − ∂V ∂t ( t, x 0 ( a ∗ ( t ))) . Then F. Jean (ENSTA) Why don’t we move slower? IHP, 2014 9 / 27

Recovering g Conclusion Given the cost of motion L ( x, u ) , the cost of time g can be deduced from simple experiments. Problems: how to determine L ( x, u ) ? Robustness of the construction of g ? F. Jean (ENSTA) Why don’t we move slower? IHP, 2014 10 / 27

Inverse optimal control Outline Cost of time in human motions 1 Recovering g 2 Inverse optimal control 3 Experimental results 4 From self paced to slow/fast motions 5 F. Jean (ENSTA) Why don’t we move slower? IHP, 2014 11 / 27

Inverse optimal control Inverse optimal control (Direct) Optimal control problem Given a dynamic ˙ x = f ( x, u ) , a cost C ( x u ) and a pair of points x 0 , x 1 , find a trajectory x u ∗ solution of inf { C ( x u ) : x u traj. s.t. x u (0) = x 0 , x u ( T ) = x 1 } . Inverse optimal control problem Given ˙ x = f ( x, u ) and a set Γ of trajectories, find a cost C ( x u ) such that every γ ∈ Γ is solution of inf { C ( x u ) : x u traj. s.t. x u (0) = γ (0) , x u ( T ) = γ ( T ) } . Applications to analysis/modelling of human motor control (physiology) → looking for optimality principles F. Jean (ENSTA) Why don’t we move slower? IHP, 2014 12 / 27

Inverse optimal control Inverse problem: Choose a class C of reasonable costs and let Φ : C ∈ C �→ Γ optimal synthesis . Inverse optimal control problem = find an inverse Φ − 1 . Well-posed problem? Φ injective? Continuity (and stability) of Φ − 1 ? → Very few general results: Calculus of Variations case [Krupkova, Prince,. . . 1990-2000’s], Linear-Quadratic case [Kalmann 64, Nori-Frezza 04], numerical methods [Mombaur-Laumond 2010, Pauwels-Henrion-Lasserre 2014] F. Jean (ENSTA) Why don’t we move slower? IHP, 2014 13 / 27

Inverse optimal control Theoretical result Dynamics = the one of the 1 doF arm motion (single-input linear system) Let SC = set of smooth functions L ( x, u ) such that ∂ 2 L ∂u 2 > 0 (strict convexity) (0 , 0) unique minimum, L (0 , 0) = 0 , and ∂ 2 L ∂u 2 (0 , 0) = 1 (normalization) Theorem There exists a dense subset Ω s.t. Φ injective on Ω . (Proof based on Thom transversality) Open questions: continuity of (Φ | Ω ) − 1 ? Φ(Ω) dense in Φ( SC ) ? F. Jean (ENSTA) Why don’t we move slower? IHP, 2014 14 / 27

Inverse optimal control Practical point of view x = Ax + Bu , x ∈ R n . Linear dynamics: ˙ Class of admissible costs = class of quadratic costs, C = { L ( x, u ) = u T Qu + x T Rx + 2 x T Su, Q ≻ 0 , L sym, � 0 } . → optimal controls in time T of the form u ( t ) = K T ( t ) x ( t ) . Remark: { K T ( · ) , T > 0 } uniquely determined by a pair ( K − , K + ) , optimal solutions in time T of the form: x ( t ) = e ( A + BK + ) t y + + e ( A + BK − )( t − T ) y − . Hyp: Single input case, i.e. u ∈ R F. Jean (ENSTA) Why don’t we move slower? IHP, 2014 15 / 27

Inverse optimal control Theorem (adapted from Nori-Frezza, 2004), single-input case For any L ∈ C , there exists a unique k ∈ R n s.t. ( u − k T x ) 2 � � Φ( L ) = Φ . Moreover, ( K − , K + ) �→ k continuous ( k = − K T + ). → the inverse optimal control problem is well-posed. Application to the computation of g Find ( K − , K + ) by identification from experimental data; + and L ( x, u ) = ( u − k T x ) 2 ; set k = − K T compute the function ∂V ∂t ( t, x ) using the Hamiltonian; set g ( t ) = − ∂V ∂t ( t, x 0 ( a ∗ ( t ))) . → Method robust w.r.t. perturbations of the data and w.r.t. the choice of cost. F. Jean (ENSTA) Why don’t we move slower? IHP, 2014 16 / 27

Inverse optimal control Quantitative result: Assume x f = 0 and x 0 ( a ) = ax 0 (1) . Then: ∂V ∂t ( T, x 0 ) = − u a ( T ) 2 , where u a ( · ) = optimal control from x 0 ( a ) to x f in time T . u a ( T ) = aν ( T ) , where e − ( A + BK − ) T − e − ( A + BK + ) T � − 1 � x 0 (1) . ν ( T ) = ( K + − K − ) g ( t ) = ν ( t ) 2 a ∗ ( t ) 2 ⇒ F. Jean (ENSTA) Why don’t we move slower? IHP, 2014 17 / 27

Experimental results Outline Cost of time in human motions 1 Recovering g 2 Inverse optimal control 3 Experimental results 4 From self paced to slow/fast motions 5 F. Jean (ENSTA) Why don’t we move slower? IHP, 2014 18 / 27

Experimental results Two kind of motions: Pointing motions of the arm in a horizontal plane (1 degree of freedom), Saccadic eye movements. In both cases: the dynamic is of the form: θ ( n ) + c n − 1 θ ( n − 1) + · · · + c 0 θ = u, → linear with a state x = ( θ, ˙ θ, . . . , θ ( n − 1) ) ( n = 2 or 3 in general) Initial and final states are equilibria, typically: x f = 0 . x 0 ( a ) = ( a, 0 , . . . , 0) and F. Jean (ENSTA) Why don’t we move slower? IHP, 2014 19 / 27