ORF -MOSAIC M. Mahdi Ghazaei Ardakani*, Henrik Jrntell**, and Rolf - PowerPoint PPT Presentation

ORF -MOSAIC M. Mahdi Ghazaei Ardakani*, Henrik Jörntell**, and Rolf Johansson* * Dep. of Automatic Control and ** Exp. Medical Science Group Lund University, Sweden

How it began! 2 LCCC Symposium, 2012-04-19 Mahdi Ghazaei

3 LCCC Symposium, 2012-04-19 Mahdi Ghazaei

Outline  Introduction  Cerebellum  Different cerebellar models  MOSAIC  Problem formulation  Modeling  Methods  Experimental Design  Simulations  Results  Discussion  Conclusions LCCC Symposium, 2012-04-19 Mahdi Ghazaei 4

Cerebellum and its Role [1 ] [1 ] Modified from Neuroscie, 3rd Ed. [2] Courtesy of H.Jörntell [2] LCCC Symposium, 2012-04-19 Mahdi Ghazaei 5

Microznoe circuitry [1 ] Figure form Handbook of Robotics, Springer 2008, LCCC Symposium, 2012-04-19 Mahdi Ghazaei 6

Principles in Cerebellum  Feedforward processing  Divergence and convergence  Modularity  Plasticity LCCC Symposium, 2012-04-19 Mahdi Ghazaei 7

Computational Models  Cerebellar Model Articulation Controller (CMAC)  Adjustable Pattern Generator (APG)  Schweighofer-Arbib  Cerebellar feedback-error-learning model (CBFELM)  Multiple paired forward-inverse model (MPFIM) [1] Figure adapted form Handbook of Robotics, Springer 2008 LCCC Symposium, 2012-04-19 Mahdi Ghazaei 8

MOSAIC Structure  Internal Model  Forward Models  Inverse Models  Modularity  Adaptation  Reduction of motor error  Efference copy  Spin-offs  HMM-MOSAIC  HMOSAIC  e-MOSAIC  MMRL  AMA-MOSAICI [1] Figure from D.M. Wolpert and M. Kawato, Neural Net.,11:1325, 1998 LCCC Symposium, 2012-04-19 Mahdi Ghazaei 9

MOSAIC based Models  Original MOSAIC  Tested for switching between 3 objects, and generalizing to a new one  No of modules are manually tuned  Requires careful tuning of parameters  The quality of forward models are critical  HMM MOSAIC  Same experiments as above  Probabilistic model using HMM  heavy computation  Fixed to linear forward models  Originally in batch mode  Improved parameter tuning and resp. estimation by EM LCCC Symposium, 2012-04-19 Mahdi Ghazaei 10

MOSAIC based Model  HMOSAIC Same experiments  Two layers of MOSAIC  Higher layer provides estimate of prior probabilities   eMOSAIC Humanoid robot control   LQG for controllers  Forward models replaced by Kalman filters No adaptation   AMA-MOSAICI  Sit-to-stand control  Clustering algorithm for determining no of modules  Clustering and training Off-line  MMRL  Controllers are replaced by RL agents  Discrete and continuous case  Self-organization of modules  Haunting task in a grid world and controlling an inverted pendulum LCCC Symposium, 2012-04-19 Mahdi Ghazaei 11

MOSAIC based Model  Toy problems  More serious problems  Resorting to classic controllers or RL  Simplification  No dealy  No adaptation LCCC Symposium, 2012-04-19 Mahdi Ghazaei 12

Objectives  Investigation of the applicability of a biologically inspired model of cerebellum to deal with:  More complex embodiments  Less accurate models (delays, noise, …) [1] Figure from F.M.M.O. Camposa, J.M.F. Calado, Ann. Rev. In Control 33 (2009), 70 LCCC Symposium, 2012-04-19 Mahdi Ghazaei 13

Problem Statement ORF-MOSAIC as a biologically inspired cerebellar model to adaptively control a human-like robotic arm with potential delays LCCC Symposium, 2012-04-19 Mahdi Ghazaei 14

Methodology Choose models Validation Simulation Fix assumptions  Models as faithful as possible to biology  Fixes according well-established theories in control engineering LCCC Symposium, 2012-04-19 Mahdi Ghazaei 15

Modeling  Motor Cortex (Trajectory Generation, …)  Cerebellum (MOSAIC )  Sensory System and Lower Motor Control  Arm  Muscle Systems LCCC Symposium, 2012-04-19 Mahdi Ghazaei 16

Lower Motor Control LCCC Symposium, 2012-04-19 Mahdi Ghazaei 17

Musculoskletal Model  Preserves essential features of an arm  Mono-articular and bi-articular  Hill-type muscle model [1] Figure from Handbook of brain theory, MIT press, 1995 LCCC Symposium, 2012-04-19 Mahdi Ghazaei 18

Assumptions  Minimum jerk trajectory by CTX  Planning in task space, control in joint space and transformation to muscle space  Minimum tension principle for muscle control  Internal model LCCC Symposium, 2012-04-19 Mahdi Ghazaei 19

Methods  Linear models representing cellular structure  Low level control represented by a feedback controller and a transformation  Approximation to known adaptive controllers LCCC Symposium, 2012-04-19 Mahdi Ghazaei 20

Customization of MOSAIC  Introduction of receptive fields for modules  Dissociation of adaptation from control in a module  Why different modules?  Taking care of different subtasks which are domains in state space  Plasticity role?  Adapt existing internal models to cope with small changes in plant  To acquire new skills but no retention LCCC Symposium, 2012-04-19 Mahdi Ghazaei 21

Cerebellar Controller n 2 1 τ ff .. . Σ θ d , θ d , θ d Linear Inverse Model × . ( θ , θ ) 0 RF 1 × .. . θ , θ , θ Linear Forward τ copy Prediction Likelihood Model λ Soft max LCCC Symposium, 2012-04-19 Mahdi Ghazaei 22

Model of Arm LCCC Symposium, 2012-04-19 Mahdi Ghazaei 23

Minimum Tension Controller  Convex optimization  With some mathematical tricks reformulated to a quadratic programming problem LCCC Symposium, 2012-04-19 Mahdi Ghazaei 24

Simulation of Arm Constant muscle activation LCCC Symposium, 2012-04-19 Mahdi Ghazaei 25

Simulation of Arm Constant muscle activation States vs. time 0.7 400 u 1 u 2 200 0.6 u 3 0 Normalized Muscle Activation u 4 0.5 u 5 -200 u 6 0.4 -400 0.3 -600 q 1 0.2 q 2 -800 w 1 0.1 w 2 -1000 0 -1200 0 1 2 3 4 5 0 1 2 3 4 5 Time (s) Time (s) LCCC Symposium, 2012-04-19 Mahdi Ghazaei 26

Simulation of Arm Minimum Tension LCCC Symposium, 2012-04-19 Mahdi Ghazaei 27

Simulation of Arm Minimum Tension States vs. time 150 1 u 1 u 2 100 0.8 u 3 Normalized Muscle Activation u 4 50 u 5 0.6 u 6 0 0.4 -50 0.2 q 1 -100 q 2 w 1 0 -150 w 2 -0.2 -200 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 Time (s) Time (s) LCCC Symposium, 2012-04-19 Mahdi Ghazaei 28

End-2-end Simulation Trajectory generator .. . X θ , θ , θ 30 Inverse Kinematics 30 Cerebellar .. . Controller θ d , θ d , θ d τ sp u T τ C3/C4 Arm + Muscles A(q) Synergy dynamics Spinal Cord . θ d , θ d Feedback feedback . Controller motor cmd θ , θ . θ , θ LCCC Symposium, 2012-04-19 Mahdi Ghazaei 29

Final Experimental Design  30 [ms] delay in the path to the cerebellar controller  PD controller with stiffness parameter of human arm and no delay  Movement 0.65 [s] , wait for 0.65 [s]  16 modules in a15x15 [cm] workspace  External translation invariant force field in task space  Object : 60 [cm] rod with 2 [Kg], perpendicular to the arm LCCC Symposium, 2012-04-19 Mahdi Ghazaei 30

Mapping of modules to workspace Numbering and color coding of Samples of receptive fields in modules static configuration LCCC Symposium, 2012-04-19 Mahdi Ghazaei 31

Hand Trajectory Before Training After Training LCCC Symposium, 2012-04-19 Mahdi Ghazaei 32

Feedback & Feedforward contributions Before Training After Training LCCC Symposium, 2012-04-19 Mahdi Ghazaei 33

Contributions from modules Before Training After Training LCCC Symposium, 2012-04-19 Mahdi Ghazaei 34

Controlling Modules Before Training After Training LCCC Symposium, 2012-04-19 Mahdi Ghazaei 35

External Field Test Before Training After Training LCCC Symposium, 2012-04-19 Mahdi Ghazaei 36

Handling an Object Adaptation Before Training After Training LCCC Symposium, 2012-04-19 Mahdi Ghazaei 37

Parameters Across Modules No object With an object LCCC Symposium, 2012-04-19 Mahdi Ghazaei 38

Parameters Across Modules w/ External Field LCCC Symposium, 2012-04-19 Mahdi Ghazaei 39

Different trajectory 0.48 0.48 0.46 0.46 0.44 0.44 y(m) y(m) 0.42 0.42 0.4 0.4 0.38 0.38 0.36 0.36 0.16 0.18 0.2 0.22 0.24 0.26 0.28 0.16 0.18 0.2 0.22 0.24 0.26 0.28 x(m) x(m) 40 LCCC Symposium, 2012-04-19 Mahdi Ghazaei

Discussion  Learning of non-linearities by the cerbellar controller  Specialization of modules with 7-parameters to different areas of the force field  Trade-offs  Unit complexity vs. the number of modules  Unit adaptation vs. effective switching or combination  How to localize the model in cerebellum and brain?  Microzones and modules  Biologically plausible signals 41 LCCC Symposium, 2012-04-19 Mahdi Ghazaei

Conclusion  Cerebellar Model for Control Inspired by the Microzonal Structure  Arm Model with Musculo-skeletal structure  Adaptation to the changes in the load and external disturbances despite delay  Highly sparse representation with not full knowledge of the model as a priority  Model for distributed control and adaptation 42 LCCC Symposium, 2012-04-19 Mahdi Ghazaei