Mobile Robotics Lab Workshop on Grasp Planning and Task Learning by Imitation IROS 2010 – Taipei, Taiwan Institute of Systems and Robotics Institute of Systems and Robitcs http://paloma.isr.uc.pt University of Coimbra
Symbolic level generalization of in-hand manipulation tasks from human demonstrations using tactile information Ricardo Martins, Diego R. Faria and Jorge Dias Ricardo Martins http://www.isr.uc.pt/~rmartins/ Taipei, Taiwan 18 th August 2010 University of Coimbra
Contents • Introduction • Approach Overview • Contact Templates Definition • Contact Templates Detection • Conclusions • Future Work University of Coimbra
I - Introduction Development of robotic systems able to interact in new dynamic , unpredictable environments: - Domestic, healthcare, entertainment, education. • New challenges: Interaction Robots & Humans , Interaction Robots & Environment • Development of mobile robotic platforms: -Multimodal sensing capabilities: + Active vision + Audition + Multi Articulated Arms + Dexterous hands + … -Emergence of integration recognition, interaction and learning issues. University of Coimbra
I - Introduction • Development of human inspired robotic hands: Anatomy: Move from simple grippers towards human inspired articulated hands . Physiology: Introduction of sensing devices (tactile, temperature, force/torque sensors) Evolution: human inspired simple gripper articulated hands University of Coimbra
I - Introduction • The neuroscience literature (Johansson and Flanagan, 2009) proposes a decomposition of a typical human manipulation movement on different stages -reach -load -lift -hold -replace -unload Object approachs Digits contact Object lifts off manipulation Object contacts Digits release object surface height surface object Reach Load Lift Holding Replace Unload • In-Hand Manipulation -The internal consecutive regrasping and release of the object -Perform object reorientation, fine positioning or more complex interaction. University of Coimbra
I - Introduction • Representation of the general contact signatures for manipulation tasks developed on some daily activities N. Kamakura, Te no ugoki, Te no katachi (in Japanese), Ishiyaku Publishers, Tokyo, Japan, 1989. University of Coimbra
I - Introduction • Challenge: -How to learn and encode the human like skills/capabilities? -Develop a compact and flexible representation for: + task recognition + task planning + task synthesis • Imitation Learning à à Task representation University of Coimbra
II – Approach Overview • Generalization at a trajectory level: Adapted from Billard et al , 2008 University of Coimbra
II – Approach Overview • Generalization at a symbolic level: Adapted from Billard et al , 2008 University of Coimbra
II – Approach Overview Overview of the global structure of the proposed system Environment Contact State Bayesion Primitives Approach Templates Contact State Decision Sequence Pior Knowledge Rule Multifingered Posterior Robotic Hand Tactile Hand Tactile Effector Task Discretization Sensing Array Sensory Constraints Sensing Devices Processing Gain/Loss Function Action Perception Sensation • The human subjects performs an in-hand manipulation task using a instrumented glove (hand joint flexure sensors and tactile sensors) • Elementary primitives sequence extraction among a pre-defined primitive set. • Verification if the raw sequence of primitives extracted from the demonstration respects the constraints of that class. University of Coimbra
III –Contact Template Definition Primitive model definition • The output of the 360 tactile sensing elements are grouped on 15 regions. • Each region corresponds to different areas of the hand. • A variable T i is assigned to each of this regions: T = { T 1 , T 2 , … , T 15 } University of Coimbra
III –Contact Template Definition Primitive model definition T 3 T 4 T 2 • A variable T i is assigned to each of this regions: T 5 T 8 T 7 T 6 T 9 T 11 T = { T 1 , T 2 , … , T 15 } T 12 T 10 T 13 T 14 T 1 • Domain definition of each variable: T i { NotActive , LowActive , HighActive } ∀ = ∈ i 1 ,..., 15 T 15 • NotActive à 0 < T i < 25 • LowActive à 26 < T i < 190 • HighActive à 190 < T i < 255 University of Coimbra
III –Contact Template Definition Primitive set definition • The set of primitives comprises a total of 7 templates. • A primitive is designed by the variable E. Primitive2 Primitive3 Primitive1 • The domain definition of E is: E ε { primitive 1 , primitive 2 , … , primitive 7 } Primitive4 Primitive5 Primitive6 • Primitive 7 à no contact between the hand and the object. University of Coimbra
III –Contact Template Definition Primitive set training • Estimation of the parameters T of each of the predefined primitives E • Human demonstration of each of the 7 pre- defined primitives à static contact Primitive1 Primitive2 Primitive3 configuration of the human hand and the object. • From each primitive demonstration a Primitive4 Primitive5 Primitive6 probabilistic distribution is built à P( T /E) University of Coimbra
III – Contact Template Definition Experimental Results • Estimation of the parameters T of each of the pre-defined contact state templates • Five demonstrations of each grasp configurations Primitive 1 trainning result Trial 01 Trial 02 Trial 03 Trial 04 Trial 05 T 1 H H H H L T 2 H H H H L Primitive 1 - Conditional Probability Density Function T 3 N N N N N t k P( T = t k / E=primitive1) T 4 N N N N N (H,H,N,N,N,N,N,N,N,N,N, 4/5 N,N,N,N) T 5 N N N N N T 6 N N N N N (L,L,N,N,N,N,N,N,N,N,N,N 1/5 ,N,N,N) T 7 N N N N N T 8 N N N N N Other 0 T 9 N N N N N T 10 N N N N N T 11 N N N N N T 12 N N N N N T 13 N N N N N T 14 N N N N N N-NotActive Primitive1 L-LowActive T 15 N N N N N H-HighActive University of Coimbra
III – Contact Template Definition Experimental Results Primitive 2 - Conditional Probability Density Function Primitive 3 - Conditional Probability Density Function t k P( T = t k / E=primitive2) t k P( T = t k / E=primitive3) (N,L,H,H,L,L,H,H,L,N,N,N, 2/5 (H,H,H,H,L,H,H,L,L,H,L,L, 3/5 N,N,H) N,L,N) (N,L,H,H,L,N,L,L,N,N,N,N, 1/5 (H,H,H,H,N,H,H,L,N,H,L,L, 2/5 N,N,H) N,L,N) (N,L,H,H,N,N,L,L,N,N,N,N, 2/5 Other 0 N,N,H) Other 0 Primitive 4 - Conditional Probability Density Function t k P( T = t k / E=primitive4) (H,N,H,N,N,N,L,N,N,N,N,N 2/5 ,N,N,N) Primitive2 Primitive3 Primitive4 (H,N,H,N,N,N,N,N,N,N,N, 3/5 N,N,N,N) Other 0 N-NotActive L-LowActive H-HighActive University of Coimbra
III – Contact Template Definition Experimental Results Primitive 5 - Conditional Probability Density Function Primitive 6 - Conditional Probability Density Function t k P( T = t k / E=primitive5) t k P( T = t k / E=primitive6) (H,H,H,H,H,N,N,N,N,N,N, 1/5 (L,L,L,L,L,H,H,H,L,N,N,N, 2/5 N,N,N,N) N,N,H) (L,L,L,L,N,H,H,H,L,N,N,N, 3/5 (H,H,H,L,L,N,N,N,N,N,N,N 2/5 N,N,H) ,N,N,N) Other 0 (H,H,L,N,N,N,N,N,N,N,N,N 2/5 ,N,N,N) Other 0 Primitive 7 - Conditional Probability Density Function t k P( T = t k / E=primitive7) (N,N,N,N,N,N,N,N,N,N,N, 1 N,N,N,N) Primitive5 Primitive6 Other 0 N-NotActive L-LowActive H-HighActive University of Coimbra
IV – Contact Template Detection Primitives detection on raw data input • The raw data input produced by the human demonstration is integrated during equal time intervals of length t . • The integrated data during each time slot t à T t • The primitive with the maximum likelihood is the template assigned to that timeslot. From the primitives demonstration trainning session P ( T / E ) P ( E ) t P ( E / T ) = P(E)=1/7 t P ( T ) t P ( T ( t ,..., t ) / E primitive ) P ( E primitive ) = = = t 1 15 i i P ( E primitive / T ( t , t ,..., t )) = = = t i 1 2 15 7 P ( T / t ,..., t ) / E primitive ) P ( E primitive ) ∑ = = = = t 1 15 j j j 1 University of Coimbra
IV – Contact Template Detection Experimental Setup • Human demonstrator • Right handed instrumented data glove • Tactile sensing • Tactile sensing array data acquisition rate: 500Hz • Cup University of Coimbra
IV - Contact Template Detection Experimental Results • Task I – Reorientation of the mug in order to place the grasp of the mug in a configuration suitable to be grasped by the handle by the subject. University of Coimbra
Recommend
More recommend
