Manipulation in HRI: How a robot: an Overview Makes physical - PDF document

3/22/19 What is Manipulation? Manipulation in HRI: How a robot: • an Overview Makes physical changes to the world around it • Physically interacts with the world and other agents • Grasping, pushing, carrying, • moving, joining, placing, dropping, throwing, … Manipulators • Arm(s) with end-effectors • Other types • slide adapted from www.cs.columbia.edu/~allen/F15/NOTES/graspingClass2_2.ppt Manipulators Manipulators Manipulators Terminology UnderactuaFon: Only some joints are directly • controlled Compliant/Compliance: SoJ, not rigid, yielding • Handover: Handing from one agent to another • IMU: Interial Measurement Unit • Feed-forward: responds (to some signal) in a pred- • defined way (no feedback incorporated) 1

3/22/19 Manipulation Manipulation in HRI Tasks Challenges Physically alter the world through contact • As a primary goal • Retrieving objects Compliant objects • • Not its posi(on • Carrying objects Compliant grippers • • When is this desirable? • Placing objects TelemanipulaFon • • Dangerous workspaces • Handoff Scaled manipulaFon • • Human-intractable workspaces • Boring, repeFFve, unpleasant work • Physical assistance Human manipulaFon • • Carrying, transfer, And in HRI? • • CollaboraFon • feeding, … Grasps Grasps Drinking Drinking Grasps vary by: • Grasp: • Hand (gripper) • A set of contact points on an object’s surface • Object being grasped • Goal: constrain object’s movement • • Topology, topography, mass, surface, … Type of moFon desired • For each hand or • Tool use ool use hand/object pair: Where to grasp it? • How hard? • Then what? • AddiFonal constraints (e.g., don’t spill) • www www.intechopen.com .intechopen.com/books/r /books/robot-ar obot-arms/r ms/robotic-grasping-of-unknown-objects1 obotic-grasping-of-unknown-objects1 www www.madry .madry.pr .pro news.nationalgeographic.com news.nationalgeographic.com/news news/2009/05/090505-r /2009/05/090505-robot-hand-pictur obot-hand-picture.html e.html León, Morales, Sancho-Bru León, Morales, Sancho- Bru. Robot Grasping Foundations. 2013 . Robot Grasping Foundations. 2013 The Grasping Problem Grasp Planning Grasps are not obvious (easy to calculate) • Grasp synthesis: Find suitable set • Any given object has arbitrary contact points • of contacts, given: Hand has geometry constraints, etc. • Object model • Constraints on allowable contacts • Synthesized trial-and-error • For a hand/object pair: Grasp points are determined • • Mostly assume point contacts • Different grasp types planned and analyzed • Larger areas usually discreFzed • Contact model defines the force the • manipulator exerts on contact areas Grasp analysis • Is that grasp stable? • Real trial and error • www.cs.columbia.edu/~cmatei/ www .cs.columbia.edu/~cmatei/graspit graspit/ / www www.pr .programmingvision.com/r ogrammingvision.com/resear esearch.html ch.html León, Morales, Sancho-Bru León, Morales, Sancho- Bru. Robot Grasping Foundations. 2013. . Robot Grasping Foundations. 2013. www.cc.gatech.edu www .cc.gatech.edu/gvu gvu/people/faculty/ /people/faculty/nancy nancy.pollar .pollard/grasp.html grasp.html www www.intechopen.com .intechopen.com/books/r /books/robot-ar obot-arms/r ms/robotic-grasping-of-unknown-objects1 obotic-grasping-of-unknown-objects1 2

3/22/19 Robocup@Home Prismatic Impactive Gripping www.robocupathome.org Takes place in simulated kitchen & living room • Examples of past tasks: • Speech and Person Recogni(on: idenFfy unknown people and answer quesFons • about them and the environment Cocktail Party: learn and recognize previously unknown people, fetch orders • Help-me-carry: Help bring groceries into the home from outside • Storing Groceries: Storing new groceries in the cupboard next to objects of the • same kind that are already there Dishwasher Challenge: remove all dishes from a table and put in dishwasher • Restaurant: two r obots move within and environment to handle human requests, • such as delivering drinks or snacks, while people are walking around ∃ several different “RoboCup” compeFFons/challenge areas • https://www.youtube.com/watch?v=qKZLx1wtFCk Soft Impactive Gripping Soft Pneumatic Impactive Gripping “…relies on two kinds of soft robot technology: pneumatics and dielectric elastomer actuators.” � [Science Magazine, Jan. 2018] https://www.youtube.com/watch?v=qPVt0bZtNAM https://www.youtube.com/watch?v=gI0tzsO8xwc And Then There’s This Reactive Gripping youtu.be/0d4f8fEysf8 React to sensaFon of touching something • Rotate to grasp • ~86% success on tennis ball High-fricFon, compliant fingers • Minimal sensor suite: IMUs only (wow!) • Works with human-shaped hands only • ArFfact of data collecFon • Don’t consider objects from above • Tested detecFon of objects from above by hi`ng it with a • wrench; 88% of the Fme it read as a whack from above 3

3/22/19 Example Reactive Grasp Data and Applications Data gathering: IMU glove, wrist locaFon via • markers, hapFc feedback First contact only with distal phalanges (fingerFps) – • is that realisFc? 15 seconds hold?? • No feedback for • unexpected events Not usually a • good HRI assumpFon Simplifications Reactive Primitives for Soft Hands Reject baseline wrist pose • Assumes no “jerk” in movement • Measure wrist 3d rotaFon, but the controller only • allows roll How is this reacFve? • Doesn’t handle addiFonal constraints (torque, • direcFon, weight) Is this really handover? • https://www.youtube.com/watch?v=N03WTBK5eMw Multimodal Shared Autonomy Pupillary Response Try to use eye gaze to predict human intent Pupils suggest joysFck is • • more (mental) effort SubstanFal data-gathering experiments • Variance high • Eyes, pupillary response • JoysFck control • Affected by blinking, • brief unrelated Heavy quality filtering • saccades, peripheral Of course, can’t filter • vision out your actual users Gaze tracking is super • hard 4

3/22/19 Eye Movements Conclusions “Planning glances”: Not moving joysFck These are noisy signals! • • “Monitoring glances”: There is useful informaFon in gaze • • Checking status Gaze and control are not randomly distributed • Mostly while moving • Can take advantage of those pakerns • the arm, not rotaFng Robot behavior has an effect on all modaliFes • Pakerns about what • kinds of gaze are associated with what acFons Discussion What is the problem addressed? • What is the approach they take? • Did it work? • Did they choose good experiments / metrics? • What are their conclusions? • What do we think of this paper? • 5