BADGr: A Toolbox for Box-based Approximation, Decomposition and - PowerPoint PPT Presentation

BADGr: A Toolbox for Box-based Approximation, Decomposition and Grasping Kai Huebner khubner@kth.se KTH – Royal Institute of Technology, Stockholm, Sweden International Conference on Intelligent RObots and Systems 2010 Slide 1 Workshop on “Grasp Planning and Task Learning by Imitation”

Overview Overview BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Motivation Boxgrasping Framework Overview BADGr System Architecture Conclusion International Conference on Intelligent RObots and Systems 2010 Slide 2 Workshop on “Grasp Planning and Task Learning by Imitation”

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Motivation EU Project PACO-PLUS EU project PACO-PLUS : Perception, Action and Cognition through Learning of Object-Action-Complexes (February 2006 - July 2010). Focus on formalizing the interplay between objects and actions . Describe an object not by its appearance only, but also by which actions it allows. International Conference on Intelligent RObots and Systems 2010 Slide 3 Workshop on “Grasp Planning and Task Learning by Imitation”

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Motivation Interactions with Objects Problem: before actions can be employed or analyzed on a higher-level, the object must already be interacted with on a lower level, through grasping . Grasping is an important module in a number of robot applications. (4 Grasping sessions = 22 talks at IROS 2010 conference) International Conference on Intelligent RObots and Systems 2010 Slide 4 Workshop on “Grasp Planning and Task Learning by Imitation”

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Motivation Grasping A good grasp is classically defined as stable object-gripper situation such that the object can be successfully lifted for further manipulation . Thus, grasp planners have to take into account a number of properties of the object (e.g. shape) and the action (e.g. hand kinematics) and more [1]. [1] Song et al ., IROS 2010, Grasping II, TuET2.4. International Conference on Intelligent RObots and Systems 2010 Slide 5 Workshop on “Grasp Planning and Task Learning by Imitation”

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Motivation Object Shape Representations From a sensor point of view, an object can be described to a reference, or to its pure appearance, i.e. 2D or 3D features, whereof one is 3D shape . Even with the simple representation of box constellations we can access rough shape, size, pose, task, not only of objects, but also object parts [2]. [2] Huebner & Kragic, IROS 2008. International Conference on Intelligent RObots and Systems 2010 Slide 6 Workshop on “Grasp Planning and Task Learning by Imitation”

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Boxgrasping Framework Overview Framework Overview 3D Shape Approximation Geometrical Heuristics Box Approximation (Task, Occlusion, Reachability, ...) Box Faces ⇒ Grasp Hypotheses Heuristical Validity Checks 3D Point Bounding Pre-Grasp Cloud Box Box Decomposition Huebner et al . [3] Huebner & Kragic [2] From this motivation, we developed a flexible framework enabling box approximation , decomposition , and grasp hypotheses generation , ◮ taking a point cloud and generating a restricted set of pre-grasps , ◮ where input can be from any source providing basic 3D data, and ◮ output are also shape representations besides grasp poses. International Conference on Intelligent RObots and Systems 2010 Slide 7 Workshop on “Grasp Planning and Task Learning by Imitation”

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Boxgrasping Framework Overview Framework Overview Graphical Modeling 3D Shape Approximation Geometrical Heuristics Sim. Box 3D Model Scene Approximation (Task, Occlusion, Reachability, ...) Box Faces ⇒ Grasp Hypotheses Heuristical Validity Checks Object 3D Real 3D Point Bounding Pre-Grasp Scene Recognition Segment Cloud Box 2D/3D Scene Box Decomposition Processor Machine Vision Rasolzadeh et al . [4] Huebner et al . [5] Huebner et al . [3] Huebner & Kragic [2] From this motivation, we developed a flexible framework enabling box approximation , decomposition , and grasp hypotheses generation , ◮ taking a point cloud and generating a restricted set of pre-grasps , ◮ where input can be from any source providing basic 3D data, and ◮ output are also shape representations besides grasp poses. International Conference on Intelligent RObots and Systems 2010 Slide 7 Workshop on “Grasp Planning and Task Learning by Imitation”

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Boxgrasping Framework Overview Framework Overview Graphical Modeling 3D Shape Approximation Geometrical Heuristics Method Evaluation (Justification) Evaluation Sim. Box 3D Model Scene Approximation (Task, Occlusion, Reachability, ...) Box Faces ⇒ Grasp Hypotheses Huebner et al . [3] Heuristical Validity Checks Stability Learning Object 3D Neural Network Learning Real 3D Point Bounding Pre-Grasp Pre-Grasp Scene Recognition Segment Cloud Box (Stability) Geidenstam et al . [6] 2D/3D Scene Box Task Constraint Learning Decomposition Processor Bayesian Network Learning Pre-Grasp (Task) Machine Vision Rasolzadeh et al . [4] Song et al . [1] Huebner et al . [5] Huebner et al . [3] Huebner & Kragic [2] From this motivation, we developed a flexible framework enabling box approximation , decomposition , and grasp hypotheses generation , ◮ taking a point cloud and generating a restricted set of pre-grasps , ◮ where input can be from any source providing basic 3D data, and ◮ output are also shape representations besides grasp poses. International Conference on Intelligent RObots and Systems 2010 Slide 7 Workshop on “Grasp Planning and Task Learning by Imitation”

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping Boxgrasping Framework Overview Framework Overview Graphical Modeling 3D Shape Approximation Geometrical Heuristics Method Evaluation (Justification) Evaluation Box Sim. 3D Model Scene Approximation Box Faces ⇒ Grasp Hypotheses (Task, Occlusion, Reachability, ...) Huebner et al . [3] Heuristical Validity Checks Stability Learning Real Object 3D 3D Point Bounding Neural Network Learning Pre-Grasp Pre-Grasp Recognition Segment (Stability) Scene Cloud Box Geidenstam et al . [6] Task Constraint Learning 2D/3D Scene Box Processor Decomposition Bayesian Network Learning Pre-Grasp (Task) Machine Vision Rasolzadeh et al . [4] Song et al . [1] Huebner et al . [5] Huebner et al . [3] Huebner & Kragic [2] 2008 Implementation of box approximation & decomposition (Huebner et al ., ICRA) Implementation of pre-grasp selection strategies (Huebner & Kragic, IROS) 2009 Integration on Armar-III, KIT (Huebner et al ., ICAR) Integration and learning of 2D grasping strategies (Geidenstam et al ., RSS) 2010 Application for task-constraint learning (Song et al ., IROS) Publication of open source software package BADGr (Huebner, IROS-WS) International Conference on Intelligent RObots and Systems 2010 Slide 7 Workshop on “Grasp Planning and Task Learning by Imitation”

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping BADGr System Architecture System Architecture BoxGrasping fit (( Point (( Bound Cloud )) Box )) split (( Pre- Grasp )) generate ◮ Input: object model, configurations, [gripper model, interface] ◮ Output: box shape representations, pre-grasp representations ◮ Tools: BoxGrasping, BoxXmlViewer, TaskLabeling http://www.csc.kth.se/~khubner/badgr/ International Conference on Intelligent RObots and Systems 2010 Slide 8 Workshop on “Grasp Planning and Task Learning by Imitation”

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping BADGr System Architecture System Architecture < Configuration > setup parametrize < Gripper > BoxGrasping fit (( Point (( Bound Shape (( Object )) Cloud )) Box )) < > split Features Expert (( Pre- Grasp )) generate Grasp < > Features ◮ Input: object model, configurations, [gripper model, interface] ◮ Output: box shape representations, pre-grasp representations ◮ Tools: BoxGrasping, BoxXmlViewer, TaskLabeling http://www.csc.kth.se/~khubner/badgr/ International Conference on Intelligent RObots and Systems 2010 Slide 8 Workshop on “Grasp Planning and Task Learning by Imitation”

BADGr: A Toolbox for Box-based Approximation, Decomposition and GRasping BADGr System Architecture System Architecture < Configuration > setup parametrize < Gripper > BoxGrasping BoxXmlViewer fit analyze (( Point (( Bound Shape (( Object )) Cloud )) Box )) < > split Features Expert User (( Pre- Grasp )) generate TaskLabeling Grasp Task (( )) < > Features Label provide ◮ Input: object model, configurations, [gripper model, interface] ◮ Output: box shape representations, pre-grasp representations ◮ Tools: BoxGrasping, BoxXmlViewer, TaskLabeling http://www.csc.kth.se/~khubner/badgr/ International Conference on Intelligent RObots and Systems 2010 Slide 8 Workshop on “Grasp Planning and Task Learning by Imitation”