how to win the Amazon Robotics Challenge T eam ACRV - PowerPoint PPT Presentation

Doug Morrison acrv | arc centre of excellence for robotic vision qut | queensland university of technology RoboticVisionAU how to win the Amazon Robotics Challenge T eam ACRV

roboticvision.org

#cartman Hardware 1.2m 1.2m 1.5m http://roboticvision.org/

Multi-modal End-Effector #cartman http://roboticvision.org/

In Action #cartman http://roboticvision.org/

Visual Perception HW #cartman http://roboticvision.org/

Unsupervised Approach #cartman Deep Metric Learning http://roboticvision.org/

Supervised Approach #cartman [Lin et al., CVPR ’17] http://roboticvision.org/

Data Collection #cartman http://roboticvision.org/

Data Collection Unseen #cartman http://roboticvision.org/

Quick Detour F0.5 = 63% F1 = 72% F 0.5 vs F 1 vs IOU in Robotic Applications IOU = 57% Precision = 57% F0.5 = 63% F1 = 72% IOU = 57% Precision = 57% F0.5 = 83% F0.5 = 63% F1 = 66% F1 = 72% IOU = 50% IOU = 57% F0.5 = 83% F1 = 66% Precision = 100% Precision = 57% IOU = 50% Precision = 100% F0.5 = 100% F0.5 = 83% F0.5 = 100% F1 = 100% F1 = 100% F1 = 66% IOU = 100% IOU = 100% IOU = 50% Precision = 100% Precision = 100% Precision = 100% http://roboticvision.org/ F0.5 = 100% F1 = 100% IOU = 100% Precision = 100%

Unsupervised vs Supervised #cartman http://roboticvision.org/

Training Data Needs #cartman http://roboticvision.org/

Perception Results #cartman http://roboticvision.org/

Perception in Clutter #cartman http://roboticvision.org/

#cartman unknown vs known http://roboticvision.org/

Grasping #cartman RGB Image Item Segment Grasp Ranking Grasp Output Surface Normals Good quality point clouds (e.g. regular, matt objects) No high quality grasps Point Cloud Centroid Incomplete point clouds 
 (e.g. reflective objects) Not enough valid points RGB Centroid no valid points for No valid point cloud 
 segment (e.g. clear or black objects) http://roboticvision.org/

Grasp Accuracy #cartman http://roboticvision.org/

Finals Scoring #cartman http://roboticvision.org/

Finals Run #cartman http://roboticvision.org/

System Robustness #cartman Active and Interactive Perception 
 Multiple viewpoints are used to help locate partially occluded items. If no wanted items are visible, the system will move objects within the storage system based on the likelihood that they are obscuring wanted items. Item Reclassification 
 Items can be reclassified to correct errors, based on consensus from two sensors (primary/ secondary visual classification and weight). Error Detection and Recovery 
 A number of sensors are used to detect failed grasps and dropped items. T oward the end of a task, visual classification is used to double-check that the location of items matches the internal state. http://roboticvision.org/

What Now? Real-time, Active and Reactive Grasping. … [Closing the Loop for Robotic Grasping, Morrison et al, RSS 2018]

Juxi The ACRV Picking Benchmark Jürgen ‘Juxi’ Leitner j.leitner@qut.edu.au #ICRA2017 http://Juxi.net/acrv-picking-benchmark/ overcome limita,ons of current robo,c system comparison 
 reproducible research on end-to-end TASKS 


Tidy Up My Room Challenge http://Juxi.net/challenge/tidy-up-my-room

#teamACRV RoboticVisionAU http://facebook.com/T eamACRV Adam Tow Doug Morrison Steve Martin Matt McTaggert Rohan Smith Zheyu Zhuang Jordan Erskine Norton Kelly-Boxall Anthony Gillespie Sean Wade-McCue Riccardo Grinover Thomas Rowntree Alec Gurman Trung Pham Tom Hunn Vijay Kumar Darryl Lee Ming Cai Nathan Perkins Saroj Weerasekera Gerard Rallos Chris Lehnert Andrew Razjigaev Anton Milan Juxi Leitner, Ian Reid, Peter Corke http://roboticvision.org/

Thanks! Juxi Leitner Doug Morrison <j.leitner@qut.edu.au> http://Juxi.net < douglas.morrison@hdr.qut.edu.au > Brisbane, Australia We are hiring 
 here in Brisbane at QUT! Come talk to me!