Model-free Approach to Garments Unfolding Based on Detection of Folded Layers Jan Stria, Vladim´ ır Petr´ ık, V´ aclav Hlav´ aˇ c Czech Institute of Informatics, Robotics and Cybernetics Czech Technical University in Prague Czech Republic 26 September 2017 1 / 14
Project CloPeMa (Clothes Perception and Manipulation) ◮ Funded by European Commission in FP7, 2012–2015 Czech Technical University , University of Genoa, ◮ CERTH, University of Glasgow ◮ Fully autonomous pipeline for garments classification , unfolding , flattening and folding using a dual-arm robot 2 / 14
Problem formulation ◮ Input: Unknown piece of garment folded once or more times with the top folded layers not overlapping ◮ Goal: Repeatedly detect one folded layer , estimate the folding axis and unfold it with two cooperating robotic arms. 3 / 14
Motivation ◮ Geometric unfolding 1 ◮ The garment is grasped and partially unfolded in the air. ◮ The remaining fold is detected after placing it on a table. 1 Dimitra Triantafyllou et al. “A geometric approach to robotic unfolding of garments”. In: Robotics and Autonomous Systems (2016) 4 / 14
Related work ◮ Foldable templates 1 ◮ Depth segmentation 2 ◮ Input: RGB image ◮ Input: Depth map ◮ Partial template fitting to ◮ Watershed segmentation ◮ Checking various the garment contour ◮ Folding axis generation unfolding directions 2 David Estevez et al. “Towards Robotic Garment Folding: A Vision Approach for Fold Detection”. In: Proc. ICARSC (2016) 5 / 14
Input ◮ RGBD sensor ASUS Xtion attached to the wrist is positioned above the garment and oriented downwards. ◮ Acquire single image and depth map of the garment. 1070 1030 990 950 [mm] 6 / 14
Layers detection 20 10 0 [mm] ◮ Layers detection is formulated as pixels labeling : p �→ z p ∈ { T , B } ◮ Combine information provided by the image and heights: ◮ Pixels from the top (folded) layer are higher above the table than the pixels from the bottom layer. ◮ The boundary between layers is coincident with image edges . 7 / 14
Energy minimization 1 ◮ Pixels labeling formulated as energy minimization problem: Z ∗ = � � arg min U p ( z p ) + V p , q ( z p , z q ) Z ∈{ T , B } | P | p ∈ P { p , q }∈ N ◮ Unary potential U p evaluates how the pixel height H ( p ) corresponds to the estimated mean height of the bottom layer µ B and the top layer µ T = 2 µ B : H ( p ); µ z p , σ 2 � U p ( z p ) = − log N � 15 12 9 6 3 0 Top Bottom 8 / 14
Energy minimization 2 ◮ Pairwise potential V p , q depends on the spatial d ( p , q ) and visual g ( I , p , q ) difference of the neighboring pixels : � z p � = z q � � − g ( I , p , q ) � V p , q ( z p , z q ) = γ 1 + γ 2 d ( p , q ) exp 2 E [ g ] ◮ Solved by finding the minimum cut of a specific graph. ◮ The largest top layer is chosen for unfolding. 9 / 14
Folding axis estimation ◮ The folding axis must form an approximate segment on the garment contour. ◮ The garment is unfolded virtually over each candidate folding axis and the best one is selected. 10 / 14
Robotic manipulation ◮ Cooperated manipulation of two robotic arms: ◮ One arm is holding the garment to prevent it from slipping. ◮ The other arm grasps the top layer and unfolds it. ◮ Test various grasping and holding candidates . ◮ The holding gripper follows a triangular unfolding path . 11 / 14
Vision experiments ◮ Dataset containing 13 garments of various types, colors and materials; each posed in 15 folded configurations Failure Garment Success Layers Axis Jacket 14 / 15 1 0 Jeans 12 / 15 3 0 Shorts 14 / 15 1 0 Skirt (2) 25 / 30 5 0 Sweater (2) 26 / 30 2 2 Sweatshirt 14 / 15 0 1 Towel 14 / 15 1 0 T-shirt (4) 51 / 60 9 0 Total 87% 11% 2% 12 / 14
Robotic experiments Reason of failure Garment Success Detection Planning Execution Shorts 3 / 5 1 1 0 Sweatshirt 4 / 5 1 0 0 Towel 5 / 5 0 0 0 T-shirt 5 / 5 0 0 0 Total 17 / 20 2 1 0 13 / 14
Thank you for your attention. Questions, please? http://bit.do/unfolding Czech Institute of Informatics, Robotics and Cybernetics Czech Technical University in Prague 14 / 14
Recommend
More recommend
