object recognition self-supervised strategies kinematics motion Human Robot Interaction is a new and natural application for visual recognition In robotics settings strong cues are often available, therefore object detectors can be avoided Recognition as tool for complex tasks: grasp, manipulation, affordances, pose S.R. Fanello, C. Ciliberto, L. Natale, G. Metta – Weakly Supervised Strategies for Natural Object Recognition in Robotics, ICRA 2013 C. Ciliberto, S.R. Fanello, M. Santoro, L. Natale, G. Metta, L. Rosasco - On the Impact of Learning Hierarchical Representations for Visual Recognition in Robotics, IROS 2013 6/25/2015 53
6/25/2015 54
dataset 6/25/2015 55
iCubWorld dataset (2.0) • Growing dataset collecting images from a real robotic setting • Provide the community with a tool for benchmarking visual recognition systems in robotics • 28 Objects, 7 categories, 4 sessions of acquisition (four different days ) • 11Hz acquisition frequency. • ~50K Images http://www.iit.it/en/projects/data-sets.html 6/25/2015 56
methods 6/25/2015 57
methods Sparse Coding [yang et al. ’09] Sparse Coding [yang et al. ’09] Learned on iCubWorld Learned on iCubWorld Overfeat [sermanet et al. ’14] Overfeat [sermanet et al. ’14] Learned on Imagenet Learned on Imagenet 6/25/2015 58
some questions • Scalability. How do iCub recognition capability decrease as we add more objects to distinguish? • Can we use assumptions on physical continuity to make recognition more stable? • Incremental Learning. How does learning during multiple sessions affect the system recognition skills? • Generalization. How well does the system recognize objects “seen” under different settings? 6/25/2015 59
first results We started by addressing ins instance ance recognition cognition 6/25/2015 60
performance wrt # of objects 6/25/2015 61
exploiting continuity in time 6/25/2015 62
incremental learning Cumulative learning on the 4 days of acquisition. Tested on: accuracy (avg. over classes) • Presen t: test on current day 1 • Causal : test on current and 0.95 0.9 past days 0.85 • Future : test on future days 0.8 (current not included) OF present 0.75 OF causal • Independent : train & test on OF future 0.7 OF Independent current day only 0.65 1 2 3 4 day 6/25/2015 63
what about a week? accuracy (avg. over classes) 1 ? 0.95 0.9 0.85 0.8 OF present 0.75 OF causal OF future 0.7 OF Independent 0.65 1 2 3 4 5 6 7 day 6/25/2015 64
… or a month? accuracy (avg. over classes) 1 0.95 ?? 0.9 0.85 0.8 OF present 0.75 OF causal OF future 0.7 OF Independent 0.65 1 6 11 16 21 26 30 day 6/25/2015 65
6/25/2015 66
6/25/2015 67
Experiments on affordances 6/25/2015 68
Experiments on affordances 6/25/2015 69
Experiments on affordances 6/25/2015 70
Experiments on affordances 6/25/2015 71
6/25/2015 72
3D vision for grasping Input Segmentation Disparity 6/25/2015 73
grasping segmentation point cloud from stereo surface extraction minimum bounding box grasp selection from experience grasp points from curvature + bias (task dependent) Gori,I; Pattacini, U; Tikhanoff, V; Metta G; (submitted 2013) Ranking the Good Points: A Comprehensive Method for Humanoid Robots to Grasp Unknown Objects 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2013) 6/25/2015 74
6/25/2015 75
force reconstruction 6/25/2015 76
raw data 6/25/2015 77
GP approximation 6/25/2015 78
6/25/2015 79
From: Graziano 1999 From: Graziano et al. 2006 from : Fogassi L., Gallese V., Fadiga L., Luppino G., Matelli M., Rizzolatti G. Coding of peripersonal space in inferior premotor cortex (area F4) . Journal of Neurophysiology 76 (1) 1996. 6/25/2015 81
spinal reflexes walking behavior: cat rehabilitated to walk after complete spinal cord transection wiping reflex: an irritating stimulus elicits a wiping movement precisely directed at the stimulus location from: Poppele, R., & Bosco, G. (2003). Sophisticated spinal contributions to motor control. Trends in Neurosciences, 26 (5), 269-276. 6/25/2015 82
visuo-tactile fusion 6/25/2015 83
double touch From two fixed-base chain to a single floating-base serial chain 12 dof 6/25/2015 84
6/25/2015 85
6/25/2015 86
receptive fields Receptive field: a cone that extends up to 0.2m and angle of 40 • 6/25/2015 87
single taxel model 6/25/2015 88
no noise 6/25/2015 89
noisy data 6/25/2015 90
with double touch 6/25/2015 91
visual tracker 6/25/2015 92
external stimulation 6/25/2015 93
6/25/2015 94
extending peripersonal space 6/25/2015 95
6/25/2015 96
6/25/2015 97
6/25/2015 98
what future? iCub new tech iCub3.0 iCub2.0 now the future? 6/25/2015 99
6/25/2015 100
6/25/2015 101
“How old are you?” she wanted to know. “Thirty-two,” I said. “Then you don’t remember a world without robots. There was a time when humanity faced the universe alone and without a friend. Now he has creatures to help him; stronger creatures than himself, more faithful, more useful, and absolutely devoted to him. Mankind is no longer alone. Have you ever thought of it that way?” 6/25/2015 102
Recommend
More recommend
