Learning on Humanoid Robots Vadym Gryshchuk 19.11.2018 Outline - PowerPoint PPT Presentation

Neural Architectures for Lif ifelong Learning on Humanoid Robots Vadym Gryshchuk 19.11.2018

Outline • Motivation • Background • Approaches • Results • Discussion • Conclusion Neural Architectures for Lifelong Learning on Humanoid Robots 2

• Motivation • Background • Approaches • Results • Discussion • Conclusion Neural Architectures for Lifelong Learning on Humanoid Robots 3

What is is Lif ifelong Learning? • Continual acquisition of knowledge • Fine-tuning of knowledge • Learning from experiences • Retaining of previously learnt experiences Figure 1.1: NICO – Neuro-Inspired COmpanion (Source: Kerzel et al. [2]). Neural Architectures for Lifelong Learning on Humanoid Robots 4

Catastrophic Forgetting • Interference of learnt representations with new information Representation 2 Representation 1 Neural Architectures for Lifelong Learning on Humanoid Robots 5

In Inspiration from Biological Systems • Neurosynaptic plasticity • Hippocampus and cerebral cortex • Transfer learning • Intrinsic motivation • Crossmodal learning • Incremental learning Neural Architectures for Lifelong Learning on Humanoid Robots 6

• Motivation • Background • Approaches • Results • Discussion • Conclusion Neural Architectures for Lifelong Learning on Humanoid Robots 7

Neural Networks Figure 2.1: Neural network representation (Source: McDonald [3]). Neural Architectures for Lifelong Learning on Humanoid Robots 8

Convolutional Neural l Networks (C (CNNs) Figure 2.2: Convolutional neural network (Source: Cavaioni [1]) Neural Architectures for Lifelong Learning on Humanoid Robots 9

Self lf-Organizing Networks • Self-Organizing Map (SOM) • Grow When Required Network (GWR Network) • Recurrent GWR Neural Architectures for Lifelong Learning on Humanoid Robots 10

• Motivation • Background • Approaches • Results • Discussion • Conclusion Neural Architectures for Lifelong Learning on Humanoid Robots 11

Object Recognition: CNN + Cla lassifier • Learning from video sequences • Visual transformations of objects • Changing environment Figure 3.1: iCub (Source: Pasquale et al. [6]). Neural Architectures for Lifelong Learning on Humanoid Robots 12

Object Recognition: CNN + Classifier apple ball CNN tomato cup Classifier Neural Architectures for Lifelong Learning on Humanoid Robots 13

iC iCub: Object Learning Source: https://www.youtube.com/watch?v=ghUFweqm7W8 Neural Architectures for Lifelong Learning on Humanoid Robots 14

iCub: Object Learning Source: https://www.youtube.com/watch?v=ghUFweqm7W8 Neural Architectures for Lifelong Learning on Humanoid Robots 15

iCub: Object Learning Source: https://www.youtube.com/watch?v=ghUFweqm7W8 Neural Architectures for Lifelong Learning on Humanoid Robots 16

iCub: Object Learning Source: https://www.youtube.com/watch?v=ghUFweqm7W8 Neural Architectures for Lifelong Learning on Humanoid Robots 17

iCub: Object Learning Source: https://www.youtube.com/watch?v=ghUFweqm7W8 Neural Architectures for Lifelong Learning on Humanoid Robots 18

iCub: Object Learning Source: https://www.youtube.com/watch?v=ghUFweqm7W8 Neural Architectures for Lifelong Learning on Humanoid Robots 19

Sensorimotor Learning: Self lf-Organization • Latency in sensorimotor systems • Predictive mechanisms for future motor states • Online learning Source: https://upload.wikimedia.org/wikipedia/commons/ 4/47/Nao_Robot_%28Robocup_2016%29.jpg Neural Architectures for Lifelong Learning on Humanoid Robots 20

Sensorimotor Learning: Self-Organization Figure 3.2: The imitation scenario (Source: Mici et al. [4]). Neural Architectures for Lifelong Learning on Humanoid Robots 21

Sensorimotor Learning: Self-Organization Figure 3.3: Visuomotor learning (Source: Mici et al. [4]). Neural Architectures for Lifelong Learning on Humanoid Robots 22

Object Recognition: CNN + Self-Organization Pre-trained CNN RGB sequence Features Self-organizing network Depth Pre-trained CNN sequence Label Figure 3.4: Recognition pipeline (Adapted from Part et al. [5]). Neural Architectures for Lifelong Learning on Humanoid Robots 23

• Motivation • Background • Approaches • Results • Discussion • Conclusion Neural Architectures for Lifelong Learning on Humanoid Robots 24

Object Recognition: CNN + Cla lassifier Figure 4.1: Classification accuracy of the model, which was trained on an incremental number of objects (Source: Pasquale et al. [6]). Neural Architectures for Lifelong Learning on Humanoid Robots 25

Object Recognition: CNN + Cla lassifier Figure 4.2: Classification accuracy of the model trained incrementally on different days (Source: Pasquale et al. [6]). Neural Architectures for Lifelong Learning on Humanoid Robots 26

Sensorimotor Learning: : Self-Organizing Architecture Figure 4.3: Behaviour of the architecture (Source: Mici et al. [4]). Neural Architectures for Lifelong Learning on Humanoid Robots 27

Object Recognition: CNN + Self-Organization Figure 4.4: Recognition pipeline (Source: Part et al. [5]). Neural Architectures for Lifelong Learning on Humanoid Robots 28

Dis iscussion • CNN + Classifier architecture for object recognition: • Features extracted from a CNN are dependent on a dataset the model was trained on • Old representations are overwritten by the new information • Self-organizing architecture for sensorimotor learning: • Incremental online learning and prediction • Unreliability of visual body tracking framework in complex body positions • CNN + self-organization for object recognition: • Self-organizing network grows when required • Temporal context is not considered Neural Architectures for Lifelong Learning on Humanoid Robots 29

Conclusion • Lifelong learning is crucial for intelligent robots • Biological systems provide a basis for the incremental learning • Self-organizing networks preserve the topology • CNNs learn efficient feature descriptors • Catastrophic forgetting increases during incremental tasks Neural Architectures for Lifelong Learning on Humanoid Robots 30

Thank You! Questions? Neural Architectures for Lifelong Learning on Humanoid Robots 31

References • [1] Cavaioni, M. Deep Learning series: Convolutional Neural Networks. https://medium.com/machine- learning- bites/deeplearning-series-convolutional-neural-networks-a9c2f2ee1524 . [Online; accessed 13-November-2018]. • [2] Kerzel, M., Strahl, E., Magg, S., Navarro-Guerrero, N., Heinrich, S., Wermter, S.NICO - neuro-inspired companion: A developmental humanoid robot platform for multimodal interaction. In26th IEEE International Symposium on Robot and Human Interactive Communication, ROMAN 2017, Lisbon, Portugal, August 28 - Sept. 1, 2017, pages 113 – 120, 2017. • [3] McDonald, C. Machine learning fundamentals (II): Neural networks. https://towardsdatascience.com/machine- learning-fundamentals-ii-neural-networks-f1e7b2cb3eef. [Online; accessed 13-November-2018]. • [4] Mici, L., Parisi, I. G., Wermter, S. An Incremental Self-Organizing Architecture for Sensorimotor Learning and Prediction. CoRR, abs/1712.08521, 2017. • [5] Part l. J., Lemon, O. Incremental online learning of objects for robots operating in real environments. In Joint IEEE International Conference on Development and Learning and Epigenetic Robotics, ICDL-EpiRob 2017, Lisbon, Portugal, September 18-21, 2017, pages 304 – 310, 2017. • [6] Pasquale, G., Ciliberto, C., Odone, F., Rosasco, L., Natale, L. Real-world object recognition with off-the-shelf deep conv nets: How many objects can iCub learn? CoRR, abs/1504.03154, 2015. Neural Architectures for Lifelong Learning on Humanoid Robots 32