Ongoing Emergence: A Core Concept in Epigenetic Robotics Christopher G. Prince University of Minnesota Duluth, Dept. of Computer Science Nathan A. Helder George J. Hollich Purdue University, Dept. of Psychological Sciences http://www.cprince.com/PubRes/EpiRob05
Acknowledgments Lakshmi Gogate, Eric Mislivec John Weng Supported in part by UROP grants and a donation from Digi-Key Corp. http://www.cprince.com/PubRes/EpiRob05 2
Outline Motivation Infant example, goals review Criteria for ongoing emergence Evaluation of recent robotics projects Bootstrapping & tasks Conclusions http://www.cprince.com/PubRes/EpiRob05 3
Ongoing Emergence Example cases : Infant locomotion, infant word learning, and infant object skills Each show progressive development of skills http://www.cprince.com/PubRes/EpiRob05 4
Locomotion skills (Adolph, 2005) Postures: sitting, crawling, cruising, walking Age & order of acquisition highly variable Skill developments occur gradually, over weeks of experience with the particular posture Adaptive responding depends on infants’ experience with the specific posture http://www.cprince.com/PubRes/EpiRob05 5
Crawling (Adolph et al., 2000, 2003; Adolph, 2005; Berger & Adolph, 2003) 9 mo 9 mo http://www.cprince.com/PubRes/EpiRob05 6
Cruising (Adolph et al., 2000, 2003; Adolph, 2005; Berger & Adolph, 2003) 11 mo http://www.cprince.com/PubRes/EpiRob05 7
(Adolph et al., 2000, 2003; First Steps Adolph, 2005; Berger & Adolph, 2003) 12 mo? 14 mo? http://www.cprince.com/PubRes/EpiRob05 8
Infants vs. Robots Infants clearly display progressive developments of skills Potential for achieving ongoing emergence is a core reason why epigenetic robotics is interesting So, propose ongoing emergence as a core concept in epigenetic robotics http://www.cprince.com/PubRes/EpiRob05 9
Goals of Recent Projects Blank, Kumar, Meeden, & Marshall (2005) Dominey & Boucher (2005) Oudeyer, Kaplan, Hafner, & Whyte (2005) Weng and colleagues (2001, 2004; Chen & Weng, 2004) http://www.cprince.com/PubRes/EpiRob05 10
Blank et al. (2005) “allow a mobile robot to incrementally progress through levels of increasingly sophisticated behavior” Developmental algorithms involve: Abstraction : to focus agent on relevant environmental features Anticipation : enable prediction of environmental change Self-motivation : pushes system to develop more complex abstractions & anticipations 11
Dominey & Boucher(2005) “successive emergence of behaviors in a developmental progression of increasing processing power and complexity” “From simple representations such as contact, support, and attachment … the infant [may]construct progressively more elaborate representations of visuospatial meaning” (p. 244; see also Mandler, 1999) 12 http://www.cprince.com/PubRes/EpiRob05
Oudeyer, Kaplan, Hafner, & Whyte (2005) Progressive increases in complexity of activities & capabilities Autonomy, self-construction of development sequences, & intrinsic motivation (e.g., play) 13 http://www.cprince.com/PubRes/EpiRob05
Weng et al. Autonomous construction of representations for previously unknown knowledge and skills (Weng et al., 2001) Agents may thus “select rules when new situations arise, e.g. in uncontrolled environments” (p. 205, Weng, 2004). Open ended and cumulative learning of complex skills by first learning simpler skills (Weng et al., 2001) 14 http://www.cprince.com/PubRes/EpiRob05
How can we make progress? 1. Create robotic systems that attempt to exhibit ongoing emergence 2. Analyze existing systems to see how they fare, and see what is present, see what is missing http://www.cprince.com/PubRes/EpiRob05 15
Criteria Requirements for an epigenetic robot to be seen as exhibiting ongoing emergence 16 http://www.cprince.com/PubRes/EpiRob05
Criterion 1 Agent creates (acquires) behaviors, representations, and perceptual capacities Call these “skills” Agent creates new skills based on current skill repertoire, physical & environmental resources Properties of these new skills include May not yet be independent components May not be usable to create further, new skills 17 http://www.cprince.com/PubRes/EpiRob05
Issue: Indefinite Skill Progress Don’t want skill creation to stop In some sense, we want an indefinite progression of skills But, how can that be measured? How can you tell if development is open-ended? In a finite experiment, how can you make sure that skill development doesn’t stop? 18 http://www.cprince.com/PubRes/EpiRob05
Criterion 2 Make new skills part of repertoire of agent’s skills New skills need to be incorporated into this repertoire Properties of incorporated skills include Is now a separate component, to some degree Can be used to create new skills, with primitive or other incorporated skills 19 http://www.cprince.com/PubRes/EpiRob05
Criterion 3 New environments may have new tasks Perception (e.g., vision), and behaviors may vary depending on these tasks Issue: Why should the agent solve these tasks? For example, why should an agent that develops skills to play soccer also learn to swim or learn social skills? The agent needs to be self-motivated Need semi-autonomous development of values & goals (motivations) http://www.cprince.com/PubRes/EpiRob05 20
Criteria 4, 5, & 6 Skills need to start somewhere Criterion 4: Bootstrapping, i.e., when system starts, some skills rapidly become available Some skill invariance is needed to create new skills based on earlier skills Criterion 5: Stability, i.e., skills persist over time Characterization of scope of behaviors Criterion 6: Reproducibility, i.e., robotic system started in similar initial states and in similar environments should produce similar effects http://www.cprince.com/PubRes/EpiRob05 21
Criteria for Ongoing Emergence 1) New skill creation 2) Skills incorporated into repertoire 3) Autonomous development of motivations 4) Bootstrapping 5) Stability 6) Reproducibility 22 http://www.cprince.com/PubRes/EpiRob05
Systems & Evaluation Scale Meaning Notation Score Achieved criterion + 2 Partially achieved ? 1 criterion Criterion not – 0 achieved 23 http://www.cprince.com/PubRes/EpiRob05
Blank et al. (2005) Connectionist models Sensor prototypes (Linaker, & Niklasson, 2000) Self-organizing maps Tasks: Wall following & navigation to goal locations http://www.cprince.com/PubRes/EpiRob05 24
Evaluation New skill creation (+) Learning wall following & navigation to locations (–) Independent sensory Skill incorporation prototypes incrementally added; otherwise 1 phase of learning Autonomous (?) Self-organizing maps specify development of goals in location navigation motivations (+) Translation & rotation Bootstrapping movements; sonar readings Stability (?) Apparently Reproducibility (+) 10 wall following simulations http://www.cprince.com/PubRes/EpiRob05 25
Dominey & Boucher (2005) Visual & auditory input Connectionist models learn syntax, based on closed class words (e.g., “a”, “the”) Generalization to new sentences with same syntax Extension to new tasks by modification of architecture 26 http://www.cprince.com/PubRes/EpiRob05
Evaluation New skill creation (+) Models acquire syntax Skill incorporation (–) Only 1 phase of learning Autonomous development of (–) No explicit value system motivations (+) Built-in speech and object Bootstrapping recognition (+) Ceiling performance & Stability generalization on most tasks (+) Combined tasks; models seem Reproducibility to have limited random elements http://www.cprince.com/PubRes/EpiRob05 27
Oudeyer et al. (2005) Sensorimotor vectors used as input (8D) Learning ‘expert’ per vector space partition f: SM -> S Rate of expert learning used to select actions http://www.cprince.com/PubRes/EpiRob05 28
Details on Results Phase 1: exploration, body babbling Phase 2: looking, but not finding objects Phase 3: biting & bashing, but not oriented to objects Time step: 3-4 seconds http://www.cprince.com/PubRes/EpiRob05 29
Evaluation New skill creation (+) Robot learns various relations between motors & sensors Skill incorporation (?) Can new skills be used in creating other skills? Autonomous (+) Rate of learning used to development of predict sensory input motivations (+) (M) biting, bashing, head turn; Bootstrapping (S) object, biting, oscillation Stability (?) Within phases (?) Similar developmental Reproducibility sequences in several experiments; sequences never exactly the same 30
Chen & Weng (2004) “Drawbridge” task (Baillargeon et al., 1985) IHDR learning (Weng & Hwang, 2003) Prediction of next sensory inputs Novelty computed from distance between actual sensory input and predicted input Higher novelty means higher action reinforcement 31 http://www.cprince.com/PubRes/EpiRob05
Recommend
More recommend
