Lifelong Learning in Optimisation Emma Hart Edinburgh Napier - PowerPoint PPT Presentation

Lifelong Learning in Optimisation Emma Hart Edinburgh Napier University http://jamesobrien.tumblr.com/post/1112777561/lifel ong-learning-illustration

Optimisation Algorithms Algorithm Algorithm Algorithm Algorithm …… Algorithm Generalist Specialist Tuned metaheuristics Evolution Strategies Offline hyper heuristics Online hyper heuristics …. Metaheuristics …

Optimisation Algorithms Algorithm Algorithm Algorithm Algorithm …… Algorithm Generalist Specialist • Generalist: incapable of adapting to new problem characteristics • Specialist: unable to learn from experience or exploit prior knowledge

Machine Learning • Contemporary ML “ it is now appropriate for systems usually exploit the AI community to move prior knowledge if faced beyond learning algorithms with new but similar to more seriously consider task systems that are capable of learning over a lifetime ” Silver, 2013 beneficial finding significant Benefits significant benefits Efficient ficiently benefits. ficient benefit ficiently

Machine Learning An LML should: What kind of approach 1. Retain and/or consolidate knowledge might provide these (long-term memory) features ? 2. Selectively transfer prior knowledge when learning new tasks 3. Adopt a systems approach that ensures effective and efficient interaction of elements of the system

Machine Learning An LML should: Natural Immune System 1. Retain and/or Basis of vaccination, can be consolidate knowledge very long term (long-term memory) 2. Selectively transfer prior knowledge when learning new tasks 3. Adopt a systems approach that ensures effective and efficient interaction of elements of the system

Machine Learning An LML should: Natural Immune System 1. Retain and/or Basis of vaccination, can be consolidate knowledge very long term (long-term memory) 2. Selectively transfer prior knowledge when learning new tasks 3. Adopt a systems approach that ensures effective and efficient interaction of elements of the system

Machine Learning An LML should: Natural Immune System 1. Retain and/or consolidate Selectively transfer prior knowledge (long-term knowledge when learning new memory) tasks 2. Selectively transfer prior knowledge when learning new tasks 3. Adopt a systems approach that ensures effective and efficient interaction of elements of the system

Machine Learning An LML should: Natural Immune System Behaviour is the result of many 1. Retain and/or interacting components consolidate knowledge (long-term memory) 2. Selectively transfer prior knowledge when learning new tasks 3. Adopt a systems approach that ensures effective and efficient interaction of elements of the system

The Role of the Immune Network • Immune cells interact with other and with antigen – Can be stimulatory or suppressive • Results in a network with dynamically changing topology • Useful cells recruited into network • Redundant ones rejected • Topology depends on past & current environment

Machine Learning An LML should: Natural Immune System 1. Retain and/or consolidate Gene recombination in bone marrow continually trials news knowledge (long-term cells leading to a useful repertoire memory) of antibodies 2. Selectively transfer prior knowledge when learning new tasks 3. Adopt a systems approach that ensures effective and efficient interaction of elements of the system 4. Generate new knowledge

Immune Systems Environment Computational Properties • Exploration Pathogens – randomly combining components from a library gives rise to many cells Network dynamics • Exploitation A4 A2 – focuses search on promising cells • Memory : – network provides a ‘map’ of the A1 antigen space A3 • Adaptable – Doubly plastic: parametric & structural • Diverse Bone marrow -> antibodies – Finite repertoire of cells has to ensure all pathogens recognised Meta-dynamics

Optimisation Systems Environment Computational Properties • Exploration – randomly combining components from a library gives rise to many heuristics • Exploitation Network dynamics – focuses search on promising heuristics • Memory : – network provides a ‘map’ of the problem space • Adaptable – Doubly plastic: parametric & structural to deal with changes in problem characteristics • Diverse – Finite repertoire of heuristics has to ensure all problems solved Meta-dynamics

Conceptual Overview • The network sustains heuristics that 2d Representation of problem space work best in distinct regions of the H1 instance space (diversity) H2 – Need to win to be in! • H4 The network sustains problems that are representative of areas of the problem space – Problems that are solved by more than H3 one heuristic are not ‘interesting’ • Problems & heuristics gain concentration through mutual stimulation – Decay mechanisms enable gradual forgetting problem instance – Lack of stimulation leads to removal • Topology of network changes over heuristic time depending on problems injected and heuristics generated

Conceptual Overview • The network sustains heuristics that 2d Representation of problem space work best in distinct regions of the H1 instance space (diversity) H2 – Need to win to be in! • The network sustains problems that are representative of areas of the problem space – Problems that are solved by more than one heuristic are not ‘interesting’ • Problems & heuristics gain concentration through mutual stimulation – Decay mechanisms enable gradual forgetting problem instance – Lack of stimulation leads to removal • Topology of network changes over heuristic time depending on problems injected and heuristics generated

NELLI – Ne twork for L ifeLong L earn i ng Environment Network dynamics ① Problem Stream ② Heuristic Generator ③ Network of heuristics & problems Meta-dynamics

NELLI – Ne twork for L ifeLong L earn i ng Environment ① Problem Stream Network dynamics At each iteration instances can be injected into the system – Single instance – Multiple instances – Frequent/infrequent Meta-dynamics

NELLI – Ne twork for L ifeLong L earn i ng Environment ② Heuristic Generator Network dynamics • Library of components s • Components can be ‘pre - defined’ or evolved • Components are combined into heuristic s Meta-dynamics

NELLI – Ne twork for L ifeLong L earn i ng Component Library ② Heuristic Generator R3 R5 R1 • Library of components R2 R4 • Components can be ‘pre - defined’ or evolved • Components are combined into R1 R3 R1 R5 Heuristics heuristic s • (few components -> lots R4 R5 of heuristics) R2 R3 R1 R4 R5 R1

NELLI – Ne twork for L ifeLong L earn i ng Bin packing Component Library ② Heuristic Generator FF DJD FFD • Library of components DJT SOS • Components can be ‘ pre-defined ’ or evolved • Components are R1 R3 R1 R5 combined into heuristic s R4 R5 R2 R3 R1 R4 R5 R1

NELLI – Ne twork for L ifeLong L earn i ng Job Shop Scheduling Component Library ② Heuristic Generator LPT SPT FIFO • Library of components SWT LWT • Components can be ‘ pre-defined ’ or evolved • Components are R1 R3 R1 R5 combined into heuristic s R4 R5 R2 R3 R1 R4 R5 R1

NELLI – Ne twork for L ifeLong L earn i ng Component Library ② Heuristic Generator R3 R5 R1 • Library of components R2 R4 • Components can be ‘pre - defined’ or evolved • Components are R1 R3 R1 R5 combined into heuristic s R4 R5 R2 R3 R1 R4 R5 R1

NELLI – Ne twork for L ifeLong L earn i ng ② Heuristic Generator Component Library Evolution R3 R5 R1 • Library of components R2 R4 • Components can be ‘pre - defined’ or evolved • Components are combined into R1 R3 R1 R5 heuristic s • Both components and Evolution R4 R5 heuristics can evolve R2 R3 R1 R4 R5 R1

NELLI – Ne twork for L ifeLong L earn i ng ② Heuristic Generator Evoluion • Library of components • Components can be ‘pre - defined’ or evolved • Components are combined into • Mutate terminal nodes • Mutate function nodes heuristic s • Remove subtree • Both components and • Swap subtrees heuristics can evolve

NELLI – Ne twork for L ifeLong L earn i ng R1 R3 R1 R2 R5 ② Heuristic Generator Evolution R4 R5 • Library of components • Components can be R2 R3 R1 R4 R5 R1 ‘pre - defined’ or evolved • Components are combined into Swap components heuristic s Change components • Both components and Remove/insert components Concatenate heuristics heuristics can evolve

NELLI – Ne twork for L ifeLong L earn i ng ② Heuristic Generator Evoluion • Library of components • Components can be ‘pre - defined’ or evolved • Components are combined into R1 R3 R1 R5 heuristic s • Both components and Evolution R4 R5 heuristics can evolve R2 R3 R1 R4 R5 R1

NELLI – Ne twork for L ifeLong L earn i ng Environment ③ Network • Heuristics are stimulated Network dynamics by winning at least one problem – The higher the win, the bigger the stimulation • Problems are stimulated if they are won by only one heuristic – The higher the win, the Meta-dynamics bigger the stimulation