Population Based Training of Neural Networks M. Jaderberg, V. - PowerPoint PPT Presentation

Sep 19, 2022 •143 likes •317 views

Presented by: Devin Taylor Population Based Training of Neural Networks M. Jaderberg, V. Dalibard, S. Osindero, W.M. Czarnecki November 14, 2018 DeepMind, London, United Kingdom Problem Statement Problem statement Neural networks suffer from

Presented by: Devin Taylor Population Based Training of Neural Networks M. Jaderberg, V. Dalibard, S. Osindero, W.M. Czarnecki November 14, 2018 DeepMind, London, United Kingdom
Problem Statement Problem statement Neural networks suffer from sensitivity to empirical choices of hyperparameters Solution Asynchronous optimisation algorithm that jointly optimises a population of models 1
Key Idea Figure 1: Overview of proposed approach 2
Population Based Training - Algorithm • step - weight update • eval - performance evaluation • ready - current path limit • exploit - compare to population • explore - adjust hyperparameters Figure 2: PBT algorithm 3
Population Base Training - Core • exploit • Replace weights and/or hyperparameters • T-test selection, truncation selection, binary tournament • explore • Adjust hyperparameters • Perturb, resample Figure 3: PBT dummy example 4
Implementation Notes • Asynchronous • No centralised orchestrator • Only current performance information, weights, hyperparameters published • No synchronisation of population 5
Experiments Experiments conducted in three areas: • Deep reinforcement learning - Find policy to maximise expected episodic return • Neural machine translation - Convert sequence of words from one language to another • Generative adversarial networks - Generative models with competing components, generator and descriminator 6
Results - Spoiler Figure 4: PBT result summary 7
Results - Deep reinforcement learning Figure 5: PBT deep reinforcement learning result - DM Lab 8
Results - Machine translation Figure 6: PBT machine translation results 9
Results - Generative Adversarial Networks Figure 7: PBT GAN results 10
Analysis Figure 8: PBT design space analysis 11
Analysis Figure 9: PBT lineage analysis 12
Analysis Figure 10: PBT development as phylogenetic tree 13
Critique • No results showing evidence quite large requirements (10 workers) • Minimum computational • Is susceptible to local minima steps, perturb, etc) hyperparameters ( ready • Added in additional of reduced time Negatives Positives • Improved training stability for hyperparameter tuning • Approximate complex paths sacrificing on time • Result improvements without unanswered some questions left • Detailed analysis - although • Well written 14
Related Work • Unique genetic algorithm approach to implementation - parallel and sequential • Author: Max Jaderberg • Mix&Match: Agent Curricula for Reinforcement Learning - boostrapping off simpler agents 15
Conclusion • Presented algorithm that asynchronously and jointly optimises a population of models • Obtained improved results on a range of different algorithms • Still certain questions unanswered but still a good contribution 16

Recommend

TRAINING NEURAL TRAINING NEURAL NETWORKS ON THE NETWORKS ON THE EDGE EDGE Navjot Kukreja,

TRAINING NEURAL TRAINING NEURAL NETWORKS ON THE NETWORKS ON THE EDGE EDGE Navjot Kukreja, Alena Shilova Also: Olivier Beaumont Jan Huckelheim Nicola Ferrier Paul Hovland Gerard Gorman BACKGROUND BACKGROUND Typical data ow pattern

779 views • 35 slides

Neural Networks and their Application to Go Neural Networks Learning Blackjack Theory Training

Neural Networks and their Application to Go A. Bausch Neural Networks and their Application to Go Neural Networks Learning Blackjack Theory Training neural networks Problems AlphaGo Anne-Marie Bausch The Game of Go Policy Network

280 views • 24 slides

CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks

CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks Neuron Non-linearity Softmax layer DNN training Loss function and regularization SGD and backprop Learning rate Overfitting

1.8k views • 86 slides

Modern Systems for Neural Networks Valentin Dalibard This talk 1.Practicalities of training

Modern Systems for Neural Networks Valentin Dalibard This talk 1.Practicalities of training Neural Networks 2.Leveraging heterogeneous hardware Source: wikipedia Modern Neural Networks Applications: Image classification Modern Neural

513 views • 20 slides

Training Neural Networks CMSC 470 Marine Carpuat Neural Networks so far Powerful non-linear

Training Neural Networks CMSC 470 Marine Carpuat Neural Networks so far Powerful non-linear models for classification Predictions are made as a sequence of simple operations matrix-vector operations non-linear activation functions

417 views • 14 slides

Neural Networks Neural networks arise from attempts to model Neural Networks human/animal

Feed-forward Networks Network Training Error Backpropagation Deep Learning Feed-forward Networks Network Training Error Backpropagation Deep Learning Neural Networks Neural networks arise from attempts to model Neural Networks

380 views • 9 slides

Training Knowledge Bots for Physics Based Simulations Using Artificial Neural Networks Jay Ming

Introduction Knowledge Bot Trajectory Analysis Conclusion Training Knowledge Bots for Physics Based Simulations Using Artificial Neural Networks Jay Ming Wong Jamshid Samareh (Mentor) Department of Computer Science Vechicle Analysis Branch

504 views • 16 slides

Computa(on through dynamics Using recurrent neural networks to unveil mechanism in neural

Computa(on through dynamics Using recurrent neural networks to unveil mechanism in neural circuits David Sussillo with Valerio Mante and Bill Newsome Table of contents Introduc(on Training recurrent neural networks(RNNs) Understanding how

1.57k views • 105 slides

DL2: Training and Querying Neural Networks with Logic Ma Marc Fischer er , Mislav Balunovi ,

DL2: Training and Querying Neural Networks with Logic Ma Marc Fischer er , Mislav Balunovi , Dana Drachsler-Cohen, Timon Gehr, Ce Zhang, Martin Vechev github.com/eth-sri/dl2 differencing neural networks finding inputs that deactivates [Pei

992 views • 13 slides

Neural Networks CS 6355: Structured Prediction Based on slides and material from Geoffrey Hinton,

Neural Networks CS 6355: Structured Prediction Based on slides and material from Geoffrey Hinton, Richard Socher, Dan Roth, Yoav Goldberg, Shai Shalev-Shwartz and Shai Ben-David, and others This lecture What is a neural network? Training

1.28k views • 93 slides

Training Neural Networks Some considerations Gaurav Kumar Center for Language and Speech

Training Neural Networks Some considerations Gaurav Kumar Center for Language and Speech Processing gkumar@cs.jhu.edu Universal Approximators Neural networks can approximate [1] function. any Capacity Layers hidden layer size

1.19k views • 17 slides

Lightweight Neural Networks from PCA & LDA Based Distilled Dense Neural Networks ICIP 2020

Lightweight Neural Networks from PCA & LDA Based Distilled Dense Neural Networks ICIP 2020 MEA. Seddik 1 , 2 , , H. Essafi 1 , A. Benzine 1 , 3 , M. Tamaazousti 1 1 CEA List, France 2 CentraleSuplec, L2S, France 3 Sorbonne University,

243 views • 5 slides

Deep Learning: Training Juhan Nam Training Deep Neural Networks Forward (hidden unit

GCT634/AI613: Musical Applications of Machine Learning (Fall 2020) Deep Learning: Training Juhan Nam Training Deep Neural Networks Forward (hidden unit activation) Parametric Gradient-based Learning Backward Non-parametric (gradient

650 views • 21 slides

Introduction to Neural Networks Slides from L. Lazebnik, B. Hariharan Outline Perceptrons

Introduction to Neural Networks Slides from L. Lazebnik, B. Hariharan Outline Perceptrons Perceptron update rule Multi-layer neural networks Training method Best practices for training classifiers After that:

505 views • 25 slides

Landscape Connectivity and Dropout Stability of SGD Solutions for Over-parameterized Neural

Landscape Connectivity and Dropout Stability of SGD Solutions for Over-parameterized Neural Networks Alexander Shevchenko Marco Mondelli Neural Network Training From theoretical perspective training of neural networks is di ffi cult

494 views • 28 slides

Neural Networks and Handwriting Recognition Background Neural Networks Neural Network Steven

Neural Networks and Handwriting Recognition Steven Sloss Math 164 Neural Networks and Handwriting Recognition Background Neural Networks Neural Network Steven Sloss Structure Training Neural Networks Math 164 Motivation Problem

889 views • 41 slides

Outline Evolution of neurocomputing Artificial neural networks Feed forward

Introduction to Neural Networks and Deep Learning Ling Guan References: S. Haykin, Neural Networks , 2 nd Edition, New Jersey: Prentice Hall, 1. 2004. C. M. Bishop, Neural Networks for Pattern Re cognition, New York: 2. Oxford University

451 views • 17 slides

FLEET: Flexible Efficient Ensemble Training for Heterogenous Deep Neural Networks Hui Guan,

FLEET: Flexible Efficient Ensemble Training for Heterogenous Deep Neural Networks Hui Guan, Laxmikant Kishor Mokadam, Xipeng Shen, Seung-Hwan Lim, Robert Patton 1 Build an image classifier? Deep Neural Network (DNN) CPU GPU Training

957 views • 41 slides

Data Parallelism in Training Sparse Neural Networks Namhoon Lee 1 , Philip Torr 1 , Martin Jaggi 2

Data Parallelism in Training Sparse Neural Networks Namhoon Lee 1 , Philip Torr 1 , Martin Jaggi 2 1 University of Oxford, 2 EPFL ICLR 2020 Workshop on PML4DC Motivation Compressing neural networks can save a large amount of memory and

570 views • 21 slides

What Can Neural Networks Teach us about Language? Graham Neubig a2-dlearn 11/18/2017 Supervised

What Can Neural Networks Teach us about Language? Graham Neubig a2-dlearn 11/18/2017 Supervised Training of Neural Networks for Language Training Data Training Model this is an example the cat went to the store Prediction Results

1.07k views • 62 slides

Evolving Neural Networks This lecture is based on Xin Yaos tutorial slides From Evolving

Evolving Neural Networks This lecture is based on Xin Yaos tutorial slides From Evolving Single Neural Networks to Evolving Ensembles presented at CEC 2007. Literature: Evolving Artificial Neural Networks , Xin Yao, Proceedings of the

281 views • 16 slides

Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks

Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they have

1.17k views • 91 slides

Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks

Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they have

852 views • 33 slides

Training Neural Networks with Local Error Signals Arild Nkland Lars H. Eidnes Local learning

Training Neural Networks with Local Error Signals Arild Nkland Lars H. Eidnes Local learning Typically we train neural networks by backpropagating errors from the loss function and back through the layers. Hard to explain how the

647 views • 13 slides