Global convergence of neuron birth-death dynamics ICML 2019 Long - PowerPoint PPT Presentation

Nov 27, 2022 •171 likes •252 views

Global convergence of neuron birth-death dynamics ICML 2019 Long Beach, CA 13 June 2019 Grant M. Rotskofg, Samy Jelassi, Joan Bruna, and Eric Vanden-Eijnden Center for Data Science, New York University Courant Institute of Mathematical

Global convergence of neuron birth-death dynamics ICML 2019 Long Beach, CA 13 June 2019 Grant M. Rotskofg, Samy Jelassi, Joan Bruna, and Eric Vanden-Eijnden Center for Data Science, New York University Courant Institute of Mathematical Sciences, New York University
Synopsis of main results →! Modifjcation of gradient descent (GD) with unbalanced optimal transport ● Parameter birth-death process ● Proof of global convergence ● Rate of convergence scales as →! Based on mean-fjeld perspective on neural networks ● Analysis of deterministic partial difgerential equation (PDE) ● PDE leads to practical, effjcient algorithm →! Experiments show faster convergence relative to GD ● Illustrative examples show efgect of transport / exploration ● Easy to implement with no additional gradient computations ICML 13 Jun 2019
Single hidden layer neural network d -dim input PDE for parameter distribution Distinct from kernel learning / NTK, dynamics leads to feature selection ICML 13 Jun 2019
Non-local mass transport (particle birth-death) Parameters are Total population killed / cloned is fjxed ICML 13 Jun 2019
Theorem [R,J,B,V-E]: global convergence Compare with [Chizat & Bach, 2018] without any restriction on homogeneity for the units ICML 13 Jun 2019
Dramatic efgect of birth/death dynamics ICML 13 Jun 2019
Come see our poster! Thu Jun 13th 06:30 -- 09:00 PM @ Pacifjc Ballroom #93 Grant Rotskofg Samy Jelassi Joan Bruna Eric Vanden-Eijnden arXiv:1902.01843 ICML 13 Jun 2019

Recommend

Birth and Death Processes Today: Birth processes Birth and Death Processes Death

Birth and Death Processes Today: Birth processes Birth and Death Processes Death processes Biarth and death processes Bo Friis Nielsen 1 Limiting behaviour of birth and death processes Next week 1 DTU Informatics Finite state

309 views • 7 slides

Birth-Death Processes Birth-Death Processes: Transient Solution Poisson Process: State

Birth-Death Processes Birth-Death Processes: Transient Solution Poisson Process: State Distribution Poisson Process: Inter-arrival Times Dr Conor McArdle EE414 - Birth-Death Processes 1/17 Birth-Death Processes Birth-Death Processes are

282 views • 17 slides

for information such as the birth place, birth date and parents' names of the deceased. What

Indexes and Sources of Info on Deaths, Obituaries, Burials Death records are primary source records because they are completed at, or close to, the time of the death by someone who was present at the death. Death records are especially helpful,

66 views • 5 slides

Using Single Neuron Dynamics to Predict Synchronous Global Network Activities Robert Kim

Using Single Neuron Dynamics to Predict Synchronous Global Network Activities Robert Kim Neurosciences Graduate Program University of California, San Diego December 8, 2017 Neurodynamics Final Project December 8, 2017 1 / 16 Outline

761 views • 29 slides

Theoretical neuroscience: From single neuron to network dynamics Nicolas Brunel Outline

Theoretical neuroscience: From single neuron to network dynamics Nicolas Brunel Outline Single neuron stochastic dynamics Network dynamics Learning and memory Single neurons in vivo seem highly stochastic Single neuron stochastic

475 views • 19 slides

Birth and Fetal Death Reporting Enhanced (BFDR-E) and Vital Records Death Reporting (VRDR)

Birth and Fetal Death Reporting Enhanced (BFDR-E) and Vital Records Death Reporting (VRDR) Alaina Elliott Lori Reed-Fourquet Consultant IT Specialist E-HealthSign Centers for Disease Control and Prevention National Center for Health

294 views • 6 slides

Calculating bounds on expected return and first passage times in finite-state imprecise

Calculating bounds on expected return and first passage times in finite-state imprecise birth-death chains Stavros Lopatatzidis, Jasper De Bock and Gert de Cooman Birth-death chains Birth-death chain special type of Markov chain X : =

404 views • 15 slides

Death before Birth Understanding, informing and supporting the choices made by people who have

Death before Birth Understanding, informing and supporting the choices made by people who have experienced miscarriage, termination, and stillbirth Overall project aim: To examine the law surrounding the disposal of the remains of pregnancy

362 views • 21 slides

TxEVER Birth and Death for Local Registrars CDCAT Fall Conference Derek Johnson, Field

TxEVER Birth and Death for Local Registrars CDCAT Fall Conference Derek Johnson, Field Services, VSS 0 2 Aug 2 0 1 8 Agenda This conference call presentation will cover the following: 1 . Birth Local Registration a. Local

527 views • 28 slides

Shift techniques for Quasi-Birth-and-Death processes: canonical factorizations and matrix

Shift techniques for Quasi-Birth-and-Death processes: canonical factorizations and matrix equations Beatrice Meini Universit` a di Pisa Joint work with D.A. Bini and G. Latouche MAM9 - Budapest QBD processes Let A A 0 0

680 views • 24 slides

Cause-of-Death Mortality: What Can Be Learned From Population Dynamics? A Study of a

Cause-of-Death Mortality: What Can Be Learned From Population Dynamics? A Study of a Heterogeneous Portfolio Dynamic Hlose Labit Hardy PhD Student, University of Lausanne, Switzerland joint work with S. Arnold, A. Boumezoued and N. El

211 views • 18 slides

Myra Dioquino Ne Neuron Ne Neuron ons ons ~30x out ~30x in 20 000 20,000x out o t ~30x

Myra Dioquino Ne Neuron Ne Neuron ons ons ~30x out ~30x in 20 000 20,000x out o t ~30x out Describes the equilibrium potential of the cell for a particular ion R Universal gas constant (8.314472 J K ) T

455 views • 9 slides

REQUIEM BY ALFRED SCHNITTKE Composer's name and birth/death dates: Alfred Schnittke (Russian,

REQUIEM BY ALFRED SCHNITTKE Composer's name and birth/death dates: Alfred Schnittke (Russian, German/Jewish, 1938-1996) Full title of composition: Requiem (from the stage music to Don Carlos by Friedich von Schiller) Placement in the

205 views • 3 slides

Spatial Birth-Death Model for Wireless Networks Abishek Sankararaman and Franois Baccelli UT

Spatial Birth-Death Model for Wireless Networks Abishek Sankararaman and Franois Baccelli UT Austin Outline Motivation and Background. Math Model - Interacting Particle System. Summary of Results. Further

1.07k views • 74 slides

Stochastic dynamics of spiking neuron models and implications for network dynamics Nicolas Brunel

Stochastic dynamics of spiking neuron models and implications for network dynamics Nicolas Brunel The question What is the input-output relationship of single neurons? The question What is the input-output relationship of single neurons?

489 views • 32 slides

Population Dynamics Deductive modelling: based on physical laws Inductive modelling: based on

Population Dynamics Deductive modelling: based on physical laws Inductive modelling: based on observation + intuition Single species: Birth (in migration) Rate, Death (out

706 views • 33 slides

Spatial Birth and Death Peer-to-Peer Point Processes F. Baccelli INRIAENS Joint work with F.

Spatial Birth and Death Peer-to-Peer Point Processes F. Baccelli INRIAENS Joint work with F. Mathieu and I. Norros Stochastic Network Conference 2012, MIT 1 STRUCTURE OF THE TALK 1. P2P Motivations 5.

968 views • 47 slides

Network determination based on birth-death MCMC inference A. Mohammadi and E. Wit February 4,

N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC METHOD E XAMPLES Network determination based on birth-death MCMC inference A. Mohammadi and E. Wit February 4, 2013 N ETWORK B IRTH - DEATH MCMC METHOD S PECIFIC ELEMENT OF BDMCMC

602 views • 28 slides

Birth, Marriage and Death statistics from registration systems Frank Dixon, Vital Events

Birth, Marriage and Death statistics from registration systems Frank Dixon, Vital Events Statistician GROS Workshop at Using and Improving Population and Household Statistics seminar 30 October 2009 Some points from the Workshop

599 views • 20 slides

Convergence in distribution of stochastic dynamics Mathias Rousset (1) (1) INRIA Roquencourt,

Convergence in distribution of stochastic dynamics Mathias Rousset (1) (1) INRIA Roquencourt, France MICMAC project-team. CEMRACS 2013 p.1 Motivation Consider a stochastic dynamical model in the form t ( X t , E t ) , where X

482 views • 33 slides

Convergence of Nash Dynamics: Equilibria and Nearly-Optimal Solutions Vahab Mirrokni Google

Convergence of Nash Dynamics: Equilibria and Nearly-Optimal Solutions Vahab Mirrokni Google Research, New York Heiko R oglin Department of Quantitative Economics Maastricht University ACM Conference on Electronic Commerce (EC09)

2.03k views • 166 slides

Population Dynamics Deductive modelling: based on physical laws Inductive modelling: based

Population Dynamics Deductive modelling: based on physical laws Inductive modelling: based on observation + intuition Single species: Birth (in migration) Rate, Death (out migration) Rate dP dt = BR DR Rates proportional to

707 views • 43 slides

STEP 5. Slides of a neuron, nerve, and spinal cord #1 Neuron slide This step can wait until

STEP 5. Slides of a neuron, nerve, and spinal cord #1 Neuron slide This step can wait until later, if youd like. But make sure you look at these Read images! Me First, we will look at the motor neuron. Note that it is difficult to

370 views • 3 slides

Some definitions: Vital rates (e.g., birth, death) are said to be density dependent if they

Some definitions: Vital rates (e.g., birth, death) are said to be density dependent if they change depending on the density of dependent if they change depending on the density of conspecifics (individuals of the same species). Important!

165 views • 3 slides