Functional resilience in neural networks Mediterranean School of - PowerPoint PPT Presentation

Functional resilience in neural networks Mediterranean School of Complex Networks Edward Laurence September 5, 2017 Département de physique, de génie physique, et d’optique Université Laval, Québec, Canada 0

Resilience Ability to recover the original state in a reasonnable short period of time. Robustness - Opposite of vulnerability Difficulty to modify the state of a system. 1

The brain is resilient Plasticity, compensation, ... The details and strategies are still unknown 2

Connectomics Nodes Neurons of activity x i ( t ) Edges Synapses of weight w ij ( t ) Perturbations ∆ x i ( t ) , ∆ w ij ( t ) , edges/nodes removal, ... 3

TOC Effective formalism � Description � Application to neural networks � Approximations and errors Adaptive connectivity � Special behaviors � Measures of resilience 4

Effective formalism | Description Effective formalism Presented by Gao et al. 2016 5

Effective formalism | Description N -dimensional complete system N � x i � F ( x i ) + � w ij G ( x i , x j ) j � 1 6

Effective formalism | Description N -dimensional complete system N � x i � F ( x i ) + � w ij G ( x i , x j ) j � 1 6

Effective formalism | Description N -dimensional complete system N � x i � F ( x i ) + � w ij G ( x i , x j ) j � 1 1-dimensional effective system x eff � F ( x eff ) + β eff G ( x eff , x eff ) � � � ij w ij x j ijk w ij w jk x eff � L � x � β eff � L � s � ; � � � ij w ij � ij w ij L � x � � Neighborhood average of x 6

Effective formalism | Neural networks 1-dimensional effective system x eff � F ( x eff ) + β eff G ( x eff , x eff ) � 7

Effective formalism | Neural networks 1-dimensional effective system x eff � F ( x eff ) + β eff G ( x eff , x eff ) � Neural dynamics - Hopfield model 1 . 0 0 . 8 � � x j − µ �� λ � x i � − x i + � w ij σ 0 . 6 σ ( y ) j 0 . 4 1 0 . 2 σ ( y ) � 1 + e − y 0 . 0 0 3 6 9 y 7

Effective formalism | Neural networks 1-dimensional effective system x eff � F ( x eff ) + β eff G ( x eff , x eff ) � Neural dynamics - Hopfield model 1 . 0 0 . 8 � � x j − µ �� λ � x i � − x i + � w ij σ 0 . 6 σ ( y ) j 0 . 4 1 0 . 2 σ ( y ) � 1 + e − y 0 . 0 0 3 6 9 y � x eff − µ �� λ � x eff � − x eff + β eff σ � 7

Effective formalism | Neural networks � x eff − µ �� λ � x eff � − x eff + β eff σ � Stationnary state � x eff � 0 8

Effective formalism | Neural networks � x eff − µ �� λ � x eff � − x eff + β eff σ � Stationnary state � x eff � 0 8

Effective formalism | Neural networks � x eff − µ �� λ � x eff � − x eff + β eff σ � Stationnary state � x eff � 0 8

Effective formalism | Neural networks � x eff − µ �� λ � x eff � − x eff + β eff σ � Stationnary state � x eff � 0 8

Effective formalism | Neural networks � x eff − µ �� λ � x eff � − x eff + β eff σ � Stationnary state � x eff � 0 8

Effective formalism | Neural networks � x eff − µ �� λ � x eff � − x eff + β eff σ � Stationnary state � x eff � 0 8

Effective formalism | Neural networks � x eff − µ �� λ � x eff � − x eff + β eff σ � Stationnary state � x eff � 0 (a) 9

Effective formalism | Neural networks � x eff − µ �� λ � x eff � − x eff + β eff σ � Good approximation for � Homogeneous network � Low inhibition � High reciprocity w ij � w ji 10

Adaptive connectivity 11

Plasticity Resilience Ability to recover the original state in a reasonnable short period of time. 12

Plasticity Resilience Ability to recover the original state in a reasonnable short period of time. Modified Hebb’s rule w ij � τ − 1 S ( σ i σ j − w ij σ 2 � j ) ; σ i � σ [ λ ( x i − µ )] 12

Recuperation w ij � τ − 1 S ( σ i σ j − w ij σ 2 � j ) σ i � σ [ λ ( x i − µ )] ; 10 τ − 1 = 1 S 8 τ − 1 = 0 . 7 S τ − 1 = 0 . 68 S 6 τ − 1 = 0 . 3 x eff S 4 2 0 2 3 4 5 6 7 8 9 10 β eff 13

Measures of resilience How to quantify resilience? 14

Measures of resilience How to quantify resilience? � Recovery time � Energy of recuperation � Maximum damage � Sensibility dx eff d β eff 14

Measures of resilience | Time of recuperation Recovery time Time to return in the surroundings of the original state No recovery 15

Measures of resilience | Time of recuperation Recovery time Time to return in the surroundings of the original state No recovery Critical slowing down 16

Conclusion 17

Conclusion Effective formalism � Simple to use on neural dynamics. � Valid for homogeneous, low inhibition and high reciprocity. Resilience 10 τ − 1 = 1 S 8 τ − 1 = 0 . 7 S τ − 1 = 0 . 68 � Introduce adaptive connectivity S 6 τ − 1 = 0 . 3 x eff S 4 � Recovery time is a good indicator of 2 0 2 3 4 5 6 7 8 9 10 catastrophe β eff 18

Thank you Collaborators Nicolas Doyon Louis J. Dubé Patrick Desrosiers dynamica.phy.ulaval.ca 19