Hebbian learning rule Wij (t) = *xj*xi Wij (t) = F(xj, xi, , t, - PowerPoint PPT Presentation

Hebbian learning rule Δ Wij (t) = γ *xj*xi Δ Wij (t) = F(xj, xi, γ , t, θ ) Time?

airpuff airpuff weight Σ Σ tone eyeblink tone eyeblink airpuffeyeblink weight tone airpuff time

What is an association? Pavlov: Eye blink: Other:

Linear associator: Notations and vectors 1. One postsynaptic neuron: 2. Many postsynaptic neurons:

xi = Ii xi = Σ wij xj Δ wij = γ * gi * fj Write out for 3 pre and 3 post neurons

Pre post Δ wij = γ * gi * fj ⎡ ⎤ ⎡ ⎤ f 1 g 1 ⎡ ⎤ ⎢ ⎥ ⎢ ⎥ g 1 f 2 g 2 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ g 2 ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ fj gi [ ] = = ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ = γ = T g w gj f 1 , f 2 ,.. fi ... fn f g . ⎢ ⎥ . ⎢ ⎥ ⎢ ⎥ f ⎢ ⎥ ⎢ ⎥ . ⎢ ⎥ . . ⎢ ⎥ ⎢ ⎥ ⎢ ⎥ ⎣ ⎦ ⎢ ⎥ . gn ⎢ ⎥ ⎣ ⎦ ⎣ ⎦ fn gn = = = ⎡ ⎤ w 11 g 1 f 1 w12 g1f2 ... w1n g1fn ⎢ ⎥ = w 21 g 2 f 1 ⎢ ⎥ = w ⎢ ⎥ ⎢ ⎥ = = = ⎣ ⎦ wn 1 gnf 1 w2n gnf2 ... wnn fngn

weight matrix w layer f layer g w11 f1 g1 w12 f2 g2 fj gi w1n fn gn g1 = x(f1) w11 + x(f2) w12 ... x(fj) w1j ... + x(fn) w1n gi = x(f1) wi1 + x(f2) wi2 ... x(fj) wij ... + x(fn) win etc ..

Grandmother baking cookies and talking Olfactory system Auditory system

Grandmother baking cookies and talking Olfactory system Auditory system Δ wij = γ * gi * fj.

Cookies Olfactory system Auditory system x(g i ) = Σ w ij * x(f j )

Bread Olfactory system Auditory system x(g i ) = Σ w ij * x(f j )

Cookies Olfactory system Auditory system x(g i ) = Σ w ij * x(f j )

Aunt cooking stew and talking Olfactory system Auditory system

Aunt baking cookies and talking Grandmother’s voice recalled! Olfactory system Auditory system

Calculate example here

input from olfactory bulb feeback interactions (association fibers intrinsic cnnection) output

input from olfactory bulb feeback interactions (association fibers intrinsic cnnection) output

input from olfactory bulb feeback interactions (association fibers intrinsic cnnection) output

"Hopfield networks", which have a recurrent structure and the development of which is inspired by statistical physics. They share the following features: Nonlinear computing units (or neurons) Symmetric synaptic connections (wij = wji) No connections of a neuron on itself (wii = 0) Abundant use of feedback (usually, all neurons are connected to all others) (Feedback means that a neurons sends a synapse to a neuron it also receives a signal from, so that there is a closed loop).

+ > = ∑ 1 if v 0 N = = i v w v * x and x { [if 0 unit j remains in its previous state] < i ij j i i - 1 if v 0 = j 1 i 1 = ≠ = = w x * x if i j w 0 if i j i j ij ij N N N 1 ∑∑ = − w v v E * * ij i j 2 = = i 1 j 1

Initial condition Trajectory Minimum

w12 w21 N1 N2 w13 w23 N3 w31 w32 v1, v2, v3: inputs to neuron 1, 2 x1, x2, x3: outputs of neuron 1,2 w12, w21 etc: connection strength Show that (1,1,1) is stable when all w = 1

w12 w21 N1 N2 w13 w23 N3 w31 w32 v1, v2, v3: inputs to neuron 1, 2 x1, x2, x3: outputs of neuron 1,2 w12, w21 etc: connection strength Show that (-1,-1,-1) is stable when all w = 1

Initial condition Trajectory Minimum Explain attractor and basin of attraction

E(-1,1,1)=1 E(1,1,1) = -3 E(-1,1,-1) = 1 (1,1,,-1) E(1,-1,1) = 1 (-1,-1,1) E(1,-1,-1) = 1 E(-1,-1,-1) = -3

Calculate example from notes

Does it work well? Is it realistic?

Show matlab simulation