INPUTS We made MNIST images - PowerPoint PPT Presentation

● ● ●

● ● ● ●

● ● ● ○ ○ ○

INPUTS ● We made MNIST images binary (white or black) and computed shannon entropy for Max Entropy each pixel ● Entropy is close to maximum (50% white) for most pixels in the middle but is close to 0 for pixels on edge Network ● We binarized activity of our hidden layer [active/not active] ● Units with equal probability of being active (firing) carry the most information

High Entropy 9.9/10 Low Entropy 9.0/10 (proportion of values >0.5 = 0.49) (proportion of values >0.5 = 0.42) Activity Activity Trial Activity Trial Activity

● ● Activation Penalty

No Activity Penalty ● Unfortunately, while this does reduce overall network activation, it disrupts our training process Moderate Activity Penalty

● ● ●

● ● ● ●

● Performance is significantly worse than fully connected network ● Overall activity levels are reduced but only marginally

● Accuracy improves with bottleneck size to an asymptote ● As expected- entropy and activity levels increase with bottleneck size

● ● ●

● ● ●

Normal MNIST images Spike trains over time

●

● ● ●

Autoencoder Spiking Neural Network

● ● ●

● ● ●