Anatomy and Interpretability of Neural Networks Leon Yin ~ Data - PowerPoint PPT Presentation

Jan 06, 2023 •204 likes •403 views

Anatomy and Interpretability of Neural Networks Leon Yin ~ Data Scientist | Research Engineer SMaPP and CDS PRG 2017-11-15 Todays talking points: How do Neural Networks work? How can we see what theyre learning? Discussion about

Anatomy and Interpretability of Neural Networks Leon Yin ~ Data Scientist | Research Engineer SMaPP and CDS PRG 2017-11-15
Today’s talking points: How do Neural Networks work? How can we see what they’re learning? Discussion about training data and policy.
First of all All models are wrong, but some are useful!
Neural Networks: Transforms one dataset (D) into another dataset (D’). The D’ is optimized for discrimination.
Basic Functions 1. Matrix multiplication 2. Thresholding
Matrix Multiplication Input gets multiplied by N randomly initialized weight s , Where N is equal to the number of nodes (neurons) in the next layer.
Convolutions Kernel or Filter https://nbviewer.jupyter.org/github/yinleon/interpreting_nerual_networks/blob/master/null_features/neural_netw ork_basics.ipynb
Thresholding
Thresholding or Activation Functions Rectified Linear Units (ReLU) remove negative values.
Downsampling Use pooling function either Max, Avg, Sum Also for simplification and amplification
Recap: Matrix multiplication creates new features. Thresholding and downsampling simplify the math and amplify signals. This is repeated and combined to identify patterns with increasing complexity.
Feature Visualization https://distill.pub/2017/feature-visualization/
Let’s Look at Logits: https://nbviewer.jupyter.org/github/yinleon/interpreting_nerual_networks/blob/mast er/null_features/model_conv_feature_evaluation.ipynb
What about Text?
Bias on Yelp Different tasks have the same outcomes: Mexican food is associated with negative reviews and negative connotations!
Training Data We build infrastructure around availability What are we feeding models? Cool paper about reducing training data gender bias: https://homes.cs.washington.edu/~my89/publications/bias.pdf
Looking for Context NLP community standardizing metadata RE: origin, app and audience.
Thoughts about Interpretability?
Thanks! @leonyin

Recommend

Interpretability and Visualization of Deep Neural Networks Au Aude Oliva MI MIT Convoluti

Interpretability and Visualization of Deep Neural Networks Au Aude Oliva MI MIT Convoluti tional Neura ral Netw twork rks convolution max-pooling normalization full connected Alexnet Ea Each ch laye yer learns prog ogressive

356 views • 34 slides

Interpretability in Convolutional Neural Networks for Building Damage Classification in Satellite

Interpretability in Convolutional Neural Networks for Building Damage Classification in Satellite Imagery NeurIPS 2020 Workshop Tackling Climate Change with Machine Learning Thomas Y. Chen Computer Vision, Satellite Imagery, and Building

401 views • 16 slides

Intro to Artificial Neural Networks Oscar Maas @oscmansan Outline 1. Perceptrons 2.

Intro to Artificial Neural Networks Oscar Maas @oscmansan Outline 1. Perceptrons 2. Sigmoid Neurons 3. Linear vs Nonlinear Classification 4. Neural Networks Anatomy 5. Training and loss a. Reducing loss b. Gradient Descent c.

832 views • 21 slides

Outline Evolution of neurocomputing Artificial neural networks Feed forward

Introduction to Neural Networks and Deep Learning Ling Guan References: S. Haykin, Neural Networks , 2 nd Edition, New Jersey: Prentice Hall, 1. 2004. C. M. Bishop, Neural Networks for Pattern Re cognition, New York: 2. Oxford University

451 views • 17 slides

Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks

Neural Networks 1. Introduction Fall 2017 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they have

1.17k views • 91 slides

Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks

Neural Networks 0. Logistics Spring 2019 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they have

852 views • 33 slides

CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks

CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks Neuron Non-linearity Softmax layer DNN training Loss function and regularization SGD and backprop Learning rate Overfitting

1.8k views • 86 slides

Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2020 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they

1.63k views • 119 slides

Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural

Neural Networks 1. Introduction Spring 2019 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In many problems they

1.57k views • 124 slides

Relaxation and Hopfield Networks Neural Networks Neural Networks - Hopfield 1 Bibliography

Relaxation and Hopfield Networks Neural Networks Neural Networks - Hopfield 1 Bibliography Hopfield, J. J., "Neural networks and physical systems with emergent collective computational abilities," Proceedings of the National Academy

367 views • 19 slides

Model Interpretation Danish Pruthi April 28, 2020 Why interpretability? Task:

CS 11-747 Neural Networks for NLP Model Interpretation Danish Pruthi April 28, 2020 Why interpretability? Task: predict probability of death for patients with pneumonia Why : so that high-risk patients can be admitted, low risk

826 views • 29 slides

Neural Networks and their Application to Go Neural Networks Learning Blackjack Theory Training

Neural Networks and their Application to Go A. Bausch Neural Networks and their Application to Go Neural Networks Learning Blackjack Theory Training neural networks Problems AlphaGo Anne-Marie Bausch The Game of Go Policy Network

280 views • 24 slides

(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning

(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning objectives What are neural networks? What are deep neural networks? How do we train neural networks? What variants of neural network

331 views • 31 slides

How Neural Networks (NN) Biological Neuron: A . . . Can (Hopefully) Learn Artificial Neural . .

Need for Machine . . . Neural Networks . . . How to Speed Up . . . Neural Networks: A . . . How Neural Networks (NN) Biological Neuron: A . . . Can (Hopefully) Learn Artificial Neural . . . Resulting Algorithm Faster by Taking Into How to

464 views • 17 slides

Outline Why model neural networks? Modeling Neural Networks A brief look at the neuron.

Outline Why model neural networks? Modeling Neural Networks A brief look at the neuron. A look at some current works. Adding an evolutionary strategy. Paul Nuytten CPSC 607 Why Model Neural Networks? The Neuron The nervous

697 views • 4 slides

Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking

Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In

987 views • 40 slides

Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Learning in Neural Networks w 1,3 w 3,5 1 3 5 w w 1,4 3,6 w w 2,3 4,5 2 4 6 w w 2,4 4,6 Class #18: Back-Propagation; } A neural network can learn a classification function by Tuning Hyper-Parameters adjusting its weights to

259 views • 4 slides

Neural Networks Hugo Larochelle ( @hugo_larochelle ) Google Brain 2 NEURAL NETWORKS What

Neural Networks Hugo Larochelle ( @hugo_larochelle ) Google Brain 2 NEURAL NETWORKS What well cover ... f ( x ) computer vision architectures - convolutional networks - data augmentation 1 - residual networks ... ...

1.29k views • 75 slides

Neural Networks Oskar Taubert (SCC) SCC 1 15.01.2020 Oskar Taubert - Neural Networks SCC

Neural Networks Oskar Taubert (SCC) SCC 1 15.01.2020 Oskar Taubert - Neural Networks SCC www.kit.edu KIT The Research University in the Helmholtz Association Neural Network Concept (very) remotely brain inspired computational system

479 views • 25 slides

Introduction to Artificial Intelligence Neural Networks - Deep Learning for NLP Janyl Jumadinova

Introduction to Artificial Intelligence Neural Networks - Deep Learning for NLP Janyl Jumadinova November 21, 2016 Neural Networks 2/20 Neural Networks 3/20 Neural Networks Neural computing requires a number of neurons , to be connected

813 views • 21 slides

7. Artificial neural networks Introduction to neural networks Despite struggling to understand

7. Artificial neural networks Introduction to neural networks Despite struggling to understand intricacies of protein, cell, and network function within the brain, neuroscientists would agree on the following simplistic description of how the

157 views • 11 slides

Neural networks Chapter 20 Chapter 20 1 Outline Brains Neural networks Perceptrons

Neural networks Chapter 20 Chapter 20 1 Outline Brains Neural networks Perceptrons Multilayer networks Applications of neural networks Chapter 20 2 Brains 10 11 neurons of > 20 types, 10 14 synapses, 1ms10ms cycle

804 views • 47 slides

Modeling Neural Networks Paul Nuytten CPSC 607 Outline Why model neural networks? A

Modeling Neural Networks Paul Nuytten CPSC 607 Outline Why model neural networks? A brief look at the neuron. A look at some current works. Adding an evolutionary strategy. Why Model Neural Networks? The nervous system is a

1.22k views • 23 slides

Modular Neural Networks CPSC 533 Franco Lee Ian Ko Modular Neural Networks What is it ? Dif

Modular Neural Networks CPSC 533 Franco Lee Ian Ko Modular Neural Networks What is it ? Dif f erent models of neural net w orks combined int o a single syst em. Each single net w ork is made int o a module t hat can be f reely int

289 views • 25 slides