Principles of neural network design Francois Belletti, CS294 RISE - PowerPoint PPT Presentation

Nov 29, 2022 •269 likes •561 views

Principles of neural network design Francois Belletti, CS294 RISE Human brains as metaphors of statistical models Biological analogies Machine learning instantiations The visual cortex of mammals Deep convolutional neural networks

Principles of neural network design Francois Belletti, CS294 RISE
Human brains as metaphors of statistical models Biological analogies Machine learning instantiations The visual cortex of mammals Deep convolutional neural networks Multiple sensing channels Multimodal neural networks Memory and attention LSTMs and GRUs
Neural Networks For Computer Vision
Neural Networks in Computer Vision Neural networks for classification of handwritten digits
Learning Mechanism: Correction of Mistakes Nature used a single tool to get to today’s success: mistake
Modularity Is Back-Prop’s Perk for Software Eng. Back-propagation is a recursive algorithm
Image Classification
Successful Architecture In Computer Vision An example of a wide network: AlexNet
Understanding What Happens Within A Deep NN Examining convolution filter banks Examining activations
Determining A Neuron’s Speciality Images that triggered the highest activations of a neuron:
Another Successful Architecture For CV “We need to go deeper”, Inception:
State of the Art
Recurrent Architectures
Learning To Leverage Context Memory in Recurrent Architectures: LSTM (Long Short Term Memory Network) Input x, output y, context c (memory) y y y Memorization Output Forget gate gate gate c c c y y y Concatenation x x x t
Other recurrent architectures Gated recurrent units:
Why Is Context Important? In language, most grammars are not context free End-to-end translation, Alex Graves
Context Is Also Important In Control Remembering what just happened is important for decision making
Memory is necessary for localization Latest experiment in asynchronous deep RL: LSTMS for maze running Memory comes at a cost: a lot of RAM or VRAM is necessary
Conclusion: the distributed brain
Interaction is crucial in enabling AI
Playing versus computers before beating humans
Bootstrapping by interaction Why would two androids casually chat one with another?
The distributed brain at the edge Distributed RL is reminiscent of the philosophical omega point of knowledge
Multiple Input Neural Networks
Multi Inputs For Inference Youtube Video Auto-encoding
Softmax Multiple Input Control Fully connected layer Multiplexing Inputs Fully connected layer Concatenated output Relu Relu Relu Fully Max Max Max connected Conv Conv Conv layer Relu Relu Relu Fully Max Max Max connected Conv Conv Conv layer Front Rear Radar Odometry Camera Camera
Multiplexing In The Human Brain

Recommend

FPGgram Diana Valverde Tonye Brown Overview Aim Design Neural Network

FPGgram Diana Valverde Tonye Brown Overview Aim Design Neural Network Structure Hardware Software Results Lesson Learned Aim Our project focuses on using convolutional neural network for image

633 views • 15 slides

A General Artificial Neural Network Extension for HTK Chao Zhang & Phil Woodland University

A General Artificial Neural Network Extension for HTK Chao Zhang & Phil Woodland University of Cambridge 15 April 2015 Overview Design Principles Implementation Details Generic ANN Support ANN Training Data Cache

600 views • 14 slides

Network Protocol Design and Evaluation 02 - Design Principles Stefan Rhrup University of

Network Protocol Design and Evaluation 02 - Design Principles Stefan Rhrup University of Freiburg Computer Networks and Telematics Summer 2009 In the last lecture... Specification: 5 Elements of a protocol Service

977 views • 58 slides

Neural Information Retrieval Wassila Lalouani 1 Plan Neural network architectures Neural

Neural Information Retrieval Wassila Lalouani 1 Plan Neural network architectures Neural IR tasks Neural IR architecture Feature Representations Neural IR query auto completion Neural IR query suggestion Neural IR document

1.48k views • 18 slides

Algorithms in Nature Pruning in neural networks Neural network development 1. Efficient signal

Algorithms in Nature Pruning in neural networks Neural network development 1. Efficient signal propagation [e.g. information processing & integration] 2. Robust to noise and failures [e.g. cell or synapse failure] 3. Cost-aware design [e.g.

874 views • 33 slides

Evolve or Die High-Availability Design Principles Drawn from Googles Network Infrastructure

Ramesh Govindan, Ina Minei, Mahesh Kallahalla, Bikash Koley and Amin Vahdat and a cast of hundreds at Google Evolve or Die High-Availability Design Principles Drawn from Googles Network Infrastructure Network availability is the

959 views • 95 slides

CS 678 Spring 2013 Network Architecture and Principles The Design Philosophy of the DARPA

CS 678 Spring 2013 Network Architecture and Principles The Design Philosophy of the DARPA Internet Protocols, Dave Clarke, 1988 Ihsan Ayyub Qazi Computer Science Department LUMS SBASSE Slides use info from Nick

620 views • 29 slides

Design of Neural Network models for screening anticancer activities in Taxol analogues Stan

Design of Neural Network models for screening anticancer activities in Taxol analogues Stan Svojanovsky, PhD Bioinformatics Coordinator Research Associate Professor Department of Molecular and Integrative Physiology University of Kansas,

558 views • 28 slides

Artificial Neural Networks By: Kodi Neumiller Overview What is an artificial neural network

Artificial Neural Networks By: Kodi Neumiller Overview What is an artificial neural network Perceptron Feed forward Backpropagation Learning rate/momentum What is a neural network? Artificial Human Neurons Each

888 views • 12 slides

DeepVision Fall Quarter Design Review Purpose We aim to build a Deep Neural Network Image

DeepVision Fall Quarter Design Review Purpose We aim to build a Deep Neural Network Image Recognition and Detection algorithm on a embedded GPU device Attending 2018 DAC Contest Sponsored and collaborated with Prof. Xie and SEAL LAB Finally

422 views • 20 slides

May 19: Design Principles and Confinement Principles of Secure Design One set; other sets

May 19: Design Principles and Confinement Principles of Secure Design One set; other sets say basically the same thing Confinement, non-VM isolation Library operating systems Sandboxes Program modification Covert

584 views • 30 slides

Same, Same But Different Recovering Neural Network Quantization Error Through Weight

Same, Same But Different Recovering Neural Network Quantization Error Through Weight Factorization Eldad Meller ICML 2019 Neural Network Quantization Quantization of Neural Networks is needed for efficient inference Quantization adds

482 views • 9 slides

Deep Learning Primer Nishith Khandwala Neural Networks Overview Neural Network Basics

Deep Learning Primer Nishith Khandwala Neural Networks Overview Neural Network Basics Activation Functions Stochastic Gradient Descent (SGD) Regularization (Dropout) Training Tips and Tricks Neural Network (NN)

828 views • 45 slides

Eco-design principles in a business network: A case study on the work of designers in developing

Eco-design principles in a business network: A case study on the work of designers in developing a large-made-to- order product Pekka Murto (Aalto University School of Arts, Design and Architecture) Oscar Person (Aalto University School of

535 views • 15 slides

Graph Neural Network Fang Yuanqiang, 2019/05/18 Graph Neural Network Why GNN? Preliminary

Graph Neural Network Fang Yuanqiang, 2019/05/18 Graph Neural Network Why GNN? Preliminary Fixed graph Vanilla Spectral Graph ConvNets ChebyNet GCN CayleyNet Multiple graphs Variable graph

817 views • 46 slides

Neural network applications ALVINN (Pomerleau, mid 1990s) Autonomous Land Vehicle in Neural

Neural network applications ALVINN (Pomerleau, mid 1990s) Autonomous Land Vehicle in Neural Network To date: Sharp Straight Sharp Left Ahead Right Neural networks: what are they 30 Output Units Backpropagation: efficient

346 views • 15 slides

Neural Networks for Machine Learning Lecture 2a An overview of the main types of neural network

Neural Networks for Machine Learning Lecture 2a An overview of the main types of neural network architecture Geoffrey Hinton with Nitish Srivastava Kevin Swersky Feed-forward neural networks These are the commonest type of neural

534 views • 32 slides

Human-Computer Interaction 12. Design Principles (2) Last class Psychological design principles

Human-Computer Interaction 12. Design Principles (2) Last class Psychological design principles Recap: Psychological principles 1. User sees what they expect to see. 2. Users have difficulty focusing on more than one activity at a time. 3. It

843 views • 55 slides

Similarity of Neural Network Representations Revisited Simon Kornblith, Mohammad Norouzi,

Similarity of Neural Network Representations Revisited Simon Kornblith, Mohammad Norouzi, Honglak Lee, Geofgrey Hinton Motivation We need tools to understand trained neural networks Neural network training involves interactions between an

588 views • 16 slides

UMBC A B M A L F T U M B C I O M Y O T R 1 (November 26, 2000 11:15 pm) I E

Principles of VLSI Design Introduction CMPE 413/CMSC 711 Principles of VLSI Design Instructor: Professor Jim Plusquellic Text: Principles of CMOS VLSI Design: A Systems Perspective, by Neil H.E. Weste and Kamran Eshraghian. Supplementary

752 views • 21 slides

Principles of Software Construction: Objects, Design, and Concurrency API Design 2: principles

Principles of Software Construction: Objects, Design, and Concurrency API Design 2: principles Josh Bloch Charlie Garrod 17-214 1 Administrivia Homework 4c due Thursday, 10/29, 11:59pm EST Homework 5 coming soon Team sign-up

620 views • 47 slides

Lecture 2: Deeper Neural Network 1 Objective In the second lecture, you will see How to

Lecture 2: Deeper Neural Network 1 Objective In the second lecture, you will see How to deepen your neural network to learn more complex functions Several techniques to fasten your learning process The most popular artificial

423 views • 40 slides

Neural network architectures for image captioning By Emily Kern Given a set of images and

Neural network architectures for image captioning By Emily Kern Given a set of images and accompanying human-generated captions, can we train a neural network to predict captions for new images? What is a neural network? A computer system

523 views • 16 slides

Unsupervised clustering with growing self-organizing neural network A comparison with non-neural

Unsupervised clustering with growing self-organizing neural network A comparison with non-neural approach Martin Hynar, Michal Burda, Jana Sarmanov a Department of Computer Science, V SB Technical University of Ostrava, 17.

473 views • 24 slides