deep neural networks
play

Deep Neural Networks Machine Learning Organizational info All - PowerPoint PPT Presentation

Sep 11, 2023 •575 likes •1.12k views

10-701: Introduction to Deep Neural Networks Machine Learning Organizational info All up-to-date info is on piazza. Instructors - Ziv Bar-Joseph - Eric Xing TAs: See info on piazza for recitations, office hours etc. See also

  1. 10-701: Introduction to Deep Neural Networks Machine Learning

  2. Organizational info • All up-to-date info is on piazza. • Instructors - Ziv Bar-Joseph - Eric Xing • TAs: See info on piazza for recitations, office hours etc. • See also piazza for contact info, office hours, etc. • Piazza would be used for questions / comments and for class quizzes. Make sure you are subscribed. • We will also use piazza for determining class participation

  3. Eric Xing (epxing@cs.cmu.edu) Models and Algorithms • Graphical • Nonparametric • Regularized • Spectral/Matrix • Sparse Structured Neural Nets • Sparse Coding • Large-Margin Models Bayesian Models Bayesian Methods Methods I/O Regression Trustworthy Personalized Meta ML Auto ML ML ML ML Compositionality Adaptive Distributed Big Data Scheduler ML Systems Tools System Compositionality Hardware and infrastructure • Network switches • Network attached storage • Server machines • GPUs • Cloud compute • Virtual Machines • Infiniband • Flash storage • Desktops/Laptops (e.g. Amazon EC2) 2020 IC, SCS@CMU • NUMA machines • Research Interests – Machine Learning: Theory & System – Healthcare and other Applications – Way to Learn: Auto, Trustworthy, Office: GHC Personalizable, and Transferable ML 8101 Office hours: TBD Course Instructor

  4. Daniel Bird (dpbird@andrew.cmu.edu) Education Associate for 10-701 Please email me if you have any issues in the course!

  5. Roger Iyengar (raiyenga@andrew.cmu.edu) PhD in Computer Science Department Interests: Edge Computing, Wearable Cognitive Assistance, Distributed Systems Research Interests in ML: Computer Vision, Natural Language Processing

  6. Abhi Adduri (aadduri@andrew.cmu.edu) PhD in Computational Biology

  7. Clay Yoo (hyungony@andrew.cmu.edu) Masters in Language Technology Institute Area of interest: Natural Language Processing, Data Visualization, Model Interpretability

  8. John Grace (jmgrace@andrew.cmu.edu) Masters in Computer Science Department Area of interest: Parallel Computing and Automated Program Synthesis.

  9. Chandreyee Bhaumik (cbhaumik@andrew.cmu.edu) Masters in the Robotics Institute Area of Interest: Reinforcement Learning, Computer Vision

  10. Bhuvan Agrawal (bhuvana@andrew.cmu.edu) Masters in Computer Science Department

  11. Jie Jiao (jiejiao@andrew.cmu.edu) Undergraduate in Computer Science Department Area of interest: Reinforcement Learning, Natural Language Processing

  12. 8/31 Intro, Three Axes of ML: Data, Algorithms, Tasks, Intro to probability Foundations and Non- 9/2 Bayesian Estimation, MAP, MLE Parametric Methods 9/7 – no class, labor day 9/9 – Decision Theory, Risk Minimization, K nearest neighbors 9/14 – Naive Bayes, Generative vs Discriminative 9/16 – Decision Trees 9/21 - Linear regression 9/23 - Logistic Regression Prediction, 9/28 – No class, Yom Kippur Parametric 9/30 – Support Vector Machines 1 Methods 10/05 – SVM2 10/07 – Neural Networks and Deep Learning 10/12 – Neural Networks and Deep Learning II 10/14 – Boosting, Surrogate Losses, Ensemble Methods Unsupervised Learning 10/19 - Clustering, Kmeans 10/21 - Clustering: Mixture of Gaussians, Expectation Maximization 10/26 – Representation Learning: Feature Transformation, Random Features, PCA Graphical and sequence 10/28 – Representation Learning: PCA Contd, ICA/ project proposals due 11/02 - Graphical Models (Bayesian Networks) models 11/04 - Graphical Models (BN2) 11/09 - Sequence Models: HMMs 11/11 - Sequence Models: State Space Models, other time series models Theoretical 11/16 – Learning Theory: Statistical Guarantees for Empirical Risk Minimization considerations 11/18 – Generalization, Model Selection 11/23 (Wednesday): Exam 11/23 - Exam 11/25 – No class, Thanksgiving break 11/30 – Industry lecture 12/02 – Reinforcement Learning 12/07 – Reinforcement Learning 2 12/09 (Wednesday) Poster presentations Actions 12/09 - Project presentations

  13. Grading • 5 Problem sets - 40% • Exam - 30% • Project - 30%

  14. Class assignments • 5 Problem sets - Both theoretical and programming assignments • Project - Select from a small list of suggested topics - We expect that multiple groups would work on a similar project - Groups of 3 - Poster session (recorded) and a short writeup • Exams - A single exams covering all topics taught in class up to the date - During class dates but likely in the afternoon (5-7pmt) • Recitations - Every Friday - Expand on material learned in class, go over problems from previous classes etc. • Office hours based on your section

  15. What is Machine Learning? Easy part: Machine Hard part: Learning - Short answer: Methods that can help generalize information from the observed data so that it can be used to make better decisions in the future

  16. What is Machine Learning? LEARNING DATA KNOWLEDGE ALGORITHMS

  17. Machine Learning LEARNING DATA KNOWLEDGE ALGORITHMS • Algorithms that improve their knowledge towards some task with data • How is it different from Statistics? - Same, but with better PR? - Statistics + Computation? • What is its relationship with AI, Data Science, Data Mining?

  18. Machine Learning • It is useful to differentiate these different fields by their goals • The goal of machine learning is the underlying mechanisms and algorithms that allow improving our knowledge with more data - Data construed broadly, e.g. “experiences” - Knowledge construed broadly e.g. possible actions 18

  19. While there is overlap, there are also differences • Statistics: the goal is the understanding of the data at hand • Artificial Intelligence: the goal is to build an intelligent agent • Data Mining: the goal is to extract patterns from large-scale data • Data Science: the science encompassing collection, analysis, and interpretation of data

  20. From Data to Understanding … Machine Learning in Action 20

  21. Machine Learning in Action Supervised learning • Decoding thoughts from brain scans Rob a bank … 21

  22. Machine Learning in Action Supervised and unsupervised learning • Stock Market Prediction Y = ? X = Feb01 22

  23. Machine Learning in Action Supervised and unsupervised learning • Document classification Sports Science News 23

  24. Machine Learning in Action Supervised learning • Spam filtering Spam/ Not spam 24

  25. semi supervised learning

  26. Machine Learning in Action Supervised and reinforcement learning • Cars navigating on their own Boss, the self-driving SUV 1st place in the DARPA Urban Challenge. Photo courtesy of Tartan Racing. 26

  27. Supervised learning (though can also be trained in an unsupervised way) Google translate

  28. Distributed gradient descent based on bacterial movement Reasoning under uncertainty

Recommend

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan

Deep Learning with Neural Networks The Structure and Optimization of Deep Neural Networks Allan Zelener Machine Learning Reading Group January 7 th 2016 The Graduate Center, CUNY Objectives Explain some of the trends of deep learning and

680 views • 44 slides
Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking

Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking

Introduction to Deep Neural Networks 0. Logistics Spring 2020 1 Neural Networks are taking over! Neural networks have become one of the major thrust areas recently in various pattern recognition, prediction, and analysis problems In

987 views • 40 slides
Visualizing and Interpreting Deep Neural Networks Bolei Zhou Department of Information

Visualizing and Interpreting Deep Neural Networks Bolei Zhou Department of Information

Visualizing and Interpreting Deep Neural Networks Bolei Zhou Department of Information Engineering The Chinese University of Hong Kong Deep Neural Networks are Everywhere Playing Go Making Medical Decision Understanding Scenes Deep Neural

1.1k views • 46 slides
On the Expressive Power of Deep Neural Networks Maithra Raghu, Ben Poole, Jon Kleinberg, Surya

On the Expressive Power of Deep Neural Networks Maithra Raghu, Ben Poole, Jon Kleinberg, Surya

On the Expressive Power of Deep Neural Networks Maithra Raghu, Ben Poole, Jon Kleinberg, Surya Ganguli, Jascha Sohl Dickstein Tom Brady Deep Neural Networks Recent successes in using Deep neural networks for image classification,

1.32k views • 17 slides
Outline Evolution of neurocomputing Artificial neural networks Feed forward

Outline Evolution of neurocomputing Artificial neural networks Feed forward

Introduction to Neural Networks and Deep Learning Ling Guan References: S. Haykin, Neural Networks , 2 nd Edition, New Jersey: Prentice Hall, 1. 2004. C. M. Bishop, Neural Networks for Pattern Re cognition, New York: 2. Oxford University

451 views • 17 slides
Introduction to Artificial Intelligence Neural Networks - Deep Learning for NLP Janyl Jumadinova

Introduction to Artificial Intelligence Neural Networks - Deep Learning for NLP Janyl Jumadinova

Introduction to Artificial Intelligence Neural Networks - Deep Learning for NLP Janyl Jumadinova November 21, 2016 Neural Networks 2/20 Neural Networks 3/20 Neural Networks Neural computing requires a number of neurons , to be connected

813 views • 21 slides
Optimizing Deep Neural Networks Leena Chennuru Vankadara 26-10-2015 Table of Contents Neural

Optimizing Deep Neural Networks Leena Chennuru Vankadara 26-10-2015 Table of Contents Neural

Optimizing Deep Neural Networks Leena Chennuru Vankadara 26-10-2015 Table of Contents Neural Networks and loss surfaces Problems of Deep Architectures Optimization in Neural Networks Under fitting Proliferation of Saddle

1.67k views • 41 slides
Deep Neural Networks and Molecular Dynamics Roberto Car Princeton University MAX Conference

Deep Neural Networks and Molecular Dynamics Roberto Car Princeton University MAX Conference

Deep Neural Networks and Molecular Dynamics Roberto Car Princeton University MAX Conference ICTP, Trieste, Jan 29-31 2018 Deep Neural Networks When properly trained, deep NN are capable of learning the complex functional dependence of

442 views • 17 slides
CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided

CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided

CS7015 (Deep Learning) : Lecture 13 Visualizing Convolutional Neural Networks, Guided Backpropagation, Deep Dream, Deep Art, Fooling Convolutional Neural Networks Mitesh M. Khapra Department of Computer Science and Engineering Indian Institute

1.07k views • 72 slides
Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and

Deep Neural Networks and Deep Reinforcement Learning Deep Neural Networks and Deep Reinforcement Learning Deep Learning, Goodfellow, Bengio and Courville [chapt. 6,7,8]; AIMA [sect. 21.1-21.3]; Sutton and Barto, Reinforcement Learning: an

527 views • 35 slides
Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Propagating Error Backward Hyperparameters for Neural Networks } Multi-layer (deep) neural

Learning in Neural Networks w 1,3 w 3,5 1 3 5 w w 1,4 3,6 w w 2,3 4,5 2 4 6 w w 2,4 4,6 Class #18: Back-Propagation; } A neural network can learn a classification function by Tuning Hyper-Parameters adjusting its weights to

259 views • 4 slides
Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the

Deep Neural Networks Reveal a Gradient in the Complexity of Neural Representations across the Ventral Stream Umut Gcl and Marcel A. J. van Gerven Article overview by Ilya Kuzovkin Computational Neuroscience Seminar University of Tartu

852 views • 63 slides
Recurrent Neural Networks Deep neural networks have enabled major advances in machine learning

Recurrent Neural Networks Deep neural networks have enabled major advances in machine learning

Recurrent Neural Networks Deep neural networks have enabled major advances in machine learning and AI y t-1 y t y t+1 Computer vision h t-1 h t h t+1 Language translation Speech recognition h t-1 h t h t+1 Question answering x t-1 x t x t+1

454 views • 43 slides
Concise Introduction to Deep Neural Networks Outline: Classification problems Motivating

Concise Introduction to Deep Neural Networks Outline: Classification problems Motivating

Concise Introduction to Deep Neural Networks Outline: Classification problems Motivating Deep (large) Neural Network (DNN) classifiers Neurons and DNN architectures Numerical training of DNNs (supervised deep learning) Spiking

630 views • 33 slides
ICASSP 2017 Tutorial on Methods for Interpreting and Understanding Deep Neural Networks G.

ICASSP 2017 Tutorial on Methods for Interpreting and Understanding Deep Neural Networks G.

ICASSP 2017 Tutorial on Methods for Interpreting and Understanding Deep Neural Networks G. Montavon, W. Samek, K.-R. Mller Part 2: Making Deep Neural Networks Transparent 5 March 2017 Making Deep Neural Nets Transparent DNN transparency

448 views • 44 slides
Deep Neural Networks CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu Deep learning slides credit:

Deep Neural Networks CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu Deep learning slides credit:

Deep Neural Networks CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu Deep learning slides credit: Vlad Morariu Training (Deep) Neural Networks Computational graphs Improvements to gradient descent Stochastic gradient descent Momentum

814 views • 21 slides
6. Convolutional Neural Networks CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from

6. Convolutional Neural Networks CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from

6. Convolutional Neural Networks CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from Zsolt Kira Quiz coming up Next Monday (2/5) 30 minutes Topics: Optimization Basic neural networks Neural Network

619 views • 41 slides
DEEP NEURAL NETWORKS FOR OBJECT DETECTION Sergey Nikolenko Steklov Institute of Mathematics at

DEEP NEURAL NETWORKS FOR OBJECT DETECTION Sergey Nikolenko Steklov Institute of Mathematics at

DEEP NEURAL NETWORKS FOR OBJECT DETECTION Sergey Nikolenko Steklov Institute of Mathematics at St. Petersburg October 10, 2017, Seoul, Korea Outline Birds eye overview of deep learning Convolutional neural networks Chris

743 views • 32 slides
Weight Parameterizations in Deep Neural Networks Sergey Zagoruyko e Paris-Est, Universit

Weight Parameterizations in Deep Neural Networks Sergey Zagoruyko e Paris-Est, Universit

Weight Parameterizations in Deep Neural Networks Weight Parameterizations in Deep Neural Networks Sergey Zagoruyko e Paris-Est, Universit Ecole des Ponts ParisTech December 26, 2017 Weight Parameterizations in Deep Neural Networks

1.05k views • 45 slides
CS480/680 Lecture 15: June 26, 2019 Deep Neural Networks [GBC] Chap. 6, 7, 8 University of

CS480/680 Lecture 15: June 26, 2019 Deep Neural Networks [GBC] Chap. 6, 7, 8 University of

CS480/680 Lecture 15: June 26, 2019 Deep Neural Networks [GBC] Chap. 6, 7, 8 University of Waterloo CS480/680 Spring 2019 Pascal Poupart 1 Outline Deep Neural Networks Gradient Vanishing Rectified linear units Overfitting

941 views • 25 slides
(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning

(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning

(Very) Brief Introduction to Neural Networks IITP-03 Algorithms for NLP 1 / 31 Learning objectives What are neural networks? What are deep neural networks? How do we train neural networks? What variants of neural network

331 views • 31 slides
Neural Networks Neural networks arise from attempts to model Neural Networks human/animal

Neural Networks Neural networks arise from attempts to model Neural Networks human/animal

Feed-forward Networks Network Training Error Backpropagation Deep Learning Feed-forward Networks Network Training Error Backpropagation Deep Learning Neural Networks Neural networks arise from attempts to model Neural Networks

380 views • 9 slides
Layer-wise Relevance Propagation in Neural Neural Networks Deep Learning Shortcomings Networks

Layer-wise Relevance Propagation in Neural Neural Networks Deep Learning Shortcomings Networks

LRP Ariyan Zarei Motivation Having More interpretable Layer-wise Relevance Propagation in Neural Neural Networks Deep Learning Shortcomings Networks to have more interpretable Papers and Demo Introduction Machine Learning models

373 views • 35 slides
Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks Yuan Cao

Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks Yuan Cao

Generalization Bounds of Stochastic Gradient Descent for Wide and Deep Neural Networks Yuan Cao and Quanquan Gu Computer Science Department 1 / 14 Learning Over-parameterized DNNs Empirical observation on extremely wide deep neural networks

631 views • 14 slides

More recommend