deep neural nets and keras
play

Deep Neural Nets and Keras Pavel Krmer 1 Data Science Summer School - PowerPoint PPT Presentation

May 12, 2023 •23 likes •313 views

Deep Neural Nets and Keras Pavel Krmer 1 Data Science Summer School @ Uni Vienna 1 Dept. of Computer Science, VB - Technical University of Ostrava, Ostrava, Czech Republic pavel.kromer@vsb.cz Outline Keras handson About Installation

  1. Deep Neural Nets and Keras Pavel Krömer 1 Data Science Summer School @ Uni Vienna 1 Dept. of Computer Science, VŠB - Technical University of Ostrava, Ostrava, Czech Republic pavel.kromer@vsb.cz

  2. Outline Keras hands–on About Installation Keras Fun with puppies, kitties, and DNNs Components September 04 2018, Vienna, AT 2

  3. About September 04 2018, Vienna, AT 2

  4. Introduction (Deep) artificial neural networks are among the most successful machine–learning models. They are universal tools that can be used for supervised and/or unsupervised learning. September 04 2018, Vienna, AT 3

  5. Artificial neural networks Artificial neural network • a computational model evaluating a parametric function composed of many other parametric (sub)functions • composed of many information processing units, organized into interconnected layers • one unit solves a linearly separable problem, i.e. draws a hyperplane in an n − dimensional space September 04 2018, Vienna, AT 4

  6. Keras September 04 2018, Vienna, AT 4

  7. • easy prototyping • support for convolutional and recurrent nets • accellerated by multicore and GPU Powered by a backend • Tensorflow (default) • Theano • others (CNTK) Keras Keras is a high-level neural networks API written in Python. September 04 2018, Vienna, AT 5

  8. Powered by a backend • Tensorflow (default) • Theano • others (CNTK) Keras Keras is a high-level neural networks API written in Python. • easy prototyping • support for convolutional and recurrent nets • accellerated by multicore and GPU September 04 2018, Vienna, AT 5

  9. Keras Keras is a high-level neural networks API written in Python. • easy prototyping • support for convolutional and recurrent nets • accellerated by multicore and GPU Powered by a backend • Tensorflow (default) • Theano • others (CNTK) September 04 2018, Vienna, AT 5

  10. • one can cheat in it Keras (cont.) My favourite because https://s3.amazonaws.com/assets.datacamp.com/ blog_assets/Keras_Cheat_Sheet_Python.pdf • sufficiently high–level (for my taste) • allows mixing–in with the wonderfull Python ecosystem (scikit, matplotlib, …) • is programmer oriented • well–documented, with lots of examples September 04 2018, Vienna, AT 6

  11. Keras (cont.) My favourite because https://s3.amazonaws.com/assets.datacamp.com/ blog_assets/Keras_Cheat_Sheet_Python.pdf • sufficiently high–level (for my taste) • allows mixing–in with the wonderfull Python ecosystem (scikit, matplotlib, …) • is programmer oriented • well–documented, with lots of examples • one can cheat in it September 04 2018, Vienna, AT 6

  12. • individual levels that define the architecture and functionality of the Model • different types, properties, params, functions • Dense layers (this is the normal, fully-connected layer) • Convolutional layers (applies convolution operations on the previous layer) • Pooling layers (used after convolutional layers) • Dropout layers (regularization, prevent overfitting) Keras components Model Layers • THE (deep) neural network you want to use • a stack of connected layers • sequential API × the bare Model class September 04 2018, Vienna, AT 7

  13. Keras components Model Layers • THE (deep) neural network you want to use • a stack of connected layers • sequential API × the bare Model class • individual levels that define the architecture and functionality of the Model • different types, properties, params, functions • Dense layers (this is the normal, fully-connected layer) • Convolutional layers (applies convolution operations on the previous layer) • Pooling layers (used after convolutional layers) • Dropout layers (regularization, prevent overfitting) September 04 2018, Vienna, AT 7

  14. • weight update strategies in the training process • stochastic gradient descent, RMSProp, Adagrad Keras components (cont.) Loss functions Optimizers • compare the predicted output with the real output in each pass of the training algorithm • tell the model how the weights should be updated • mean–squared error, cross–entropy, … September 04 2018, Vienna, AT 8

  15. Keras components (cont.) Loss functions Optimizers • compare the predicted output with the real output in each pass of the training algorithm • tell the model how the weights should be updated • mean–squared error, cross–entropy, … • weight update strategies in the training process • stochastic gradient descent, RMSProp, Adagrad September 04 2018, Vienna, AT 8

  16. Keras hands–on September 04 2018, Vienna, AT 8

  17. pip install tensorflow pip install keras pip install msgpack argparse pydot conda install keras conda install pydot Installation (Fairly) easy steps • Get Python (Anaconda highly recommended: https://www.anaconda.com/download/ ) • Get TensorFlow ( https://www.tensorflow.org/install/ ) • Get Keras ( https://keras.io/ ) September 04 2018, Vienna, AT 9

  18. pip install msgpack argparse pydot conda install keras conda install pydot Installation (Fairly) easy steps • Get Python (Anaconda highly recommended: https://www.anaconda.com/download/ ) • Get TensorFlow ( https://www.tensorflow.org/install/ ) • Get Keras ( https://keras.io/ ) pip install tensorflow pip install keras September 04 2018, Vienna, AT 9

  19. conda install keras conda install pydot Installation (Fairly) easy steps • Get Python (Anaconda highly recommended: https://www.anaconda.com/download/ ) • Get TensorFlow ( https://www.tensorflow.org/install/ ) • Get Keras ( https://keras.io/ ) pip install tensorflow pip install keras pip install msgpack argparse pydot September 04 2018, Vienna, AT 9

  20. Installation (Fairly) easy steps • Get Python (Anaconda highly recommended: https://www.anaconda.com/download/ ) • Get TensorFlow ( https://www.tensorflow.org/install/ ) • Get Keras ( https://keras.io/ ) pip install tensorflow pip install keras pip install msgpack argparse pydot conda install keras conda install pydot September 04 2018, Vienna, AT 9

  21. Published on Kaggle in 2014, contains 25,000 images of cats and dogs. To make it a bit harder, we use only 1000 training images of each class. The mother of all classification demos: cats vs. dogs September 04 2018, Vienna, AT 10

  22. To make it a bit harder, we use only 1000 training images of each class. The mother of all classification demos: cats vs. dogs Published on Kaggle in 2014, contains 25,000 images of cats and dogs. September 04 2018, Vienna, AT 10

  23. The mother of all classification demos: cats vs. dogs Published on Kaggle in 2014, contains 25,000 images of cats and dogs. To make it a bit harder, we use only 1000 training images of each class. September 04 2018, Vienna, AT 10

  24. Computer demo … https://goo.gl/M5ShF3

  25. From scratch TM 1 0 . 9 0 . 8 Accurracy 0 . 7 0 . 6 training validation 0 . 5 0 10 20 30 40 50 September 04 2018, Vienna, AT 11 Epoch

  26. From scratch TM September 04 2018, Vienna, AT 12

  27. VGG16 / ImageNet 1 0 . 9 0 . 8 Accurracy 0 . 7 0 . 6 training validation 0 . 5 0 10 20 30 40 50 September 04 2018, Vienna, AT 13 Epoch

  28. What VGG16 dreams about? September 04 2018, Vienna, AT 14

  29. What VGG16 dreams about? September 04 2018, Vienna, AT 14

Recommend

Deep Nets with http://vem.quantumunlimited.org/the-gates-of-horn/ Keras Professor

Deep Nets with http://vem.quantumunlimited.org/the-gates-of-horn/ Keras Professor

docs https://keras.io Deep Nets with http://vem.quantumunlimited.org/the-gates-of-horn/ Keras Professor Marie Roch These slides only cover enough to get started with feed-forward networks and do not cover regularization which is

134 views • 10 slides
Deep Neural Nets and Features Sung-Eui Yoon ( ) Course URL:

Deep Neural Nets and Features Sung-Eui Yoon ( ) Course URL:

CS688: Web-Scale Image Search Deep Neural Nets and Features Sung-Eui Yoon ( ) Course URL: http://sgvr.kaist.ac.kr/~sungeui/IR Class Objectives Browse main components of deep neural nets Does not aim for giving in-depth

609 views • 32 slides
The Fundamentals of Deep Learning Building Blocks Theory with Applications Neural Units Neural

The Fundamentals of Deep Learning Building Blocks Theory with Applications Neural Units Neural

Deep Learning Antecedents Vision Case Study The Fundamentals of Deep Learning Building Blocks Theory with Applications Neural Units Neural Nets Deep Neural Nets Application ConvNets Jon Krohn LSTMs untapt jon@untapt.com

1.08k views • 106 slides
MorphNet Elad Eban Faster Neural Nets with Hardware-Aware Architecture Learning Where Do

MorphNet Elad Eban Faster Neural Nets with Hardware-Aware Architecture Learning Where Do

MorphNet Elad Eban Faster Neural Nets with Hardware-Aware Architecture Learning Where Do Deep-Nets Come From? VGG: Chatfield et al. 2014 Image from: http://www.paddlepaddle.org/ How Do We Improve Deep Nets? Inception - Szegedy et al. 2015

967 views • 43 slides
Deep Convolutional Neural Nets COMPSCI 371D Machine Learning COMPSCI 371D Machine

Deep Convolutional Neural Nets COMPSCI 371D Machine Learning COMPSCI 371D Machine

Deep Convolutional Neural Nets COMPSCI 371D Machine Learning COMPSCI 371D Machine Learning Deep Convolutional Neural Nets 1 / 25 Outline 1 Why Neural Networks? 2 Circuits 3 Neurons, Layers, and Networks 4 Correlation and Convolution 5

287 views • 25 slides
Neural Nets Paul Humphreys July 10 2018 Deep learning methods: those that are formed by the

Neural Nets Paul Humphreys July 10 2018 Deep learning methods: those that are formed by the

Representations in Deep Neural Nets Paul Humphreys July 10 2018 Deep learning methods: those that are formed by the composition of multiple non-linear transformations, with the goal of yielding more abstract -- and ultimately more useful --

319 views • 12 slides
Evolving Keras Architectures for Sensor Data Analysis Petra Vidnerov Roman Neruda Institute

Evolving Keras Architectures for Sensor Data Analysis Petra Vidnerov Roman Neruda Institute

Evolving Keras Architectures for Sensor Data Analysis Petra Vidnerov Roman Neruda Institute of Computer Science The Czech Academy of Sciences FedCSIS 2017 Outline Introduction Deep Neural Networks KERAS library Related Work Our

497 views • 21 slides
RNN Recitation 10/27/17 Recurrent nets are very deep nets Y(T) h f (-1) X(0) The relation

RNN Recitation 10/27/17 Recurrent nets are very deep nets Y(T) h f (-1) X(0) The relation

RNN Recitation 10/27/17 Recurrent nets are very deep nets Y(T) h f (-1) X(0) The relation between and is one of a very deep network Gradients from errors at will vanish by the time theyre propagated to Recall: Vanishing stuff..

805 views • 58 slides
Deep Learning Barun Patra Index Convolutional Networks Introduction to Neural Nets

Deep Learning Barun Patra Index Convolutional Networks Introduction to Neural Nets

Deep Learning Barun Patra Index Convolutional Networks Introduction to Neural Nets Inspiration Activations Kernels Sigmoid Idea Tanh As used in NLP Relu (Derivatives) Paper Discussion

806 views • 46 slides
Splines and imaging: From compressed sensing to deep neural nets Michael Unser Biomedical

Splines and imaging: From compressed sensing to deep neural nets Michael Unser Biomedical

Splines and imaging: From compressed sensing to deep neural nets Michael Unser Biomedical Imaging Group EPFL, Lausanne, Switzerland Plenary talk: Int. Conf. Signal Processing and Communications (SPCOM20), IISc Bangalore, July 20-23, 2020

661 views • 22 slides
TRANSFERABLE CLEAN-LABEL POISONING ATTACKS ON DEEP NEURAL NETS Chen Zhu*, W. Ronny Huang*^, Ali

TRANSFERABLE CLEAN-LABEL POISONING ATTACKS ON DEEP NEURAL NETS Chen Zhu*, W. Ronny Huang*^, Ali

TRANSFERABLE CLEAN-LABEL POISONING ATTACKS ON DEEP NEURAL NETS Chen Zhu*, W. Ronny Huang*^, Ali Shafahi, Hengduo Li, Gavin Taylor, Christoph Studer, Tom Goldstein * equal contribution ^ presenter WHAT IS POISONING? Training data Testing

218 views • 19 slides
Deep neural nets for human pose estimation in videos Tomas Pfister, James Charles, Andrew

Deep neural nets for human pose estimation in videos Tomas Pfister, James Charles, Andrew

Deep neural nets for human pose estimation in videos Tomas Pfister, James Charles, Andrew Zisserman Department of Engineering Science University of Oxford http://www.robots.ox.ac.uk/~vgg Aim: Estimate 2D upper body joint positions (wrist,

519 views • 28 slides
ICASSP 2017 Tutorial on Methods for Interpreting and Understanding Deep Neural Networks G.

ICASSP 2017 Tutorial on Methods for Interpreting and Understanding Deep Neural Networks G.

ICASSP 2017 Tutorial on Methods for Interpreting and Understanding Deep Neural Networks G. Montavon, W. Samek, K.-R. Mller Part 2: Making Deep Neural Networks Transparent 5 March 2017 Making Deep Neural Nets Transparent DNN transparency

448 views • 44 slides
Today CS 188: Artificial Intelligence Neural Nets (wrap-up) and Decision Trees Neural Nets --

Today CS 188: Artificial Intelligence Neural Nets (wrap-up) and Decision Trees Neural Nets --

Today CS 188: Artificial Intelligence Neural Nets (wrap-up) and Decision Trees Neural Nets -- wrap Formalizing Learning Consistency Simplicity Decision Trees Expressiveness Information Gain Overfitting Instructors: Pieter

451 views • 13 slides
AUTOMATED STORY GENERATION WITH DEEP NEURAL NETS LARA J. MARTIN , PRITHVIRAJ AMMANABROLU, WILLIAM

AUTOMATED STORY GENERATION WITH DEEP NEURAL NETS LARA J. MARTIN , PRITHVIRAJ AMMANABROLU, WILLIAM

EVENT REPRESENTATIONS FOR AUTOMATED STORY GENERATION WITH DEEP NEURAL NETS LARA J. MARTIN , PRITHVIRAJ AMMANABROLU, WILLIAM HANCOCK, SHRUTI SINGH, BRENT HARRISON, AND MARK O. RIEDL GEORGIA INSTITUTE OF TECHNOLOGY AUTOMATED STORY GENERATION

486 views • 15 slides
Convolutional Neural Nets CS447 Natural Language Processing (J. Hockenmaier)

Convolutional Neural Nets CS447 Natural Language Processing (J. Hockenmaier)

Lecture 8: Convolutional Neural Nets Convolutional Neural Nets CS447 Natural Language Processing (J. Hockenmaier) https://courses.grainger.illinois.edu/cs447/ 1 Convolutional Neural Nets (ConvNets, CNNs) [4 parameters, applied 3

615 views • 24 slides
outline of this tutorial motivations 1 ACISS09 tutorial on deep belief nets deep

outline of this tutorial motivations 1 ACISS09 tutorial on deep belief nets deep

outline of this tutorial motivations 1 ACISS09 tutorial on deep belief nets deep autoencoders deep belief nets sigmoid belief nets 2 Marcus Frean why are they hard to train? could layer-by-layer training work? Melbourne, 2009

509 views • 15 slides
Compiling Deep Nets Scott Sanner Goal of this talk Will not evangelize deep networks /

Compiling Deep Nets Scott Sanner Goal of this talk Will not evangelize deep networks /

Compiling Deep Nets Scott Sanner Goal of this talk Will not evangelize deep networks / successes Go to ICML, NIPS, Silicon Valley, read tech news Just believe But deep nets do not solve all problems Yet Lack

393 views • 37 slides
Collapse of Deep and Narrow ReLU Neural Nets Lu Lu , Yeonjong Shin, Yanhui Su, George Karniadakis

Collapse of Deep and Narrow ReLU Neural Nets Lu Lu , Yeonjong Shin, Yanhui Su, George Karniadakis

Collapse of Deep and Narrow ReLU Neural Nets Lu Lu , Yeonjong Shin, Yanhui Su, George Karniadakis Division of Applied Mathematics, Brown University Scientific Machine Learning, ICERM January 28, 2019 Lu (Brown) ReLU NN Collapse Scientific ML

589 views • 32 slides
Feature extraction from deep models Olgert Denas Synopsis Intro to deep models Applications

Feature extraction from deep models Olgert Denas Synopsis Intro to deep models Applications

Feature extraction from deep models Olgert Denas Synopsis Intro to deep models Applications Neurons & Nets dimer Learning & Depth G1E model Feature extraction Theory 1 Layer Nets Neural computation

1.32k views • 45 slides
Training Deep Neural Nets Pawe Liskowski Institute of Computing Science, Pozna University of

Training Deep Neural Nets Pawe Liskowski Institute of Computing Science, Pozna University of

Training Deep Neural Nets Pawe Liskowski Institute of Computing Science, Pozna University of Technology 29 October 2018 1 / 20 Bias/Variance 2 / 20 Bias/Variance Train set error 3% 18% 18% 1% Test set error 14% 20% 30% 2% High

322 views • 20 slides
Deep Nets: What have they ever done for Vision? Alan Yuille Dept. Cognitive Science and

Deep Nets: What have they ever done for Vision? Alan Yuille Dept. Cognitive Science and

Deep Nets: What have they ever done for Vision? Alan Yuille Dept. Cognitive Science and Computer Science Johns Hopkins University What Have Deep Nets done to Computer Vision? Compared to human observers, Deep Nets are brittle and rely

460 views • 26 slides
Learning curves IN TRODUCTION TO DEEP LEARN IN G W ITH K ERAS Miguel Esteban Data Scientist

Learning curves IN TRODUCTION TO DEEP LEARN IN G W ITH K ERAS Miguel Esteban Data Scientist

Learning curves IN TRODUCTION TO DEEP LEARN IN G W ITH K ERAS Miguel Esteban Data Scientist & Founder INTRODUCTION TO DEEP LEARNING WITH KERAS INTRODUCTION TO DEEP LEARNING WITH KERAS INTRODUCTION TO DEEP LEARNING WITH KERAS

521 views • 48 slides
Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly

Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly

Convolutional Neural Nets 4-25-16 Reading Quiz Convolutional neural networks are most commonly used for what sort of task? a) Recognizing images b) Transcribing speech c) Translating documents d) Playing games Neural network review

431 views • 13 slides

More recommend