Recurrent Neural Networks using TensorFlow Jindich Libovick - PowerPoint PPT Presentation

Jan 31, 2024 •106 likes •194 views

Recurrent Neural Networks using TensorFlow Jindich Libovick December 5, 2018 B4M36NLP Introduction to Natural Language Processing Charles University Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics unless

Recurrent Neural Networks using TensorFlow Jindřich Libovický December 5, 2018 B4M36NLP Introduction to Natural Language Processing Charles University Faculty of Mathematics and Physics Institute of Formal and Applied Linguistics unless otherwise stated
The Taks Train a model that will correctly decide whether you should write ‘i’ or ‘y’ in a Czech sentence. Recurrent Neural Networks using TensorFlow 1/7
Prepare Python Environment virtualenv -p python3 env source env/bin/activate pip3 install tensorflow pip3 install jupyter Recurrent Neural Networks using TensorFlow 2/7 • create a new environment • activate the environment • install TensorFlow and Jupyter
Alternatively using Anaconda wget http://repo.continuum.io/archive/Anaconda3-4.2.0-Linux-x86_64.sh bash Anaconda3-4.2.0-Linux-x86_64.sh export PATH=$PATH:$HOME/anaconda3/bin conda create -n tf python=3.5 anaconda source activate tf pip3 install tensorflow pip3 install jupyter Recurrent Neural Networks using TensorFlow 3/7 • download and install Anaconda • create a new environment • activate the environment • install TensorFlow 0.12
Download the Lab Notebook wget http://ufallab.ms.mff.cuni.cz/~libovicky/ctu_lab.ipynb jupyter notebook Recurrent Neural Networks using TensorFlow 4/7 • download the notebook • run jupyter in the same directory
The Data The text: aristotelés dále určil poloměr země, kterí ale odhadl na dvojnásobek… v aristotelovském modelu země stojí a měsíc se sluncem a hvězdami krouží… mišlenki aristotelovi rozvinul ve 2. století našeho letopočtu klaudios… Correct solution: 00001000000000000000000000000000001000000100000000000000000000001000000100000… 02000002000100000000200000100000000000000001000000000000000000000001000000000… 00000000000000000010000000000001000002000000000000000000020000000000000000000… 1 = ‘i’, 2 = ‘y’, 0 = ‘others’ Recurrent Neural Networks using TensorFlow 5/7 • 500k sentences from Czech Wikipedia (in general the more, the better) • only character from Czech alphabet, sentence-splitted, lower-cased • randomly shuffmed, seperated validation data
Baselines 70 % 80 % 90 % Recurrent Neural Networks using TensorFlow 6/7 • just leave ‘i’ everywhere • simple rules: ‘y’ after ‘h’, ‘k’, ‘r’ and for words starting with ‘v’ • remember the most frequent spelling for each word
Learning curves Recurrent Neural Networks using TensorFlow 7/7 1 0.95 0.9 0.85 0.8 0.75 0.7 Accuracy 0.65 0.6 0.55 # Training sentences (log scale) 0.5 500 5000 50000 500000 5000000 Most frequent spelling Recurrent network Baseline Simple rules

Recommend

TensorFlow and Recurrent Neural Networks CSE392 - Spring 2019 Special Topic in CS Task

TensorFlow and Recurrent Neural Networks CSE392 - Spring 2019 Special Topic in CS Task Recurrent Neural Network how? Language Modeling and Implementation toolkit: (Most Tasks) TensorFlow Language Modeling Building a model

1.1k views • 60 slides

CHAPTER II I CHAPTER I Recurrent Neural Networks Recurrent Neural Networks CHAPTER II : I :

Ugur HALICI - METU EEE - ANKARA 11/18/2004 CHAPTER II I CHAPTER I Recurrent Neural Networks Recurrent Neural Networks CHAPTER II : I : Recurrent Neural Networks CHAPTER I Recurrent Neural Networks Introduction In this chapter first the

404 views • 27 slides

Neural Networks with Googles TensorFlow Shuo Zhang Computational discourse analysis 11/22/16

Neural Networks with Googles TensorFlow Shuo Zhang Computational discourse analysis 11/22/16 Overview 1. Neural Networks basics 2. Neural Networks specifics 3. Neural Networks with Googes TensorFlow 4. Coreference: Singleton

878 views • 30 slides

Computa(on through dynamics Using recurrent neural networks to unveil mechanism in neural

Computa(on through dynamics Using recurrent neural networks to unveil mechanism in neural circuits David Sussillo with Valerio Mante and Bill Newsome Table of contents Introduc(on Training recurrent neural networks(RNNs) Understanding how

1.57k views • 105 slides

CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks

CONVOLUTIONAL AND RECURRENT NEURAL NETWORKS Neural networks Fully connected networks Neuron Non-linearity Softmax layer DNN training Loss function and regularization SGD and backprop Learning rate Overfitting

1.8k views • 86 slides

Understanding LSTM Networks Recurrent Neural Networks An unrolled recurrent neural network The

Understanding LSTM Networks Recurrent Neural Networks An unrolled recurrent neural network The Problem of Long-Term Dependencies RNN short-term dependencies Language model trying to predict the next word based on the previous ones the clouds

729 views • 27 slides

Recurrent Neural Networks Greg Mori - CMPT 419/726 Goodfellow, Bengio, and Courville: Deep

Recurrent Neural Networks Long Short-Term Memory Temporal Convolutional Networks Examples Recurrent Neural Networks Greg Mori - CMPT 419/726 Goodfellow, Bengio, and Courville: Deep Learning textbook Ch. 10 Recurrent Neural Networks Long

667 views • 18 slides

Lecture 9: Recurrent Neural Networks Princeton University COS 495 Instructor: Yingyu Liang

Deep Learning Basics Lecture 9: Recurrent Neural Networks Princeton University COS 495 Instructor: Yingyu Liang Introduction Recurrent neural networks Dates back to (Rumelhart et al. , 1986) A family of neural networks for handling

348 views • 34 slides

Recurrent Neural Networks CS60010: Deep Learning Abir Das IIT Kharagpur Mar 11, 2020

Recurrent Neural Networks CS60010: Deep Learning Abir Das IIT Kharagpur Mar 11, 2020 Introduction Recurrent Neural Network LSTM Agenda Get introduced to different recurrent neural architecture e.g. , RNNs, LSTMs, GRUs etc. Get

743 views • 48 slides

CS6501: Deep Learning for Visual Recognition Recurrent Neural Networks (RNNs) Todays Class

CS6501: Deep Learning for Visual Recognition Recurrent Neural Networks (RNNs) Todays Class Recurrent Neural Network Cell Recurrent Neural Networks (RNNs) Bi-Directional Recurrent Neural Networks (Bi-RNNs) Multiple-layer /

793 views • 21 slides

CS6501: Deep Learning for Visual Recognition Recurrent Neural Networks (RNNs) Todays Class

583 views • 47 slides

CSEP 517: Natural Language Processing Recurrent Neural Networks Autumn 2018 Luke Zettlemoyer

CSEP 517: Natural Language Processing Recurrent Neural Networks Autumn 2018 Luke Zettlemoyer University of Washington [most slides from Yejin Choi] RECURRENT NEURAL L NE NETWOR WORKS Recurrent Neural Networks (RNNs) Each input

787 views • 29 slides

Sequential Data with Neural Networks Recurrent Neural Networks Sequential input / output Greg

Recurrent Neural Networks Long Short-Term Memory Temporal Convolutional Networks Examples Recurrent Neural Networks Long Short-Term Memory Temporal Convolutional Networks Examples Sequential Data with Neural Networks Recurrent Neural

303 views • 4 slides

IN5550 Neural Methods in Natural Language Processing Recurrent Neural Networks Stephan Oepen

IN5550 Neural Methods in Natural Language Processing Recurrent Neural Networks Stephan Oepen University of Oslo March 10, 2019 Our Roadmap Today Language structure: sequences, trees, graphs Recurrent Neural Networks Different

780 views • 76 slides

IN5550 Neural Methods in Natural Language Processing Applications of Recurrent Neural Networks

IN5550 Neural Methods in Natural Language Processing Applications of Recurrent Neural Networks Stephan Oepen University of Oslo March 24, 2019 Our Roadmap Last Week Language structure: sequences, trees, graphs Recurrent Neural

217 views • 19 slides

Convolutional Neural Networks (CNNs) Recurrent Neural Networks (RNNs) L1 Scalar Processor L0

Convolutional Neural Networks (CNNs) Recurrent Neural Networks (RNNs) L1 Scalar Processor L0 L0 M*V Processor N N N N N N Project Brainwave Pretrained DNN Model Scalable DNN Hardware Neural Processing Unit Microservice Extract

525 views • 30 slides

Recurrent Neural Networks (RNN) Artificial Intelligence @ Allegheny College Janyl Jumadinova

Recurrent Neural Networks (RNN) Artificial Intelligence @ Allegheny College Janyl Jumadinova March 9, 2020 Alex Graves, Supervised Sequence Labelling with Recurrent Neural Networks http://colah.github.io/posts/2015-08-Understanding-LSTMs/

538 views • 18 slides

Machine Learning for Computational Linguistics Recurrent neural networks (RNNs) ar

Machine Learning for Computational Linguistics Recurrent neural networks (RNNs) ar ltekin University of Tbingen Seminar fr Sprachwissenschaft July 5, 2016 Neural networks: a quick summary Recurrent neural networks July 5, 2016

570 views • 35 slides

The Power of Linear Recurrent Neural Networks Neural Networks Was knnen lineare rekurrente

The Power of Linear Recurrent The Power of Linear Recurrent Neural Networks Neural Networks Was knnen lineare rekurrente neuronale Netze? Frieder Stolzenburg Overview Frieder Stolzenburg Introduction Recurrent Neural Networks

1.17k views • 66 slides

Introduction to Recurrent Neural Networks Jakob Verbeek Modeling sequential data with Recurrent

Introduction to Recurrent Neural Networks Jakob Verbeek Modeling sequential data with Recurrent Neural Networks Compact schematic drawing of standard multi-layer perceptron (MLP) output hidden input Modeling sequential data So far we

796 views • 50 slides

INF5820: Language Technological Applications Applications of Recurrent Neural Networks Stephan

INF5820: Language Technological Applications Applications of Recurrent Neural Networks Stephan Oepen University of Oslo November 6, 2018 Most Recently: Variants of Recurrent Neural Networks To a first approximation, the de facto consensus in

339 views • 23 slides

RNN Input Layer RNN Hidden Layer RNN h t-1 h t x t (Picture adapted from Andrej

Outline Recurrent Neural Networks (RNNs) Neural Machine Transla/on NMT basics (Sutskever et al., 2014) ABenCon mechanism (Bahdanau et al., 2015) Marcello Federico 2016 Recurrent Neural Networks (RNNs) Based on slides kindly provided

437 views • 24 slides

Recurrent Neural Networks LING572 Advanced Statistical Methods for NLP March 5 2020 1 Outline

Recurrent Neural Networks LING572 Advanced Statistical Methods for NLP March 5 2020 1 Outline Word representations and MLPs for NLP tasks Recurrent neural networks for sequences Fancier RNNs Vanishing/exploding gradients

921 views • 91 slides

Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) CMSC 678 UMBC Recap

Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) CMSC 678 UMBC Recap from last time Feed-Forward Neural Network: Multilayer Perceptron + 0 ) = (

1.04k views • 93 slides