boosting the deep multidimensional long short
play

Boosting the deep multidimensional long short- term memory network - PowerPoint PPT Presentation

Jul 31, 2023 •366 likes •693 views

Boosting the deep multidimensional long short- term memory network for handwritten recognition systems Dayvid Castro 1 Byron L. D. Bezerra 1 Muser Valena 1 1 Polytechnic School of Pernambuco University of Pernambuco Recife, Brazil The 16th

  1. Boosting the deep multidimensional long short- term memory network for handwritten recognition systems Dayvid Castro 1 Byron L. D. Bezerra 1 Mêuser Valença 1 1 Polytechnic School of Pernambuco University of Pernambuco Recife, Brazil

  2. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) Handwriting Text Recognition (HTR) ❖ Handwritten entry digital representation ❖ Offline Recognition 250 251 251 ... 255 255 255 251 251 251 ... 255 255 255 251 251 250 ... 255 255 255 ... 59 74 177 ... 255 255 255 59 140 204 ... 255 255 255 74 177 217 ... 255 255 255 3

  3. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) Offline HTR Challenges ❖ Variability ➢ Different writing styles ➢ Instrument (pen/pencil) ➢ Paper type and quality ➢ Space and time available ➢ Vocabulary ❖ Similarity ➢ Similar shapes 4

  4. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) Unconstrained Offline HTR ❖ Long text line sequences ❖ Cursive nature Open Problem ❖ Different writing styles ❖ Large vocabulary Segmentation-free approaches 5

  5. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) Deep Neural Networks for Unconstrained HTR ❖ Multiple Layers ❖ Representation Learning ❖ Building Blocks: ➢ Convolutional and Pooling Layers ➢ Recurrent Layers ➢ Long Short-Term Memory (LSTM) ➢ (Bi x Multi)dimentional flow ➢ CTC Graves et al. 2009 6

  6. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) MDLSTM Network Hierarchy in HTR Pham et al. 2014 7

  7. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) MDLSTM Network Hierarchy in HTR Voigtlaender et al. 2016 GPU implementation of MDLSTM (RETURNN tool) -> Deeper configurations 8

  8. Hypothesis ❖ The goal ❖ Optical model proposal and Proposal 9

  9. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) Main Goal The main goal of this work was to investigate alternative optical modeling approaches that can contribute to the optimization of offline and unconstrained HTR systems.  New hierarchical representations for a MDLSTM optical model  Speed-ups the training and inference time at the hierarchical-level 10

  10. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) Proposal and hypothesis 1. Repositioning convolutional and recurrent aspects of the state-of- the-art MDLSTM Voigtlaender model may be useful to discard low- frequency features and send to the MDLSTM layers a richer representation of the input data 2. Adding an extra max pooling to decrease computational time and improve the invariance to small shifts and distortions 11

  11. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) Optical Model (six hidden layers) Baseline Proposal 12

  12. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) Optical Model (eight hidden layers) Baseline Proposal 13

  13. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) Optical Model (ten hidden layers) Baseline Proposal 14

  14. ❖ Evaluating the MDLSTM optical model ❖ Including Linguistic Experiments Knowledge ❖ Comparison with the state-of- the-art 15

  15. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) Experiments Dataset detailed information Partition Train. Train. Width Height Dataset Language Training Validation Test # Symbols (Avg) (Avg) IAM English 6.161 (747) 976 (116) 2.781 (336) 79 1.751 124 RIMES French 10.203 (1351) 1.130 (149) 778 (100) 99 1.658 113 16

  16. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) Network Training Tool : RETURNN Batch size : 600.000 pixels Weight Initialization : Glorot or Xavier Initialization Gradient Descent : Nadam optimizer Learning Rates Schedule : 0.0005 (1-24), 0.0003 (25-34), 0.0001 (35-Early Stopping) Training Duration: Early Stopping with patience=20 17

  17. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) C = single conv. layer Optimizing network topologies LP = conv with pooling followed by MDLSTM L = conv without pooling followed by MDLSTM on the IAM dataset M = single MDLSTM Layer

  18. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) Experimental Results Summary ● The modifications did not hurt the recognition performance (hypothesis test confirmed this results) ● Faster model ○ Reduction of roughly 50% and 30% in training and classification times respectively. ● Optimal configuration obtained with eight-layers while the baseline presents ten-layers. ● The proposal presents generalization benefits on larger models. 19

  19. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) The complete HTR system 20

  20. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) Preprocessing ➢ No preprocessing ➢ Dislanting ➢ Inversion of pixel values 21

  21. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) Linguistic knowledge-based decoding Hybrid ANN/HMM scheme Finite-state transducers (FST): ❖ HMM transducers (H): each character is represented by an HMM. ❖ Lexicon FST (L): maps a sequence of characters to a valid word. ❖ Grammar FST (G): represents the n -gram language model on computing the probability of word sequences. Compose the H, L, and, G in a decoding graph and search for the most likely transcription using a beam search algorithm. 22

  22. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) Language Model Experimental Setup Tool: SRILM ● Language model: 3-gram language model ● Smoothing technique : modified Kneser-Ney ● Text source: Brown, LOB, and Wellington corpus. ● Vocabulary : 50.000 words ● Perplexity and OOV on the valid set : 270 (3.1% OOV) ● Perplexity and OOV rate on test set : 304 (2.9% OOV) ● 23

  23. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) Decoding Experimental Setup ● Decoders : ○ Best path decoding for tuning the network topology ○ Linguistic knowledge-based decoding for final results ■ The HMM, lexicon, and language models are represented as Finite-state transducers (FST) ● Tool: Kaldi toolkit 24

  24. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) Experimental Results Including Linguistic Knowledge - Prior scale tuning Optical scale fixed at 1.0 Optimal value : 0.7 25

  25. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) Experimental Results Including Linguistic Knowledge - Optical scale tuning Prior scale fixed at 0.7 Optimal value : 0.6 26

  26. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) Experimental Results Second fine-tuning for the optical scale Prior scale fixed at 0.70 Optimal result: 0.65 27

  27. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) Experimental Results Including Linguistic Knowledge Baseline system (without Ling. Know.) 24 6.64 28

  28. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) Comparison with the state-of-the-art - IAM 29

  29. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) Brand new results ● According with the published results of the ICFHR2018 Competition on Automated Text Recognition on a READ Dataset, our approach achieved the best rate when using only the general dataset provided in the first round of this competition!!! ● We have verified our proposed optical model architecture outperforms the baseline system in the Rimes dataset with a confidence level of 95%. 30

  30. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) Conclusion Main Contributions ● New MDLSTM hierarchical representation able to reduce the training and classification times without affecting the recognition quality. Faster experimental investigations Faster model Faster HTR systems ● Important tradeoff information between the depth and width of the proposed MDLSTM model. ● Evaluation of the MDLSTM variant in a hybrid ANN/HMM scheme with linguistic knowledge. 31

  31. The 16th International Conference on Frontiers in Handwriting Recognition (ICFHR 2018) Future Works ○ Apply the convolutional layer repositioning strategy with the (1D,B)LSTM HTR system, taking advantage of the recent results presented by Puigcerver et al. (2017) in ICDAR. ○ Explore the Open-vocabulary scenario ○ Evaluate the model with data augmentation 32

  32. Boosting the deep multidimensional long short- term memory network for handwritten recognition systems Prof. Byron L. D. Bezerra byronleite@ecomp.poli.br

Recommend

Identification and quantification of isoforms in RNAseq data : deep short reads Vs shallow long

Identification and quantification of isoforms in RNAseq data : deep short reads Vs shallow long

Identification and quantification of isoforms in RNAseq data : deep short reads Vs shallow long reads Vincent Lacroix Laboratoire de Biomtrie et Biologie volutve INRIA ERABLE What do we do in Lyon We are interested in developing

531 views • 35 slides
CS7015 (Deep Learning) : Lecture 15 Long Short Term Memory Cells (LSTMs), Gated Recurrent Units

CS7015 (Deep Learning) : Lecture 15 Long Short Term Memory Cells (LSTMs), Gated Recurrent Units

CS7015 (Deep Learning) : Lecture 15 Long Short Term Memory Cells (LSTMs), Gated Recurrent Units (GRUs) Mitesh M. Khapra Department of Computer Science and Engineering Indian Institute of Technology Madras 1/43 Mitesh M. Khapra CS7015 (Deep

4.9k views • 196 slides
CS7015 (Deep Learning) : Lecture 15 Long Short Term Memory Cells (LSTMs), Gated Recurrent Units

CS7015 (Deep Learning) : Lecture 15 Long Short Term Memory Cells (LSTMs), Gated Recurrent Units

CS7015 (Deep Learning) : Lecture 15 Long Short Term Memory Cells (LSTMs), Gated Recurrent Units (GRUs) Mitesh M. Khapra Department of Computer Science and Engineering Indian Institute of Technology Madras 1/43 Mitesh M. Khapra CS7015 (Deep

467 views • 43 slides
Cluster Computing: Cluster Computing: You've Come A Long Way You've Come A Long Way In A Short

Cluster Computing: Cluster Computing: You've Come A Long Way You've Come A Long Way In A Short

LCSC 5th Annual Workshop on Linux Clusters for Super Computing October 18-21, 2004 Linkping University, Sweden Cluster Computing: Cluster Computing: You've Come A Long Way You've Come A Long Way In A Short Time In A Short Time Jack

565 views • 22 slides
Boosting Recommender Systems with Deep Learning Joo Gomes RecSys 2017 Como, Italy 200

Boosting Recommender Systems with Deep Learning Joo Gomes RecSys 2017 Como, Italy 200

Boosting Recommender Systems with Deep Learning Joo Gomes RecSys 2017 Como, Italy 200 clickstream Platform for 230 Countries 2500 Brands 300K Products 1800+ employees events / sec Luxury Fashion 500 Boutiques 4M users 20+ in Data

193 views • 17 slides
Boosting Visual Object Tracking Using Deep Features and GPU Implementations Michael Felsberg

Boosting Visual Object Tracking Using Deep Features and GPU Implementations Michael Felsberg

Boosting Visual Object Tracking Using Deep Features and GPU Implementations Michael Felsberg Computer Vision Laboratory Department of Electrical Engineering Linkping University michael.felsberg@liu.se Martin Danelljan Fahad Khan

440 views • 28 slides
Boosting New Physics Searches with Deep Learning David Shih NHETC, Rutgers University

Boosting New Physics Searches with Deep Learning David Shih NHETC, Rutgers University

Boosting New Physics Searches with Deep Learning David Shih NHETC, Rutgers University Accelerating the Search for Dark Matter with Machine Learning ICTP , Trieste April 9, 2019 Announcement You are invited to submit an abstract for the ML

729 views • 62 slides
ECG782: Multidimensional Digital Signal Processing Object Recognition

ECG782: Multidimensional Digital Signal Processing Object Recognition

ECG782: Multidimensional Digital Signal Processing Object Recognition http://www.ee.unlv.edu/~b1morris/ecg782/ 2 Outline Knowledge Representation Statistical Pattern Recognition Neural Networks Boosting 3 Object Recognition

551 views • 21 slides
The Long and Short of Passwords Rich Shay November 5, 2009 1 / 34 The Long and Short of

The Long and Short of Passwords Rich Shay November 5, 2009 1 / 34 The Long and Short of

The Long and Short of Passwords The Long and Short of Passwords Rich Shay November 5, 2009 1 / 34 The Long and Short of Passwords Motivation for Studying Passwords Outline Motivation for Studying Passwords 1 2 Overview of Password Failure

373 views • 36 slides
EE 355 Unit 5 Multidimensional Arrays Mark Redekopp 2 MULTIDIMENSIONAL ARRAYS 3

EE 355 Unit 5 Multidimensional Arrays Mark Redekopp 2 MULTIDIMENSIONAL ARRAYS 3

1 EE 355 Unit 5 Multidimensional Arrays Mark Redekopp 2 MULTIDIMENSIONAL ARRAYS 3 Multidimensional Arrays Thus far arrays can be thought of 1-dimensional (linear) sets 0 1 2 3 4 5 only indexed with 1 value (coordinate) 01

534 views • 28 slides
The Short- and Long-Term Economic Outlook Short-Term Update Jobs: PDX v US 3 3 Job Growth by

The Short- and Long-Term Economic Outlook Short-Term Update Jobs: PDX v US 3 3 Job Growth by

The Short- and Long-Term Economic Outlook Short-Term Update Jobs: PDX v US 3 3 Job Growth by Sector 2013Q2 v 2014Q2 12% 10% 8% 6% 4% 2% 0% -2% 4 4 Source: Brookings Metropolitan Monitor Unemployment 5 5 Median Household Income,

527 views • 15 slides
MWV Corp May 2015 Availability of short fiber near the mill Cost of long hauling short

MWV Corp May 2015 Availability of short fiber near the mill Cost of long hauling short

Peter W. Hart, John Johnson, Ricardo Paim MWV Corp May 2015 Availability of short fiber near the mill Cost of long hauling short fiber to the mill Texas mill uses about 2/3 short fiber Southern Brazil has significant

583 views • 21 slides
Deep learning for natural language processing A short primer on deep learning Benoit Favre <

Deep learning for natural language processing A short primer on deep learning Benoit Favre <

Deep learning for natural language processing A short primer on deep learning Benoit Favre < benoit.favre@univ-mrs.fr > Aix-Marseille Universit, LIF/CNRS 20 Feb 2017 Benoit Favre (AMU) DL4NLP: deep learning 20 Feb 2017 1 / 25 Deep

530 views • 25 slides
Deep Learning: multi-layer neural networks Recurrent Neural Networks: sequence data Long

Deep Learning: multi-layer neural networks Recurrent Neural Networks: sequence data Long

Deep Learning: multi-layer neural networks Recurrent Neural Networks: sequence data Long Short-Term Memory: LSTM

366 views • 18 slides
RECSM Summer School: Machine Learning for Social Sciences Session 2.4: Boosting Reto West

RECSM Summer School: Machine Learning for Social Sciences Session 2.4: Boosting Reto West

RECSM Summer School: Machine Learning for Social Sciences Session 2.4: Boosting Reto West Department of Political Science and International Relations University of Geneva 1 Boosting Boosting Like bagging, boosting is a general

373 views • 13 slides
Recurrent Neural Networks Greg Mori - CMPT 419/726 Goodfellow, Bengio, and Courville: Deep

Recurrent Neural Networks Greg Mori - CMPT 419/726 Goodfellow, Bengio, and Courville: Deep

Recurrent Neural Networks Long Short-Term Memory Temporal Convolutional Networks Examples Recurrent Neural Networks Greg Mori - CMPT 419/726 Goodfellow, Bengio, and Courville: Deep Learning textbook Ch. 10 Recurrent Neural Networks Long

667 views • 18 slides
Milestoning: Use of short trajectories to compute long 6me

Milestoning: Use of short trajectories to compute long 6me

Milestoning: Use of short trajectories to compute long 6me kine6cs and thermodynamics Anthony West, Peter Majek, Serdal Kirmizial:n, Alfredo Cardenas, Steven

768 views • 34 slides
CS885 Reinforcement Learning Lecture 12: June 8, 2018 Deep Recurrent Q-Networks [GBC] Chap. 10

CS885 Reinforcement Learning Lecture 12: June 8, 2018 Deep Recurrent Q-Networks [GBC] Chap. 10

CS885 Reinforcement Learning Lecture 12: June 8, 2018 Deep Recurrent Q-Networks [GBC] Chap. 10 University of Waterloo CS885 Spring 2018 Pascal Poupart 1 Outline Recurrent neural networks Long short term memory (LSTM) networks Deep

116 views • 11 slides
Multidimensional SAR SAR Imaging Imaging: : Multidimensional Studies in the in the Framework

Multidimensional SAR SAR Imaging Imaging: : Multidimensional Studies in the in the Framework

FRINGE 2007 Workshop - European Space Agency Multidimensional SAR SAR Imaging Imaging: : Multidimensional Studies in the in the Framework Framework Studies of LIMES Project of LIMES Project M. Costantini 1 , G. Fornaro 2 , F. Lombardini 3

442 views • 16 slides
Boosting (ensemble) Module 4 - Ensemble classifiers - Objectives module 4: boosting (ensemble

Boosting (ensemble) Module 4 - Ensemble classifiers - Objectives module 4: boosting (ensemble

Boosting (ensemble) Module 4 - Ensemble classifiers - Objectives module 4: boosting (ensemble models) LEARNING PERFORMANCE REPRESENTATION DATA PROBLEM RAW DATA CLUSTERING EVALUATION FEATURES UCI datasets unigrams 20newsgroups SUPERVISED

926 views • 34 slides
Deep Neural Networks for PDEs Philipp Grohs DL and Vis, September 2018 Short Reading List 1 Ian

Deep Neural Networks for PDEs Philipp Grohs DL and Vis, September 2018 Short Reading List 1 Ian

Deep Neural Networks for PDEs Philipp Grohs DL and Vis, September 2018 Short Reading List 1 Ian Goodfellow and Yoshua Bengio and Aaron Courville: Deep Learning; MIT Press, 2016 2 Aurelien Geron: Hands-On Machine Learning with Scikit-Learn and

1.44k views • 117 slides
Boosting Methods: Implicit Combinatorial Optimization via First-Order Convex Optimization Robert

Boosting Methods: Implicit Combinatorial Optimization via First-Order Convex Optimization Robert

Motivation Review of SD and FW(CG) Binary Classification and Boosting Linear Regression and Boosting Boosting Methods: Implicit Combinatorial Optimization via First-Order Convex Optimization Robert M. Freund Paul Grigas Rahul Mazumder

707 views • 49 slides
High Capability Multidimensional Data High Capability Multidimensional Data Compression on GPUs

High Capability Multidimensional Data High Capability Multidimensional Data Compression on GPUs

S5455, GTC 2015 High Capability Multidimensional Data High Capability Multidimensional Data Compression on GPUs Sergio E. Zarantonello szarantonello@scu.edu Ed Karrels ed.karrels@gmail.com School of Engineering, Santa Clara University Part 1:

650 views • 25 slides
Beyond Short Snippets: Deep Networks for Video Classification Joe Yue-Hei Ng, Matthew Hausknecht,

Beyond Short Snippets: Deep Networks for Video Classification Joe Yue-Hei Ng, Matthew Hausknecht,

Beyond Short Snippets: Deep Networks for Video Classification Joe Yue-Hei Ng, Matthew Hausknecht, Sudheendra Vijayanarasimhan, Oriol Vinyals, Rajat Monga, George Toderici zge Yalnkaya Introduction Many attempts to apply CNNs to action

690 views • 20 slides

More recommend