machine learning for computational linguistics
play

Machine Learning for Computational Linguistics ar ltekin University - PowerPoint PPT Presentation

Sep 12, 2023 •523 likes •778 views

Machine Learning for Computational Linguistics ar ltekin University of Tbingen Seminar fr Sprachwissenschaft April 12, 2016 Practical matters What and why The course plan When/where ccoltekin@sfs.uni-tuebingen.de).

  1. Machine Learning for Computational Linguistics Çağrı Çöltekin University of Tübingen Seminar für Sprachwissenschaft April 12, 2016

  2. Practical matters What and why The course plan When/where ccoltekin@sfs.uni-tuebingen.de). http://coltekin.net/cagri/courses/ml . include pointers to reading material for each lecture. Ç. Çöltekin, SfS / University of Tübingen April 12, 2016 1 / 16 ▶ Lectures: Tuesday/Thursday 08:30 at Hörsaal 0.02 ▶ Offjce hours: Tuesday 10:00-12:00, or by appointment (email ▶ Course web page: ▶ Reading material: no (single) textbook. Course web page will

  3. Practical matters What and why April 12, 2016 SfS / University of Tübingen Ç. Çöltekin, 2 / 16 Literature The course plan ▶ James et al. (2013) [online!] ▶ Hastie, Tibshirani, and J. Friedman (2009) [online!] ▶ Barber (2012) ▶ Murphy (2012) ▶ Bishop (2006) ▶ Mitchell (1997) ▶ Goodfellow, Bengio, and Courville (2016) [online copy] ▶ Alpaydın (2004) ▶ Witten and Frank (2005) ▶ Richert (2015) ▶ Lantz (2015) ▶ Cho (2015) ▶ Goldberg (2015)

  4. Practical matters What and why April 12, 2016 SfS / University of Tübingen Ç. Çöltekin, forming teams announced later) 3 / 16 Evaluation 6 (maybe 7) homeworks, to be done individually. The course plan ▶ Homeworks (30%) ▶ Term project / term paper (70%) ▶ Team work (up to 3 team members) is encouraged ▶ The project has to include a machine learning ‘experiment’ ▶ The results should be presented in a term paper (details will be ▶ You should already start thinking about project topics, and

  5. Practical matters information processing tasks. April 12, 2016 SfS / University of Tübingen Ç. Çöltekin, —James et al. (2013) data. Statistical learning refers to a vast set of tools for understanding —Barber (2012) mimicking, understanding and aiding human and biological What and why Machine Learning is the study of data-driven methods capable of —Mitchell (1997) with experience. how to construct computer programs that automatically improve The fjeld of machine learning is concerned with the question of Machine learning is … The course plan 4 / 16

  6. Practical matters information processing tasks. April 12, 2016 SfS / University of Tübingen Ç. Çöltekin, —James et al. (2013) data. Statistical learning refers to a vast set of tools for understanding —Barber (2012) mimicking, understanding and aiding human and biological What and why Machine Learning is the study of data-driven methods capable of —Mitchell (1997) with experience. how to construct computer programs that automatically improve The fjeld of machine learning is concerned with the question of Machine learning is … The course plan 4 / 16

  7. Practical matters information processing tasks. April 12, 2016 SfS / University of Tübingen Ç. Çöltekin, —James et al. (2013) data. Statistical learning refers to a vast set of tools for understanding —Barber (2012) mimicking, understanding and aiding human and biological What and why Machine Learning is the study of data-driven methods capable of —Mitchell (1997) with experience. how to construct computer programs that automatically improve The fjeld of machine learning is concerned with the question of Machine learning is … The course plan 4 / 16

  8. Practical matters What and why April 12, 2016 SfS / University of Tübingen Ç. Çöltekin, 5 / 16 are based on machine learning Machine learning and computational linguistics The course plan ▶ Majority of the computational linguistic tasks and applications ▶ Tokenization ▶ Part of speech tagging ▶ Parsing ▶ … ▶ Speech recognition ▶ Named Entity recognition ▶ Document classifjcation ▶ Question answering ▶ Machine translation ▶ …

  9. Practical matters What and why The course plan Refresher: linear algebra Thursday! Ç. Çöltekin, SfS / University of Tübingen April 12, 2016 6 / 16 ▶ Vectors, vector operations, their geometric interpretations ▶ Vector norms, distances between vectors ▶ Matrices, matrix operations ▶ Some useful matrix properties ▶ Linear transformations

  10. Practical matters What and why The course plan Refresher: probability and statistics Next week Ç. Çöltekin, SfS / University of Tübingen April 12, 2016 7 / 16 ▶ Probabilities: where do they come from? ▶ Random variables, probability distributions ▶ Joint, conditional, marginal probabilities, chain rule ▶ Bayes’ formula ▶ Some concepts from information theory

  11. Practical matters What and why April 12, 2016 SfS / University of Tübingen Ç. Çöltekin, 80 60 40 20 80 60 40 20 Regression The course plan 8 / 16 y x

  12. Practical matters – April 12, 2016 SfS / University of Tübingen Ç. Çöltekin, – – – + What and why + + + Classifjcation The course plan 9 / 16 x 2 x 1

  13. Practical matters + April 12, 2016 SfS / University of Tübingen Ç. Çöltekin, – – – – + What and why + + ? Classifjcation The course plan 9 / 16 x 2 x 1

  14. Practical matters + April 12, 2016 SfS / University of Tübingen Ç. Çöltekin, – – – – + What and why + + ? Classifjcation The course plan 9 / 16 x 2 x 1

  15. Practical matters What and why The course plan Machine learning basics Ç. Çöltekin, SfS / University of Tübingen April 12, 2016 10 / 16 ▶ How to measure success in an ML experiment? ▶ Variance and bias ▶ Overfjtting and underfjtting ▶ Cross validation ▶ Training/test/development set split

  16. Practical matters What and why The course plan Unsupervised learning Ç. Çöltekin, SfS / University of Tübingen April 12, 2016 11 / 16 ▶ Clustering ▶ Density estimation ▶ Dimensionality reduction

  17. Practical matters Input April 12, 2016 SfS / University of Tübingen Ç. Çöltekin, layer Output layer Hidden layer Output What and why Output Output Neural networks The course plan 12 / 16 x 1 x 2 x 3 x 4

  18. Practical matters What and why The course plan Distributed representations Ç. Çöltekin, SfS / University of Tübingen April 12, 2016 13 / 16 ▶ Sparse feature representations ▶ Dense representations ▶ Word/character embeddings ▶ How to obtain meaningful combinations?

  19. Practical matters What and why The course plan Deep learning Ç. Çöltekin, SfS / University of Tübingen April 12, 2016 14 / 16 ▶ Convolutional networks ▶ Recurrent networks ▶ Auto-encoder/decoders ▶ …

  20. Practical matters What and why The course plan Bayesian learning (if time allows) Ç. Çöltekin, SfS / University of Tübingen April 12, 2016 15 / 16 ▶ Bayesian inference ▶ Graphical models

  21. Practical matters What and why April 12, 2016 SfS / University of Tübingen Ç. Çöltekin, 16 / 16 some that was not covered Summary The course plan ▶ Besides what what is covered during the course, we will note ▶ Decision trees, random forests ▶ Rule learning ▶ Memory based learning ▶ Support vector machines ▶ Local regression / generalized additive models ▶ Learning sequences (e.g., HMMs) ▶ Active learning ▶ Reinforcement learning ▶ Ensemble methods ▶ …

  22. Machine learning books/resources arXiv:1511.07916. April 12, 2016 SfS / University of Tübingen Ç. Çöltekin, Grus, Joel (2015). Data Science from Scratch: First Principles with Python . O’Reilly Media. isbn : 9781491904404. Goodfellow, Ian, Yoshua Bengio, and Aaron Courville (2016). “Deep Learning”. Book in preparation for MIT Press. http://www.cs.biu.ac.il/~yogo/nnlp.pdf . Goldberg, Yoav (2015). A Primer on Neural Network Models for Natural Language Processing . url : University Press. isbn : 9781107096394. Flach, Peter (2012). Machine Learning: The Art and Science of Algorithms that Make Sense of Data . Cambridge Cho, Kyunghyun (2015). Natural Language Understanding with Distributed Representation . arXiv preprint The following is an unsorted list of machine lerning related books and 9781118961766. Bowles, Michael (2015). Machine Learning in Python: Essential Techniques for Predictive Analysis . Wiley. isbn : Bishop, Christopher M. (2006). Pattern Recognition and Machine Learning . Springer. isbn : 978-0387-31073-2. 9780521518147. Barber, David (2012). Bayesian Reasoning and Machine Learning . Cambridge University Press. isbn : Press. isbn : 0262012111,9780262012119. Alpaydın, Ethem (2004). Introduction to machine learning . Adaptive computation and machine learning. MIT AMLBook.com. isbn : 9781600490064. Abu-Mostafa, Yaser S., Malik Magdon-Ismail, and Hsuan-Tien Lin (2012). Learning from Data: A Short Course . resources. A.1 url : http://www.deeplearningbook.org .

Recommend

One-Shot Learning: Language Acquisition for Machine SS16 Computational Linguistics for

One-Shot Learning: Language Acquisition for Machine SS16 Computational Linguistics for

One-Shot Learning: Language Acquisition for Machine SS16 Computational Linguistics for Low-Resource Languages Mayumi Ohta July 6, 2016 Institute for Computational Linguistics Heidelberg University Table of contents 1. Introduction 2.

985 views • 70 slides
Machine Learning for Computational Linguistics A refresher on linear algebra ar ltekin

Machine Learning for Computational Linguistics A refresher on linear algebra ar ltekin

Machine Learning for Computational Linguistics A refresher on linear algebra ar ltekin University of Tbingen Seminar fr Sprachwissenschaft April 14, 2016 Practical matters A bit of machine learning Linear algebra Frequently

658 views • 42 slides
Machine Learning for Computational Linguistics May 3, 2016 regression non-parametric neighbors

Machine Learning for Computational Linguistics May 3, 2016 regression non-parametric neighbors

Machine Learning for Computational Linguistics May 3, 2016 regression non-parametric neighbors the instances Instance/memory based methods A quick survey of some solutions More than two classes Logistic Regression Classifjcation Practical

563 views • 4 slides
Machine Learning for Computational Linguistics Autoencoders + deep learning summary ar

Machine Learning for Computational Linguistics Autoencoders + deep learning summary ar

Machine Learning for Computational Linguistics Autoencoders + deep learning summary ar ltekin University of Tbingen Seminar fr Sprachwissenschaft July 12, 2016 Restricted Boltzmann machines Autoencoders July 12, 2016 SfS /

355 views • 17 slides
Machine Learning for Computational Linguistics Some probability distributions .

Machine Learning for Computational Linguistics Some probability distributions .

Machine Learning for Computational Linguistics Some probability distributions . ltekin, SfS / University of Tbingen April 19/21, 2016 5 / 48 Probability theory Information theory 0 0 1 0 0 Probability mass function Example:

507 views • 7 slides
Foundations of Computational Linguistics man-machine communication in natural language R OLAND H

Foundations of Computational Linguistics man-machine communication in natural language R OLAND H

Foundations of Computational Linguistics man-machine communication in natural language R OLAND H AUSSER Computational Linguistics Universitt Erlangen Nrnberg Germany Foundations of Computational Linguistics viii Part IV Semantics and

1.53k views • 130 slides
Foundations of Computational Linguistics man-machine communication in natural language R OLAND H

Foundations of Computational Linguistics man-machine communication in natural language R OLAND H

Foundations of Computational Linguistics man-machine communication in natural language R OLAND H AUSSER Computational Linguistics Universitt Erlangen Nrnberg Germany Foundations of Computational Linguistics iv Part II Theory of Grammar 7.

1.29k views • 107 slides
Machine Learning for Computational Linguistics Classifjcation ar ltekin University of

Machine Learning for Computational Linguistics Classifjcation ar ltekin University of

Machine Learning for Computational Linguistics Classifjcation ar ltekin University of Tbingen Seminar fr Sprachwissenschaft May 3, 2016 Practical matters Classifjcation Logistic Regression More than two classes Practical

588 views • 32 slides
Machine Learning for Computational Linguistics May 24, 2016 . ltekin, label of an unknown

Machine Learning for Computational Linguistics May 24, 2016 . ltekin, label of an unknown

Machine Learning for Computational Linguistics May 24, 2016 . ltekin, label of an unknown label. In the example: Supervised learning: classifjcation PCA Hierarchical clustering Mixture densities K-means Clustering Introduction

103 views • 7 slides
Foundations of Computational Linguistics man-machine communication in natural language R OLAND H

Foundations of Computational Linguistics man-machine communication in natural language R OLAND H

Foundations of Computational Linguistics man-machine communication in natural language R OLAND H AUSSER Computational Linguistics Universitt Erlangen Nrnberg Germany Foundations of Computational Linguistics vi Part III Morphology and

1.64k views • 141 slides
Understanding Machine Learning with Language and Tensors Jon Rawski Linguistics Department

Understanding Machine Learning with Language and Tensors Jon Rawski Linguistics Department

Machine Learning Language Tensors Reduplication Results References Understanding Machine Learning with Language and Tensors Jon Rawski Linguistics Department Institute for Advanced Computational Science Stony Brook University 1 Machine

612 views • 46 slides
Foundations of Computational Linguistics man-machine communication in natural language R OLAND H

Foundations of Computational Linguistics man-machine communication in natural language R OLAND H

Foundations of Computational Linguistics man-machine communication in natural language R OLAND H AUSSER Computational Linguistics Universitt Erlangen Nrnberg Germany Foundations of Computational Linguistics ii Part I Theory of Language 1.

957 views • 92 slides
Understanding Machine Learning with Language and Tensors Jon Rawski Linguistics Department

Understanding Machine Learning with Language and Tensors Jon Rawski Linguistics Department

Machine Learning Language Tensors Reduplication Understanding Machine Learning with Language and Tensors Jon Rawski Linguistics Department Institute for Advanced Computational Science Stony Brook University 1 Machine Learning Language

602 views • 36 slides
Machine Learning for Computational Linguistics Regression ar ltekin University of

Machine Learning for Computational Linguistics Regression ar ltekin University of

Machine Learning for Computational Linguistics Regression ar ltekin University of Tbingen Seminar fr Sprachwissenschaft April 26/28, 2016 Practical matters Revisiting entropy April 26/28, 2016 SfS / University of Tbingen

953 views • 59 slides
Machine Learning for Computational Linguistics Distributed representations ar ltekin

Machine Learning for Computational Linguistics Distributed representations ar ltekin

Machine Learning for Computational Linguistics Distributed representations ar ltekin University of Tbingen Seminar fr Sprachwissenschaft June 14, 2016 Introduction SVD June 14, 2016 SfS / University of Tbingen .

958 views • 32 slides
Machine Learning for Computational Linguistics Recurrent neural networks (RNNs) ar

Machine Learning for Computational Linguistics Recurrent neural networks (RNNs) ar

Machine Learning for Computational Linguistics Recurrent neural networks (RNNs) ar ltekin University of Tbingen Seminar fr Sprachwissenschaft July 5, 2016 Neural networks: a quick summary Recurrent neural networks July 5, 2016

570 views • 35 slides
Overview CS 446 What is machine learning? Machine learning : study of computational

Overview CS 446 What is machine learning? Machine learning : study of computational

Overview CS 446 What is machine learning? Machine learning : study of computational mechanisms that learn from data to make predictions and decisions. 1 / 21 Application 1: image classification Birdwatcher takes photos of birds,

1.08k views • 83 slides
Subregular Complexity and Machine Learning Jeffrey Heinz Linguistics Department Institute for

Subregular Complexity and Machine Learning Jeffrey Heinz Linguistics Department Institute for

Subregular Complexity and Machine Learning Jeffrey Heinz Linguistics Department Institute for Advanced Computational Science Stony Brook University IACS Seminar September 14, 2017 1 Joint work Enes Avcu, University of Delaware

606 views • 56 slides
Computational Learning Theory: Probably Approximately Correct (PAC) Learning Machine Learning 1

Computational Learning Theory: Probably Approximately Correct (PAC) Learning Machine Learning 1

Computational Learning Theory: Probably Approximately Correct (PAC) Learning Machine Learning 1 Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others Computational Learning Theory The Theory of Generalization

413 views • 28 slides
Topics in Computational Linguistics Learning to Paraphrase: An Unsupervised Approach Using

Topics in Computational Linguistics Learning to Paraphrase: An Unsupervised Approach Using

Topics in Computational Linguistics Learning to Paraphrase: An Unsupervised Approach Using Multiple-Sequence Alignment Regina Barzilay and Lillian Lee Presented By: Mohammad Saif Department of Computer Science, University of Toronto 10

232 views • 22 slides
Topics in Computational Linguistics Topics in Computational Linguistics March 28, 2014 GIL,

Topics in Computational Linguistics Topics in Computational Linguistics March 28, 2014 GIL,

. . . . .. . . .. . . .. . . .. . . .. .. . . Topics in Computational Linguistics Topics in Computational Linguistics March 28, 2014 GIL, National Taiwan University Lab of Ontologies, Language Processing and e-Humanities

783 views • 56 slides
Dictionaries Christian Chiarcos Applied Computational Linguistics Lab

Dictionaries Christian Chiarcos Applied Computational Linguistics Lab

Digital Humanities Workshop, Sep 9 11, 2014, Batumi, Georgia Linking Machine-Readable Dictionaries Christian Chiarcos Applied Computational Linguistics Lab chiarcos@informatik.uni-frankfurt.de 1 Linking Machine-Readable Dictionaries

719 views • 71 slides
Computational Learning Theory - HT 2018 formerly Advanced Machine Learning (HT 2017) Introduction

Computational Learning Theory - HT 2018 formerly Advanced Machine Learning (HT 2017) Introduction

Computational Learning Theory - HT 2018 formerly Advanced Machine Learning (HT 2017) Introduction and Course Details Varun Kanade University of Oxford January 15, 2018 What is Machine Learning? 200-basic level categories Here: Six

466 views • 18 slides
4CSLL5 Advanced Computational Linguistics Introduction Phrase Based Machine Trans Martin

4CSLL5 Advanced Computational Linguistics Introduction Phrase Based Machine Trans Martin

4CSLL5 Advanced Computational Linguistics Phrase Based Machine Trans 4CSLL5 Advanced Computational Linguistics Phrase Based Machine Trans 4CSLL5 Advanced Computational Linguistics Introduction Phrase Based Machine Trans

390 views • 7 slides

More recommend