computational learning theory mt 2018 introduction and
play

Computational Learning Theory - MT 2018 Introduction and Course - PowerPoint PPT Presentation

Aug 28, 2022 •133 likes •330 views

Computational Learning Theory - MT 2018 Introduction and Course Details Varun Kanade University of Oxford October 8, 2018 What is Machine Learning? 200-basic level categories Here: Six pictures containing airplanes and people

  1. Computational Learning Theory - MT 2018 Introduction and Course Details Varun Kanade University of Oxford October 8, 2018

  2. What is Machine Learning? ◮ 200-basic level categories ◮ Here: Six pictures containing airplanes and people ◮ Dataset contains hundres of thousands of images ◮ Imagenet competition (2010-) ◮ All recent successes through very deep neural networks! 1

  3. What is Machine Learning? Movie / User Alice Bob Charlie Dean Eve The Shawshank Redemption 7 9 9 5 2 The Godfather 3 ? 10 4 3 The Dark Knight 5 9 ? 6 ? Pulp Fiction ? 5 ? ? 10 Schindler’s List ? 6 ? 9 ? Netflix competition to predict user-ratings (2008-09) Any individual user will not have used most products Most products will have been used by some individual 2

  4. Supervised Learning Training data has inputs x (numerical, categorical) as well as outputs y (target) Regression: When the output is real-valued, e.g., housing price Classification: Output is a category ◮ Binary classification: only two classes e.g., spam ◮ Multi-class classification: several classes e.g., object detection 3

  5. Unsupervised Learning : Group Similar News Articles Group similar articles into categories such as politics, music, sport, etc. In the dataset, there are no labels for the articles 4

  6. Active and Semi-Supervised Learning Active Learning ◮ Initially all data is unlabelled ◮ Learning algorithm can ask a human to label some data Semi-supervised Learning ◮ Limited labelled data, lots of unlabelled data ◮ How to use the two together to improve learning? 5

  7. Outline What is Machine Learning? What is Learning Theory? Course Logistics

  8. What is Learning Theory? The goal of (computational) learning theory is to develop formal models to analyse questions arising in machine learning ◮ How much data do we need to learn? ◮ What amount of computational resources are necessary for learning? ◮ Are there hard learning problems? 6

  9. What is Learning Theory? In this course we’ll cover several models that aim to capture questions that are of interest in modern machine learning ◮ (How) can we learn in the presence of noisy data? ◮ What can we learn when data is obtained in an online manner? ◮ (How) can we do useful machine learning while preserving privacy? ◮ Can we learn when data and computational power is distributed? 7

  10. What is Learning Theory? Towards the end of the course we’ll cover some of the latest topics in the area ◮ Can we develop a theoretical understanding of neural networks? ◮ Connections to information theory, game theory, etc. ◮ Conference on Learning Theory (COLT) 8

  11. Outline What is Machine Learning? What is Learning Theory? Course Logistics

  12. Course Information Website www.cs.ox.ac.uk/people/varun.kanade/teaching/CLT-MT2018/ Lectures Mon 15h-16h, Thu 16h-18h (about 20 contact hours) Classes Weeks 3-7 Instructors: Alexandros Hollender, Philip Lazos, Francisco Marmolejo, David Martínez Office Hours After Monday lecture (16-17h) 9

  13. Course Information Textbooks Kearns and Vazirani - An Introduction to Computational Learning Theory Several additional texts for suggested reading on website Papers and (rough) lecture notes will be posted Assessment Take Home Exam Piazza Use for course-related queries Sign-up at piazza.com/ox.ac.uk/other/cltmt2018 10

  14. Who should take this course? In this course, we will cover ◮ Mathematical formulations for different learning paradigms ◮ Definitions, theorems, proofs ◮ Design and analysis of learning algorithms ◮ Provable guarantees on run-time and sample complexity 11

  15. Who should take this course? In this course, we will cover ◮ Mathematical formulations for different learning paradigms ◮ Definitions, theorems, proofs ◮ Design and analysis of learning algorithms ◮ Provable guarantees on run-time and sample complexity In this course, we will not cover ◮ Practical applications of learning algorithms - although understanding the theory will likely make you a better practitioner 11

  16. Who should take this course? It is expected that you will be familiar with most of the following ◮ The notion polynomial time, space, etc. ◮ Big O notation ◮ Basic probability theory - expectation, independence, etc. 12

  17. Who should take this course? It is expected that you will be familiar with most of the following ◮ The notion polynomial time, space, etc. ◮ Big O notation ◮ Basic probability theory - expectation, independence, etc. It’d be helpful if (though not necessary that) you’ve seen at least some of the following ◮ Basic complexity theory such as NP-completeness ◮ Applied Machine Learning ◮ Optimisation algorithms - Linear Programming 12

  18. Who should take this course? This is an advanced theoretical course. If you are taking this course, you should ◮ Be keen to understand the theory behind machine learning algorithms ◮ Be able to fill in details of algorithms and proofs omitted in the lectures ◮ Develop an ability to read research papers 13

Recommend

Computational Learning Theory - HT 2018 formerly Advanced Machine Learning (HT 2017) Introduction

Computational Learning Theory - HT 2018 formerly Advanced Machine Learning (HT 2017) Introduction

Computational Learning Theory - HT 2018 formerly Advanced Machine Learning (HT 2017) Introduction and Course Details Varun Kanade University of Oxford January 15, 2018 What is Machine Learning? 200-basic level categories Here: Six

466 views • 18 slides
Computational Learning Theory: Probably Approximately Correct (PAC) Learning Machine Learning 1

Computational Learning Theory: Probably Approximately Correct (PAC) Learning Machine Learning 1

Computational Learning Theory: Probably Approximately Correct (PAC) Learning Machine Learning 1 Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others Computational Learning Theory The Theory of Generalization

413 views • 28 slides
Computational Learning Theory: Positive and negative learnability results Machine Learning 1

Computational Learning Theory: Positive and negative learnability results Machine Learning 1

Computational Learning Theory: Positive and negative learnability results Machine Learning 1 Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others Computational Learning Theory The Theory of Generalization Probably

534 views • 40 slides
Computational Learning Theory: An Analysis of a Conjunction Learner Machine Learning Slides

Computational Learning Theory: An Analysis of a Conjunction Learner Machine Learning Slides

Computational Learning Theory: An Analysis of a Conjunction Learner Machine Learning Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others 1 This lecture: Computational Learning Theory The Theory of Generalization

845 views • 49 slides
Computational Learning Theory: Agnostic Learning Machine Learning 1 Slides based on material

Computational Learning Theory: Agnostic Learning Machine Learning 1 Slides based on material

Computational Learning Theory: Agnostic Learning Machine Learning 1 Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others This lecture: Computational Learning Theory The Theory of Generalization Probably

653 views • 44 slides
CSCE 478/878 Lecture 3: Computational Learning Theory Stephen D. Scott (Adapted from Tom

CSCE 478/878 Lecture 3: Computational Learning Theory Stephen D. Scott (Adapted from Tom

CSCE 478/878 Lecture 3: Computational Learning Theory Stephen D. Scott (Adapted from Tom Mitchells slides) September 8, 2003 1 Introduction Combines machine learning with: Algorithm design and analysis Computational complexity

393 views • 27 slides
Computational Learning Theory: The Theory of Generalization Machine Learning 1 Slides based on

Computational Learning Theory: The Theory of Generalization Machine Learning 1 Slides based on

Computational Learning Theory: The Theory of Generalization Machine Learning 1 Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others Checkpoint: The bigger picture Supervised learning: instances, concepts, and

895 views • 42 slides
Computational Learning Theory: Shattering and VC Dimensions Machine Learning 1 Slides based on

Computational Learning Theory: Shattering and VC Dimensions Machine Learning 1 Slides based on

Computational Learning Theory: Shattering and VC Dimensions Machine Learning 1 Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others This lecture: Computational Learning Theory The Theory of Generalization Probably

1.09k views • 81 slides
Applying Computational Learning Theory to Software Testing Neil Walkinshaw Computational

Applying Computational Learning Theory to Software Testing Neil Walkinshaw Computational

Applying Computational Learning Theory to Software Testing Neil Walkinshaw Computational Learning Theory (CLT) Theoretical analysis of questions of learnability . Is it feasible to learn a given type of concept from a set of examples?

1.14k views • 4 slides
CSCE 478/878 Lecture 3: Computational Learning Theory Examines the worst-case minimum and

CSCE 478/878 Lecture 3: Computational Learning Theory Examines the worst-case minimum and

Introduction Combines machine learning with: Algorithm design and analysis Computational complexity CSCE 478/878 Lecture 3: Computational Learning Theory Examines the worst-case minimum and maximum data and time requirements for

222 views • 7 slides
Computational Learning Theory: Occams Razor Machine Learning 1 Slides based on material from

Computational Learning Theory: Occams Razor Machine Learning 1 Slides based on material from

Computational Learning Theory: Occams Razor Machine Learning 1 Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others This lecture: Computational Learning Theory The Theory of Generalization Probably

585 views • 35 slides
Computational Learning Theory For which tasks is successful learning possible? Under what

Computational Learning Theory For which tasks is successful learning possible? Under what

Computational Learning Theory For which tasks is successful learning possible? Under what conditions is successful learning guaranteed? What is successful learning? Probably approximately correct (PAC) framework Bounds on number

422 views • 27 slides
Computational Learning Theory Based on Machine Learning, T. Mitchell, McGRAW Hill, 1997, ch.

Computational Learning Theory Based on Machine Learning, T. Mitchell, McGRAW Hill, 1997, ch.

0. Computational Learning Theory Based on Machine Learning, T. Mitchell, McGRAW Hill, 1997, ch. 7 Acknowledgement: The present slides are an adaptation of slides drawn by T. Mitchell 1. Main Questions in Computational Learning Theory

682 views • 42 slides
Computational Learning Theory 1 / 22 Decidability Computation Decidability which

Computational Learning Theory 1 / 22 Decidability Computation Decidability which

Computational Learning Theory 1 / 22 Decidability Computation Decidability which problems have algorithmic solutions Machine Learning Feasibility what assumptions must we make to trust that we can learn an unknown target

246 views • 22 slides
Computational and Statistical Learning Theory TTIC 31120 Prof. Nati Srebro Lecture 7:

Computational and Statistical Learning Theory TTIC 31120 Prof. Nati Srebro Lecture 7:

Computational and Statistical Learning Theory TTIC 31120 Prof. Nati Srebro Lecture 7: Computational Complexity of Learning Hardness of Improper Learning (continued) Agnostic Learning Hardness of Learning via Crypto Easy to generate

433 views • 24 slides
Learning Equivalence Structures Luca San Mauro (Vienna University of Technology) Logic Colloquium

Learning Equivalence Structures Luca San Mauro (Vienna University of Technology) Logic Colloquium

Learning Equivalence Structures Luca San Mauro (Vienna University of Technology) Logic Colloquium 2018 joint work with Ekaterina Fokina and Timo Koetzing Computational Learning Theory Computational Learning Theory (CLT) is a vast research

900 views • 69 slides
Statistical and Computational Statistical and Computational Learning Theory Learning Theory

Statistical and Computational Statistical and Computational Learning Theory Learning Theory

Statistical and Computational Statistical and Computational Learning Theory Learning Theory Fundamental Question: Predict Error Rates Fundamental Question: Predict Error Rates Given: Given: The space H of hypotheses The space H of

497 views • 29 slides
Computational Learning Theory [read Chapter 7] [Suggested exercises: 7.1, 7.2, 7.5, 7.8]

Computational Learning Theory [read Chapter 7] [Suggested exercises: 7.1, 7.2, 7.5, 7.8]

Computational Learning Theory [read Chapter 7] [Suggested exercises: 7.1, 7.2, 7.5, 7.8] Computational learning theory Setting 1: learner p oses queries to teac her Setting 2: teac her c ho oses examples

491 views • 24 slides
Machine Learning for Computational Linguistics Some probability distributions .

Machine Learning for Computational Linguistics Some probability distributions .

Machine Learning for Computational Linguistics Some probability distributions . ltekin, SfS / University of Tbingen April 19/21, 2016 5 / 48 Probability theory Information theory 0 0 1 0 0 Probability mass function Example:

507 views • 7 slides
Syntactic Theory Introduction Yi Zhang & Antske Fokkens Department of Computational

Syntactic Theory Introduction Yi Zhang & Antske Fokkens Department of Computational

Syntactic Theory Introduction Yi Zhang & Antske Fokkens Department of Computational Linguistics Saarland University October 15, 2009 Syntax: What does it mean? We can view syntax/syntactic theory in a number of ways, two of which are the

703 views • 27 slides
Texts Complexity Theory The main text for the course is: Computational Complexity . Christos H.

Texts Complexity Theory The main text for the course is: Computational Complexity . Christos H.

Complexity Theory 1 Complexity Theory 2 Texts Complexity Theory The main text for the course is: Computational Complexity . Christos H. Papadimitriou. Introduction to the Theory of Computation . Michael Sipser. Anuj Dawar Other useful

628 views • 30 slides
T-61.182 Information Theory and Machine Learning 38. Introduction to Neural Networks 40.

T-61.182 Information Theory and Machine Learning 38. Introduction to Neural Networks 40.

T-61.182 Information Theory and Machine Learning 38. Introduction to Neural Networks 40. Capacity of a Single Neuron 41. Learning as Inference Presented by Yang, Zhi-rong on 22, April 2004 T-61.182 Information Theory and Machine Learning

423 views • 20 slides
Introduction to Machine Learning CMU-10701 11. Learning Theory Barnabs Pczos Learning

Introduction to Machine Learning CMU-10701 11. Learning Theory Barnabs Pczos Learning

Introduction to Machine Learning CMU-10701 11. Learning Theory Barnabs Pczos Learning Theory We have explored many ways of learning from data But How good is our classifier, really? How much data do we need to make it good

862 views • 48 slides
MIT 9.520/6.860, Fall 2018 Statistical Learning Theory and Applications Class 06: Learning with

MIT 9.520/6.860, Fall 2018 Statistical Learning Theory and Applications Class 06: Learning with

MIT 9.520/6.860, Fall 2018 Statistical Learning Theory and Applications Class 06: Learning with Stochastic Gradients Sasha Rakhlin A. Rakhlin, 9.520/6.860 2018 Why Optimization? Much (but not all) of Machine Learning: write down objective

481 views • 24 slides

More recommend