computational learning theory ht 2018
play

Computational Learning Theory - HT 2018 formerly Advanced Machine - PowerPoint PPT Presentation

Aug 17, 2023 •276 likes •466 views

Computational Learning Theory - HT 2018 formerly Advanced Machine Learning (HT 2017) Introduction and Course Details Varun Kanade University of Oxford January 15, 2018 What is Machine Learning? 200-basic level categories Here: Six

  1. Computational Learning Theory - HT 2018 formerly Advanced Machine Learning (HT 2017) Introduction and Course Details Varun Kanade University of Oxford January 15, 2018

  2. What is Machine Learning? ◮ 200-basic level categories ◮ Here: Six pictures containing airplanes and people ◮ Dataset contains over 400,000 images ◮ Imagenet competition (2010-17) ◮ All recent successes through very deep neural networks! 1

  3. What is Machine Learning? Movie / User Alice Bob Charlie Dean Eve The Shawshank Redemption 7 9 9 5 2 The Godfather 3 ? 10 4 3 The Dark Knight 5 9 ? 6 ? Pulp Fiction ? 5 ? ? 10 Schindler’s List ? 6 ? 9 ? Netflix competition to predict user-ratings (2008-09) Any individual user will not have used most products Most products will have been used by some individual 2

  4. Supervised Learning Training data has inputs x (numerical, categorical) as well as outputs y (target) Regression: When the output is real-valued, e.g., housing price Classification: Output is a category ◮ Binary classification: only two classes e.g., spam ◮ Multi-class classification: several classes e.g., object detection 3

  5. Unsupervised Learning : Group Similar News Articles Group similar articles into categories such as politics, music, sport, etc. In the dataset, there are no labels for the articles 4

  6. Active and Semi-Supervised Learning Active Learning ◮ Initially all data is unlabelled ◮ Learning algorithm can ask a human to label some data Semi-supervised Learning ◮ Limited labelled data, lots of unlabelled data ◮ How to use the two together to improve learning? 5

  7. Outline What is Machine Learning? What is Learning Theory? Course Logistics

  8. What is Learning Theory? The goal of (computational) learning theory is to develop formal models to analyse questions arising in machine learning ◮ How much data do we need to learn? ◮ What amount of computational resources are necessary for learning? ◮ Are there hard learning problems? 6

  9. What is Learning Theory? In this course we’ll cover several models that aim to capture questions that are of interest in modern machine learning ◮ (How) can we learn in the presence of noisy data? ◮ What can we learn when data is obtained in an online manner? ◮ (How) can we do useful machine learning while preserving privacy? ◮ Can we learn when data and computational power is distributed? 7

  10. What is Learning Theory? Towards the end of the course we’ll cover some of the latest topics in the area ◮ Can we develop a theoretical understanding of neural networks? ◮ Connections to information theory, game theory, etc. ◮ Conference on Learning Theory (COLT) 8

  11. Outline What is Machine Learning? What is Learning Theory? Course Logistics

  12. Course Information Website www.cs.ox.ac.uk/people/varun.kanade/teaching/CLT-HT2018/ Lectures Mon, Thu 16h-18h (between 20-24 contact hours) Classes Weeks 3-8 (possibly skip one) Instructors: Francisco Marmolejo, David Martínez, Ninad Rajgopal Office Hours After lectures or by appointment 9

  13. Course Information Textbooks Kearns and Vazirani - An Introduction to Computational Learning Theory Mohri, Rostamizadeh, Talwalkar - Foundations of Machine Learning Shalev-Shwartz and Ben-David - Understanding Machine Learning Papers and (rough) lecture notes will be posted Assessment Take Home Exam Piazza Use for course-related queries Sign-up at piazza.com/ox.ac.uk/other/cltht2018 10

  14. Who should take this course? In this course, we will cover ◮ Mathematical formulations for different learning paradigms ◮ Definitions, theorems, proofs ◮ Design and analysis of learning algorithms ◮ Provable guarantees on run-time and sample complexity 11

  15. Who should take this course? In this course, we will cover ◮ Mathematical formulations for different learning paradigms ◮ Definitions, theorems, proofs ◮ Design and analysis of learning algorithms ◮ Provable guarantees on run-time and sample complexity In this course, we will not cover ◮ Practical applications of learning algorithms - although understanding the theory will likely make you a better practitioner 11

  16. Who should take this course? It is expected that you will be familiar with most of the following ◮ The notion polynomial time, space, etc. ◮ Big O notation ◮ Basic probability theory - expectation, independence, etc. 12

  17. Who should take this course? It is expected that you will be familiar with most of the following ◮ The notion polynomial time, space, etc. ◮ Big O notation ◮ Basic probability theory - expectation, independence, etc. It’d be helpful if (though not necessary that) you’ve seen at least some of the following ◮ Basic complexity theory such as NP-completeness ◮ Applied Machine Learning ◮ Optimisation algorithms - Linear Programming 12

  18. Who should take this course? This is an advanced theoretical course. If you are taking this course, you should ◮ Be keen to understand the theory behind machine learning algorithms ◮ Be able to fill in details of algorithms and proofs omitted in the lectures ◮ Develop an ability to read research papers 13

Recommend

Computational Learning Theory: Probably Approximately Correct (PAC) Learning Machine Learning 1

Computational Learning Theory: Probably Approximately Correct (PAC) Learning Machine Learning 1

Computational Learning Theory: Probably Approximately Correct (PAC) Learning Machine Learning 1 Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others Computational Learning Theory The Theory of Generalization

413 views • 28 slides
Computational Learning Theory: An Analysis of a Conjunction Learner Machine Learning Slides

Computational Learning Theory: An Analysis of a Conjunction Learner Machine Learning Slides

Computational Learning Theory: An Analysis of a Conjunction Learner Machine Learning Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others 1 This lecture: Computational Learning Theory The Theory of Generalization

845 views • 49 slides
Computational Learning Theory: Positive and negative learnability results Machine Learning 1

Computational Learning Theory: Positive and negative learnability results Machine Learning 1

Computational Learning Theory: Positive and negative learnability results Machine Learning 1 Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others Computational Learning Theory The Theory of Generalization Probably

534 views • 40 slides
Computational Learning Theory: Agnostic Learning Machine Learning 1 Slides based on material

Computational Learning Theory: Agnostic Learning Machine Learning 1 Slides based on material

Computational Learning Theory: Agnostic Learning Machine Learning 1 Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others This lecture: Computational Learning Theory The Theory of Generalization Probably

653 views • 44 slides
Computational Learning Theory - MT 2018 Introduction and Course Details Varun Kanade University

Computational Learning Theory - MT 2018 Introduction and Course Details Varun Kanade University

Computational Learning Theory - MT 2018 Introduction and Course Details Varun Kanade University of Oxford October 8, 2018 What is Machine Learning? 200-basic level categories Here: Six pictures containing airplanes and people

330 views • 18 slides
Computational Learning Theory: The Theory of Generalization Machine Learning 1 Slides based on

Computational Learning Theory: The Theory of Generalization Machine Learning 1 Slides based on

Computational Learning Theory: The Theory of Generalization Machine Learning 1 Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others Checkpoint: The bigger picture Supervised learning: instances, concepts, and

895 views • 42 slides
Computational Learning Theory: Shattering and VC Dimensions Machine Learning 1 Slides based on

Computational Learning Theory: Shattering and VC Dimensions Machine Learning 1 Slides based on

Computational Learning Theory: Shattering and VC Dimensions Machine Learning 1 Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others This lecture: Computational Learning Theory The Theory of Generalization Probably

1.09k views • 81 slides
Applying Computational Learning Theory to Software Testing Neil Walkinshaw Computational

Applying Computational Learning Theory to Software Testing Neil Walkinshaw Computational

Applying Computational Learning Theory to Software Testing Neil Walkinshaw Computational Learning Theory (CLT) Theoretical analysis of questions of learnability . Is it feasible to learn a given type of concept from a set of examples?

1.14k views • 4 slides
Computational Learning Theory: Occams Razor Machine Learning 1 Slides based on material from

Computational Learning Theory: Occams Razor Machine Learning 1 Slides based on material from

Computational Learning Theory: Occams Razor Machine Learning 1 Slides based on material from Dan Roth, Avrim Blum, Tom Mitchell and others This lecture: Computational Learning Theory The Theory of Generalization Probably

585 views • 35 slides
Computational Learning Theory For which tasks is successful learning possible? Under what

Computational Learning Theory For which tasks is successful learning possible? Under what

Computational Learning Theory For which tasks is successful learning possible? Under what conditions is successful learning guaranteed? What is successful learning? Probably approximately correct (PAC) framework Bounds on number

422 views • 27 slides
CSCE 478/878 Lecture 3: Computational Learning Theory Stephen D. Scott (Adapted from Tom

CSCE 478/878 Lecture 3: Computational Learning Theory Stephen D. Scott (Adapted from Tom

CSCE 478/878 Lecture 3: Computational Learning Theory Stephen D. Scott (Adapted from Tom Mitchells slides) September 8, 2003 1 Introduction Combines machine learning with: Algorithm design and analysis Computational complexity

393 views • 27 slides
Computational Learning Theory Based on Machine Learning, T. Mitchell, McGRAW Hill, 1997, ch.

Computational Learning Theory Based on Machine Learning, T. Mitchell, McGRAW Hill, 1997, ch.

0. Computational Learning Theory Based on Machine Learning, T. Mitchell, McGRAW Hill, 1997, ch. 7 Acknowledgement: The present slides are an adaptation of slides drawn by T. Mitchell 1. Main Questions in Computational Learning Theory

682 views • 42 slides
Computational Learning Theory 1 / 22 Decidability Computation Decidability which

Computational Learning Theory 1 / 22 Decidability Computation Decidability which

Computational Learning Theory 1 / 22 Decidability Computation Decidability which problems have algorithmic solutions Machine Learning Feasibility what assumptions must we make to trust that we can learn an unknown target

246 views • 22 slides
Computational and Statistical Learning Theory TTIC 31120 Prof. Nati Srebro Lecture 7:

Computational and Statistical Learning Theory TTIC 31120 Prof. Nati Srebro Lecture 7:

Computational and Statistical Learning Theory TTIC 31120 Prof. Nati Srebro Lecture 7: Computational Complexity of Learning Hardness of Improper Learning (continued) Agnostic Learning Hardness of Learning via Crypto Easy to generate

433 views • 24 slides
CSCE 478/878 Lecture 3: Computational Learning Theory Examines the worst-case minimum and

CSCE 478/878 Lecture 3: Computational Learning Theory Examines the worst-case minimum and

Introduction Combines machine learning with: Algorithm design and analysis Computational complexity CSCE 478/878 Lecture 3: Computational Learning Theory Examines the worst-case minimum and maximum data and time requirements for

222 views • 7 slides
Learning Equivalence Structures Luca San Mauro (Vienna University of Technology) Logic Colloquium

Learning Equivalence Structures Luca San Mauro (Vienna University of Technology) Logic Colloquium

Learning Equivalence Structures Luca San Mauro (Vienna University of Technology) Logic Colloquium 2018 joint work with Ekaterina Fokina and Timo Koetzing Computational Learning Theory Computational Learning Theory (CLT) is a vast research

900 views • 69 slides
Statistical and Computational Statistical and Computational Learning Theory Learning Theory

Statistical and Computational Statistical and Computational Learning Theory Learning Theory

Statistical and Computational Statistical and Computational Learning Theory Learning Theory Fundamental Question: Predict Error Rates Fundamental Question: Predict Error Rates Given: Given: The space H of hypotheses The space H of

497 views • 29 slides
Computational Learning Theory [read Chapter 7] [Suggested exercises: 7.1, 7.2, 7.5, 7.8]

Computational Learning Theory [read Chapter 7] [Suggested exercises: 7.1, 7.2, 7.5, 7.8]

Computational Learning Theory [read Chapter 7] [Suggested exercises: 7.1, 7.2, 7.5, 7.8] Computational learning theory Setting 1: learner p oses queries to teac her Setting 2: teac her c ho oses examples

491 views • 24 slides
Machine Learning for Computational Linguistics Some probability distributions .

Machine Learning for Computational Linguistics Some probability distributions .

Machine Learning for Computational Linguistics Some probability distributions . ltekin, SfS / University of Tbingen April 19/21, 2016 5 / 48 Probability theory Information theory 0 0 1 0 0 Probability mass function Example:

507 views • 7 slides
MIT 9.520/6.860, Fall 2018 Statistical Learning Theory and Applications Class 06: Learning with

MIT 9.520/6.860, Fall 2018 Statistical Learning Theory and Applications Class 06: Learning with

MIT 9.520/6.860, Fall 2018 Statistical Learning Theory and Applications Class 06: Learning with Stochastic Gradients Sasha Rakhlin A. Rakhlin, 9.520/6.860 2018 Why Optimization? Much (but not all) of Machine Learning: write down objective

481 views • 24 slides
Cartesian Cubical Computational Type Carlo Angiuli Theory Evan Cavallo (*) Favonia Robert

Cartesian Cubical Computational Type Carlo Angiuli Theory Evan Cavallo (*) Favonia Robert

2018.07.24 Cartesian Cubical Computational Type Carlo Angiuli Theory Evan Cavallo (*) Favonia Robert Harper Jonathan Sterling Todd Wilson 1 Cubical features of homotopy type theory univalence, higher inductive types + Computational

1.01k views • 98 slides
11 { < t (0), t (1), t ( n -1)> Correct Concept: Learn a decent approximation most of t (

11 { < t (0), t (1), t ( n -1)> Correct Concept: Learn a decent approximation most of t (

Learning Theory Theorems that characterize classes of learning problems or specific algorithms in terms of computational complexity or sample complexity , i.e. the number of training examples CS 391L: Machine Learning: necessary or

450 views • 7 slides
Peer Prediction Mechanisms and their Connections to Machine Learning Jens Witkowski ETH

Peer Prediction Mechanisms and their Connections to Machine Learning Jens Witkowski ETH

Peer Prediction Mechanisms and their Connections to Machine Learning Jens Witkowski ETH Zurich Game Theory Meets Computational Learning Theory Dagstuhl June 22, 2017 Joint work with David C. Parkes (Harvard) and Rafael Frongillo

831 views • 16 slides
Model Theory and Freges Philosophy of Language September 13, 2018 CSCI 2952C: Computational

Model Theory and Freges Philosophy of Language September 13, 2018 CSCI 2952C: Computational

Model Theory and Freges Philosophy of Language September 13, 2018 CSCI 2952C: Computational Semantics Instructor: Ellie Pavlick HTA: Arun Drelich UTA: Jonathan Chang The basic aim of semantics is to characterize the notion of a true

962 views • 40 slides

More recommend