optimal minimal margin maximization with boosting
play

Optimal Minimal Margin Maximization with Boosting ndr 2. linje i - PowerPoint PPT Presentation

Aug 10, 2023 •383 likes •782 views

Optimal Minimal Margin Maximization with Boosting ndr 2. linje i overskriften til AU Passata Light Allan Grnlund Kasper Green Larsen Alexander Mathiasen (me) AU AARHUS UNIVERSITY DENMARK What is boosting? Overskrift n linje Bold

  1. Optimal Minimal Margin Maximization with Boosting Ændr 2. linje i overskriften til AU Passata Light Allan Grønlund Kasper Green Larsen Alexander Mathiasen (me) AU AARHUS UNIVERSITY DENMARK

  2. What is boosting? Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

  3. What is boosting? Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

  4. What is boosting? Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

  5. What is boosting? Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

  6. What is boosting? Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

  7. What is boosting? Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

  8. What is boosting? Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

  9. What is boosting? Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

  10. What is boosting? Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

  11. What is boosting? Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

  12. What is boosting? Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

  13. What is boosting? Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

  14. What is boosting? Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

  15. No Overfitting? Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

  16. Schapire, Robert E.; Freund, Yoav; Bartlett, Peter; Lee, Wee Sun 1998 No Overfitting? Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

  17. Schapire, Robert E.; Freund, Yoav; Bartlett, Peter; Lee, Wee Sun 1998 No Overfitting? Overskrift én linje Bold eller Regular # Hypotheses AU AARHUS UNIVERSITY DENMARK

  18. Schapire, Robert E.; Freund, Yoav; Bartlett, Peter; Lee, Wee Sun 1998 No Overfitting? Overskrift én linje Bold eller Regular Classification Error # Hypotheses AU AARHUS UNIVERSITY DENMARK

  19. Schapire, Robert E.; Freund, Yoav; Bartlett, Peter; Lee, Wee Sun 1998 No Overfitting? Overskrift én linje Bold eller Regular Classification Error Training Error # Hypotheses AU AARHUS UNIVERSITY DENMARK

  20. Schapire, Robert E.; Freund, Yoav; Bartlett, Peter; Lee, Wee Sun 1998 No Overfitting? Overskrift én linje Bold eller Regular Classification Error Test Error Training Error # Hypotheses AU AARHUS UNIVERSITY DENMARK

  21. Schapire, Robert E.; Freund, Yoav; Bartlett, Peter; Lee, Wee Sun 1998 No Overfitting? Overskrift én linje Bold eller Regular Classification Error Test Error Training Error # Hypotheses Perfectly classify training data AU AARHUS UNIVERSITY DENMARK

  22. Schapire, Robert E.; Freund, Yoav; Bartlett, Peter; Lee, Wee Sun 1998 No Overfitting? Overskrift én linje Bold eller Regular Classification Error Test error still improves! Test Error Training Error # Hypotheses Perfectly classify training data AU AARHUS UNIVERSITY DENMARK

  23. Schapire, Robert E.; Freund, Yoav; Bartlett, Peter; Lee, Wee Sun 1998 No Overfitting? Overskrift én linje Bold eller Regular How do we explain this? Classification Error Test error still improves! Test Error Training Error # Hypotheses Perfectly classify training data AU AARHUS UNIVERSITY DENMARK

  24. An explanation by the minimal margin Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

  25. An explanation by the minimal margin Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

  26. An explanation by the minimal margin Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

  27. An explanation by the minimal margin Overskrift én linje Bold eller Regular * Not technically correct, see paper for definition. AU AARHUS UNIVERSITY DENMARK

  28. Breiman 1998 An explanation by the minimal margin Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

  29. Breiman 1998 An explanation by the minimal margin Overskrift én linje Bold eller Regular Generalization error AU AARHUS UNIVERSITY DENMARK

  30. Breiman 1998 An explanation by the minimal margin Overskrift én linje Bold eller Regular Generalization error Minimal margin AU AARHUS UNIVERSITY DENMARK

  31. Breiman 1998 An explanation by the minimal margin Overskrift én linje Bold eller Regular Number of hypotheses Generalization error Minimal margin AU AARHUS UNIVERSITY DENMARK

  32. Breiman 1998 An explanation by the minimal margin Overskrift én linje Bold eller Regular Number of points Number of hypotheses Generalization error Minimal margin AU AARHUS UNIVERSITY DENMARK

  33. Breiman 1998 An explanation by the minimal margin Overskrift én linje Bold eller Regular Number of points Number of hypotheses Generalization error Minimal margin AdaBoostV: Finds a classifier with a provable bound on the trade-off between the number of hypotheses and the minimal margin. [Rätsch and Warmuth 2005] AU AARHUS UNIVERSITY DENMARK

  34. Breiman 1998 An explanation by the minimal margin Overskrift én linje Bold eller Regular Number of points Number of hypotheses Generalization error Minimal margin AdaBoostV: Finds a classifier with a provable bound on the trade-off between the number of hypotheses and the minimal margin. [Rätsch and Warmuth 2005] JMLR: Conjecture: there is a lower bound which matches AdaBoostV. [Nie et al. 2013] AU AARHUS UNIVERSITY DENMARK

  35. Breiman 1998 An explanation by the minimal margin Overskrift én linje Bold eller Regular Number of points Number of hypotheses Generalization error Minimal margin AdaBoostV: Finds a classifier with a provable bound on the trade-off between the number of hypotheses and the minimal margin. [Rätsch and Warmuth 2005] JMLR: Conjecture: there is a lower bound which matches AdaBoostV. [Nie et al. 2013] FALSE. AU AARHUS UNIVERSITY DENMARK

  36. Breiman 1998 An explanation by the minimal margin Overskrift én linje Bold eller Regular Number of points Number of hypotheses Generalization error Minimal margin AdaBoostV: Finds a classifier with a provable bound on the trade-off between the number of hypotheses and the minimal margin. [Rätsch and Warmuth 2005] JMLR: Conjecture: there is a lower bound which matches AdaBoostV. [Nie et al. 2013] FALSE. SparsiBoost: Obtains a slightly better bound than AdaBoostV. AU AARHUS UNIVERSITY DENMARK

  37. Breiman 1998 An explanation by the minimal margin Overskrift én linje Bold eller Regular Number of points Number of hypotheses Generalization error Minimal margin AdaBoostV: Finds a classifier with a provable bound on the trade-off between the number of hypotheses and the minimal margin. [Rätsch and Warmuth 2005] JMLR: Conjecture: there is a lower bound which matches AdaBoostV. [Nie et al. 2013] FALSE. SparsiBoost: Obtains a slightly better bound than AdaBoostV. Proved matching lower bound, so SparsiBoost is optimal. AU AARHUS UNIVERSITY DENMARK

  38. Experiments Overskrift én linje Bold eller Regular AU AARHUS UNIVERSITY DENMARK

  39. AU AARHUS UNIVERSITY DENMARK

Recommend

lti Is convex Perceptron Boosting Max-Margin Conditional Likelihood MIRA Based on Uses a

lti Is convex Perceptron Boosting Max-Margin Conditional Likelihood MIRA Based on Uses a

Softmax-Margin CRFs: Training Log-Linear Models with Cost Functions Kevin Gimpel and Noah A. Smith lti Is convex Perceptron Boosting Max-Margin Conditional Likelihood MIRA Based on Uses a cost probabilistic function inference

603 views • 26 slides
Minimal Sufficient Conditions for a Primal Optimizer in Nonsmooth Utility Maximization Harry

Minimal Sufficient Conditions for a Primal Optimizer in Nonsmooth Utility Maximization Harry

Minimal Sufficient Conditions for a Primal Optimizer in Nonsmooth Utility Maximization Harry Zheng Imperial College (Joint work with Nicholas Westray, Humboldt Univ. Berlin) Workshop on Stochastic Analysis and Finance Hong Kong City University

287 views • 25 slides
Optimal and Adaptive Algorithms for Online Boosting Alina Beygelzimer 1 Satyen Kale 1 Haipeng Luo 2

Optimal and Adaptive Algorithms for Online Boosting Alina Beygelzimer 1 Satyen Kale 1 Haipeng Luo 2

Optimal and Adaptive Algorithms for Online Boosting Alina Beygelzimer 1 Satyen Kale 1 Haipeng Luo 2 1 Yahoo! Labs, NYC 2 Computer Science Department, Princeton University December 11, 2015 Boosting: An Example Idea: combine weak rules of

614 views • 58 slides
Optimal Continuous DR-Submodular Maximization and Applications to Provable Mean Field Inference

Optimal Continuous DR-Submodular Maximization and Applications to Provable Mean Field Inference

Optimal Continuous DR-Submodular Maximization and Applications to Provable Mean Field Inference Yatao (An) Bian, Joachim M. Buhmann, Andreas Krause ETH Zurich Motivation and Background Product recommendation Given a parameterized submodular

420 views • 5 slides
Non-monotone Submodular Maximization with Nearly Optimal Adaptivity and Query Complexity Matthew

Non-monotone Submodular Maximization with Nearly Optimal Adaptivity and Query Complexity Matthew

Non-monotone Submodular Maximization with Nearly Optimal Adaptivity and Query Complexity Matthew Fahrbach 1 Vahab Mirrokni 2 Morteza Zadimoghaddam 2 June 13, 2019 1 Georgia Tech 2 Google 1/4 Submodular Functions Def. A function f : 2 N R is

214 views • 7 slides
Boosting, Min-Norm Interpolated Classifiers, and Overparametrization: a precise asymptotic theory

Boosting, Min-Norm Interpolated Classifiers, and Overparametrization: a precise asymptotic theory

Intro. Min-Norm Interpolation Boosting and Margin Main Results: Precise Asymptotics Proof Sketch Boosting, Min-Norm Interpolated Classifiers, and Overparametrization: a precise asymptotic theory Tengyuan Liang joint work with Pragya Sur

712 views • 60 slides
Optimal Left and Right Additive Schwarz Preconditioning for Minimal Residual Methods with

Optimal Left and Right Additive Schwarz Preconditioning for Minimal Residual Methods with

Optimal Left and Right Additive Schwarz Preconditioning for Minimal Residual Methods with Euclidean and Energy Norms Marcus Sarkis Worcester Polytechnic Inst., Mass. and IMPA, Rio de Janeiro and Daniel B. Szyld Temple University,

321 views • 28 slides
Optimal ranking of online search requests for long-term revenue maximization Pierre LEcuyer

Optimal ranking of online search requests for long-term revenue maximization Pierre LEcuyer

1 Optimal ranking of online search requests for long-term revenue maximization Pierre LEcuyer Patrick Maill e, Nicol as Stier-Moses, Bruno Tuffin Technion, Israel, June 2016 2 Search engines Major role in the Internet economy

735 views • 50 slides
Minimal Dataset to obtain optimal registration data from Pathologists Dr. Mia Slabbaert BWP

Minimal Dataset to obtain optimal registration data from Pathologists Dr. Mia Slabbaert BWP

Minimal Dataset to obtain optimal registration data from Pathologists Dr. Mia Slabbaert BWP 25/10/2014 Law 13/06/2006 INTRODUCTION All malignancies have to be registered, using specific classifications and delivered with the complete

728 views • 45 slides
Learning near-optimal hyperparameters with minimal overhead Gellrt Weisz Andrs Gyrgy

Learning near-optimal hyperparameters with minimal overhead Gellrt Weisz Andrs Gyrgy

Learning near-optimal hyperparameters with minimal overhead Gellrt Weisz Andrs Gyrgy Csaba Szepesvri Workshop on Automated Algorithm Design (TTIC 2019) August 7, 2019 1 / 22 Introduction Problem: find good parameter settings

804 views • 58 slides
Optimal Composition of Characteristic Modes For Minimal Quality Factor Q Miloslav Capek Luk

Optimal Composition of Characteristic Modes For Minimal Quality Factor Q Miloslav Capek Luk

Optimal Composition of Characteristic Modes For Minimal Quality Factor Q Miloslav Capek Luk a s Jel nek Department of Electromagnetic Field CTU in Prague, Czech Republic miloslav.capek@fel.cvut.cz 2016 IEEE International

580 views • 44 slides
Geometric and numerical methods in optimal control for the time minimal saturation in Magnetic

Geometric and numerical methods in optimal control for the time minimal saturation in Magnetic

Geometric and numerical methods in optimal control for the time minimal saturation in Magnetic Resonance Imaging DYNAMICS, CONTROL, and GEOMETRY In honor of Bronisaw Jakubczyks 70th birthday 12.09.2018 - 15.09.2018 | Banach Center, Warsaw

324 views • 30 slides
Regularization Paths Boosting fits a regularization path toward a max-margin classifier.

Regularization Paths Boosting fits a regularization path toward a max-margin classifier.

June 2006 Trevor Hastie, Stanford Statistics 1 June 2006 Trevor Hastie, Stanford Statistics 2 Theme Regularization Paths Boosting fits a regularization path toward a max-margin classifier. Svmpath does as well. Trevor Hastie In

112 views • 9 slides
Round Optimal Secure Multiparty Computation from Minimal Assumptions Arka Rai Choudhuri Michele

Round Optimal Secure Multiparty Computation from Minimal Assumptions Arka Rai Choudhuri Michele

Round Optimal Secure Multiparty Computation from Minimal Assumptions Arka Rai Choudhuri Michele Ciampi Vipul Goyal Johns Hopkins University The University of Edinburgh Carnegie Mellon University and NTT Research Abhishek Jain Rafail

1.42k views • 119 slides
Optimal Supply Networks Introduction Optimal branching Complex Networks, Course 295A, Spring,

Optimal Supply Networks Introduction Optimal branching Complex Networks, Course 295A, Spring,

Supply Networks Optimal Supply Networks Introduction Optimal branching Complex Networks, Course 295A, Spring, 2008 Murray meets Tokunaga Single Source History Reframing the question Minimal volume calculation Prof. Peter Dodds Blood

1.06k views • 76 slides
Expectation Maximization CMSC 691 UMBC Outline EM (Expectation Maximization) Basic idea Three

Expectation Maximization CMSC 691 UMBC Outline EM (Expectation Maximization) Basic idea Three

Expectation Maximization CMSC 691 UMBC Outline EM (Expectation Maximization) Basic idea Three coins example Why EM works Expectation Maximization (EM) 0. Assume some value for your parameters Two step, iterative algorithm 1. E-step: count

665 views • 55 slides
About this class Maximizing the Margin Maximum margin classifiers Picture of large and small

About this class Maximizing the Margin Maximum margin classifiers Picture of large and small

About this class Maximizing the Margin Maximum margin classifiers Picture of large and small margin hyperplanes SVMs: geometric derivation of the primal prob- lem Intuition: large margin condition acts as a reg- ularizer and should generalize

219 views • 9 slides
RECSM Summer School: Machine Learning for Social Sciences Session 2.4: Boosting Reto West

RECSM Summer School: Machine Learning for Social Sciences Session 2.4: Boosting Reto West

RECSM Summer School: Machine Learning for Social Sciences Session 2.4: Boosting Reto West Department of Political Science and International Relations University of Geneva 1 Boosting Boosting Like bagging, boosting is a general

373 views • 13 slides
Maximization of a Function of One Variable Economic theories assume that Economic agents

Maximization of a Function of One Variable Economic theories assume that Economic agents

Maximization of a Function of One Variable Economic theories assume that Economic agents seek the optimal value of some objective function Consumers maximize utility Firms maximize profit Simple example, = f(q) Manager

1.7k views • 136 slides
Boosting (ensemble) Module 4 - Ensemble classifiers - Objectives module 4: boosting (ensemble

Boosting (ensemble) Module 4 - Ensemble classifiers - Objectives module 4: boosting (ensemble

Boosting (ensemble) Module 4 - Ensemble classifiers - Objectives module 4: boosting (ensemble models) LEARNING PERFORMANCE REPRESENTATION DATA PROBLEM RAW DATA CLUSTERING EVALUATION FEATURES UCI datasets unigrams 20newsgroups SUPERVISED

926 views • 34 slides
Boosting Methods: Implicit Combinatorial Optimization via First-Order Convex Optimization Robert

Boosting Methods: Implicit Combinatorial Optimization via First-Order Convex Optimization Robert

Motivation Review of SD and FW(CG) Binary Classification and Boosting Linear Regression and Boosting Boosting Methods: Implicit Combinatorial Optimization via First-Order Convex Optimization Robert M. Freund Paul Grigas Rahul Mazumder

707 views • 49 slides
An overview of Boosting Yoav Freund UCSD Plan of talk Generative vs. non-generative

An overview of Boosting Yoav Freund UCSD Plan of talk Generative vs. non-generative

An overview of Boosting Yoav Freund UCSD Plan of talk Generative vs. non-generative modeling Boosting Alternating decision trees Boosting and over-fitting Applications 2 Toy Example Computer receives telephone call

735 views • 61 slides
The Boosting Approach to Machine Learning Maria-Florina Balcan 03/16/2015 Boosting General

The Boosting Approach to Machine Learning Maria-Florina Balcan 03/16/2015 Boosting General

The Boosting Approach to Machine Learning Maria-Florina Balcan 03/16/2015 Boosting General method for improving the accuracy of any given learning algorithm. Works by creating a series of challenge datasets s.t. even modest

715 views • 32 slides
Submodular Maximization Seffi Naor Lecture 2 4th Cargese Workshop on Combinatorial Optimization

Submodular Maximization Seffi Naor Lecture 2 4th Cargese Workshop on Combinatorial Optimization

Submodular Maximization Seffi Naor Lecture 2 4th Cargese Workshop on Combinatorial Optimization Seffi Naor Submodular Maximization Submodular Maximization Constrained Submodular Maximization Family of allowed subsets M 2 N . f ( S ) max

477 views • 35 slides

More recommend