Nearly Tight Bounds for Robust Proper Learning of Halfspaces with a - PowerPoint PPT Presentation

Nearly Tight Bounds for Robust Proper Learning of Halfspaces with a Margin Ilias Diakonikolas Daniel M. Kane Pasin Manurangsi UW Madison UC San Diego Google Nearly Tight Bound for Robust Proper Learning of Halfspaces with a Margin Diakonikolas, Kane, Manurangsi

Agnostic Proper Learning of Halfspaces Input + + - - + - Labeled samples (x 1 , y 1 ), (x 2 , y 2 ), … ∈ 𝓒 (d) × {±1} + from distribution 𝓔 + - + - - - + - - - + - Nearly Tight Bound for Robust Proper Learning of Halfspaces with a Margin Diakonikolas, Kane, Manurangsi

Agnostic Proper Learning of Halfspaces Input + + - - + - Labeled samples (x 1 , y 1 ), (x 2 , y 2 ), … ∈ 𝓒 (d) × {±1} + from distribution 𝓔 + - Positive real number ε - + - - - + - - - + - Nearly Tight Bound for Robust Proper Learning of Halfspaces with a Margin Diakonikolas, Kane, Manurangsi

Agnostic Proper Learning of Halfspaces Input + + - - + - Labeled samples (x 1 , y 1 ), (x 2 , y 2 ), … ∈ 𝓒 (d) × {±1} + from distribution 𝓔 + - Positive real number ε - + Output A halfspace w with “small” classifjcation error - - - + - - - + - Nearly Tight Bound for Robust Proper Learning of Halfspaces with a Margin Diakonikolas, Kane, Manurangsi

Agnostic Proper Learning of Halfspaces Input + + - - + - Labeled samples (x 1 , y 1 ), (x 2 , y 2 ), … ∈ 𝓒 (d) × {±1} + from distribution 𝓔 + - Positive real number ε - + Output A halfspace w with “small” classifjcation error - - - + - - - + - OPT = Min classifjcation error among all halfspaces = min w Pr (x, y)~ 𝓔 [<w, x> ・ y ＜ 0] Nearly Tight Bound for Robust Proper Learning of Halfspaces with a Margin Diakonikolas, Kane, Manurangsi

Agnostic Proper Learning of Halfspaces w* Input + + - - + - Labeled samples (x 1 , y 1 ), (x 2 , y 2 ), … ∈ 𝓒 (d) × {±1} + from distribution 𝓔 + - Positive real number ε - + Output A halfspace w with “small” classifjcation error - - - + - - - + - OPT = Min classifjcation error among all halfspaces = min w Pr (x, y)~ 𝓔 [<w, x> ・ y ＜ 0] Nearly Tight Bound for Robust Proper Learning of Halfspaces with a Margin Diakonikolas, Kane, Manurangsi

Agnostic Proper Learning of Halfspaces w* Input + + - - + - Labeled samples (x 1 , y 1 ), (x 2 , y 2 ), … ∈ 𝓒 (d) × {±1} + from distribution 𝓔 + - Positive real number ε - + Output A halfspace w with “small” classifjcation error - - - + - An algorithm is a 𝛽 -learner if it outputs w with - classifjcation error at most 𝛽 ・ OPT + ε - + - OPT = Min classifjcation error among all halfspaces = min w Pr (x, y)~ 𝓔 [<w, x> ・ y ＜ 0] Nearly Tight Bound for Robust Proper Learning of Halfspaces with a Margin Diakonikolas, Kane, Manurangsi

Agnostic Proper Learning of Halfspaces w* Input + w + - - + - Labeled samples (x 1 , y 1 ), (x 2 , y 2 ), … ∈ 𝓒 (d) × {±1} + from distribution 𝓔 + - Positive real number ε - + Output A halfspace w with “small” classifjcation error - - - + - An algorithm is a 𝛽 -learner if it outputs w with - classifjcation error at most 𝛽 ・ OPT + ε - + - OPT = Min classifjcation error among all halfspaces = min w Pr (x, y)~ 𝓔 [<w, x> ・ y ＜ 0] Nearly Tight Bound for Robust Proper Learning of Halfspaces with a Margin Diakonikolas, Kane, Manurangsi

Agnostic Proper Learning of Halfspaces w* Input + w + - - + - Labeled samples (x 1 , y 1 ), (x 2 , y 2 ), … ∈ 𝓒 (d) × {±1} + from distribution 𝓔 + - Positive real number ε - + Output A halfspace w with “small” classifjcation error - - - + - An algorithm is a 𝛽 -learner if it outputs w with - classifjcation error at most 𝛽 ・ OPT + ε - + - Bad news: [Arora et al.’97] Unless NP = RP, no poly-time 𝛽 -learner for all constants 𝛽 . OPT = Min classifjcation error among all halfspaces = min w Pr (x, y)~ 𝓔 [<w, x> ・ y ＜ 0] [Guruswami-Raghavendra’ 06, Feldman et al.’06] Even weak learning is NP-hard. Nearly Tight Bound for Robust Proper Learning of Halfspaces with a Margin Diakonikolas, Kane, Manurangsi

Agnostic Proper Learning of Halfspaces with a Margin Input + + - - + - Labeled samples (x 1 , y 1 ), (x 2 , y 2 ), … ∈ 𝓒 (d) × {±1} + from distribution 𝓔 + - Positive real number ε - + Output A halfspace w with “small” classifjcation error - - - + - - - + - Nearly Tight Bound for Robust Proper Learning of Halfspaces with a Margin Diakonikolas, Kane, Manurangsi

Agnostic Proper Learning of Halfspaces with a Margin w* Input + + - - + - Labeled samples (x 1 , y 1 ), (x 2 , y 2 ), … ∈ 𝓒 (d) × {±1} + from distribution 𝓔 + - Positive real number ε - + Output A halfspace w with “small” classifjcation error - - - + - - - + - OPT 𝛿 = Min 𝛿 -margin error among all halfspaces = min w Pr (x, y)~ 𝓔 [<w, x> ・ y ＜ 𝛿 ] Nearly Tight Bound for Robust Proper Learning of Halfspaces with a Margin Diakonikolas, Kane, Manurangsi

Agnostic Proper Learning of Halfspaces with a Margin w* Input + + - - + - Labeled samples (x 1 , y 1 ), (x 2 , y 2 ), … ∈ 𝓒 (d) × {±1} + from distribution 𝓔 + - Positive real number ε - 𝛿 𝛿 + Output A halfspace w with “small” classifjcation error - - - + - - - + - OPT 𝛿 = Min 𝛿 -margin error among all halfspaces = min w Pr (x, y)~ 𝓔 [<w, x> ・ y ＜ 𝛿 ] Nearly Tight Bound for Robust Proper Learning of Halfspaces with a Margin Diakonikolas, Kane, Manurangsi

Agnostic Proper Learning of Halfspaces with a Margin w* Input + + - - + - Labeled samples (x 1 , y 1 ), (x 2 , y 2 ), … ∈ 𝓒 (d) × {±1} + from distribution 𝓔 + - Positive real number ε - 𝛿 𝛿 + Output A halfspace w with “small” classifjcation error - - - + - An algorithm is a 𝛽 -learner if it outputs w with - classifjcation error at most 𝛽 ・ OPT 𝛿 + ε - + - OPT 𝛿 = Min 𝛿 -margin error among all halfspaces = min w Pr (x, y)~ 𝓔 [<w, x> ・ y ＜ 𝛿 ] Nearly Tight Bound for Robust Proper Learning of Halfspaces with a Margin Diakonikolas, Kane, Manurangsi

Agnostic Proper Learning of Halfspaces with a Margin w* Input + + - - + - Labeled samples (x 1 , y 1 ), (x 2 , y 2 ), … ∈ 𝓒 (d) × {±1} + from distribution 𝓔 + - Positive real number ε - 𝛿 𝛿 + Output A halfspace w with “small” classifjcation error - - - + - An algorithm is a 𝛽 -learner if it outputs w with - classifjcation error at most 𝛽 ・ OPT 𝛿 + ε - + - Margin Assumption OPT 𝛿 = Min 𝛿 -margin error among all halfspaces - “Robustness” of the optimal halfspace to ℓ 2 noise = min w Pr (x, y)~ 𝓔 [<w, x> ・ y ＜ 𝛿 ] Nearly Tight Bound for Robust Proper Learning of Halfspaces with a Margin Diakonikolas, Kane, Manurangsi

Agnostic Proper Learning of Halfspaces with a Margin w* Input + + - - + - Labeled samples (x 1 , y 1 ), (x 2 , y 2 ), … ∈ 𝓒 (d) × {±1} + from distribution 𝓔 + - Positive real number ε - 𝛿 𝛿 + Output A halfspace w with “small” classifjcation error - - - + - An algorithm is a 𝛽 -learner if it outputs w with - classifjcation error at most 𝛽 ・ OPT 𝛿 + ε - + - Margin Assumption OPT 𝛿 = Min 𝛿 -margin error among all halfspaces - “Robustness” of the optimal halfspace to ℓ 2 noise = min w Pr (x, y)~ 𝓔 [<w, x> ・ y ＜ 𝛿 ] - Variants used in Perceptron, SVMs Nearly Tight Bound for Robust Proper Learning of Halfspaces with a Margin Diakonikolas, Kane, Manurangsi

Nearly Tight Bounds for Robust Proper Learning of Halfspaces with a - PowerPoint PPT Presentation

Nearly Tight Bounds for Robust Proper Learning of Halfspaces with a Margin Ilias Diakonikolas Daniel M. Kane Pasin Manurangsi UW Madison UC San Diego Google Nearly Tight Bound for Robust Proper Learning of Halfspaces with a Margin

Collaborative Learning with Limited Interaction: Tight Bounds for Distributed Exploration in

Learning and Optimization: Lower Bounds and Tight Connections Nati Srebro TTI-Chicago On The

Tight Bounds on Minimax Regret under Logarithmic Loss via Self-Concordance Blair Bilodeau 1,2,3 ,

Tight Bounds for Learning a Mixture of Two Gaussians Moritz Hardt Eric Price Google Research

Non-convex Robust PCA: Provable Bounds Anima Anandkumar U.C. Irvine Joint work with Praneeth

Tight bounds for Communication assisted agreement distillation Jaikumar Radhakrishnan Tata

Nearly-tight VC-dimension bounds for piecewise linear neural networks Nicholas J. A. Harvey,

Computing Tight Bounds for Insurance Payments with Nonlinear Risk Man Hong WONG 1 Shuzhong ZHANG 2

Tight Erds-Psa bounds for minors Jean-Florent Raymond (TU Berlin) Joint work with Wouter

Tight Bounds for Cost-Sharing in Weighted Congestion Games Martin Gairing University of

Finer Tight Bounds for Coloring on Clique-width Michael Lampis LAMSADE Universit e Paris

Tight Upper and Lower Bounds for Leakage-Resilient Locally Decodable and Updatable Non-Malleable

Tight Bounds on the Relative Performance of Pricing Mechanisms in Storable Good Markets Gerardo

PAC-learning, VC Dimension and Margin-based Bounds Machine Learning 10701/15781 Carlos

Tight Bounds for Single-Pass Streaming Complexity of the Set Cover Problem Sepehr Assadi

Short Course in Supervised Learning Robust Optimization and Machine Learning Robust Supervised

Weakly Supervised Classification Weakly Supervised Classification and Robust Learning and Robust

, , Weakly Supervised Classification Robust Learning and More: Robust Learning and More:

Robust Bounds for Emptiness Formation Probability for Dimers Shannon Starr University of Alabama

Robust Lower Bounds for Communication and Stream Computation Amit Chakrabarti Dartmouth

What is this talk about? What is this talk about? Deriving tight and safe task execution time

Conspiracies between Learning Algorithms, Lower Bounds, and Pseudorandomness Igor Carboni

Regret Bounds for Lifelong Learning Pierre Alquier Groupe de Travail de Machine learning du CMLA

Adversarial Learning Bounds for Linear Classes and Neural Nets Understanding Adversarial Learning