An adaptive nearest neighbor rule for classification Akshay - PowerPoint PPT Presentation

Dec 23, 2022 •352 likes •411 views

An adaptive nearest neighbor rule for classification Akshay Balsubramani, Stanford Sanjoy Dasgupta, UCSD Yoav Freund, UCSD Shay Moran, Google AI Princeton Main Idea: Modify k -NN Algorithm by Choosing k Adaptively for Each Query Classical

An adaptive nearest neighbor rule for classification Akshay Balsubramani, Stanford Sanjoy Dasgupta, UCSD Yoav Freund, UCSD Shay Moran, Google AI Princeton
Main Idea: Modify k -NN Algorithm by Choosing k Adaptively for Each Query • Classical k -NN: classify x by the majority vote of its k nearest in the training set. x is the green point in the middle. The label assigned to x is determined by its k nearest neighbors (inside the big circle, in this example k =13+12=25 )
Main Idea: Modify k -NN Algorithm by Choosing k Adaptively for Each Query • Adaptive k -NN: • Iterate over the neighbors of x from nearest to furthest and query their labels. • If one of the label-classes obtains a significant majority then exit the loop and use this label to classify x . Points x that are far from the boundary observe a significant Points x that are close to the boundary require querying a large number of neighbors advantage after querying a small number of neighbors
Main Result s Theoretical Results 1. Adaptive k-NN rule is consistent (i.e. achieves Bayes optimality in the limit). 2. Instance-dependent generalization bounds Number of examples required to classify x correctly depends on its “local- • margin” (a formal notion introduced in the paper). Points far from the boundary are correctly classified fast. • Practical Results 1. Adaptive k -NN rule is competitive with Classical k -NN with the best choice of k Thus, this method circumvents the need to tune the meta-parameter k. •

Recommend

Nearest Neighbor Classification Machine Learning 1 This lecture K-nearest neighbor

Nearest Neighbor Classification Machine Learning 1 This lecture K-nearest neighbor classification The basic algorithm Different distance measures Some practical aspects Voronoi Diagrams and Decision Boundaries What is the

893 views • 57 slides

NEAREST NEIGHBOR RULE Jeff Robble, Brian Renzenbrink, Doug Roberts Nearest Neighbor Rule

NEAREST NEIGHBOR RULE Jeff Robble, Brian Renzenbrink, Doug Roberts Nearest Neighbor Rule Consider a test point x. x is the closest point to x out of the rest of the test points. Nearest Neighbor Rule selects the class for x with the

573 views • 33 slides

Nearest Neighbor Classification Seed classification by area and What should we compactness

Nearest Neighbor Classification Seed classification by area and What should we compactness predict for unlabeled Lecture 6 KNN and Decision Trees test points (stars)? Nearest neighbor classification: predict CS 335 label of

203 views • 3 slides

Nearest neighbor classification in metric spaces: universal consistency and rates of convergence

Nearest neighbor classification in metric spaces: universal consistency and rates of convergence Sanjoy Dasgupta University of California, San Diego Nearest neighbor The primeval approach to classification. Given: a training set { ( x 1 , y

926 views • 20 slides

Classification James H. Steiger Department of Psychology and Human Development Vanderbilt

Introduction The Linear Classification Function Quadratic Classification Functions Estimating Misclassification Rates Bias in Error Rate Estimation Error Rates in Variable Selection Classification via the k Nearest Neighbor Rule Classification

417 views • 31 slides

CSCI 447/547 MACHINE LEARNING Outline Nearest Neighbor K-Nearest Neighbor Algorithm

Nearest Neighbor CSCI 447/547 MACHINE LEARNING Outline Nearest Neighbor K-Nearest Neighbor Algorithm Note: Slides were adapted from David Sontag, New York University (who adapted them from Vibhav Gogate, Carlos Questrin, Mehryar

200 views • 16 slides

DATA MINING LECTURE 10 Classification Nearest Neighbor Classification Support Vector Machines

DATA MINING LECTURE 10 Classification Nearest Neighbor Classification Support Vector Machines Logistic Regression Nave Bayes Classifier Supervised Learning Illustrating Classification Task Learning Tid Attrib1 Attrib2 Attrib3 Class

954 views • 79 slides

DATA MINING LECTURE 11 Classification Nearest Neighbor Classification Support Vector Machines

DATA MINING LECTURE 11 Classification Nearest Neighbor Classification Support Vector Machines Logistic Regression Nave Bayes Classifier Supervised Learning Illustrating Classification Task Learning Tid Attrib1 Attrib2 Attrib3 Class

1.03k views • 75 slides

Data Mining Techniques: Statistical Decision Theory Nearest Neighbor Classification and

Classification and Prediction Overview Introduction Decision Trees Data Mining Techniques: Statistical Decision Theory Nearest Neighbor Classification and Prediction Bayesian Classification Artificial Neural Networks Mirek

701 views • 40 slides

Data Mining Techniques: Statistical Decision Theory Nearest Neighbor Classification and

548 views • 42 slides

Learning From Data Lecture 16 Similarity and Nearest Neighbor Similarity Nearest Neighbor M.

Learning From Data Lecture 16 Similarity and Nearest Neighbor Similarity Nearest Neighbor M. Magdon-Ismail CSCI 4100/6100 My 5-Year-Old Called It A ManoHorse The simplest method of learning that we know. Classify according to similar

331 views • 16 slides

Classification K-nearest neighbor classification D istance functions Choice of k Choice of k

Classification K-nearest neighbor classification D istance functions Choice of k Choice of k Leave-one-out cross validation K-fold cross validation Classification Error = Average classification error on K folds Linear Classification Linear

395 views • 22 slides

Nearest Neighbor Classifiers CSE 4308/5360: Artificial Intelligence I University of Texas at

Nearest Neighbor Classifiers CSE 4308/5360: Artificial Intelligence I University of Texas at Arlington 1 The Nearest Neighbor Classifier Let X be the space of all possible patterns for some classification problem. Let F be a distance

512 views • 24 slides

Nearest-Neighbor Methods Store all training examples Given a new test example, find the k that are

Nearest-Neighbor Methods Store all training examples Given a new test example, find the k that are closest to it in feature space (distance: Eu- About this class clidean/Mahalanobis?) k -Nearest-Neighbors Return majority classification among

368 views • 6 slides

Finding Predictors: Nearest Neighbor Modern Motivations: Be Lazy! Classification Regression

Finding Predictors: Nearest Neighbor Modern Motivations: Be Lazy! Classification Regression Choosing the right number of neighbors Some Optimizations Other types of lazy algorithms Compendium slides for Guide to Intelligent Data

1.53k views • 109 slides

Proximity in the Age of Distraction: Robust Approximate Nearest Neighbor Search Sariel Har-Peled

Proximity in the Age of Distraction: Robust Approximate Nearest Neighbor Search Sariel Har-Peled Sepideh Mahabadi UIUC MIT Nearest Neighbor Problem Nearest Neighbor Dataset of points in a metric space (, ) , e.g.

967 views • 81 slides

BAYES AND NEAREST NEIGHBOR BAYES AND NEAREST NEIGHBOR CLASSIFIERS CLASSIFIERS Matthieu R Bloch

BAYES AND NEAREST NEIGHBOR BAYES AND NEAREST NEIGHBOR CLASSIFIERS CLASSIFIERS Matthieu R Bloch Tuesday, January 21, 2020 1 LOGISTICS LOGISTICS TAs and Office hours Monday: Mehrdad (TSRB 523a) - 2pm-3:15pm Tuesday: TJ (VL C449 Cubicle D) -

362 views • 12 slides

High-Dimensional Nearest Neighbor Search High-Dimensional Nearest Neighbor Search Who?

High-Dimensional Nearest Neighbor Search High-Dimensional Nearest Neighbor Search Who? About Cliqz and me What? Problem statement Why? Applications How? Exact solutions in low dimensions Approximate

690 views • 19 slides

9/28/2009 Nearest Neighbor Queries What are the two nearest stars to Andromeda? Reverse

9/28/2009 Nearest Neighbor Queries What are the two nearest stars to Andromeda? Reverse kNNsearch in Arbitrary Dimensionality Where is the nearest restaurant? Seyed Jalal Kazemitabar Original paper by Y. Tao, D. Papadias, and X. Lian Where

228 views • 6 slides

Simultaneous Nearest Neighbor Search Piotr Indyk Robert Kleinberg MIT Cornell Sepideh

Simultaneous Nearest Neighbor Search Piotr Indyk Robert Kleinberg MIT Cornell Sepideh Mahabadi Yang Yuan MIT Cornell Nearest Neighbor Dataset of points in a metric space (, ) 6/17/2016 2 Nearest Neighbor

883 views • 69 slides

Nearest neighbor methods Lecture 11 David Sontag New York

Nearest neighbor methods Lecture 11 David Sontag New York University Slides adapted from Vibhav Gogate, Carlos Guestrin, Mehryar Mohri, & Luke Zettlemoyer Nearest Neighbor Algorithm

311 views • 20 slides

Nearest Neighbor and Locality-Sensitive Hashing Nearest Neighbor Set Similarity

Nearest Neighbor and Locality-Sensitive Hashing Nearest Neighbor Set Similarity Locality-Sensitive Hashing Document Similarity Philip Bille Nearest Neighbor and Locality-Sensitive Hashing Nearest Neighbor Set Similarity

623 views • 17 slides

Outline Density Estimation 1 Nonparametric Methods Bins Kernel Estimators k-Nearest Neighbor

Density Estimation Classification Regression Density Estimation Classification Regression Outline Density Estimation 1 Nonparametric Methods Bins Kernel Estimators k-Nearest Neighbor Steven J Zeil Multivariate Data Old Dominion Univ.

414 views • 7 slides

1 Nearest Neighbor Heuristic Nearest Neighbor Heuristic Start at some point p 0 p 1 p 0

Algorithms: definition &6($OJRULWKPVDQG &RPSXWDWLRQDO&RPSOH[LW\ Procedure to accomplish a task or solve a well-specified problem $OJRULWKPVDQG(IILFLHQF\ Well-specified: know what

51 views • 3 slides