k nearest neighbors
play

K-Nearest Neighbors Nicolas Indelicato K-Nearest Neighbors Dataset - PowerPoint PPT Presentation

Sep 18, 2023 •135 likes •353 views

K-Nearest Neighbors Nicolas Indelicato K-Nearest Neighbors Dataset Background How the Algorithm Works Optimizing the Algorithm Results Issues Summary Dataset Background Wine Dataset 13 Attributes Alcohol, Malic

  1. K-Nearest Neighbors Nicolas Indelicato

  2. K-Nearest Neighbors • Dataset Background • How the Algorithm Works • Optimizing the Algorithm • Results • Issues • Summary

  3. Dataset Background • Wine Dataset – 13 Attributes • Alcohol, Malic Acid, Ash, Alcalinity of Ash, Magnesium, Total Phenols, Flavanoids, NonFlavanoid Phenols, Proanthocyanins, Color Intensity, Hue, OD280/D315 of Diluted Wines, Proline – Wide Range of Correlations • 2% in Ash to 83% in Flavanoids

  4. Dataset Background Wine (continued) – 3 Classes • Class 1, Class 2, Class 3 wine – Attribute Weights • Nonflavanoid Phenols from 0.13 to 0.66 • Proline from 290 to 1680

  5. Dataset Background • Iris Dataset – 4 Attributes • Sepal Length, Sepal Width, Petal Length, Petal Width – Range of Correlations • Sepal Width of 42% to Petal Lenth of 95% and Petal Width of 96% – 3 Classes • Iris-Setosa, Versicolor, and Virginica – Attribute Weights • Petal Width from 0.1 to 2.5 • Sepal Lentrh from 4.3 to 7.9

  6. Dataset Background • Datasets include entities with similar attributes. • Determining the class cannot be done easily or quickly. • Descriptive Statistics is inefficient and cumbersome.

  7. How the Algorithm Works • Instance-based • Used in classification and pattern recognition since the 1960s. • Minor training phase. • Customizable – Distance Method – k

  8. How the Algorithm Works • K – Fixed constant – Determines number of elements to be included in each neighborhood. • Neighborhood determines classification • Different k values can and will produce different classifications

  9. How the Algorithm Works • 1 Nearest Neighbor – Point x q classified as a “ + ” • 5 Nearest Neighbors – Point x q classified as a “ - ”

  10. How the Algorithm Works • Euclidean Distance in n space. • a r (x) = r th attribute of instance x • x I and x J represent two separate instances • Distance = Square Root of the Sum of the Squares.

  11. Optimizing the Algorithm • Correlation – Does low correlation mean irrelevant attributes? • Missing values – Will missing values make the results erroneous? • Normalization – Will normalization of the attributes make the results more accurate? • Size – How efficiently does the algorithm classify data?

  12. Results • Iris Dataset – Non-normalized • All attributes – Misclassification rate = 6% – 94% Accuracy » Setosa misclassified = 0/150 = 0% » Versicolor misclassified = 0/150 = 0% » Virginica misclassified = 9/150 = 6%

  13. Results • Iris Dataset – Normalized • All attributes – Misclassification rate = 7.33% – 92.67% Accuracy » Setosa misclassified = 0/150 = 0% » Versicolor misclassified = 1/150 = 0.67% » Virginica misclassified = 10/150 = 6.67%

  14. Results • Iris Dataset – Non-normalized • Petal Length and Petal Width – Misclassification rate = 4.67% – 95.33% Accuracy » Setosa misclassified = 0/150 = 0% » Versicolor misclassified = 0/150 = 0% » Virginica misclassified = 7/150 = 4.67%

  15. Results • Iris Dataset – Normalized • Petal Length and Petal Width – Misclassification rate = 7.33% – 92.67% Accuracy » Setosa misclassified = 0/150 = 0% » Versicolor misclassified = 0/150 = 0% » Virginica misclassified = 11/150 = 7.33%

  16. Results • Wine Dataset – Non-normalized • All attributes – Misclassification rate = 27.45% – 72.55% Accuracy » Class 1 wine misclassified = 7/153 = 4.58% » Class 2 wine misclassified = 23/153 = 15.08% » Class 3 wine misclassified = 12/153 = 7.84%

  17. Results • Wine Dataset – Normalized • All attributes – Misclassification rate = 5.88% – 94.12% Accuracy » Class 1 wine misclassified = 0/153 = 0% » Class 2 wine misclassified = 9/153 = 5.88% » Class 3 wine misclassified = 0/153 = 0%

  18. Results • Wine Dataset – Non-normalized • Phenols, Flavanoids, OD280/OD315 – Misclassification rate = 20.92% – 79.08% Accuracy » Class 1 wine misclassified = 1/153 = 0.65% » Class 2 wine misclassified = 31/153 = 20.26% » Class 3 wine misclassified = 0/153 = 0%

  19. Results • Wine Dataset – Normalized • Phenols, Flavanoids, OD280/OD315 – Misclassification rate = 20.92% – 79.08% Accuracy » Class 1 wine misclassified = 2/153 = 1.31% » Class 2 wine misclassified = 30/153 = 19.61% » Class 3 wine misclassified = 0/153 = 0%

  20. Issues • Nearest neighbors include equal amount of neighbors from two classes. – Classified into class with nearest neighbor.

  21. Summary • Dataset Background • How the Algorithm Works • Optimizing the Algorithm • Results • Issues

Recommend

c i,j max k,m c k,m 4 Wednesday, 26 Feb. 2020 Machine Learning (COMP 135) 3 Wednesday, 26

c i,j max k,m c k,m 4 Wednesday, 26 Feb. 2020 Machine Learning (COMP 135) 3 Wednesday, 26

Uses of Nearest Neighbors } Once we have found the k -nearest neighbors of a point, we can use this information: In and of itself : sometimes we just want to know what 1. those nearest neighbors actually are (items that are similar to a given

221 views • 4 slides
c i,j max k,m c k,m 4 Wednesday, 2 Oct. 2019 Machine Learning (COMP 135) 3 Wednesday, 2

c i,j max k,m c k,m 4 Wednesday, 2 Oct. 2019 Machine Learning (COMP 135) 3 Wednesday, 2

Uses of Nearest Neighbors } Once we have found the k -nearest neighbors of a point, we can use this information: In and of itself : sometimes we just want to know what 1. those nearest neighbors actually are (items that are similar to a given

439 views • 4 slides
Approximate Nearest Neighbors Search Approximate Nearest Neighbors Search in High Dimensions in

Approximate Nearest Neighbors Search Approximate Nearest Neighbors Search in High Dimensions in

Approximate Nearest Neighbors Search Approximate Nearest Neighbors Search in High Dimensions in High Dimensions and Locality-Sensitive Hashing and Locality-Sensitive Hashing PAPERS Piotr Indyk, Rajeev Motwani: Approximate Nearest Neighbors:

793 views • 68 slides
FAST APPROXIMATE NEAREST NEIGHBORS WITH AUTOMATIC ALGORITHM CONFIGURATION Marius Muja, David G.

FAST APPROXIMATE NEAREST NEIGHBORS WITH AUTOMATIC ALGORITHM CONFIGURATION Marius Muja, David G.

FAST APPROXIMATE NEAREST NEIGHBORS WITH AUTOMATIC ALGORITHM CONFIGURATION Marius Muja, David G. Lowe, 2009 Presented by: Gautam Gunjala & Jordan Zhang Nearest Neighbors? Given points P={p 1 , ,p n } in a vector space X,preprocess such

560 views • 33 slides
Approximate Nearest Neighbors Sariel Har Peled: Notes Arya, Mount, Netenyahu, Silverman, Wu An

Approximate Nearest Neighbors Sariel Har Peled: Notes Arya, Mount, Netenyahu, Silverman, Wu An

Approximate Nearest Neighbors Sariel Har Peled: Notes Arya, Mount, Netenyahu, Silverman, Wu An Optimal Algorithm for Approximate Nearest Neighbor Searching in Fixed Dimensions Approximate Nearest Neighbors What we want O(n log n)

447 views • 25 slides
New directions in approximate nearest neighbors for the angular distance Thijs Laarhoven

New directions in approximate nearest neighbors for the angular distance Thijs Laarhoven

New directions in approximate nearest neighbors for the angular distance Thijs Laarhoven mail@thijs.com http://www.thijs.com/ Proximity Workshop, College Park (MD), USA (January 13, 2016) Nearest neighbor searching O Nearest neighbor

1.24k views • 102 slides
CSC 411: Lecture 05: Nearest Neighbors Class based on Raquel Urtasun & Rich Zemels lectures

CSC 411: Lecture 05: Nearest Neighbors Class based on Raquel Urtasun & Rich Zemels lectures

CSC 411: Lecture 05: Nearest Neighbors Class based on Raquel Urtasun & Rich Zemels lectures Sanja Fidler University of Toronto Jan 25, 2016 Urtasun, Zemel, Fidler (UofT) CSC 411: 05-Nearest Neighbors Jan 25, 2016 1 / 21 Today

508 views • 21 slides
k-Nearest Neighbors Lecture 2 k-Nearest Neighbors September 16, 2015 1 Wentworth Institute of

k-Nearest Neighbors Lecture 2 k-Nearest Neighbors September 16, 2015 1 Wentworth Institute of

Wentworth Institute of Technology COMP4050 Machine Learning | Fall 2015 | Derbinsky k-Nearest Neighbors Lecture 2 k-Nearest Neighbors September 16, 2015 1 Wentworth Institute of Technology COMP4050 Machine Learning | Fall 2015 |

385 views • 34 slides
Inference and Estimation Using Nearest Neighbors 2019 The Second Korea-Japan Machine Learning

Inference and Estimation Using Nearest Neighbors 2019 The Second Korea-Japan Machine Learning

Inference and Estimation Using Nearest Neighbors 2019 The Second Korea-Japan Machine Learning Workshop 2019. 2. 22 (Fri.) Yung-Kyun Noh Seoul National University Hanyang University Seoul National University Nearest Neighbors Similar

374 views • 25 slides
k -Reciprocal Nearest Neighbors Algorithm for One-Class Collaborative Filtering Wei Cai a , b , c

k -Reciprocal Nearest Neighbors Algorithm for One-Class Collaborative Filtering Wei Cai a , b , c

k -Reciprocal Nearest Neighbors Algorithm for One-Class Collaborative Filtering Wei Cai a , b , c , Weike Pan a , b , c , Jixiong Liu a , b , c , Zixiang Chen a , b , c , Zhong Ming a , b , c { caiwei2016, liujixiong, chenzixiang2016 }

678 views • 35 slides
CSC 311: Introduction to Machine Learning Lecture 1 - Introduction and Nearest Neighbors Roger

CSC 311: Introduction to Machine Learning Lecture 1 - Introduction and Nearest Neighbors Roger

CSC 311: Introduction to Machine Learning Lecture 1 - Introduction and Nearest Neighbors Roger Grosse Chris Maddison Juhan Bae Silviu Pitis University of Toronto, Fall 2020 Intro ML (UofT) CSC311-Lec1 1 / 55 This course Broad

1.38k views • 58 slides
Classification with Nearest Neighbors CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu What we know

Classification with Nearest Neighbors CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu What we know

Classification with Nearest Neighbors CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu What we know so far Decision Trees What is a decision tree, and how to induce it from data Fundamental Machine Learning Concepts Difference between

560 views • 21 slides
Clustering, K-Means, and K-Nearest Neighbors CMSC 678 UMBC Most slides courtesy Hamed

Clustering, K-Means, and K-Nearest Neighbors CMSC 678 UMBC Most slides courtesy Hamed

Clustering, K-Means, and K-Nearest Neighbors CMSC 678 UMBC Most slides courtesy Hamed Pirsiavash Recap from last time Geometric Rationale of LDiscA & PCA Objective: to rigidly rotate the axes of the D-dimensional space to new

1.31k views • 65 slides
Finding Finding Al All l Nearest earest Neighb Neighbors ors wi with th a Single a Single

Finding Finding Al All l Nearest earest Neighb Neighbors ors wi with th a Single a Single

Finding Finding Al All l Nearest earest Neighb Neighbors ors wi with th a Single a Single Graph Traversal raph Traversal Yixin Xu, Jianzhong Qi, Renata Borovica-Gajic, and Lars Kulik Motivation Parking undersupply? 65% oversupply [1]

417 views • 22 slides
Simple and Fast Nearest Neighbor Search Marcel Birn, Manuel Holtgrewe, Peter Sanders , Johannes

Simple and Fast Nearest Neighbor Search Marcel Birn, Manuel Holtgrewe, Peter Sanders , Johannes

Birn et al.: Nearest Neighbors 1 Simple and Fast Nearest Neighbor Search Marcel Birn, Manuel Holtgrewe, Peter Sanders , Johannes Singler Birn et al.: Nearest Neighbors 2 2D Nearest Neighbor Search Preprocess S = { ( x 1 , y 1 ) , . . . , ( x

201 views • 19 slides
Spatial-Temporal K Nearest Neighbors Model on MapReduce for Traffic Flow Prediction A. Agafonov,

Spatial-Temporal K Nearest Neighbors Model on MapReduce for Traffic Flow Prediction A. Agafonov,

Spatial-Temporal K Nearest Neighbors Model on MapReduce for Traffic Flow Prediction A. Agafonov, A. Yumaganov Samara National Research University The 19th International Conference on Intelligent Data Engineering and Automated Learning (IDEAL

437 views • 15 slides
Experimental Design + k-Nearest Neighbors KNN Readings: Prob. Readings: (next

Experimental Design + k-Nearest Neighbors KNN Readings: Prob. Readings: (next

10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University Experimental Design + k-Nearest Neighbors KNN Readings: Prob. Readings: (next

757 views • 65 slides
Final Project Specifications CMPE 650 kNN Overview K-N earest N eighbors (kNN) is a

Final Project Specifications CMPE 650 kNN Overview K-N earest N eighbors (kNN) is a

Final Project Specifications CMPE 650 kNN Overview K-N earest N eighbors (kNN) is a relatively simple ML algorithm in which an object is classified by a majority vote of its k nearest neighbors. Given a new input, its nearest neighbors are

70 views • 5 slides
K-Nearest Neighbors Jia-Bin Huang Virginia Tech Spring 2019 ECE-5424G / CS-5824 Administrative

K-Nearest Neighbors Jia-Bin Huang Virginia Tech Spring 2019 ECE-5424G / CS-5824 Administrative

K-Nearest Neighbors Jia-Bin Huang Virginia Tech Spring 2019 ECE-5424G / CS-5824 Administrative Check out review materials Probability Linear algebra Python and NumPy Start your HW 0 On your Local machine: Install

730 views • 57 slides
k-Nearest Neighbors + Model Selection Matt Gormley Lecture 5 Jan. 29, 2020 1 Q&A Q: Why

k-Nearest Neighbors + Model Selection Matt Gormley Lecture 5 Jan. 29, 2020 1 Q&A Q: Why

10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University k-Nearest Neighbors + Model Selection Matt Gormley Lecture 5 Jan. 29, 2020 1 Q&A Q: Why dont my entropy

1.12k views • 87 slides
Introduction to Machine Learning k -Nearest Neighbors Regression Learning goals Understand the

Introduction to Machine Learning k -Nearest Neighbors Regression Learning goals Understand the

Introduction to Machine Learning k -Nearest Neighbors Regression Learning goals Understand the basic idea of k-NN Know different distance measures for different scales of feature variables Understand that k-NN has no optimization step

569 views • 13 slides
Announcements CS 188: Artificial Intelligence On-going: contest (optional and FUN!) Spring

Announcements CS 188: Artificial Intelligence On-going: contest (optional and FUN!) Spring

Announcements CS 188: Artificial Intelligence On-going: contest (optional and FUN!) Spring 2010 Remaining lectures: Today: Machine Learning: Nearest Neighbors, Kernels Lecture 22: Nearest Neighbors, Kernels Wednesday: Machine

108 views • 6 slides
Overfitting + k-Nearest Neighbors Matt Gormley Lecture 4 Jan. 27, 2020 1 Course Staff 3

Overfitting + k-Nearest Neighbors Matt Gormley Lecture 4 Jan. 27, 2020 1 Course Staff 3

10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University Overfitting + k-Nearest Neighbors Matt Gormley Lecture 4 Jan. 27, 2020 1 Course Staff 3 Course Staff Team A 4

752 views • 49 slides
Decision Trees + k-Nearest Neighbors Matt Gormley Lecture 3 January 24, 2018 1 Q&A Q:

Decision Trees + k-Nearest Neighbors Matt Gormley Lecture 3 January 24, 2018 1 Q&A Q:

10-601 Introduction to Machine Learning Machine Learning Department School of Computer Science Carnegie Mellon University Decision Trees + k-Nearest Neighbors Matt Gormley Lecture 3 January 24, 2018 1 Q&A Q: Why dont my entropy

582 views • 29 slides

More recommend