Spatial Data: Dimensionality Reduction CS444 Techniques, Lecture 3 - PowerPoint PPT Presentation

Mar 29, 2024 •812 likes •1.02k views

Spatial Data: Dimensionality Reduction CS444 Techniques, Lecture 3 In this subfield, we think of a data point as a vector in R^n (what could possibly go wrong?) Linear dimensionality reduction: Reduction is achieved by is a single

Spatial Data: Dimensionality Reduction CS444 Techniques, Lecture 3
In this subfield, we think of a data point as a vector in R^n (what could possibly go wrong?)
“Linear” dimensionality reduction: Reduction is achieved by is a single matrix for every point.
Regular Scatterplots • Every data point is a vector:   v 0 v 1     v 2   v 3 • Every scatterplot is produced by a very simple matrix:  1 � 0 0 0 0 1 0 0  1 � 0 0 0 0 0 1 0
What about other matrices?
Grand Tour (Asimov, 1985) http://cscheid.github.io/lux/demos/tour/tour.html
Is there a best matrix? How do we think about that?
Linear Algebra review • Vectors • Inner Products • Lengths • Angles • Bases • Linear Transformations and Eigenvectors
Principal Component Analysis 0.2 0.1 Species Petal.Length setosa Petal.Width PC2 0.0 versicolor virginica Sepal.Length − 0.1 Sepal.Width − 0.2 − 0.10 − 0.05 0.00 0.05 0.10 0.15 PC1
Principal Component Analysis • Algorithm: • Given data set as matrix X in R^(d x n), ~ 1 1 T ) = XH ˜ • Center matrix: ~ X = X ( I − n X T ˜ • Compute eigendecomposition of ˜ X • X T ˜ ˜ X = U Σ U T • The principal components are the first few rows of U Σ 1 / 2
What if we don’t have coordinates, but distances? “Classical” Multidimensional Scaling
http://www.math.pku.edu.cn/teachers/yaoy/Fall2011/ lecture11.pdf
Borg and Groenen, Modern Multidimensional Scaling
Borg and Groenen, Modern Multidimensional Scaling
“Classical” Multidimensional Scaling • Algorithm: B = − 1 • Given , create D ij = | X i − X j | 2 2 HDH T • PCA of B is equal to the PCA of X • Huh?!
“Nonlinear” dimensionality reduction (ie: projection is not a matrix operation)
Data might have “high- order” structure
http://isomap.stanford.edu/Supplemental_Fig.pdf
We might want to minimize something else besides “di ff erence between squared distances” t-SNE: di ff erence between neighbor ordering Why not distances?
The curse of Dimensionality • High dimensional space looks nothing like low- dimensional space • Most distances become meaningless

Recommend

Spatial Data: Dimensionality Reduction CSC444 Techniques In this subfield, we think of a data

Spatial Data: Dimensionality Reduction CSC444 Techniques In this subfield, we think of a data point as a vector in R^n (what could possibly go wrong?) Linear dimensionality reduction: Reduction is achieved by multiplying a point by

354 views • 22 slides

STAT 209 Dimensionality Reduction November 26, 2019 Colin Reimer Dawson 1 / 24 Dimensionality

Dimensionality Reduction STAT 209 Dimensionality Reduction November 26, 2019 Colin Reimer Dawson 1 / 24 Dimensionality Reduction Outline Dimensionality Reduction 2 / 24 Dimensionality Reduction High Dimensional Data Modern datasets

470 views • 24 slides

Kernel-Based Dimensionality Reduction Methods on Synthesized and Facial Image Data Jonathan L.

High Dimensionality Dimensionality Reduction Methods Application to Simulated Data Application to Morph-II Wrap-up Kernel-Based Dimensionality Reduction Methods on Synthesized and Facial Image Data Jonathan L. Fabish Statistical Data Mining

340 views • 18 slides

Dimensionality reduction AI F UN DAMEN TALS Nemanja Radojkovic Senior Data Scientist

Dimensionality reduction AI F UN DAMEN TALS Nemanja Radojkovic Senior Data Scientist Denition "Dimensionality reduction is the process of reducing the number of variables under consideration by obtaining a set of principal

693 views • 33 slides

Dimensionality Reduction for Data Mining - Techniques, Applications and Trends Lei Yu Binghamton

Dimensionality Reduction for Data Mining - Techniques, Applications and Trends Lei Yu Binghamton University Jieping Ye, Huan Liu Arizona State University Outline Introduction to dimensionality reduction Feature selection (part I)

1.5k views • 135 slides

Preprocessing and Dimensionality Reduction J er emy Fix CentraleSup elec

Datasets Preprocessing Dimensionality reduction Preprocessing and Dimensionality Reduction J er emy Fix CentraleSup elec jeremy.fix@centralesupelec.fr 2017 1 / 73 Datasets Preprocessing Dimensionality reduction Where to get data

1.22k views • 73 slides

Lecture 7 Jan-Willem van de Meent DIMENSIONALITY REDUCTION Borrowing from : Percy Liang

Unsupervised Machine Learning and Data Mining DS 5230 / DS 4420 - Fall 2018 Lecture 7 Jan-Willem van de Meent DIMENSIONALITY REDUCTION Borrowing from : Percy Liang (Stanford) Dimensionality Reduction Goal: Map high dimensional

1.17k views • 70 slides

Visualization ( Nonlinear dimensionality reduction ) Fei Sha Yahoo! Research

Visualization ( Nonlinear dimensionality reduction ) Fei Sha Yahoo! Research feisha@yahoo-inc.com CS294 March 18, 2008 Dimensionality reduction Question: How can we detect low dimensional structure in high dimensional data?

713 views • 59 slides

Dimensionality Reduction Algorithms (and how to interpret their output) Dalya Baron (Tel Aviv

Dimensionality Reduction Algorithms (and how to interpret their output) Dalya Baron (Tel Aviv University) XXX Winter School, November 2018 What is Dimensionality Reduction? Dimensionality Reduction algorithm 28 x 28 features per object 2

437 views • 33 slides

Dimensionality Reduction for Visualization Lecture 13 April 8, 2020 Outline High-dimensional

CS53000-Spring 2020 Introduction to Scientific Visualization Dimensionality Reduction for Visualization Lecture 13 April 8, 2020 Outline High-dimensional data Dimensionality reduction Manifold learning Topological data analysis 2 CS530 /

676 views • 30 slides

Dimensionality Reduction Alexandros Tantos Assistant Professor Aristotle University of

DataCamp Dimensionality Reduction in R DIMENSIONALITY REDUCTION IN R Dimensionality Reduction Alexandros Tantos Assistant Professor Aristotle University of Thessaloniki DataCamp Dimensionality Reduction in R Curse of Dimensionality

550 views • 35 slides

6.1 Dimensionality reduction Previously in the course, we have discussed algorithms suited for a

CS/CNS/EE 253: Advanced Topics in Machine Learning Topic: Dimensionality Reduction Lecturer: Andreas Krause Scribe: Matt Faulkner Date: 25 January 2010 6.1 Dimensionality reduction Previously in the course, we have discussed algorithms suited for

442 views • 6 slides

DIMENSIONALITY REDUCTION DIMENSIONALITY REDUCTION MATTHIEU BLOCH April 21, 2020 1 / 26

DIMENSIONALITY REDUCTION DIMENSIONALITY REDUCTION MATTHIEU BLOCH April 21, 2020 1 / 26 MULTIDIMENSIONAL SCALING MULTIDIMENSIONAL SCALING There are situations for which Euclidean distance is not appropriate Suppose we have access to a

500 views • 26 slides

The Beginner's Guide to Dimensionality Reduction Explore the methods that data scientists use to

The Beginner's Guide to Dimensionality Reduction Explore the methods that data scientists use to visualize high-dimensional data. By: Matthew Conlen and Fred Hohman Workshop on Visualization for AI Explainability October 22, 2018 Who it Reached

400 views • 13 slides

Random Projections & Applications To Dimensionality Reduction Aditya Krishna Menon (BSc.

Random Projections & Applications To Dimensionality Reduction Aditya Krishna Menon (BSc. Advanced) Supervisors: Dr. Sanjay Chawla Dr. Anastasios Viglas High-dimensionality Lots of data objects/items with some attributes i.e.

429 views • 17 slides

Applied Machine Learning Dimensionality reduction using PCA Siamak Ravanbakhsh COMP 551 (Fall

Applied Machine Learning Dimensionality reduction using PCA Siamak Ravanbakhsh COMP 551 (Fall 2020) Learning objectives What is dimensionality reduction? What is it good for? Linear dimensionality reduction: Principal Component Analysis

661 views • 26 slides

Exploring Multivariate Data with Clustering and Dimensionality Reduction Marco Baroni Practical

Exploring Multivariate Data with Clustering and Dimensionality Reduction Marco Baroni Practical Statistics in R Outline Introduction Clustering Clustering in R Dimensionality reduction Dimensionality reduction in R Outline Introduction

773 views • 65 slides

Dimensionality Reduction; PCA & SVD Kalev Kask Motivation High-dimensional data

+ Machine Learning and Data Mining Dimensionality Reduction; PCA & SVD Kalev Kask Motivation High-dimensional data Images of faces Text from articles All S&P 500 stocks Can we describe them in a simpler way?

707 views • 32 slides

Introduction to Big Data and Machine Learning Dimensionality Reduction Continuous Latent

Introduction to Big Data and Machine Learning Dimensionality Reduction Continuous Latent Variables Dr. Mihail October 8, 2019 (Dr. Mihail) Intro Big Data October 8, 2019 1 / 20 Data Dimensionality Idea Many datasets have the property that

444 views • 21 slides

Dimensionality Reduction INFO-4604, Applied Machine Learning University of Colorado Boulder

Dimensionality Reduction INFO-4604, Applied Machine Learning University of Colorado Boulder October 25, 2018 Prof. Michael Paul Dimensionality The dimensionality of data is the number of variables Usually this refers to the number of input

438 views • 42 slides

Nonlinear Dimensionality Reduction Donovan Parks Overview Direct visualization vs.

Nonlinear Dimensionality Reduction Donovan Parks Overview Direct visualization vs. dimensionality reduction Nonlinear dimensionality reduction techniques: ISOMAP, LLE, Charting A fun example that uses non- metric, replicated

882 views • 38 slides

topology and data topological data analysis and manifold learning what is dimensionality

by Joshua Tan, for Ufora & NYU Capstone, 12/16/2014 a library for topology and data topological data analysis and manifold learning what is dimensionality reduction Given some input space X and a sample set S , dimensionality

955 views • 16 slides

Dimensionality reduction Outline From distances to points : MultiDimensional Scaling (MDS)

Dimensionality reduction Outline From distances to points : MultiDimensional Scaling (MDS) Dimensionality Reductions or data projections Random projections Singular Value Decomposition and Principal Component Analysis (PCA)

394 views • 36 slides

An Algorithm for Sample and Data Dimensionality Reduction Using Fast Simulated Annealing Szymon

7th International Conference on Advanced Data Mining and Applications An Algorithm for Sample and Data Dimensionality Reduction Using Fast Simulated Annealing Szymon ukasik , Piotr Kulczycki Department of Automatic Control and IT, Cracow

589 views • 16 slides