manifold alignment of high dimensional datasets
play

Manifold Alignment of High- Dimensional Datasets Sridhar Mahadevan - PowerPoint PPT Presentation

Nov 15, 2022 •142 likes •500 views

Manifold Alignment of High- Dimensional Datasets Sridhar Mahadevan (PI) & Rui Wang (co-PI) Thomas Boucher, Clifton Carey, Stefan Dernbach, Blake Foster, Hoa Vu, Chang Wang (IBM) School of Computer Science University of Massachusetts,

  1. Manifold Alignment of High- Dimensional Datasets Sridhar Mahadevan (PI) & Rui Wang (co-PI) Thomas Boucher, Clifton Carey, Stefan Dernbach, Blake Foster, Hoa Vu, Chang Wang (IBM) School of Computer Science University of Massachusetts, Amherst Thursday, December 13, 12

  2. Learning from Multiple Datasets • In many applications, multiple “views” or multiple datasets are constructed • Bioinformatics • Activity recognition • Computer graphics • Scientific exploration (MARS rover) • Cross-lingual information retrieval • Spectral methods for learning latent variable models Thursday, December 13, 12

  3. Canonical Correlation Analysis (Hotelling, 1936) Acceleration MPG Displacement Horsepower Weight Thursday, December 13, 12

  4. Canonical Correlation Analysis (Hotelling, 1936) Acceleration MPG Displacement Horsepower Weight Thursday, December 13, 12

  5. Canonical Correlation Analysis (Hotelling, 1936) Acceleration MPG Displacement Horsepower Weight Thursday, December 13, 12

  6. Canonical Correlation Analysis (Hotelling, 1936) Acceleration MPG Displacement Horsepower Weight Find u,v that maximizes Thursday, December 13, 12

  7. Canonical Correlation Analysis (Hotelling, 1936) Acceleration MPG Displacement Horsepower Weight Find u,v that maximizes Thursday, December 13, 12

  8. Canonical Correlation Analysis (Hotelling, 1936) Acceleration MPG Pioneer of the first two statistics departments in the US! UNC, Chapel Hill Displacement Horsepower Weight Columbia University Find u,v that maximizes Thursday, December 13, 12

  9. FODAVA project: main contribution • We developed a new class of methods, called manifold alignment, that outperforms CCA in many domains • Linear + Nonlinear • Local + Global • Supervised + Unsupervised • If you use multiple datasets, you should try manifold alignment! Thursday, December 13, 12

  10. Manifold Projections Thursday, December 13, 12

  11. Manifold Projections We want to find mapping functions α , β to minimize the cost function C ( α , β ) , where ∑∑ ∑ ∑ T T 2 i , j T T 2 i , j T T 2 i , j C ( α , β ) = µ ( α x − β y ) W + 0 . 5 ( α x − α x ) W + 0 . 5 ( β y − β y ) W i j i j x i j y i j i , j i , j Thursday, December 13, 12

  12. Manifold Projections We want to find mapping functions α , β to minimize the cost function C ( α , β ) , where ∑∑ ∑ ∑ T T 2 i , j T T 2 i , j T T 2 i , j C ( α , β ) = µ ( α x − β y ) W + 0 . 5 ( α x − α x ) W + 0 . 5 ( β y − β y ) W i j i j x i j y i j i , j i , j Thursday, December 13, 12

  13. Manifold Projections We want to find mapping functions α , β to minimize the cost function C ( α , β ) , where ∑∑ ∑ ∑ T T 2 i , j T T 2 i , j T T 2 i , j C ( α , β ) = µ ( α x − β y ) W + 0 . 5 ( α x − α x ) W + 0 . 5 ( β y − β y ) W i j i j x i j y i j i , j i , j Thursday, December 13, 12

  14. Manifold Projections We want to find mapping functions α , β to minimize the cost function C ( α , β ) , where ∑∑ ∑ ∑ T T 2 i , j T T 2 i , j T T 2 i , j C ( α , β ) = µ ( α x − β y ) W + 0 . 5 ( α x − α x ) W + 0 . 5 ( β y − β y ) W i j i j x i j y i j i , j i , j Preserve correspondences Thursday, December 13, 12

  15. Manifold Projections We want to find mapping functions α , β to minimize the cost function C ( α , β ) , where ∑∑ ∑ ∑ T T 2 i , j T T 2 i , j T T 2 i , j C ( α , β ) = µ ( α x − β y ) W + 0 . 5 ( α x − α x ) W + 0 . 5 ( β y − β y ) W i j i j x i j y i j i , j i , j Preserve correspondences Thursday, December 13, 12

  16. Manifold Projections We want to find mapping functions α , β to minimize the cost function C ( α , β ) , where ∑∑ ∑ ∑ T T 2 i , j T T 2 i , j T T 2 i , j C ( α , β ) = µ ( α x − β y ) W + 0 . 5 ( α x − α x ) W + 0 . 5 ( β y − β y ) W i j i j x i j y i j i , j i , j Preserve correspondences Preserve local geometry Thursday, December 13, 12

  17. Manifold Projections We want to find mapping functions α , β to minimize the cost function C ( α , β ) , where ∑∑ ∑ ∑ T T 2 i , j T T 2 i , j T T 2 i , j C ( α , β ) = µ ( α x − β y ) W + 0 . 5 ( α x − α x ) W + 0 . 5 ( β y − β y ) W i j i j x i j y i j i , j i , j Preserve correspondences Preserve local geometry (2) Theorem 1 : α , β to minimize C ( α , β ) are given by the eigenvecto rs correspond ing to the smallest eigenvalue s of T T ZLZ γ = λ ZDZ γ . Thursday, December 13, 12

  18. Thursday, December 13, 12

  19. A Summary of Manifold Alignment Approaches Given Given Unsupervised correspondences labels alignment Preserve Local geometry Preserve Global geometry One-step alignment Two-step alignment Feature-level Instance-level Procrustes alignment Manifold Projections (MP) Extensions of MP Thursday, December 13, 12

  20. Manifold Warping (Hoa, Carey, Mahadevan: AAAI, 2012) Dynamic Time Warping + Iterate: • Find projection to lower-dimensional space • Find new set of correspondences Manifold Alignment Thursday, December 13, 12

  21. Activity Recognition CCA+DTW (Zhou, NIPS 2009) The resulted alignment path of manifold warping is much closer to the ground truth alignment Vu, Carey, and Mahadevan, AAAI 2012 Thursday, December 13, 12 • • • • • ’ •

  22. Social Network Alignment Sparse Manifold Alignment Use Lasso to find a sparse solution. DBLP Social Network • • • Wang, Liu, Vu, and Mahadevan, 2012 Thursday, December 13, 12 • • ’ •

  23. Cross-Lingual Transfer in IR Thursday, December 13, 12

  24. Cross-Lingual Transfer in IR Madam President, on a point of order. You will be aware from the press and television that there have been a number of bomb English explosions and killings in Sri Lanka. documents Signora Presidente, intervengo per una mozione d'ordine.Come avrà letto sui giornali o sentito alla Italian televisione, in Sri Lanka si sono verificati numerosi documents assassinii ed esplosioni di ordigni. Frau Präsidentin, zur Geschäftsordnung. German Wie Sie sicher aus der Presse und dem Fernsehen documents wissen, gab es in Sri Lanka mehrere Bombenexplosionen mit zahlreichen Toten. Thursday, December 13, 12

  25. Cross-Lingual Transfer in IR Madam President, on a point of order. You will be aware from the press and television that there have been a number of bomb English explosions and killings in Sri Lanka. documents Signora Presidente, intervengo per una mozione d'ordine.Come avrà letto sui giornali o sentito alla Italian televisione, in Sri Lanka si sono verificati numerosi documents assassinii ed esplosioni di ordigni. Frau Präsidentin, zur Geschäftsordnung. German Wie Sie sicher aus der Presse und dem Fernsehen documents wissen, gab es in Sri Lanka mehrere Bombenexplosionen mit zahlreichen Toten. Proceedings of the EU Thursday, December 13, 12

  26. Cross-lingual IR Thursday, December 13, 12

  27. Cross-lingual IR Thursday, December 13, 12

  28. Impact of Work Thursday, December 13, 12

  29. Impact of Work • The most useful research I have done in 20 years! Thursday, December 13, 12

  30. Impact of Work • The most useful research I have done in 20 years! • Led to several new collaborations Thursday, December 13, 12

  31. Impact of Work • The most useful research I have done in 20 years! • Led to several new collaborations • Mars rover Curiosity (Darby Dyar, Mount Holyoke, NASA/ JPL scientific team) Thursday, December 13, 12

  32. Impact of Work • The most useful research I have done in 20 years! • Led to several new collaborations • Mars rover Curiosity (Darby Dyar, Mount Holyoke, NASA/ JPL scientific team) • Proposals submitted to CDS&E and BIGDATA Thursday, December 13, 12

  33. Impact of Work • The most useful research I have done in 20 years! • Led to several new collaborations • Mars rover Curiosity (Darby Dyar, Mount Holyoke, NASA/ JPL scientific team) • Proposals submitted to CDS&E and BIGDATA • Papers: 100+ citations on Google Scholar Thursday, December 13, 12

  34. Impact of Work • The most useful research I have done in 20 years! • Led to several new collaborations • Mars rover Curiosity (Darby Dyar, Mount Holyoke, NASA/ JPL scientific team) • Proposals submitted to CDS&E and BIGDATA • Papers: 100+ citations on Google Scholar • Many many applications (bioinformatics, graphics, robotics, science, IR) Thursday, December 13, 12

Recommend

HIGH DIMENSIONAL MANIFOLD TOPOLOGY THEN AND NOW Andrew Ranicki (Edinburgh)

HIGH DIMENSIONAL MANIFOLD TOPOLOGY THEN AND NOW Andrew Ranicki (Edinburgh)

1 HIGH DIMENSIONAL MANIFOLD TOPOLOGY THEN AND NOW Andrew Ranicki (Edinburgh) http://www.maths.ed.ac.uk/ aar Orsay 7,8,9 December 2005 An n -dimensional topological manifold M is a paracompact Hausdorff topological space which is locally

1.17k views • 71 slides
Three-dimensional Radial Visualization of High-dimensional Continuous or Discrete Datasets Fan

Three-dimensional Radial Visualization of High-dimensional Continuous or Discrete Datasets Fan

Three-dimensional Radial Visualization of High-dimensional Continuous or Discrete Datasets Fan Dai, Yifan Zhu and Ranjan Maitra Department of Statistics Iowa State University {fd43,yifanzhu,maitra} @ iastate.edu Motivation Multivariate

640 views • 34 slides
Manifold Regularization Lorenzo Rosasco MIT, 9.520 L. Rosasco Manifold Regularization About

Manifold Regularization Lorenzo Rosasco MIT, 9.520 L. Rosasco Manifold Regularization About

Manifold Regularization Lorenzo Rosasco MIT, 9.520 L. Rosasco Manifold Regularization About this class Goal To analyze the limits of learning from examples in high dimensional spaces. To introduce the semi-supervised setting and the use of

437 views • 31 slides
n -dimensional manifold M with T := TM n -dimensional manifold M with T := TM T n -dimensional

n -dimensional manifold M with T := TM n -dimensional manifold M with T := TM T n -dimensional

n -dimensional manifold M with T := TM n -dimensional manifold M with T := TM T n -dimensional manifold M with T := TM T frame bundle GL ( R n , T ) is a principal GL ( n )-bundle n -dimensional manifold M with T := TM T + T frame bundle

1.34k views • 79 slides
A Tracy-Widom Empirical Estimator For Valid P-values With High-Dimensional Datasets Maxime

A Tracy-Widom Empirical Estimator For Valid P-values With High-Dimensional Datasets Maxime

A Tracy-Widom Empirical Estimator For Valid P-values With High-Dimensional Datasets Maxime Turgeon 10 August 2019 University of Manitoba Departments of Statistics and Computer Science 1/21 Motivating Example Systemic Autoimmune Diseases

455 views • 27 slides
Manifold Regularization Lorenzo Rosasco 9.520 Class 10 March 6, 2011 L. Rosasco Manifold

Manifold Regularization Lorenzo Rosasco 9.520 Class 10 March 6, 2011 L. Rosasco Manifold

Manifold Regularization Lorenzo Rosasco 9.520 Class 10 March 6, 2011 L. Rosasco Manifold Regularization About this class Goal To analyze the limits of learning from examples in high dimensional spaces. To introduce the semi-supervised

1.16k views • 31 slides
SpRay an R-based visual-analytics platform for large and high-dimensional datasets J. Heinrich 1

SpRay an R-based visual-analytics platform for large and high-dimensional datasets J. Heinrich 1

Introduction SpRay Discussion Future Work SpRay an R-based visual-analytics platform for large and high-dimensional datasets J. Heinrich 1 J. Dietzsch 1 D. Bartz 2 K. Nieselt 1 1 Center for Bioinformatics, University of Tbingen 2 ICCAS/VCM,

220 views • 20 slides
Nonlinear Manifold Learning Part One: Background, LLE, IsoMap 6.454 Area One Seminar October 8

Nonlinear Manifold Learning Part One: Background, LLE, IsoMap 6.454 Area One Seminar October 8

Nonlinear Manifold Learning Part One: Background, LLE, IsoMap 6.454 Area One Seminar October 8 th 2003 Alexander Ihler Introduction Motivation Observe high-dimensional data Hopefully, a low-dimensional (simple) underlying process

426 views • 28 slides
Advanced Introduction to Machine Learning, CMU-10715 Manifold Learning Barnabs Pczos

Advanced Introduction to Machine Learning, CMU-10715 Manifold Learning Barnabs Pczos

Advanced Introduction to Machine Learning, CMU-10715 Manifold Learning Barnabs Pczos Motivation Find meaningful low-dimensional structures hidden in high-dimensional observations. The human brain confronts the same problem in

1.06k views • 85 slides
1 Statement of the problem Let ( M; g ) be a n -dimensional smooth compact Riemannian manifold

1 Statement of the problem Let ( M; g ) be a n -dimensional smooth compact Riemannian manifold

How to nd a copy of the Hamiltonian from the scattering map Leonid Pestov Immanuel Kant Baltic Federal University 1 Statement of the problem Let ( M; g ) be a n -dimensional smooth compact Riemannian manifold with boundary @M and g t ( x;

191 views • 5 slides
Advances and Applications of Intrinsic Low Dimensional Manifold Theory by Joseph M. Powers,

Advances and Applications of Intrinsic Low Dimensional Manifold Theory by Joseph M. Powers,

Advances and Applications of Intrinsic Low Dimensional Manifold Theory by Joseph M. Powers, Samuel Paolucci, and Sandeep Singh Department of Aerospace and Mechanical Engineering University of Notre Dame presented to the Work in Progress

309 views • 28 slides
Search in High-Dimensional spaces and Dimensionality Reduction i i li d i D. Gunopulos 1

Search in High-Dimensional spaces and Dimensionality Reduction i i li d i D. Gunopulos 1

Search in High-Dimensional spaces and Dimensionality Reduction i i li d i D. Gunopulos 1 Retrieval techniques for high- dimensional datasets The retrieval problem: The retrieval problem: Given a set of objects S , and a query

543 views • 25 slides
Hard Problems in 3-Manifold Topology School on Low-Dimensional Geometry and Topology: Discrete

Hard Problems in 3-Manifold Topology School on Low-Dimensional Geometry and Topology: Discrete

Hard Problems in 3-Manifold Topology School on Low-Dimensional Geometry and Topology: Discrete and Algorithmic Aspects Arnaud de Mesmay 1 Yoav Rieck 2 Eric Sedgwick 3 Martin Tancer 4 1 CNRS, GIPSA-Lab 2 University of Arkansas 3 DePaul

1.37k views • 83 slides
What can we say about high- dimensional objects from a low-dimensional representation? 2

What can we say about high- dimensional objects from a low-dimensional representation? 2

Dimension Reduction with Certainty Rasmus Pagh IT University of Copenhagen S CALABLE S IMILARITY ECML-PKDD S EARCH September 21, 2016 Slides: goo.gl/hZoRWo 1 What can we say about high- dimensional objects from a low-dimensional

940 views • 75 slides
http://cs246.stanford.edu High dimensional == many features Find

http://cs246.stanford.edu High dimensional == many features Find

CS246: Mining Massive Datasets Jure Leskovec, Stanford University http://cs246.stanford.edu High dimensional == many features Find concepts/topics/genres: Documents: Features: Thousands of words, millions of word pairs

606 views • 57 slides
Nonlinear Methods Data often lies on or near a nonlinear low-dimensional curve aka manifold. 27

Nonlinear Methods Data often lies on or near a nonlinear low-dimensional curve aka manifold. 27

Nonlinear Methods Data often lies on or near a nonlinear low-dimensional curve aka manifold. 27 Laplacian Eigenmaps Linear methods Lower-dimensional linear projection that preserves distances between all points Laplacian Eigenmaps (key idea)

533 views • 14 slides
Manifold-Adaptive Dimension Estimation Amir massoud Farahmand (1) , Csaba Szepesvri (1) ,

Manifold-Adaptive Dimension Estimation Amir massoud Farahmand (1) , Csaba Szepesvri (1) ,

Manifold-Adaptive Dimension Estimation Amir massoud Farahmand (1) , Csaba Szepesvri (1) , Jean-Yves Audibert (2) (1) Department of Computing Science, University of Alberta, Canada (2) CERTIS, Ecole Nationale des Ponts, France High-Dimensional

494 views • 35 slides
Feature Selection and Classification Pairwise Combinations for High-dimensional Tumour Biomedical

Feature Selection and Classification Pairwise Combinations for High-dimensional Tumour Biomedical

Feature Selection and Classification Pairwise Combinations for High-dimensional Tumour Biomedical Datasets A. Dziomdziora A. Wosiak Lodz University of Technology Institute of Information Technology Theoretical Foundations of Machine Learning,

625 views • 33 slides
Dimensionality Reduction for Visualization Lecture 13 April 8, 2020 Outline High-dimensional

Dimensionality Reduction for Visualization Lecture 13 April 8, 2020 Outline High-dimensional

CS53000-Spring 2020 Introduction to Scientific Visualization Dimensionality Reduction for Visualization Lecture 13 April 8, 2020 Outline High-dimensional data Dimensionality reduction Manifold learning Topological data analysis 2 CS530 /

676 views • 30 slides
High Dimensional Data, Covariance Matrices High Dimensional Data Examples and Application to

High Dimensional Data, Covariance Matrices High Dimensional Data Examples and Application to

Outline Motivation High Dimensional Data, Covariance Matrices High Dimensional Data Examples and Application to Genetics Theoretical Underpinnings Random Matrices Shrinkage Estimation Decision Theory Samprit Banerjee, Ph.D Bayesian

1.34k views • 80 slides
Analysis of hierarchical metric-tree indexing schemes for similarity search in high-dimensional

Analysis of hierarchical metric-tree indexing schemes for similarity search in high-dimensional

Analysis of hierarchical metric-tree indexing schemes for similarity search in high-dimensional datasets Vladimir Pestov vpest283@uottawa.ca http://aix1.uottawa.ca/ vpest283 Department of Mathematics and Statistics University of Ottawa

1.22k views • 106 slides
Change detection in multi-dimensional datasets and time series Andrea De Simone

Change detection in multi-dimensional datasets and time series Andrea De Simone

Change detection in multi-dimensional datasets and time series Andrea De Simone andrea.desimone@sissa.it Univ. Camerino, 2019-02-26 [DS, Jacques arXiv:1807.06038] Outline 1 Two-Sample Test: Intro & Motivation 2 Nearest

659 views • 26 slides
Visualizing Multi-dimensional Data S E T H H O R R I G A N C O M P U T E R V I S U A L I Z A T

Visualizing Multi-dimensional Data S E T H H O R R I G A N C O M P U T E R V I S U A L I Z A T

Visualizing Multi-dimensional Data S E T H H O R R I G A N C O M P U T E R V I S U A L I Z A T I O N F A L L 2 0 0 8 Motivation Multi-dimensional datasets are common Digital cameras Wall-street stocks Motor vehicles

571 views • 19 slides
Statistics for High-Dimensional Data: Selected Topics Peter B uhlmann Seminar f ur

Statistics for High-Dimensional Data: Selected Topics Peter B uhlmann Seminar f ur

Statistics for High-Dimensional Data: Selected Topics Peter B uhlmann Seminar f ur Statistik, ETH Z urich June 2014 High-dimensional linear model: basic concepts in methodology, theory and computation High-dimensional data

998 views • 48 slides

More recommend