abstraction by structure
play

Abstraction by Structure Carl Henrik Ek, Danica Kragic { chek, danik - PowerPoint PPT Presentation

Introduction Structural Representations Structural Models Conclusion References Abstraction by Structure Carl Henrik Ek, Danica Kragic { chek, danik } @csc.kth.se Royal Institute of Technology April 18, 2012 Ek, Kragic KTH Abstraction by


  1. Introduction Structural Representations Structural Models Conclusion References Action Representation 94 . 8 1 . 6 0 . 4 3 . 2 75 . 2 4 . 0 18 . 8 2 . 0 47 . 6 7 . 2 11 . 6 33 . 6 57 . 2 3 . 6 0 . 8 38 . 4 99 . 9% 98 . 0% 94 . 6% 94 . 4% 97 . 5% 0 . 4 99 . 2 0 . 4 0 6 . 0 81 . 2 12 . 8 0 34 . 8 34 . 0 10 . 0 21 . 2 53 . 6 10 . 8 6 . 4 29 . 2 81 . 8% 2 . 0 1 . 2 96 . 8 0 10 . 0 6 . 8 81 . 2 2 . 0 40 . 8 6 . 4 30 . 0 22 . 8 61 . 6 0 . 8 8 . 0 29 . 6 0 . 8 0 0 99 . 3 1 . 6 0 . 4 8 . 4 89 . 6 10 . 0 1 . 2 1 . 2 87 . 6 22 . 8 0 . 8 0 . 8 75 . 6 49 . 8% 37 . 9% 99 . 7 0 0 . 3 0 99 . 3 0 0 . 6 0 . 1 91 . 7 2 . 8 4 . 4 1 . 1 90 . 2 2 . 2 7 . 1 0 . 4 0 100 0 0 0 99 . 7 0 . 3 0 1 . 3 96 . 7 1 . 6 0 . 4 0 . 4 96 . 4 2 . 9 0 . 3 0 0 . 1 99 . 9 0 1 . 6 1 . 4 97 0 4 . 2 2 . 4 93 . 3 0 . 1 6 . 6 1 . 2 92 . 1 0 . 3 0 0 0 100 1 . 3 2 . 7 0 . 1 95 . 9 1 . 7 1 . 7 0 . 1 96 . 5 0 . 5 0 . 7 0 98 . 8 Ek, Kragic KTH Abstraction by Structure

  2. Introduction Structural Representations Structural Models Conclusion References Objects 4 Feature Representation • Local representation • Encode order • Distribution of order 4 In submission: Marianna Madry, Renaud Detry, Kaiyu Hang Ek, Kragic KTH Abstraction by Structure

  3. Introduction Structural Representations Structural Models Conclusion References Objects 4 Feature Representation • Local representation • Encode order • Distribution of order 4 In submission: Marianna Madry, Renaud Detry, Kaiyu Hang Ek, Kragic KTH Abstraction by Structure

  4. Introduction Structural Representations Structural Models Conclusion References Objects 4 Feature Representation • Local representation • Encode order • Distribution of order 4 In submission: Marianna Madry, Renaud Detry, Kaiyu Hang Ek, Kragic KTH Abstraction by Structure

  5. Introduction Structural Representations Structural Models Conclusion References Objects 4 4 In submission: Marianna Madry, Renaud Detry, Kaiyu Hang Ek, Kragic KTH Abstraction by Structure

  6. Introduction Structural Representations Structural Models Conclusion References Objects 4 distance histogram between points of type 1 and 2 (marked as 1-2) etc. Images are best viewed in color. 4 In submission: Marianna Madry, Renaud Detry, Kaiyu Hang Ek, Kragic KTH Abstraction by Structure

  7. Introduction Structural Representations Structural Models Conclusion References Objects 4 Test Setting • Synthetic • Real • Real & different pose,scale • Synthetic & full,partial 4 In submission: Marianna Madry, Renaud Detry, Kaiyu Hang Ek, Kragic KTH Abstraction by Structure

  8. Introduction Structural Representations Structural Models Conclusion References Objects 4 Test Setting • Synthetic • Real • Real & different pose,scale • Synthetic & full,partial 4 In submission: Marianna Madry, Renaud Detry, Kaiyu Hang Ek, Kragic KTH Abstraction by Structure

  9. Introduction Structural Representations Structural Models Conclusion References Objects 4 Test Setting • Synthetic • Real • Real & different pose,scale • Synthetic & full,partial 4 In submission: Marianna Madry, Renaud Detry, Kaiyu Hang Ek, Kragic KTH Abstraction by Structure

  10. Introduction Structural Representations Structural Models Conclusion References Objects 4 Test Setting • Synthetic • Real • Real & different pose,scale • Synthetic & full,partial 4 In submission: Marianna Madry, Renaud Detry, Kaiyu Hang Ek, Kragic KTH Abstraction by Structure

  11. Introduction Structural Representations Structural Models Conclusion References Grasping 5 • Pre-segmentation of objects 0.3 0.2 • Exploit structure in joint object 0.1 0 and grasp space − 0.1 − 0.2 • Part based generalisation 0.15 0.16 0.17 0.18 0.19 0.2 0.21 5 Detry et al. [2012] Ek, Kragic KTH Abstraction by Structure

  12. Introduction Structural Representations Structural Models Conclusion References Grasping 5 5 Detry et al. [2012] Ek, Kragic KTH Abstraction by Structure

  13. Introduction Structural Representations Structural Models Conclusion References Introduction Structural Representations Structural Models Conclusion Ek, Kragic KTH Abstraction by Structure

  14. Introduction Structural Representations Structural Models Conclusion References Re-representations • Preference: low-dimensional • Linearity • Observed data Y ∈ ℜ N · D • Underlying intrinsic representation X ∈ ℜ N · q • Generative mapping: y i = f ( x i ) Ek, Kragic KTH Abstraction by Structure

  15. Introduction Structural Representations Structural Models Conclusion References Gaussian Processes: Prior • Distribution over infinite objects: functions . Ek, Kragic KTH Abstraction by Structure

  16. Introduction Structural Representations Structural Models Conclusion References Gaussian Processes: Prior Linear Kernel 1.5 1 0.5 0 ! 0.5 ! 1 ! 1.5 ! 0.5 ! 0.4 ! 0.3 ! 0.2 ! 0.1 0 0.1 0.2 0.3 0.4 0.5 • Distribution over infinite objects: functions . Ek, Kragic KTH Abstraction by Structure

  17. Introduction Structural Representations Structural Models Conclusion References Gaussian Processes: Prior RBF Kernel width=1 2.5 2 1.5 1 0.5 0 ! 0.5 ! 1 ! 1.5 ! 2 ! 0.5 ! 0.4 ! 0.3 ! 0.2 ! 0.1 0 0.1 0.2 0.3 0.4 0.5 • Distribution over infinite objects: functions . Ek, Kragic KTH Abstraction by Structure

  18. Introduction Structural Representations Structural Models Conclusion References Gaussian Processes: Prior RBF Kernel width=1e ! 1 2 1.5 1 0.5 0 ! 0.5 ! 1 ! 1.5 ! 2 ! 2.5 ! 3 ! 0.5 ! 0.4 ! 0.3 ! 0.2 ! 0.1 0 0.1 0.2 0.3 0.4 0.5 • Distribution over infinite objects: functions . Ek, Kragic KTH Abstraction by Structure

  19. Introduction Structural Representations Structural Models Conclusion References Gaussian Processes: Prior ()* ,e./e0 1i3t561e ! 2 3 2 1 0 ! 1 ! 2 ! 3 ! 0"# ! 0"4 ! 0"3 ! 0"2 ! 0"1 0 0"1 0"2 0"3 0"4 0"# • Distribution over infinite objects: functions . Ek, Kragic KTH Abstraction by Structure

  20. Introduction Structural Representations Structural Models Conclusion References Gaussian Processes: Prior MLP Kernel 2 1.5 1 0.5 0 ! 0.5 ! 1 ! 1.5 ! 2 ! 2.5 ! 0.5 ! 0.4 ! 0.3 ! 0.2 ! 0.1 0 0.1 0.2 0.3 0.4 0.5 • Distribution over infinite objects: functions . Ek, Kragic KTH Abstraction by Structure

  21. Introduction Structural Representations Structural Models Conclusion References Gaussian Processes: Prior RBF, Linear, Noise 4 3 2 1 0 ! 1 ! 2 ! 3 ! 4 ! 5 ! 6 ! 0.5 ! 0.4 ! 0.3 ! 0.2 ! 0.1 0 0.1 0.2 0.3 0.4 0.5 • Distribution over infinite objects: functions . Ek, Kragic KTH Abstraction by Structure

  22. Introduction Structural Representations Structural Models Conclusion References Gaussian Processes: Posterior Combine prior with observed data y ∗ | X ∗ , X , y ∼N ( K ( X ∗ , X ) K ( X , X ) − 1 y , , K ( X ∗ , X ∗ ) − K ( X ∗ , X ) K ( X , X ) − 1 K ( X , X ∗ )) Ek, Kragic KTH Abstraction by Structure

  23. Introduction Structural Representations Structural Models Conclusion References Gaussian Processes: Posterior Combine prior with observed data y ∗ | X ∗ , X , y ∼N ( K ( X ∗ , X ) K ( X , X ) − 1 y , , K ( X ∗ , X ∗ ) − K ( X ∗ , X ) K ( X , X ) − 1 K ( X , X ∗ )) Ek, Kragic KTH Abstraction by Structure

  24. Introduction Structural Representations Structural Models Conclusion References Gaussian Processes: Posterior Combine prior with observed data y ∗ | X ∗ , X , y ∼N ( K ( X ∗ , X ) K ( X , X ) − 1 y , , K ( X ∗ , X ∗ ) − K ( X ∗ , X ) K ( X , X ) − 1 K ( X , X ∗ )) Ek, Kragic KTH Abstraction by Structure

  25. Introduction Structural Representations Structural Models Conclusion References Gaussian Processes: Posterior Combine prior with observed data y ∗ | X ∗ , X , y ∼N ( K ( X ∗ , X ) K ( X , X ) − 1 y , , K ( X ∗ , X ∗ ) − K ( X ∗ , X ) K ( X , X ) − 1 K ( X , X ∗ )) Ek, Kragic KTH Abstraction by Structure

  26. Introduction Structural Representations Structural Models Conclusion References Gaussian Processes: Marginal Likelihood 6 � � � � − 1 − 1 − N y T ( K + β − 1 I ) − 1 y det ( K + β − 1 I ) 2 log 2 π 2 tr 2 log � �� � � �� � data − fit complexity 6 Images: Neil Lawrence Ek, Kragic KTH Abstraction by Structure

  27. Introduction Structural Representations Structural Models Conclusion References Gaussian Processes: Marginal Likelihood 6 � 4 1 � 5 0.5 � 6 log � likelihood � 7 � 1.5 � 1 � 0.5 0.5 1 1.5 � 8 � 0.5 � 9 � 1 � 10 � 11 � 1.5 � 12 � 1 0 1 10 10 10 � 2 length scale − 1 � � − 1 � � − N y T ( K + β − 1 I ) − 1 y det ( K + β − 1 I ) 2 log 2 π 2 tr 2 log � �� � � �� � data − fit complexity 6 Images: Neil Lawrence Ek, Kragic KTH Abstraction by Structure

  28. Introduction Structural Representations Structural Models Conclusion References Gaussian Processes: Marginal Likelihood 6 � 4 1 � 5 0.5 � 6 log � likelihood � 7 � 1.5 � 1 � 0.5 0.5 1 1.5 � 8 � 0.5 � 9 � 1 � 10 � 11 � 1.5 � 12 � 1 0 1 10 10 10 � 2 length scale − 1 � � − 1 � � − N y T ( K + β − 1 I ) − 1 y det ( K + β − 1 I ) 2 log 2 π 2 tr 2 log � �� � � �� � data − fit complexity 6 Images: Neil Lawrence Ek, Kragic KTH Abstraction by Structure

  29. Introduction Structural Representations Structural Models Conclusion References Gaussian Processes: Marginal Likelihood 6 � 4 1 � 5 0.5 � 6 log � likelihood � 7 � 1.5 � 1 � 0.5 0.5 1 1.5 � 8 � 0.5 � 9 � 1 � 10 � 11 � 1.5 � 12 � 1 0 1 10 10 10 � 2 length scale − 1 � � − 1 � � − N y T ( K + β − 1 I ) − 1 y det ( K + β − 1 I ) 2 log 2 π 2 tr 2 log � �� � � �� � data − fit complexity 6 Images: Neil Lawrence Ek, Kragic KTH Abstraction by Structure

  30. Introduction Structural Representations Structural Models Conclusion References Gaussian Processes: Marginal Likelihood 6 � 4 1 � 5 0.5 � 6 log � likelihood � 7 � 1.5 � 1 � 0.5 0.5 1 1.5 � 8 � 0.5 � 9 � 1 � 10 � 11 � 1.5 � 12 � 1 0 1 10 10 10 � 2 length scale − 1 � � − 1 � � − N y T ( K + β − 1 I ) − 1 y det ( K + β − 1 I ) 2 log 2 π 2 tr 2 log � �� � � �� � data − fit complexity 6 Images: Neil Lawrence Ek, Kragic KTH Abstraction by Structure

  31. Introduction Structural Representations Structural Models Conclusion References Gaussian Processes: Marginal Likelihood 6 � 4 1 � 5 0.5 � 6 log � likelihood � 7 � 1.5 � 1 � 0.5 0.5 1 1.5 � 8 � 0.5 � 9 � 1 � 10 � 11 � 1.5 � 12 � 1 0 1 10 10 10 � 2 length scale − 1 � � − 1 � � − N y T ( K + β − 1 I ) − 1 y det ( K + β − 1 I ) 2 log 2 π 2 tr 2 log � �� � � �� � data − fit complexity 6 Images: Neil Lawrence Ek, Kragic KTH Abstraction by Structure

  32. Introduction Structural Representations Structural Models Conclusion References Gaussian Processes: Marginal Likelihood 6 � 4 1 � 5 0.5 � 6 log � likelihood � 7 � 1.5 � 1 � 0.5 0.5 1 1.5 � 8 � 0.5 � 9 � 1 � 10 � 11 � 1.5 � 12 � 1 0 1 10 10 10 � 2 length scale − 1 � � − 1 � � − N y T ( K + β − 1 I ) − 1 y det ( K + β − 1 I ) 2 log 2 π 2 tr 2 log � �� � � �� � data − fit complexity 6 Images: Neil Lawrence Ek, Kragic KTH Abstraction by Structure

  33. Introduction Structural Representations Structural Models Conclusion References Gaussian Processes: Marginal Likelihood 6 � 4 1 � 5 0.5 � 6 log � likelihood � 7 � 1.5 � 1 � 0.5 0.5 1 1.5 � 8 � 0.5 � 9 � 1 � 10 � 11 � 1.5 � 12 � 1 0 1 10 10 10 � 2 length scale − 1 � � − 1 � � − N y T ( K + β − 1 I ) − 1 y det ( K + β − 1 I ) 2 log 2 π 2 tr 2 log � �� � � �� � data − fit complexity 6 Images: Neil Lawrence Ek, Kragic KTH Abstraction by Structure

  34. Introduction Structural Representations Structural Models Conclusion References Gaussian Processes: Marginal Likelihood 6 � 4 1 � 5 0.5 � 6 log � likelihood � 7 � 1.5 � 1 � 0.5 0.5 1 1.5 � 8 � 0.5 � 9 � 1 � 10 � 11 � 1.5 � 12 � 1 0 1 10 10 10 � 2 length scale − 1 � � − 1 � � − N y T ( K + β − 1 I ) − 1 y det ( K + β − 1 I ) 2 log 2 π 2 tr 2 log � �� � � �� � data − fit complexity 6 Images: Neil Lawrence Ek, Kragic KTH Abstraction by Structure

  35. Introduction Structural Representations Structural Models Conclusion References Gaussian Processes: Marginal Likelihood 6 � 4 1 � 5 0.5 � 6 log � likelihood � 7 � 1.5 � 1 � 0.5 0.5 1 1.5 � 8 � 0.5 � 9 � 1 � 10 � 11 � 1.5 � 12 � 1 0 1 10 10 10 � 2 length scale − 1 � � − 1 � � − N y T ( K + β − 1 I ) − 1 y det ( K + β − 1 I ) 2 log 2 π 2 tr 2 log � �� � � �� � data − fit complexity 6 Images: Neil Lawrence Ek, Kragic KTH Abstraction by Structure

  36. Introduction Structural Representations Structural Models Conclusion References Re-representation GP-LVM a θ Y X a Lawrence [2005] • Occam’s Razor ◮ Dimensionality ◮ Co-variance function • Sufficiently regularises Y problem Ek, Kragic KTH Abstraction by Structure

  37. Introduction Structural Representations Structural Models Conclusion References Shared Representations 7 θ Y θ Z X GP-LVM • Fully shared ◮ not CCA style • Shared/Private Y Z 7 Ek [2008]Salzmann et al. [2010] Ek, Kragic KTH Abstraction by Structure

  38. Introduction Structural Representations Structural Models Conclusion References Shared Representations 7 GP-LVM X Y X X Z • Fully shared ◮ not CCA style θ Y Y Z θ Z • Shared/Private 7 Ek [2008]Salzmann et al. [2010] Ek, Kragic KTH Abstraction by Structure

  39. Introduction Structural Representations Structural Models Conclusion References Factorized Variance 8 • Bayesian GP-LVM a ◮ Prior on X W Y θ Y X θ Z W Z ◮ ARD 2 � � ard ) 2 e − 1 � Q q = 1 w Y q ( x i , q − x j , q ) = ( σ Y k x i , x j 2 Y Z a Titsias and Lawrence [2010] 8 In submission: Damianou, Lawrence, Titsias Ek, Kragic KTH Abstraction by Structure

  40. Introduction Structural Representations Structural Models Conclusion References Factorized Variance 8 8 In submission: Damianou, Lawrence, Titsias Ek, Kragic KTH Abstraction by Structure

  41. Introduction Structural Representations Structural Models Conclusion References Factorized Variance 8 0.08 0.08 0.07 0.07 0.06 0.06 0.05 0.05 0.04 0.04 0.03 0.03 0.02 0.02 0.01 0.01 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 1 2 3 4 5 6 7 8 9 10 11 12 13 14 8 In submission: Damianou, Lawrence, Titsias Ek, Kragic KTH Abstraction by Structure

  42. Introduction Structural Representations Structural Models Conclusion References Factorized Variance 8 8 In submission: Damianou, Lawrence, Titsias Ek, Kragic KTH Abstraction by Structure

  43. Introduction Structural Representations Structural Models Conclusion References Factorized Variance 8                                     8 In submission: Damianou, Lawrence, Titsias Ek, Kragic KTH Abstraction by Structure

  44. Introduction Structural Representations Structural Models Conclusion References Factorised Density 9 Dimensionality Reduction I • Conditional dependency structures, � p ( X ) = p ( x i | π ( x i ) , θ i , S ) i • Learning, ◮ Parameters: θ i ◮ Structure: S � Priors? � Carnality • Heuristics for discrete data 9 Song, Huebner, Hjelm Ek, Kragic KTH Abstraction by Structure

  45. Introduction Structural Representations Structural Models Conclusion References Factorised Density 9 Dimensionality Reduction II T • Very ill-defined X • Re-representation X W X B ◮ “a mapping and configuration” • Prefer “clustered” Y Y re-representation 9 Song, Huebner, Hjelm Ek, Kragic KTH Abstraction by Structure

  46. Introduction Structural Representations Structural Models Conclusion References Factorised Density 9 Objective � p ( Y , X , U | θ ) = p ( Y | f , θ ) p ( f | f U , X , θ ) p ( f U | U | θ ) p ( X ) p ( U | θ ) d f d f U 9 Song, Huebner, Hjelm Ek, Kragic KTH Abstraction by Structure

  47. Introduction Structural Representations Structural Models Conclusion References Factorised Density 9 9 Song, Huebner, Hjelm Ek, Kragic KTH Abstraction by Structure

  48. Introduction Structural Representations Structural Models Conclusion References Factorised Density 9 T O 1 A 1 O 2 A 3 O 3 A 2 C 1 O 4 C 2 C 3 C 4 C 5 C 6 A 4 C 7 A 5 9 Song, Huebner, Hjelm Ek, Kragic KTH Abstraction by Structure

  49. Introduction Structural Representations Structural Models Conclusion References Factorised Density 10 Glass Hammer Knife Hand Over 10 Ek et al. [2011]Song et al. [2011b]Song et al. [2011a] Ek, Kragic KTH Abstraction by Structure

  50. Introduction Structural Representations Structural Models Conclusion References Factorised Density 10 Glass Hammer Knife Pouring 10 Ek et al. [2011]Song et al. [2011b]Song et al. [2011a] Ek, Kragic KTH Abstraction by Structure

  51. Introduction Structural Representations Structural Models Conclusion References Factorised Density 10 Glass Hammer Knife Tool-Use 10 Ek et al. [2011]Song et al. [2011b]Song et al. [2011a] Ek, Kragic KTH Abstraction by Structure

  52. Introduction Structural Representations Structural Models Conclusion References Factorised Density 10 Glass Hammer Knife Hand Over 10 Ek et al. [2011]Song et al. [2011b]Song et al. [2011a] Ek, Kragic KTH Abstraction by Structure

  53. Introduction Structural Representations Structural Models Conclusion References Factorised Density 10 Glass Hammer Knife Pouring 10 Ek et al. [2011]Song et al. [2011b]Song et al. [2011a] Ek, Kragic KTH Abstraction by Structure

  54. Introduction Structural Representations Structural Models Conclusion References Factorised Density 10 Glass Hammer Knife Tool-Use 10 Ek et al. [2011]Song et al. [2011b]Song et al. [2011a] Ek, Kragic KTH Abstraction by Structure

  55. Introduction Structural Representations Structural Models Conclusion References Structural Density 11 Topology Respecting • Structural properties • Geometrical notion irrelevant • Topological information • Barcodes a a Carlsson [2009] 11 In submission: Pokorny, Kjellstr¨ om Ek, Kragic KTH Abstraction by Structure

  56. Introduction Structural Representations Structural Models Conclusion References Structural Density 11 Topology Respecting • Structural properties 0.2 0.1 • Geometrical notion irrelevant 0 − 10 0 10 20 30 40 0.2 • Topological information 0.1 0 − 10 0 10 20 30 40 • Barcodes a a Carlsson [2009] 11 In submission: Pokorny, Kjellstr¨ om Ek, Kragic KTH Abstraction by Structure

  57. Introduction Structural Representations Structural Models Conclusion References Structural Density 11 Topology Respecting • Structural properties • Geometrical notion irrelevant 0.2 0.2 0.1 0.1 • Topological information 0 0 − 10 0 10 20 30 40 − 10 0 10 20 30 40 • Barcodes a a Carlsson [2009] 11 In submission: Pokorny, Kjellstr¨ om Ek, Kragic KTH Abstraction by Structure

  58. Introduction Structural Representations Structural Models Conclusion References Structural Density 11 Topology Respecting 250 200 • Structural properties 150 100 • Geometrical notion irrelevant 50 • Topological information 0 −50 • Barcodes a −100 −150 −150 −100 −50 0 50 100 150 200 250 a Carlsson [2009] 11 In submission: Pokorny, Kjellstr¨ om Ek, Kragic KTH Abstraction by Structure

Recommend


More recommend