cmp722
play

CMP722 ADVANCED COMPUTER VISION Lecture #5 Language and Vision - PowerPoint PPT Presentation

Illustration: William Joel CMP722 ADVANCED COMPUTER VISION Lecture #5 Language and Vision Aykut Erdem // Hacettepe University // Spring 2019 Illustration: Detail from Fritz Kahns Der Mensch als Industriepalast Previously on CMP722


  1. Representing meaning Is there a red shape above a circle? 57

  2. Representing meaning Is there a red shape above a circle? 58

  3. Sets encode meaning Is there a red shape above a circle? 59

  4. Sets encode meaning Is there a red shape above a circle? 60

  5. Set transformations encode meaning Is there a red shape above a circle? 61

  6. Set transformations encode meaning Is there a red shape above a circle? 62

  7. Sentence meanings are computations Is there a red shape above a circle? exists and red above circle 63

  8. Sentence meanings are computations Is there a red shape above a circle? exists and red above circle 64

  9. Computations are built from set functions exists ↦ red and ↦ true exists ↦ red above above circle 65

  10. …or relaxed to vector functions 0.0 0.9 1.0 exists ↦ red and ↦ true exists ↦ red above above circle 66

  11. Composing vector functions exists ↦ red and ↦ true exists ↦ red above above circle 67

  12. Composing vector functions exists ↦ red and ↦ true exists ↦ red above above circle 68

  13. Composing vector functions ↦ exists red and ↦ true red above exists ↦ circle above 69

  14. Compositions of vector functions are neural nets ↦ ↦ true ↦ 70

  15. Compositions of vector functions are neural nets ↦ exists red and ↦ true red above exists ↦ circle above 71

  16. Outline yes Is there a red shape above a circle? exists and red above ↦ circle red ↦ true exists ↦ above 72

  17. Outline yes Is there a red shape above a circle? exists and red above ↦ circle red ↦ true exists ↦ above 73

  18. Outline yes Is there a red shape above a circle? exists and red above ↦ circle red ↦ true exists ↦ above 74

  19. Outline yes Is there a red shape above a circle? exists and red above ↦ circle red ↦ true exists ↦ above 75

  20. Outline yes Is there a red shape above a circle? exists and red above ↦ circle red ↦ true exists ↦ above 76

  21. Anatomy of a module above 77

  22. Anatomy of a module red color 78

  23. The [find] module red 79

  24. The [find] module necktie [Xu et al. 2015] 80

  25. The [find] module 0.9 Columbia 0.1 Cooper 0.8 Myrtle Beach city nam ame typ ype coast co astal al Co Columbia ci city no no Cooper Co rive ver yes ye Myr yrtle ci city ye yes Beach ach 81

  26. The [find] module red 82

  27. The [find] module red 83

  28. The [find] module red red 84

  29. The [find] module 0.9 red red 85

  30. The [find] module 0.9 red red 86

  31. The [find] module 0.1 red red 87

  32. The [describe] module red color 88

  33. The [describe] module necktie what 89

  34. The [describe] module red color 90

  35. The [describe] module red . . . color 91

  36. The [describe] module red . . . color 92

  37. What modules do we need? Is there a red shape above a circle? What color is the triangle? Who is running in the grass? What cities are south of San Diego? 93

  38. A module for predicates Is there a red shape above a circle? [find] What color is the triangle? Who is running in the grass? What cities are south of San Diego? 94

  39. A module for relations Is there a red shape above a circle? [relate] What color is the triangle? Who is running in the grass? What cities are south of San Diego? [find] 95

  40. Module inventory Is there a red shape above a circle? What color is the triangle? Who is running in the grass? [and] [describe] What cities are south of San Diego? [find] [relate] [exists] 96

  41. Outline yes Is there a red shape above a circle? exists and red above ↦ circle red ↦ true exists ↦ above 97

  42. Learning yes blue exists color and right_of red above circle circle What color is the shape right of a Is there a red shape above a circle? circle? 98

  43. Learning yes blue What color is the shape right of a Is there a red shape above a circle? circle? 99

  44. Parameter tying yes blue circle circle What color is the shape right of a Is there a red shape above a circle? circle? 100

Recommend


More recommend