Object detection as supervised classification Thurs April 13 - PDF document

4/12/2017 Object detection as supervised classification Thurs April 13 Kristen Grauman UT Austin Last time • Discovering visual patterns • Randomized hashing algorithms • Mining large-scale image collections Review questions: on your own • What kind of input data is searchable with min- hash hashing? • What kind of input data is searchable with LSH using random projections? • For Visual “PageRank” what do weights between nodes (images) signify? 1

4/12/2017 What does recognition involve? Fei-Fei Li Detection: are there people? Activity: What are they doing? 2

4/12/2017 Object categorization mountain tree building banner street lamp vendor people Instance recognition Potala Palace A particular sign Scene and context categorization • outdoor • city • … 3

4/12/2017 Attribute recognition gray made of fabric crowded flat Object Categorization • Task Description  “Given a small number of training images of a category, recognize a-priori unknown instances of that category and assign Perceptual and Sensory Augmented Computing the correct category label.” • Which categories are feasible visually? Visual Object Recognition Tutorial “Fido” German dog animal living shepherd being K. Grauman, B. Leibe K. Grauman, B. Leibe Visual Object Categories • Basic Level Categories in human categorization [Rosch 76, Lakoff 87] Perceptual and Sensory Augmented Computing  The highest level at which category members have similar perceived shape  The highest level at which a single mental image reflects the Visual Object Recognition Tutorial entire category  The level at which human subjects are usually fastest at identifying category members  The first level named and understood by children  The highest level at which a person uses similar motor actions for interaction with category members K. Grauman, B. Leibe K. Grauman, B. Leibe 4

4/12/2017 Visual Object Categories • Basic-level categories in humans seem to be defined predominantly visually. Perceptual and Sensory Augmented Computing • There is evidence that humans (usually) … start with basic-level categorization before doing identification. animal Visual Object Recognition Tutorial  Basic-level categorization is easier Abstract and faster for humans than object … … levels identification! quadruped  How does this transfer to automatic … classification algorithms? Basic level dog cat cow German Doberman shepherd Individual … … “Fido” level K. Grauman, B. Leibe K. Grauman, B. Leibe How many object categories are there? Biederman 1987 Source: Fei-Fei Li, Rob Fergus, Antonio Torralba. 5

4/12/2017 Other Types of Categories • Functional Categories  e.g. chairs = “something you can sit on” Perceptual and Sensory Augmented Computing Visual Object Recognition Tutorial K. Grauman, B. Leibe K. Grauman, B. Leibe Why recognition? – Recognition a fundamental part of perception • e.g., robots, autonomous agents – Organize and give access to visual content • Connect to information • Detect trends and themes Autonomous agents able to detect objects Slide: Kristen Grauman http://www.darpa.mil/grandchallenge/gallery.asp 6

4/12/2017 Posing visual queries Yeh et al., MIT Belhumeur et al. Kooaba, Bay & Quack et al. Slide: Kristen Grauman Finding visually similar objects Slide: Kristen Grauman Exploring community photo collections Snavely et al. Slide: Kristen Grauman Simon & Seitz 7

4/12/2017 Discovering visual patterns Sivic & Zisserman Objects Lee & Grauman Categories Wang et al. Actions Slide: Kristen Grauman Auto-annotation Gammeter et al. T. Berg et al. Slide: Kristen Grauman Challenges: robustness Illumination Object pose Clutter Intra-class Occlusions Viewpoint appearance Slide: Kristen Grauman 8

4/12/2017 Challenges: context and human experience Context cues Slide: Kristen Grauman Challenges: context and human experience Function Dynamics Context cues Slide: Kristen Grauman Video credit: J. Davis Challenges: complexity • Millions of pixels in an image • 30,000 human recognizable object categories • 30+ degrees of freedom in the pose of articulated objects (humans) • Billions of images online • 82 years to watch all videos uploaded to YouTube per day! … • About half of the cerebral cortex in primates is devoted to processing visual information [Felleman and van Essen 1991] Slide: Kristen Grauman 9

4/12/2017 Challenges: learning with minimal supervision More Less Slide: Kristen Grauman Slide from Pietro Perona, 2004 Object Recognition workshop Slide from Pietro Perona, 2004 Object Recognition workshop 10

4/12/2017 What kinds of things work best today? Reading license plates, zip codes, checks Frontal face detection Recognizing flat, textured objects (like books, CD Fingerprint recognition covers, posters) What kinds of things work best today? Progress charted by datasets Roberts 1963 COIL 1963 … 1996 Slide: Kristen Grauman 11

4/12/2017 Progress charted by datasets MIT-CMU Faces MIT-CMU Faces MIT-CMU Faces INRIA Pedestrians INRIA Pedestrians INRIA Pedestrians UIUC Cars UIUC Cars UIUC Cars 1963 … 1996 2000 Slide: Kristen Grauman Progress charted by datasets MSRC 21 Objects MSRC 21 Objects MSRC 21 Objects Caltech-101 Caltech-101 Caltech-101 Caltech-256 Caltech-256 Caltech-256 1963 … 2005 1996 2000 Slide: Kristen Grauman Progress charted by datasets ImageNet ImageNet ImageNet 80M Tiny Images 80M Tiny Images 80M Tiny Images PASCAL VOC PASCAL VOC PASCAL VOC PASCAL VOC PASCAL VOC Birds-200 Birds-200 Birds-200 Faces in the Wild Faces in the Wild Faces in the Wild 1963 … 1996 2000 2005 2007 2008 2013 Slide: Kristen Grauman 12

4/12/2017 Evolution of methods • Hand-crafted models • Hand-crafted features • “End-to-end” • 3D geometry • Learned models learning of features and • Hypothesize and align • Data-driven models*,** * Labeled data availability ** Architecture design decisions, parameters. Slide: Kristen Grauman Next • Supervised classification • Window-based generic object detection – basic pipeline – boosting classifiers – face detection as case study Supervised classification • Given a collection of labeled examples, come up with a function that will predict the labels of new examples. “four” “nine” ? Novel input Training examples • How good is some function we come up with to do the classification? • Depends on – Mistakes made – Cost associated with the mistakes 13

4/12/2017 Supervised classification • Given a collection of labeled examples, come up with a function that will predict the labels of new examples. • Consider the two-class (binary) decision problem – L(4→9): Loss of classifying a 4 as a 9 – L(9→4): Loss of classifying a 9 as a 4 • Risk of a classifier s is expected loss:               R ( s ) Pr 4 9 | using s L 4 9 Pr 9 4 | using s L 9 4 • We want to choose a classifier so as to minimize this total risk Supervised classification Optimal classifier will minimize total risk. At decision boundary, either choice of label yields same expected Feature value x loss. If we choose class “four” at boundary, expected loss is:     P ( class is 9 | x ) L (9 4) P (class is 4 | x ) L (4 4)   P ( class is 9 | x ) L (9 4) If we choose class “nine” at boundary, expected loss is:  P ( class is 4 | x ) L (4  9) Supervised classification Optimal classifier will minimize total risk. At decision boundary, either choice of label yields same expected Feature value x loss. So, best decision boundary is at point x where    P ( class is 9 | x ) L (9 4) P(class is 4 | x ) L (4 9) To classify a new point, choose class with lowest expected loss; i.e., choose “four” if    P ( 4 | x ) L ( 4 9 ) P ( 9 | x ) L ( 9 4 ) 14

4/12/2017 Supervised classification Optimal classifier will P(4 | x ) P(9 | x ) minimize total risk. At decision boundary, either choice of label yields same expected Feature value x loss. So, best decision boundary is at point x where P ( class is 9 | x ) L (9  4)  P(class is 4 | x ) L (4  9) To classify a new point, choose class with lowest expected loss; i.e., choose “four” if    P ( 4 | x ) L ( 4 9 ) P ( 9 | x ) L ( 9 4 ) Example: learning skin colors • We can represent a class-conditional density using a histogram (a “non-parametric” distribution) Percentage of skin pixels in each bin P(x|skin) Feature x = Hue P(x|not skin) Feature x = Hue Slide: Kristen Grauman Example: learning skin colors • We can represent a class-conditional density using a histogram (a “non-parametric” distribution) P(x|skin) Feature x = Hue Now we get a new image, P(x|not skin) and want to label each pixel as skin or non-skin. What’s the probability we care about to do skin detection? Feature x = Hue Slide: Kristen Grauman 15