Developing Objects Segregation capabilities and the notion of - PowerPoint PPT Presentation

Developing Objects Segregation capabilities and the notion of Object Containment from unlabeled natural videos Daniel Harari Joint work with Nimrod Dorfman and Shimon Ullman

Object Segregation Object 2 Object 1 Background

Object segregation is learned [Kellman & Spelke 1983; Spelke 1990; Kestenbaum et al., 1987] Even basic Gestalt cues are initially missing [Schmidt et al. 1986] 5 months 3

Object segregation is learned Adults 4

It all begins with motion 5

It all begins with motion Grouping by common motion precedes figural goodness [Spelke 1990 - review] Motion discontinuities provide an early cue for occlusion boundaries [Granrud et al. 1984] 6

Our model Motion-based segregation Motion Common Boundary Global discontinuities motion General Object-specific Accurate Complete Noisy Inaccurate Local occlusion Object form Incomplete boundaries Static segregation 7 CogSci 2013

Boundary Intensity edges? 8

Boundary Occlusion cues T-junctions Convexity Extremal edges [Ghose & Palmer 2010] 9

Global Familiar object 10

How does it actually work? 11

Motion Moving object 12

Motion Figure Ground Unknown Boundary Global 13

Boundary Informative boundary features Need many examples for good results (1000+) 14

Boundary Prediction Figure Figure or or Ground? Ground? Novel object, novel background 78% success Using 100,000 training examples 15

Boundary Entire image Figure Background 16

Global Learning an object Standard object recognition algorithm Learns local features and their relative locations 17

Global Detection 18

Combining information sources Figure Background Boundary Global Combined Accurate Complete Noisy & Incomplete Inaccurate 19

More complex algorithms Default GrabCut With segregation cue 20 [Rother et al. 2004]

Summary • Static segregation is learned from motion • Two simple mechanisms: Boundary Motion discontinuities  Occlusion boundaries (Need a rich library, including extremal edges) Global Common motion  Object form • These mechanisms work in synergy • This is enough to get started, adult segregation is much more complex 21

Object Containment Object 2 Object 2 Object 1 Object 2

Occlusion vs. Containment Occlusion A occludes C Containment = a paradoxical occlusion A C A C C occludes A A A occludes C & C occludes A C

Developmental path Dynamic occlusion High angle containment Static containment Dynamic containment Tight and loose fit 2.5 months 6 months 18 months

Reasoning about spatial relations Occlusion and Containment at 2.5 months [Hespos & Baillargeon 2001]

Dynamic containment

Familiarization events External boundary Internal boundary External object region Internal object region

Motion boundaries Optical flow Flow boundary t- Δ t t t+ Δ t

Motion boundaries ownership Between (t- Δ t) and t

Motion boundaries ownership Between t and (t+ Δ t) The boundary is owned by the GREEN object (basket)

Temporal dynamics of containment Occlusion Containment A A A occludes C A occludes C A occludes C & C C C occludes A

Detection of dynamic containment Occluder/total Simple Occlusion Event Frame number Container/total Containment Event Frame number

Static containment 33

Familiarization events External boundary Internal boundary External object region Internal object region

Main idea Use motion to learn about object regions and boundaries 35

Advanced notions of containment: Tight vs. Loose Fit and High angle At 6 months High Angle Tight/loose fit [Casasola & Cohen 2002]

Tight vs. Loose Fit

The computational challenge • Learning to identify containers and containment events in dynamic and static visual inputs without labels. • Learning more conceptual aspects of what containment means, for example that a contained object moves together with the container to a new location, loose- and tight-fit, etc. • In the future, learn other aspects of conceptual knowledge about container (containing liquids, pouring from a container, etc.)

Summary Object Segregation Object Containment • Dynamic grouping - • Dynamic grouping – global object model object occlusion • Dynamic discontinuities- • Dynamic discontinuities- figure-ground internal and external object boundaries • Acquiring the capability • Acquiring the notion of to segregate objects in static images paradoxical occlusion, object containment, in both dynamic and static images

Thank you!