[PDF] - Visual Hacks Henrik I. Christensen Robotics and Intelligent PDF Document

SLIDE 1 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Visual Hacks

Henrik I. Christensen

Robotics and Intelligent Machines @ GT College of Computing Georgia Institute of Technology Atlanta, GA hic@cc.gatech.edu

February 12, 2008

H. I. Christensen (RIM@GT)

Vision February 12, 2008 1 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Outline

1

Introduction

2

Imaging Geometry

3

Features

4

Feature Description

5

Recognition

6

Wrap-up

H. I. Christensen (RIM@GT)

Vision February 12, 2008 2 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Camera

Most flexible sensory modality Complex sensory processing Not discussed in any detail Offers wide range Diverse tasking of sensor Relatively inexpensive Computationally demanding

H. I. Christensen (RIM@GT)

Vision February 12, 2008 3 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Image processing chain

ptics

image acquisition image enhancement/ processing Image segmentation Image Description (features) Recognition/ Estimation

H. I. Christensen (RIM@GT)

Vision February 12, 2008 4 / 37

SLIDE 2 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Color Models

There are quite a few image color models RGB Red, Green, Blue HSI Hue, Saturation, Intensity CMY Cyan, Magenta, Yellow For this course RGB is adequate Check carefully on the selected / available color models

H. I. Christensen (RIM@GT)

Vision February 12, 2008 5 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Color combinations

R B G Magenta Cyan Yellow Black

H. I. Christensen (RIM@GT)

Vision February 12, 2008 6 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Outline

1

Introduction

2

Imaging Geometry

3

Features

4

Feature Description

5

Recognition

6

Wrap-up

H. I. Christensen (RIM@GT)

Vision February 12, 2008 7 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Pin-Hole Model

H. I. Christensen (RIM@GT)

Vision February 12, 2008 8 / 37

SLIDE 3 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Pin-Hole Model

The relations are then: x λ = X Z y λ = Y Z ⇒ x = λX Z y = λY Z

H. I. Christensen (RIM@GT)

Vision February 12, 2008 9 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Pin-Hole Model

Remember Homogeneous Coordinates?

P =

    X Y Z 1     Define the Perspective transform as T =     1 1 1

1 λ

   

H. I. Christensen (RIM@GT)

Vision February 12, 2008 10 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Camera models

The pin-hole model is a fair approximation for medium focal lengths For extreme values of the focal length various kinds of distortion should be taken into account Calibration is usually required to ensure use of the camera for real-world tasks MATLAB has a great toolbox for image calibration http://www.vision.caltech.edu/bouguetj/calib_doc/

H. I. Christensen (RIM@GT)

Vision February 12, 2008 11 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Outline

1

Introduction

2

Imaging Geometry

3

Features

4

Feature Description

5

Recognition

6

Wrap-up

H. I. Christensen (RIM@GT)

Vision February 12, 2008 12 / 37

SLIDE 4 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Image Features

Computer Vision has been studied for 4 decades. It might offer the biggest potential. Robustness is the major challenge SLAM is termed “Structure from motion” (SFM) in the vision literature.

H. I. Christensen (RIM@GT)

Vision February 12, 2008 13 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Harris Features

Detection of high curvature corner/edge points Standard in several image processing packages. Noise sensitive M = I 2

x

IxIy IxIy I 2

y

R = det M − k(traceM)2

Lukas-Tomasi-Kanade (condition number of M)

H. I. Christensen (RIM@GT)

Vision February 12, 2008 14 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Harris

H. I. Christensen (RIM@GT)

Vision February 12, 2008 15 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Harris example

H. I. Christensen (RIM@GT)

Vision February 12, 2008 16 / 37

SLIDE 5 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

SIFT features

(Lowe, 2004) Scale-space top-points Coding of local orientation at top-points Example code is available (linux) and several Matlab versions.

H. I. Christensen (RIM@GT)

Vision February 12, 2008 17 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

SIFT Example

H. I. Christensen (RIM@GT)

Vision February 12, 2008 18 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Harris Example

H. I. Christensen (RIM@GT)

Vision February 12, 2008 19 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Outline

1

Introduction

2

Imaging Geometry

3

Features

4

Feature Description

5

Recognition

6

Wrap-up

H. I. Christensen (RIM@GT)

Vision February 12, 2008 20 / 37

SLIDE 6 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Feature Detection

Covered in more detail in future lectures Basics of feature detection to get started The idea is to condense regions into a compact representation.

H. I. Christensen (RIM@GT)

Vision February 12, 2008 21 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Example features

Euler Number / #holes in a region Ellipse approximation (major/minor axes) Line parameters Compactness Perimeter length

H. I. Christensen (RIM@GT)

Vision February 12, 2008 22 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Feature / Region Description

Description of two different types of regions

1

Contour based

2

Region based

Advantages / Disadvantages?

H. I. Christensen (RIM@GT)

Vision February 12, 2008 23 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Contour descriptors

Chain codes Signatures (polar) Polygonal approximation

H. I. Christensen (RIM@GT)

Vision February 12, 2008 24 / 37

SLIDE 7 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Moments

Statistical moments are in general defined as: mn =

L−1

i=0

xn

i g(xi)

The mean is then m1 =

L−1

i=0

xig(xi) Central moments are defined as µn =

L−1

i=0

(xi − m1)ng(xi) In two dimensions µmn

L−1

i=0

K−1

j=0

(xi − m10)m(yj − m01)ng(xi, yj)

H. I. Christensen (RIM@GT)

Vision February 12, 2008 25 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Well-known moments

Standard deviation σ = √µ2 =

L−1
i=0

(xi − m1)2g(xi) Skewness - µ3 - indicates the symmetry of the distribution, value 0 = perfect symmetry Normalized central moments of order (p+q) νpq = µpq mγ

pq

where γ = p+q

2

+ 1 Moments are widely used for characterization of regions and for standard tasks

H. I. Christensen (RIM@GT)

Vision February 12, 2008 26 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Least Square Line Fitting

Assume g(x,y) = 1 where we have the line elements If we have computed µ20, µ11, µ02, µ10, µ01, µ00 We want to estimate a line of the form y = a + bx The regressions can be computed as b = µ11 µ20 a = µ01 − bµ10 Quality of fit defined as r2 r2 = µ2

11

µ02µ20

H. I. Christensen (RIM@GT)

Vision February 12, 2008 27 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Other use of moments

Characterization of image texture Variance, Cross correlation, ... allow pattern matching Ellipse matching - major / minor axes of a region A rich and easy to use descriptor

H. I. Christensen (RIM@GT)

Vision February 12, 2008 28 / 37

SLIDE 8 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Outline

1

Introduction

2

Imaging Geometry

3

Features

4

Feature Description

5

Recognition

6

Wrap-up

H. I. Christensen (RIM@GT)

Vision February 12, 2008 29 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Recognition

Supervised recognition A prior database of samples has been recorded and processed Strategy

1

Compute a set of descriptors

2

Determine how well the “features” match against prototypes

3

Choose the best fit

H. I. Christensen (RIM@GT)

Vision February 12, 2008 30 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Example - Bolts or Needles

H. I. Christensen (RIM@GT)

Vision February 12, 2008 31 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Minimum distance classification

Suppose we have computed the mean value for each of the classes mneedle = [0.86, 2.34]T and mbolt = [5.74, 5, 85]T We can then compute the minimum distance dj(x) = ||x − mj|| argminidi(x) is the best fit Decision functions can be derived

H. I. Christensen (RIM@GT)

Vision February 12, 2008 32 / 37

SLIDE 9 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Bolts / Needle Decision Functions

Needle dneedle(x) = 0.86x1 + 2.34x2 − 3.10 Bolt dbolt(x) = 5.74x1 + 5.85x2 − 33.59 Decision boundary di(x) − dj(x) = 0 dneedle/bolt(x) = −4.88x1 − 3.51x2 + 30.49

H. I. Christensen (RIM@GT)

Vision February 12, 2008 33 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Example decision surface

H. I. Christensen (RIM@GT)

Vision February 12, 2008 34 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Nearest neighbor

Assume you have a set of prototypes (fi) that are classified Estimate distance to prototypes: di(x) = ||x − fi|| Assign the label that corresponds to the closest prototype Easy to implement, can often be pre-processed or simplified

H. I. Christensen (RIM@GT)

Vision February 12, 2008 35 / 37 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Outline

1

Introduction

2

Imaging Geometry

3

Features

4

Feature Description

5

Recognition

6

Wrap-up

H. I. Christensen (RIM@GT)

Vision February 12, 2008 36 / 37

SLIDE 10 Introduction Imaging Geometry Features Feature Description Recognition Wrap-up

Wrap-up

Select tools to fit your task Quick tour of vision 101 Think about color spaces Use off-the-shelf calibration Consider simple robust features This is not a project about recognition ....

H. I. Christensen (RIM@GT)

Vision February 12, 2008 37 / 37