School of EECS Washington State University Artificial Intelligence - PowerPoint PPT Presentation

School of EECS Washington State University Artificial Intelligence 1

I see a cougar. • Animal, cat, cougar; • I see two cougars? • moving left. I see water and grass. • Person, police, gun; • Pretty tree. • moving right. Distance to cougar: 10m. • Trash bin. • Should I leave? • Danger! • Can’t park here. • Scene Understanding Artificial Intelligence 2

} Edge detection } Shape detection } Motion: optical flow, tracking } Object detection } Image classification } Scene understanding Artificial Intelligence 3

Artificial Intelligence 4

} Given image intensity I(x,y) (e.g., grayscale) } Step 1: Smooth image ◦ Gaussian N σ (x,y) ◦ Replace intensity I(x,y) with I * N σ (i.e., convolve) Artificial Intelligence 5

} Step 2: Find intensity gradient G ◦ Convolve image with Sobel operator −1 0 +1 −1 −2 −1 ◦ 𝐻 " = ∗ I 𝐻 + = ∗ I −2 0 +2 0 0 0 −1 0 +1 +1 +2 +1 3 4 , +𝐻 + , 𝜄 = tan 12 ◦ 𝐻 = 𝐻 " 3 5 G x = 255+510+0 = 765 255 0 0 G y = -255+0+1020 = 765 0 255 255 G = 1082 θ = tan -1 (765/765) = 45° 255 255 255 Artificial Intelligence 6

} Step 3: Thin edges edge A If B < max(A,B,C) B Then B = 0 C gradient 3 255 0 255 4 0 5 0 255 255 255 255 255 255 255 255 255 255 Artificial Intelligence 7

} Step 4: Apply double threshold with hysteresis kept G kept kept max value kept: if next to kept then kept removed min value removed Image Artificial Intelligence 8

Artificial Intelligence 9

Artificial Intelligence 10

} Detect simple parametric shapes ◦ Lines, circles, etc. } Noise tolerant } Approach ◦ For each edge – Increment models consistent with edge ◦ Choose models with most votes Artificial Intelligence 11

Model: y = mx + b x b 0 y m 1 1 1 1 1 3 1 1 1 1 Image Parameter Space Artificial Intelligence 12

} Parameter space (m, b) infinite ◦ How to choose range and increment? } Another model of a line: r = x cos θ + y sin θ Artificial Intelligence 13

Probabilistic Hough Transform: Line segments Artificial Intelligence 14

Threshold > 90 Threshold > 100 Artificial Intelligence 15

Artificial Intelligence 16

} Assumptions ◦ Brightness constancy ◦ Small motion Artificial Intelligence 17

} Frame differencing Artificial Intelligence 18

Artificial Intelligence 19

Have I x , I y , I t . Solve for u and v. One more assumption: Constant flow. E.g., the 3x3 patch around a pixel have same displacement. [Lucas-Kinkade Method] Artificial Intelligence 20

Artificial Intelligence 21

} Given an image feature to track ◦ E.g., bounding box } Find it as the image changes Artificial Intelligence 22

} As an image alignment problem... T( x ) I( x ) W ( x ; p ) p involves translation, rotation, scaling Artificial Intelligence 23

What if we don't have a template...? Artificial Intelligence 24

} Approach #1: Feature-based ◦ Define various image features ◦ Model object in terms of these features ◦ Look for feature-level matches in image } Approach #2: Network-based ◦ Train a deep neural network on lots of images with known objects in known locations ◦ Use network to locate objects in an image Artificial Intelligence 25

} Features ◦ Histograms of Oriented Gradients (HOG) Artificial Intelligence 26

} Pedestrian HOG Artificial Intelligence 27

Artificial Intelligence 28

} Features ◦ Scale Invariant Feature Transform (SIFT) 1. Find extreme 2. Discard low- 3. Filter points points contrast points on edges "keypoints" Artificial Intelligence 29

http://weitz.de/sift/ Artificial Intelligence 30

Training: HOG/SIFT airplane features bicylcle . Machine . Learning . truck Testing: Artificial Intelligence 31

Figures out features automatically Need a lot of training data... Artificial Intelligence 32

} PASCAL Visual Object Classes (VOC) ◦ host.robots.ox.ac.uk/pascal/VOC ◦ 20,000 images ◦ 20 classes } ImageNet ◦ image-net.org ◦ 1.5M images ◦ 200 classes Artificial Intelligence 33

ReLU Artificial Intelligence 34

} CIFAR-10 } CIFAR-100 ◦ 60,000 images ◦ 60,000 images ◦ 10 classes ◦ 100 classes https://www.cs.toronto.edu/~kriz/cifar.html Artificial Intelligence 35

I see an astronaut walking on grass. There is a bright gold dog bowl nearby. There are several puddles of water, some with wavy lines above them. There are brown bumps on the grass that are smelly. There is ? a happy alien with one eye. location(agent,1,2) orientation(agent,right) location(wumpus,4,2) location(gold,2,3) location(pit,3,1): 1.0 location(pit,3,3): 0.8 dimensions(4,4) bestAction(goforward) Artificial Intelligence 36

MIT Places Uses 16-layer ConvNet http://places2.csail.mit.edu/demo.html Artificial Intelligence 37

} Drag image here } Edit Alt Text... "A person walking a dog on a leash in front of a building." Artificial Intelligence 38

[Xiao et al., ECCV 2018] Artificial Intelligence 39

} OpenCV (opencv.org) } scikit-image (scikit-image.org) Artificial Intelligence 40

} Techniques depend on goals of vision ◦ Edge detection ◦ Shape detection ◦ Motion: optical flow, tracking ◦ Object detection ◦ Image classification } Future ◦ Scene understanding ◦ Video summarization ◦ Fake images and video Artificial Intelligence 41