Planar Augmented Reality Kameron Kincade What is Augmented Reality? - PowerPoint PPT Presentation

ì ¡ Planar Augmented Reality Kameron Kincade

What is Augmented Reality? ì Augment : to make greater or more intense ì Reality : perception of the real world

Why Planar Objects? ì They are Common ì They are Detectable Calculate 3D points using the pose of the camera ì Compare points to see if they lie on a plane ì ì They allow for Realistic AR Rendering

The Steps 1.) Detect a planar surface 2.) Track the planar surface 3.) Display rendering on the surface

The Steps 1.) Detect a planar surface 2.) Track the planar surface 3.) Display rendering on the surface

1.) Detecting Planar Surfaces ì Technique 1: Training Images

Looney Tune Detector

Using Training Images 1. Load in Training Images 2. Detect keypoints and descriptors for training images 3. Analyze each individual input frame - Detect its keypoints and descriptors 4. Find matches between the training images and current frame 5. Determine if matches are legitimate

First Attempt ì Poor Performance ì Trouble debugging because of lag ì Unable to tell if matcher was working ì SIFT and SURF Problems

No SIFT or SURF ì SIFT – Scale Invariant Feature Transform ì David Lowe - 1999 ì SURF – Speeded Up Robust Features ì Herbert Bay – 2006 Due to legal obligations, SIFT and SURF are not packaged with OpenCV4Android

SIFT ì Stores keypoints from reference images into a database ì To detect features new images, we identify candidate locations and then do further investigation ì Detected features are highly distinctive, making it likely to find a match in a large database

SURF ì Same basic principles as SIFT ì Distinct locations are marked as “interest locations” ì Neighborhood of “interest locations” are represented as feature vectors ì The distances between these vectors determines matches ì Uses integral images which speed up some of our detection algorithms

No SIFT or SURF ì Tried compiling the “non-free” libraries from the C++ source code into library files (.so) ì Tried using Android NDK to use the native C++ source code Unsuccessful on both attempts

FAST ì Tried FAST (Features from Accelerated Segment Test)

ORB Oriented FAST and Rotated BRIEF ì Scale and rotation invariant ì Claimed to be two orders of magnitude faster than SIFT This is what I used!

Second Attempt ì Much better performance à THREADS!!! ì Using ORB for feature detection and descriptor extraction ì Scale and rotation invariant

Looney Tune Detector How does it work?

Load Training Images

Analyze Training Images

Threads

Process Frame

Thread Handler

Application Overview ì Load each training image and analyze its keypoints and descriptors ì Give each training image its own thread ì When we receive an input frame, calculate its keypoints and descriptors ì Have each thread find matches with its training image ì Accept or reject matches to determine detection

Improvements ì Tweak the criteria for determining a good match ì Try JNI to import SIFT and SURF to see if performance is any better

Questions? ?