Augmented Reality made More Real Amitabh Varshney Institute for - PowerPoint PPT Presentation

Augmented Reality made More Real Amitabh Varshney Institute for Advanced Computer Studies (UMIACS) and Computer Science

Augmented Reality • An Interface to facilitate our awareness of the Digital (Virtual) World coexisting with the Real World in a natural and intuitive manner • All senses, but most advances thus far have been visual. • Location-based Spatio-Temporal reality

Head-Mounted Displays Ron Azuma 1995

Augmented Reality in the Battlefield

Augmented Reality for Surgery

Augmented Reality in Daily Life

Augmented Reality in Daily Life

Visual Challenges in AR • Thin, Lightweight, High resolution, wide field of view displays • Registration of real and virtual worlds • Rendering of virtual worlds mixed with real worlds • Perception management for AR GPUs will play a central role in all of the above!

AR Prototype (in 3 months) • We came across this Oculus HD Prototype (don't ask how...) • 840 x 975 pixels for each eye • Approx. 110 degree diagonal field of view (comparatively wide) • Device orientation tracking – accelerometer- gyroscope combo • Prototype 0 – 20fps HD webcams: horrible!

Customizing the AR Bracket • We made a CAD model of the front shell • We modeled a bracket around it • We 3D printed it

Assembling the AR Prototype • We got a couple of Point Grey Flea3 Cameras • 3.2 MP @ 60 fps via USB 3 • Low-latency camera design (thanks to Point Grey) • Added some wide-angle lenses (80-185 ° FOV) • Wrote some code for it...

Greg Kramida

GPUs for De-Warping • Lens distortion: really bad in our case • Have to calibrate and "undistort": – compute mapping once (calibrate) – apply mapping to each successive frame (remap) • Remap operation is a simple kernel • OpenCV 3.0 alpha already provides this kernel for both CUDA and OpenCL Top graphic adapted from "Computer Vision Acceleration Using GPUs" by Shehrzad Qureshi

Disparity Computation • Depth from stereo • Compute how far each point in right image is from the same point in left image • OpenCV 3.0 alpha routines for CUDA and OpenCL: – Block matching – Belief propagation Top graphic adapted from "Computer Vision Acceleration Using GPUs" by Shehrzad Qureshi

Need for AR Registration

Dynamic Registration

AR Registration Extraction of objects using contours 1. Find color range of the interesting objects – HSV, RGB, LAB, LUB 2. Create mask based on the range – Noise reduced with erosion, dilation, and blurring 3. Detect edges of the mask 4. Find contours – Draw lines at the border of screen to find partial views 5. Draw contours

Original Image Range Monitor Frame Mask Extracted Edge Filled Contours with Final Output Detection partial view fix

Original Image Range Mask Monitor Frame Edge Extracted Detection Filled Contours Filled Contours Filled Contours Final Output Currently: 40-50 msec/frame with GPUs

Curing Phobias through Augmented Reality Botella et al. 2010

Next Steps: Augmentarium

Last Night …

Last Night …

Closing Remarks • Confluence of a number of technological drivers – robots, HMDs, projectors, imaging sensors, and GPU processing, .. • VR/AR tools and technologies need to co-develop with driving applications in consumer applications, entertainment, science, medicine, and engineering. • Critical to leverage human perceptual and cognitive processes in development of VR/AR tools • Very exciting times ahead!

Acknowledgements Greg Kramida Sujal Bista Tim Gray Fritz McCall Derek Yarnell Mike van Opstal Eric Krokos Hsueh-Chien Cheng Patricia Sazama Barbara Brawn-Cinani Larry Rosenblum NSF, NVIDIA, Mpower Maryland, and UMD CMNS