Mobile Augmented Reality Tássio Knop de Castro Visgraf Lab, IMPA, Rio de Janeiro 2011
Intro – Mobile Augmented Reality AR Toolkit Nearest Tube App ● Enhance the perception of the real world
Motivation Trip Advisor ● How to provide useful information without confusing the user? ● Is it possible to really insert a virtual object onto a real scene in a realistic way?
Goal To explore the possibilities and limitations of Mobile Augmented Reality concerning information insertion and objects rendering
Mobile Development
The device: Nokia N900 ● 600 MHz ARM processor ● 256 MB RAM ● 5MP programmable camera
Maemo Development ● MADDE (Maemo Application Development and Debugging Environment) ● Cross-compiling ● Multi-platform support (Linux (32-bit/64-bit), Windows, Mac OS X) ● Comes with Nokia Qt SDK =) ● It's a technology preview =( ● Scratchbox
Maemo Development ● Main challenges: ● Small community, poor documentation ● How to compile and configure external libraries for the N900 using this Qt-based environment? (however, Qt is a good tool to create interfaces)
AR Toolkit Plus ● Marker-based augmented reality: ● Given a physical marker, this toolkit is able to compute the camera position and orientation in real time ● Hundreds of markers templates ● There are marker-less methods as well (more expensive) ● We just need a good transformation matrix.
AR Toolkit Plus ● Main challenges: ● It's an external library: really tricky on MADDE ● No longer updated since June 2006
Coding in OpengGL ES 2.0 ● OpenGL ES 2.0 ● Programmable Shaders (GLSL) ● No fixed function pipeline
OpenGL ES 2.0
Overview Objects Setup VBO Setup Texture Detect Markers, Capture estimate Render! (GStreamer) transf. matrix (ARToolkitPlus) Compile and link shaders
GStreamer Powerful pipeline-based multimedia framework ●
Experiments ● 2D Text ● Texture Overlay ● 3D Text ● Shadow Mapping 2D Text
Texture Overlay
3D text ● More immersive than the 2D approach ● Array of vertices ● Simple shader Video: 3D text
Shadow Mapping Lance Williams, Casting curved shadows on curved surfaces. ● SIGGRAPH Comput. Graph. 12, 3 (August 1978) – objects light camera
Shadow Mapping Render objects from Light POV into a buffer ● This buffer shows which regions are illuminated ● objects light camera
Shadow Mapping Render from the Camera POV using the computed depth buffer ● The (x,y,z) coordinates which aren't visible from light POV are in shadow. ● objects light camera
Shadow Mapping Video: Shadow Mapping
Main Challenges ● Tracking stability ● Limited memory and speed ● Interface-related: one hand is always holding the device ● Small community ● OpenGL ES 2.0 and GLSL sintax
Future Work ● Improve shadow mapping and 2D text ● Experiment with arbitrary complex meshes ● Acquire light position in real time ● Render soft shadows ● Marker-less AR
Special Thanks ● Luiz Velho ● Djalma Lucio ● Aldo Zang ● Leonardo Carvalho
References ● Tangible Interfaces using Handheld Augmented Reality. P. Rojtberg and A. Olwal ● OpenGL ES specs: http://www.khronos.org/opengles/spec/ ● Understanding Virtual Reality. William R. Sherman and Alan Craig
Thank you -------- tknop@impa.br www.impa.br/~tknop
Recommend
More recommend
