Augmented Reality: Fusing the Real and Synthetic Worlds Mauro - PowerPoint PPT Presentation

Augmented Reality: Fusing the Real and Synthetic Worlds Mauro Dalla Mura, Michele Zanin, Claudio Andreatta, Paul Chippendale, Fondazione Bruno Kessler, Trento, Italy

Morphological and Attribute Profiles Spectral/spatial analysis What is Augmented Reality? IEEE IGARSS’12 M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale 2

Morphological and Attribute Profiles Spectral/spatial analysis What is Augmented Reality? Can it be useful for the GRS community? IEEE IGARSS’12 M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale 3

Outline 1. Introduction to Augmented Reality 2. AR and GRS: the proposed concept 3. Augmenting the user’s view 4. Augmenting geo-data 5. Discussion and conclusion IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

Augmented Reality Augmented Reality (AR) blends virtual and real scenes in both time and space IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

Augmented Reality Augmented Reality (AR) blends virtual and real scenes in both time and space Nowadays, AR technology starts to run on consumer level mobile devices (e.g., smartphones and tablets) in real-time 1966 1950s IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

Augmented Reality Augmented Reality (AR) blends virtual and real scenes in both time and space Nowadays, AR technology starts to run on consumer level mobile devices (e.g., smartphones and tablets) in real-time 2000 1966 1950s IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

Augmented Reality Augmented Reality (AR) blends virtual and real scenes in both time and space Nowadays, AR technology starts to run on consumer level mobile devices (e.g., smartphones and tablets) in real-time 2008 2000 1966 1950s IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

Augmented Reality Augmented Reality (AR) blends virtual and real scenes in both time and space Nowadays, AR technology starts to run on consumer level mobile devices (e.g., smartphones and tablets) in real-time 2008 2014 2000 1966 1950s IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

Augmented Reality Augmented Reality (AR) blends virtual and real scenes in both time and space Nowadays, AR technology starts to run on consumer level mobile devices (e.g., smartphones and tablets) in real-time . . . 2008 2014 2000 1966 1950s IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

AR applications  Gaming IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

AR applications  Gaming  Entertainment IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

AR applications  Gaming  Entertainment  Sport IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

AR applications  Gaming  Entertainment  Sport  Commercial IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

AR applications  Gaming  Urban planning  Entertainment  Sport  Commercial IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

AR applications  Gaming  Urban planning  Entertainment  Tourism  Sport  Commercial IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

AR applications  Gaming  Urban planning  Entertainment  Tourism  Sport  Navigation  Commercial IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

AR applications  Gaming  Urban planning  Entertainment  Tourism  Sport  Navigation  Commercial  . . . . . . IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

Augmented Reality for GRS Augmented Reality exploiting geo-data  Availability of massive archives of geo-referenced data (e.g., remote sensing, geo-referenced social photo websites, GIS, geo-data) A possible exploitation of geo-data Augmentation of a 2D map superimposing the corresponding 3D model  Effective integration of AR and geo-data is becoming feasible on consumer- level portable devices IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

Augmented Reality for GRS Available applications showing geo-data on smart-phones Layar (since Jun 2009) Peak.AR (since Oct 2009) SwissPeaks (since Nov 2009) Peaks Mixare (since Mar 2010) Wikitude (since May 2009)  Device-world registration is usually not accurate (e.g., errors in horizontal orientation of 30 deg if using the on-board compass) IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

Photo-world registration Accurate alignment of the device and the surrounding panorama is necessary  Camera position ( ϕ 0 , λ 0 , h 0 ) and Captured Photograph DEM pose ( α , β , γ ) should be estimated Salient feature Panorama extraction rendering  Vision based approaches operates by finding the best match between visual features and synthetic ones (from a 3D model) 2D Rendered DEM  Hardware information (e.g., Photo magnetometer, compass, GPS) can be registration used as initial clue of the pose  After photo-world registration, each pixel in the image can be associated to geographic coordinates Aligned Photograph IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

AR for GRS: the proposed concept IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

Reality augmentation Augmenting the user’s view IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

Augmenting the user’s view  Once registered, geo-data can be delivered to the user  Geographical toponyms  GIS layers  Topographic maps  3D building models  . . .  New modality to query and access geo-data  Fast interactive modality (e.g., touchscreen) IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

Augmenting the geoscientist’s view  Support tool for field campaigns:  Real-time guidance for field surveys (e.g., ground truth collection)  In-situ validation of remote sensing data and products (e.g., land cover maps)  Aid the interpretation of geo-referenced RS data  data not in the visible range  time series Forest regions from Corine land cover 2000 (25m resolution) Topographic map IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

Reality augmentation Augmenting the geo-data archives IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

Augmenting the geo-data archives c r Digital Elevation Model (r, c)  ( ϕ , λ ) f(F, ϕ 0 , λ 0 , h 0 , α , β , γ , DEM) focal position pose length Users’ portable devices become ϕ ground-based sensors λ IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

Ground-based sensing  Extract Information on the land cover Oct. 2007 Nov. 2007 Dec. 2007 Snow cover map 1 Sept Nov Jan 0.8 November Snow (percentage) 0.6 snowline >1785 m 0.4 0.2 December snowline 0 860 1060 1260 1460 1660 1860 2060 2260 2460 2660 2860 3060 >1212 m Altitude (m) IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

Ground-based sensors  Extract Information on the land cover  Availability of multiple images on the same area   3D models with photogrammetry  Analysis of the land cover evolution IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

Ground-based sensors  Extract Information on the land cover  Availability of multiple images on the same area   3D models with photogrammetry  Analysis of the land cover evolution  Capture vertical surfaces  Massive databases of geo-tagged images (for free!)  Near-live acquisition (e.g., for damage assessment) IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale

Ground-based sensors  Extract Information on the land cover  Availability of multiple images on the same area   3D models with photogrammetry  Analysis of the land cover evolution  Capture vertical surfaces  Massive databases of geo-tagged images (for free!)  Near-live acquisition (e.g., for damage assessment)  Limited spatial coverage  Uneven resolution  Occlusions IEEE IGARSS’12 dallamura@fbk.eu M. Dalla Mura, M. Zanin, C. Andreatta, P. Chippendale