1 Aug me nte d Re a lity Sung -e ui Yo o n
Project Guidelines: Project Topics ● Any topics related to the course theme are okay ● You can find topics by browsing recent papers 2
Expectations ● Mid-term project presentation ● Introduce problems and explain why it is important ● Give an overall idea on the related work ● Explain what problems those existing techniques have ● (Optional) explain how you can address those problems ● Explain roles of each member ● Ack. any external slides/codes, etc. 3
Expectations ● Final-term project presentation ● Review materials that you talked for your mid-term project ● Present your ideas that can address problems of those state-of- the-art techniques ● Give your qualitatively (or intuitive) reasons how your ideas address them ● Also, explain expected benefits and drawbacks of your approach ● (Optional) backup your claims with quantitative results collected by some implementations ● Explain roles of each members ● Ack. any external slides/codes, etc. 4
A few more comments ● Start to implement a paper, if you don’t have any clear ideas ● While you implement it, you may get ideas about improving it 5
Project evaluation sheet You name: ID: Score table: higher score is better. Role of each student is clear and well Completeness Novelty of the Practical benefits Total balanced? level of the Speaker project and idea of the method score (Yes or No) project (1 ~ 5) (1 ~ 5) (3 ~ 15) (1 ~ 5) XXX YYY 6
Go a ls fo r T o da y Goals • Ge tting a c q ua inte d with AR • Ma king yo ur o wn Aug me nte d Re a lity • So me o f re c e nt te c hniq ue s 7
Wha t is Aug me nte d Re a lity? Re a l Wo rld Virtua l Aug me nte d E nviro nme nt Re a lity Re a lity Re a l Wo rld Virtua l Aug me nte d E nviro nme nt Re a lity Re a lity Milgram, Paul, et al. "Augmented reality: A class of displays on the reality-virtuality continuum." Photonics for industrial applications . International Society for Optics and Photonics, 1995. Photo Source: http://smartideasblog.trekk.com/augmented-or-virtual-how-do-you-like-your-reality 8
Brie f Histo ry • 1968 I va n Suthe rla nd inve nts first he a d-mo unte d displa y “Swo rd-o f-Da mo c le s” a t Unive rsity o f Uta h. • 2010 Vufo ria fo r AR Mo b ile Apps wa s re le a se d b y Qua lc o mm. • 2013 Go o g le a nno unc e s Go o g le Gla ss. • 2015 Mic ro so ft a nno unc e s the Ho lo L e ns. • 2016 Nia ntic re le a se d Po ké mo n Go . Original slides are from Frend, Chauncey. ”Augmented Reality & the UITS Advanced Visualization Lab." 9
AR to da y – I ndustry E xa mple s AR Color ing Book (0:00-0:30) HoloL e ns Live Texturing of Augmented Reality Augme nte d Climbing Wall Characters from Colored Drawings IEEE International Symposium on Mixed and (0:00-0:33) Augmented Reality (ISMAR) 2015 10
dedicated workshops on Mar. 24 th & 31st Building E xpe rie nc e s Asse ts + Displa y + I nte rfa c e Sour c ing or Building Data • • Yo u c a n a lso a ug me nt with me dia 3D Scanning – Audio 3D Scanning Photogrammetry – 2D me dia (I ma g e & vide o ) • 3D Authoring – T e xt Autodesk Maya or Max Sketchup TinkerCAD • 3D Repositories Sketchup 3D Warehouse Smithsonian X 3D https://3d.si.edu/browser Sketchfab* Thingiverse* 11
Building E xpe rie nc e s Asse ts + Display + I nte rfa c e Mobile De vic e s PC or Mac with We bc am HoloL e ns He adse t 12
Building E xpe rie nc e s Asse ts + Displa y + Inte r fac e Onboar ding AR – Pr ototyping T ools No pro g ra mming re q uire d 13
Building E xpe rie nc e s Asse ts + Displa y + Inte r fac e De ve lopme nt T ools AR T e c hnology Base Pac kage AR Plugins • Mo b ile De vic e Apps • I ma g e o r Ob je c t Ma rke rs • De skto p o r L a pto p Apps • F iduc ia l o r I ma g e Ma rke rs • Ho lo L e ns de vic e Apps • Are a L e a rning o r I ma g e Ma rke rs 14
Wha t the future ma y ho ld Cybe r Physic al AR We b Hoste d AR Applic ations Consume r -gr ade he adse ts Applic ations (2:16-3:04) 15
Re so urc e s fo r yo u! Base Pac kage T uto ria ls https:/ / unity3d.c o m/ le a rn AR Plugins Vuforia Basics Tutorial By: Chauncey https://www.youtube.com/watch?v=qbI9PrSUo5w AR YouTube Playlist 16
Technical Components ● Robotics parts ● Simultaneous localization and mapping (SLAM) ● Computer vision parts ● Geometry, light, and materials estimation ● Graphics parts ● Rendering virtual objects ● Machine learning ● Understand various things ● AR spans various fields 17
Modeling Surface Appearance from a Single Photography using Self-Augmentation [Li et al.] ● Assume the anisotropic Ward BRDF model ● Diffuse albedo and normal map per each pixel ● Specular parameters for each image Ground truth Reconstructed ones Original images with relighted images 18
Network Architectures ● Two separate architectures ● Self-augmentation ● Use unlabeled image and reconstruct parameters ● Generate its corresponding image ● Use them as training pairs with labeled ones 19
LIME: Live Intrinsic Material Estimation [Meka et al. CVPR 18] ● Estimate specular information of an object in the RGB image ● Starts with the rendering equation, but ends up with assuming the Phone illumination: diffuse and specular terms 20
Real-Time Geometry, Albedo, and Motion Reconstruction Using a Single RGB-D Camera Kaiwen Guo 1 , Feng Xu 1 , Tao Yu 1,2 , Xiaoyang Liu 1 , Qionghai Dai 1 , Yebin Liu 1 Tsinghua University 1 Beihang University 2 21
Offline Volumetric Performance Capture [Starck and Hilton, 2007], [Liu et al., [Vlasic et al. 2009], [Debevec, Light Stage], [Collet et al. 2015] 2009] 22
Real-time Volumetric Performance Capture [Fusion4D, Dou et al. 2016], [Holoportation, Orts‐Escolano et al. 2016] 23
Real-time Single-view Volumetric Capture 24
Key Idea: Joint Optimization considering Shading Geometry & Appearance Motion (diffuse objects) 25
Overview Color Depth Albedo (color) Geometry Motion Assume diffuse objects Input Output 26
27
28
29
30
31
32
Near-Eye Light Field Display [ToG 13] ● Use a microlens array for supporting the light field 33
Goals for Today were: ● Getting acquainted with AR ● Making your own Augmented Reality ● Some of recent techniques 34
Recommend
More recommend
