Spatial Reconstruction Using Microsoft HoloLens GUPTA Aman ZAFAR Waleed
AGENDA ▫ Introduction ▫ Application Design ▫ Results ▫ Looking Back ▫ Moving Forward ▫ Conclusion ▫ Feedback & Demo HISS Holo Indoor Spatial Scanner 2
WHAT WHY To Capture A Challenge WHAT? To Reconstruct An Opportunity WHY? To Visualize Tackling a Real Problem WHO? WHO Interior Designers and VR Environment Real-Estate Agents Engineers Developers 3
Existing Technologies LIDAR Microsoft Kinect Fusion Google Tango 4
The HoloLens ▫ Spatial Understanding ▫ Mobility ▫ Accuracy ▫ Cost Effectiveness ▫ 3D Visualization 5 5
Software and Technologies
Application Design HISS: Holo Indoor Spatial Scanner
The Deliverables HoloLens Record & Visualize Processing Server Process and Store Meshes on the Cloud Web Portal Interact with the Server HoloLens Mesh Web Portal Application Processing Server 8
HoloLens Application A modular approach. Recording Library Takes care of the recording Populates list of available and mesh generation blobs from Azure Storage features using Spatial Understanding Prefab Input/Output Visualization Handles the serialization and Handles the modelling and deserialization of OBJ files manipulation of generated from Mesh models. 9
The Recording Module Uses the Spatial Understanding DLLs built by Microsoft Gesture and Speech Input Helpful Mesh Insights Minimum Criteria for Mesh Quality 10 10
Visualization View The Modelling Module 11
The Processing Server System Design CGAL Library Azure Storage and Functions RANSAC Algorithm Advancing Front Surface Reconstruction Poisson Surface Reconstruction Web Portal
PRPU Design Core C++ processing library ▫ Managed Interface ▫ Azure Function App ▫ 13 13
RANSAC Algorithm Point set shape detection ▫ Plane detection using point and ▫ normal set Outer hull reconstruction using point ▫ set with oriented normal 14 14
Advancing Front Surface Reconstruction ▪ RANSAC for Plane Identification ▪ Priority Structure Functor ▪ Advancing Front Reconstruction
Poisson Surface Reconstruction Operates on 3D point sets with ▫ oriented normal Computes an implicit Poisson function ▫ and extracts an isosurface Doesn’t handle sharp features, noise ▫ or outliers well Use for interior element ▫ reconstruction 16 16
Processing System Flow Shape Surface File Read File Write Detection Reconstruction Push to Azure Parse arguments RANSAC Advancing Front Point Set Shape Poisson Detection Outer Hull Construction
Web Portal 18
Spatial Reconstruction Visualize Record Process Download Upload 19
Results
Dirk’s Office Original Scan Processed Model
Originals Results
Results HLA Web Portal Processing Server Mesh Recording Server Connection Processing Algorithms Visualization & Modelling Server Connection Front End IO Module Throughput Response
Looking Back Moving Forward ▫ Enhance Feasibility Study ▫ Integration of Classification Algorithms (Machine Learning) ▫ Better Integration ▫ Incorporate Textures and Coloring of ▫ Better User Interface surfaces ▫ Manual capture of model features 24 24
CONCLUSION Spatial Reconstruction is possible! The proof of concept works! We implemented various modules and interactions with multiple systems! The system has amazing applications! We need to refine the system! It must be implemented fully to become user acceptable!
Thank You! Demo & Questions 26
27
28
29
Dirk’s Office Advancing Front 30
Recommend
More recommend
