Model-Based Teeth Reconstruction Chenglei Wu 2 , Derek Bradley 1 , Pablo Garrido 3 , Michael Zollhöfer 3 , Christian Theobalt 3 , Markus Gross 1,2 , Thabo Beeler 1 1 Disney Research 2 ETH Zurich 3 Max Planck Institute for Informatics
MOTIVATION Michael Zollhöfer Michael Zollhöfer Model-Based Teeth Reconstruction Model-Based Teeth Reconstruction 2 2
MOTIVATION Input Image Beeler11 Michael Zollhöfer Michael Zollhöfer Model-Based Teeth Reconstruction Model-Based Teeth Reconstruction 3 3
MOTIVATION Berard 16 Input Image Beeler11 Michael Zollhöfer Michael Zollhöfer Model-Based Teeth Reconstruction Model-Based Teeth Reconstruction 4 4
MOTIVATION Berard 16 Bermano 16 Input Image Beeler11 Michael Zollhöfer Michael Zollhöfer Model-Based Teeth Reconstruction Model-Based Teeth Reconstruction 5 5
MOTIVATION Berard 16 Bermano 16 Beeler12 Input Image Beeler11 Michael Zollhöfer Michael Zollhöfer Model-Based Teeth Reconstruction Model-Based Teeth Reconstruction 6 6
MOTIVATION Berard 16 Bermano 16 Beeler12 Nagano15 Input Image Beeler11 Michael Zollhöfer Michael Zollhöfer Model-Based Teeth Reconstruction Model-Based Teeth Reconstruction 7 7
MOTIVATION Berard 16 Bermano 16 Beeler12 Nagano15 Input Image Beeler11 Hu14 Michael Zollhöfer Michael Zollhöfer Model-Based Teeth Reconstruction Model-Based Teeth Reconstruction 8 8
MOTIVATION Teeth are missing! Input Image Beeler11 Michael Zollhöfer Model-Based Teeth Reconstruction 9
MOTIVATION Teeth are missing! Input Image Beeler11 Michael Zollhöfer Model-Based Teeth Reconstruction 10
MOTIVATION Teeth are missing! Complex Appearance Input Image Beeler11 Michael Zollhöfer Model-Based Teeth Reconstruction 11
MOTIVATION Teeth are missing! Complex Appearance Featureless Input Image Beeler11 Michael Zollhöfer Model-Based Teeth Reconstruction 12
MOTIVATION Teeth are missing! Complex Appearance Featureless Occlusions Input Image Beeler11 Michael Zollhöfer Model-Based Teeth Reconstruction 13
MOTIVATION Teeth are missing! Complex Appearance Featureless Occlusions Input Image Our Reconstruction Michael Zollhöfer Model-Based Teeth Reconstruction 14
RELATED WORK Michael Zollhöfer Model-Based Teeth Reconstruction 15
RELATED WORK Medical Dentistry From CT images [ Omachi07 , Yanagisawa14] Michael Zollhöfer Model-Based Teeth Reconstruction 16
RELATED WORK Medical Dentistry From CT images [ Omachi07 , Yanagisawa14] Intra-Oral Scanners [3M True Definition, iTero , 3Shape TRIOS] Michael Zollhöfer Model-Based Teeth Reconstruction 17
RELATED WORK Medical Dentistry Photogrammetric Reconstruction From CT images Shape from Shading [ Omachi07 , Yanagisawa14] [Carter10, Farag13 , Mostafa14] Intra-Oral Scanners [3M True Definition, iTero , 3Shape TRIOS] Michael Zollhöfer Model-Based Teeth Reconstruction 18
RELATED WORK Medical Dentistry Photogrammetric Reconstruction From CT images Shape from Shading [ Omachi07 , Yanagisawa14] [Carter10, Farag13 , Mostafa14] Contours and Feature Points [ Zheng11 ] Intra-Oral Scanners [3M True Definition, iTero , 3Shape TRIOS] Michael Zollhöfer Model-Based Teeth Reconstruction 19
RELATED WORK Medical Dentistry Photogrammetric Reconstruction From CT images Shape from Shading [ Omachi07 , Yanagisawa14] [Carter10, Farag13 , Mostafa14] Contours and Feature Points [ Zheng11 ] Intra-Oral Scanners [3M True Definition, iTero , 3Shape TRIOS] Statistical Models [ Mehl05 , Buchaillard07] Michael Zollhöfer Model-Based Teeth Reconstruction 20
HUMAN TEETH Scans Michael Zollhöfer Model-Based Teeth Reconstruction 21
HUMAN TEETH Scans Michael Zollhöfer Model-Based Teeth Reconstruction 22
HUMAN TEETH Scans Michael Zollhöfer Model-Based Teeth Reconstruction 23
OVERVIEW Scans Michael Zollhöfer Model-Based Teeth Reconstruction 24
OVERVIEW Static Images/Video Clip Scans Michael Zollhöfer Model-Based Teeth Reconstruction 25
OVERVIEW Contour Extraction Static Images/Video Clip Scans Michael Zollhöfer Model-Based Teeth Reconstruction 26
OVERVIEW Teeth Prior Model Dental Scans Contour Extraction Static Images/Video Clip Scans Michael Zollhöfer Model-Based Teeth Reconstruction 27
OVERVIEW Teeth Prior Model Teeth Reconstruction Dental Scans Contour Extraction Static Images/Video Clip Scans Michael Zollhöfer Model-Based Teeth Reconstruction 28
OVERVIEW Teeth Prior Model Teeth Reconstruction 3D Reconstruction Dental Scans Contour Extraction Static Images/Video Clip Scans Michael Zollhöfer Model-Based Teeth Reconstruction 29
OVERVIEW Teeth Prior Model Teeth Reconstruction 3D Reconstruction Dental Scans Contour Extraction Static Images/Video Clip Scans Michael Zollhöfer Model-Based Teeth Reconstruction 30
OVERVIEW Teeth Prior Model Teeth Reconstruction 3D Reconstruction Dental Scans Contour Extraction Static Images/Video Clip Scans Michael Zollhöfer Model-Based Teeth Reconstruction 31
TEETH MODEL • 86 high resolution plaster cast scans Michael Zollhöfer Model-Based Teeth Reconstruction 32
TEETH MODEL • 86 high resolution plaster cast scans Michael Zollhöfer Model-Based Teeth Reconstruction 33
PREPROCESSING Teeth Segmentation Fitted Template Michael Zollhöfer Model-Based Teeth Reconstruction 34
PREPROCESSING Teeth Segmentation Fitted Template Input Scan Michael Zollhöfer Model-Based Teeth Reconstruction 35
PREPROCESSING Teeth Segmentation Fitted Template Input Scan Michael Zollhöfer Model-Based Teeth Reconstruction 36
PREPROCESSING Teeth Segmentation Fitted Template Input Scan Michael Zollhöfer Model-Based Teeth Reconstruction 37
PARAMETRIC TEETH MODEL 𝐣 𝐂 𝛖 𝐣 𝐚 𝛖 = 𝐓 𝑼 τ 𝐁 𝛖 + 𝛃 𝛖 𝐣 Michael Zollhöfer Model-Based Teeth Reconstruction 38
PARAMETRIC TEETH MODEL 𝐣 𝐂 𝛖 𝐣 𝐚 𝛖 = 𝐓 𝑼 τ 𝐁 𝛖 + 𝛃 𝛖 𝐣 𝐁 𝛖 Michael Zollhöfer Model-Based Teeth Reconstruction 39
PARAMETRIC TEETH MODEL 𝐣 𝐂 𝛖 𝐣 𝐚 𝛖 = 𝐓 𝑼 τ 𝐁 𝛖 + 𝛃 𝛖 𝐣 𝐁 𝛖 𝐣 𝐂 𝛖 𝐣 𝛃 𝛖 shape of tooth 𝛖 Michael Zollhöfer Model-Based Teeth Reconstruction 40
PARAMETRIC TEETH MODEL 𝐣 𝐂 𝛖 𝐣 𝐚 𝛖 = 𝐓 𝑼 τ 𝐁 𝛖 + 𝛃 𝛖 𝐣 𝐁 𝛖 𝑼 τ 𝐣 𝐂 𝛖 𝐣 𝛃 𝛖 position of tooth 𝛖 shape of tooth 𝛖 Michael Zollhöfer Model-Based Teeth Reconstruction 41
PARAMETRIC TEETH MODEL 𝐣 𝐂 𝛖 𝐣 𝐚 𝛖 = 𝐓 𝑼 τ 𝐁 𝛖 + 𝛃 𝛖 𝐣 𝐁 𝛖 𝐓 𝑼 τ 𝐣 𝐂 𝛖 𝐣 𝛃 𝛖 Scale and position position of tooth 𝛖 shape of tooth 𝛖 Michael Zollhöfer Model-Based Teeth Reconstruction 42
GAUSSIAN DISTRIBUTION 𝐣 }) 𝒀 = (𝐓 , 𝑼 τ , {𝛃 𝛖 Michael Zollhöfer Model-Based Teeth Reconstruction 43
GAUSSIAN DISTRIBUTION • All dimensions modeled via multivariate Gaussians 𝐣 }) 𝒀 = (𝐓 , 𝑼 τ , {𝛃 𝛖 Michael Zollhöfer Model-Based Teeth Reconstruction 44
GAUSSIAN DISTRIBUTION • All dimensions modeled via multivariate Gaussians 𝐣 }) 𝒀 = (𝐓 , 𝑼 τ , {𝛃 𝛖 Michael Zollhöfer Model-Based Teeth Reconstruction 45
GAUSSIAN DISTRIBUTION • All dimensions modeled via multivariate Gaussians 𝐣 }) 𝒀 = (𝐓 , 𝑼 τ , {𝛃 𝛖 𝒒 𝒀 Teeth Prior Michael Zollhöfer Model-Based Teeth Reconstruction 46
OVERVIEW Teeth Prior Model Teeth Reconstruction 3D Reconstruction Dental Scans Contour Extraction Static Images/Video Clip Scans Michael Zollhöfer Model-Based Teeth Reconstruction 47
OVERVIEW Teeth Prior Model Teeth Reconstruction 3D Reconstruction Dental Scans Contour Extraction Static Images/Video Clip Scans Michael Zollhöfer Model-Based Teeth Reconstruction 48
CONTOUR EXTRACTION Scans Michael Zollhöfer Model-Based Teeth Reconstruction 49
CONTOUR EXTRACTION • Teeth are featureless Scans Michael Zollhöfer Model-Based Teeth Reconstruction 50
CONTOUR EXTRACTION • Teeth are featureless Except the silhouette Scans Michael Zollhöfer Model-Based Teeth Reconstruction 51
CONTOUR EXTRACTION • Teeth are featureless Except the silhouette . Scans Michael Zollhöfer Model-Based Teeth Reconstruction 52
CONTOUR EXTRACTION • Teeth are featureless Except the silhouette . Boosted Edge Learning (BEL) Scans Michael Zollhöfer Model-Based Teeth Reconstruction 53
FEATURE FUSION Scans Michael Zollhöfer Model-Based Teeth Reconstruction 54
FEATURE FUSION Input Scans Michael Zollhöfer Model-Based Teeth Reconstruction 55
FEATURE FUSION BEL Input Scans Michael Zollhöfer Model-Based Teeth Reconstruction 56
FEATURE FUSION BEL Input Gabor Scans Michael Zollhöfer Model-Based Teeth Reconstruction 57
FEATURE FUSION BEL Fusion Input Gabor Scans Michael Zollhöfer Model-Based Teeth Reconstruction 58
FEATURE FUSION Formulas in BEL the paper! Fusion Input Gabor Scans Michael Zollhöfer Model-Based Teeth Reconstruction 59
Recommend
More recommend