<latexit sha1_base64="63ak468AXCor9nBfBLOpfhlLeqM=">AB7XicbZDLSgMxFIbP1Fut6pLN8Ei1E2ZKQVdFt24s4K9QDuWTJpYzPJkGSEMvQd3LhQxK3v4863MW1noa0/BD7+cw45w9izrRx3W8nt7a+sbmV3y7s7O7tHxQPj1paJorQJpFcqk6ANeVM0KZhtNOrCiOAk7bwfh6Vm8/UaWZFPdmElM/wkPBQkawsVbrtiwequf9YsmtuHOhVfAyKEGmRr/41RtIkRUGMKx1l3PjY2fYmUY4XRa6CWaxpiM8ZB2LQocUe2n82n6Mw6AxRKZ8waO7+nkhxpPUkCmxnhM1IL9dm5n+1bmLCSz9lIk4MFWTxUZhwZCSanY4GTFi+MQCJorZXREZYWJsQEVbAje8smr0KpWPMt3tVL9KosjDydwCmXw4ALqcAMNaAKBR3iGV3hzpPivDsfi9ack80cwx85nz9g645U</latexit> <latexit sha1_base64="63ak468AXCor9nBfBLOpfhlLeqM=">AB7XicbZDLSgMxFIbP1Fut6pLN8Ei1E2ZKQVdFt24s4K9QDuWTJpYzPJkGSEMvQd3LhQxK3v4863MW1noa0/BD7+cw45w9izrRx3W8nt7a+sbmV3y7s7O7tHxQPj1paJorQJpFcqk6ANeVM0KZhtNOrCiOAk7bwfh6Vm8/UaWZFPdmElM/wkPBQkawsVbrtiwequf9YsmtuHOhVfAyKEGmRr/41RtIkRUGMKx1l3PjY2fYmUY4XRa6CWaxpiM8ZB2LQocUe2n82n6Mw6AxRKZ8waO7+nkhxpPUkCmxnhM1IL9dm5n+1bmLCSz9lIk4MFWTxUZhwZCSanY4GTFi+MQCJorZXREZYWJsQEVbAje8smr0KpWPMt3tVL9KosjDydwCmXw4ALqcAMNaAKBR3iGV3hzpPivDsfi9ack80cwx85nz9g645U</latexit> <latexit sha1_base64="63ak468AXCor9nBfBLOpfhlLeqM=">AB7XicbZDLSgMxFIbP1Fut6pLN8Ei1E2ZKQVdFt24s4K9QDuWTJpYzPJkGSEMvQd3LhQxK3v4863MW1noa0/BD7+cw45w9izrRx3W8nt7a+sbmV3y7s7O7tHxQPj1paJorQJpFcqk6ANeVM0KZhtNOrCiOAk7bwfh6Vm8/UaWZFPdmElM/wkPBQkawsVbrtiwequf9YsmtuHOhVfAyKEGmRr/41RtIkRUGMKx1l3PjY2fYmUY4XRa6CWaxpiM8ZB2LQocUe2n82n6Mw6AxRKZ8waO7+nkhxpPUkCmxnhM1IL9dm5n+1bmLCSz9lIk4MFWTxUZhwZCSanY4GTFi+MQCJorZXREZYWJsQEVbAje8smr0KpWPMt3tVL9KosjDydwCmXw4ALqcAMNaAKBR3iGV3hzpPivDsfi9ack80cwx85nz9g645U</latexit> <latexit sha1_base64="63ak468AXCor9nBfBLOpfhlLeqM=">AB7XicbZDLSgMxFIbP1Fut6pLN8Ei1E2ZKQVdFt24s4K9QDuWTJpYzPJkGSEMvQd3LhQxK3v4863MW1noa0/BD7+cw45w9izrRx3W8nt7a+sbmV3y7s7O7tHxQPj1paJorQJpFcqk6ANeVM0KZhtNOrCiOAk7bwfh6Vm8/UaWZFPdmElM/wkPBQkawsVbrtiwequf9YsmtuHOhVfAyKEGmRr/41RtIkRUGMKx1l3PjY2fYmUY4XRa6CWaxpiM8ZB2LQocUe2n82n6Mw6AxRKZ8waO7+nkhxpPUkCmxnhM1IL9dm5n+1bmLCSz9lIk4MFWTxUZhwZCSanY4GTFi+MQCJorZXREZYWJsQEVbAje8smr0KpWPMt3tVL9KosjDydwCmXw4ALqcAMNaAKBR3iGV3hzpPivDsfi9ack80cwx85nz9g645U</latexit> <latexit sha1_base64="pPG/+UG2ht75Yz5Ajs7KOd+Azk=">AB7XicbZDLSgMxFIbP1Fut6pLN8Ei1E2ZsYIui27cWcFeoB1LJs20sZlkSDJCGfoOblwo4tb3cefbmLaz0NYfAh/OYec8wcxZ9q47reTW1ldW9/Ibxa2tnd294r7B0tE0Vog0guVTvAmnImaMw2k7VhRHAaetYHQ9rbeqNJMinszjqkf4YFgISPYWKt5WxYP1dNeseRW3JnQMngZlCBTvVf86vYlSIqDOFY647nxsZPsTKMcDopdBNY0xGeEA7FgWOqPbT2bYTdGKdPgqlsk8YNHN/T6Q40nocBbYzwmaoF2tT879aJzHhpZ8yESeGCjL/KEw4MhJNT0d9pigxfGwBE8XsrogMscLE2IAKNgRv8eRlaJ5VPMt356XaVRZHo7gGMrgwQXU4Abq0ACj/AMr/DmSOfFeXc+5q05J5s5hD9yPn8AYnCOVQ=</latexit> <latexit sha1_base64="pPG/+UG2ht75Yz5Ajs7KOd+Azk=">AB7XicbZDLSgMxFIbP1Fut6pLN8Ei1E2ZsYIui27cWcFeoB1LJs20sZlkSDJCGfoOblwo4tb3cefbmLaz0NYfAh/OYec8wcxZ9q47reTW1ldW9/Ibxa2tnd294r7B0tE0Vog0guVTvAmnImaMw2k7VhRHAaetYHQ9rbeqNJMinszjqkf4YFgISPYWKt5WxYP1dNeseRW3JnQMngZlCBTvVf86vYlSIqDOFY647nxsZPsTKMcDopdBNY0xGeEA7FgWOqPbT2bYTdGKdPgqlsk8YNHN/T6Q40nocBbYzwmaoF2tT879aJzHhpZ8yESeGCjL/KEw4MhJNT0d9pigxfGwBE8XsrogMscLE2IAKNgRv8eRlaJ5VPMt356XaVRZHo7gGMrgwQXU4Abq0ACj/AMr/DmSOfFeXc+5q05J5s5hD9yPn8AYnCOVQ=</latexit> <latexit sha1_base64="pPG/+UG2ht75Yz5Ajs7KOd+Azk=">AB7XicbZDLSgMxFIbP1Fut6pLN8Ei1E2ZsYIui27cWcFeoB1LJs20sZlkSDJCGfoOblwo4tb3cefbmLaz0NYfAh/OYec8wcxZ9q47reTW1ldW9/Ibxa2tnd294r7B0tE0Vog0guVTvAmnImaMw2k7VhRHAaetYHQ9rbeqNJMinszjqkf4YFgISPYWKt5WxYP1dNeseRW3JnQMngZlCBTvVf86vYlSIqDOFY647nxsZPsTKMcDopdBNY0xGeEA7FgWOqPbT2bYTdGKdPgqlsk8YNHN/T6Q40nocBbYzwmaoF2tT879aJzHhpZ8yESeGCjL/KEw4MhJNT0d9pigxfGwBE8XsrogMscLE2IAKNgRv8eRlaJ5VPMt356XaVRZHo7gGMrgwQXU4Abq0ACj/AMr/DmSOfFeXc+5q05J5s5hD9yPn8AYnCOVQ=</latexit> <latexit sha1_base64="pPG/+UG2ht75Yz5Ajs7KOd+Azk=">AB7XicbZDLSgMxFIbP1Fut6pLN8Ei1E2ZsYIui27cWcFeoB1LJs20sZlkSDJCGfoOblwo4tb3cefbmLaz0NYfAh/OYec8wcxZ9q47reTW1ldW9/Ibxa2tnd294r7B0tE0Vog0guVTvAmnImaMw2k7VhRHAaetYHQ9rbeqNJMinszjqkf4YFgISPYWKt5WxYP1dNeseRW3JnQMngZlCBTvVf86vYlSIqDOFY647nxsZPsTKMcDopdBNY0xGeEA7FgWOqPbT2bYTdGKdPgqlsk8YNHN/T6Q40nocBbYzwmaoF2tT879aJzHhpZ8yESeGCjL/KEw4MhJNT0d9pigxfGwBE8XsrogMscLE2IAKNgRv8eRlaJ5VPMt356XaVRZHo7gGMrgwQXU4Abq0ACj/AMr/DmSOfFeXc+5q05J5s5hD9yPn8AYnCOVQ=</latexit> What’s Different in 3D? • Number of Voxels grows as versus occupied surface O ( n 3 ) O ( n 2 ) �7
Data Representation .. Many Possibilities! points �8
Data Representation .. Many Possibilities! points voxels cells patches �8
Challenges �9
Challenges 1. Representation �9
Challenges 1. Representation 2. Neighborhood information • who are the neighbouring elements • how are the elements ordered �9
Challenges 1. Representation 2. Neighborhood information • who are the neighbouring elements • how are the elements ordered 3. Extrinsic versus intrinsic representation �9
Challenges 1. Representation 2. Neighborhood information • who are the neighbouring elements • how are the elements ordered 3. Extrinsic versus intrinsic representation 4. Simplicity versus memory/runtime tradeoff �9
Representation for 3D • Image-based • Volumetric • Surface-based • Point-based �10
Representation for 3D • Image-based • Volumetric • Surface-based • Point-based �11
Representation for 3D: Multi-view CNN [Kalogerakis et al. 2015] �12
Representation for 3D: Multi-view CNN [Kalogerakis et al. 2015] �12
Representation for 3D: Multi-view CNN [Kalogerakis et al. 2015] �12
Representation for 3D: Multi-view CNN regular image analysis networks [Kalogerakis et al. 2015] �12
3DMV: Joint 3D-Multi-View Prediction for 3D Semantic Scene Segmentation �13
Integrating View Information �14
Representation for 3D: Local Multi-view CNN [Huang et al. 2018] �15
Representation for 3D: Local Multi-view CNN Segmentation Correspondence Feature matching Predicting semantic functions [Huang et al. 2018] �15
Representation for 3D: Local Multi-view CNN Segmentation Correspondence Feature matching Predicting semantic functions [Huang et al. 2018] localized renderings for point-wise features �15
Tangent Convolutions loses information due to occlusion [Tatarchenko et al. 2018] �16
Tangent Convolutions loses information due to occlusion project to local patches (contrast with PCPNet construction) [Tatarchenko et al. 2018] �16
Tangent Convolutions loses information due to occlusion project to local patches (contrast with PCPNet construction) [Tatarchenko et al. 2018] �16
Dealing with Sparse Points �17
Dealing with Sparse Points �18
Improved Performance �19
Representation for 3D • Image-based • PROS: directly use image networks, good performance • CONS: rendering is slow and memory-heavy, not very geometric • Volumetric • Point-based • Surface-based �20
Representation for 3D • Image-based • Volumetric • Surface-based • Point-based �21
3D CNNs : Direct Approach [Xiao et al. 2014] �22
VoxNet [Maturana et al. 15] ▸ Binary occupancy, density grid, etc. rotational invariance �23
Visualization of First Level Filters �24
Visualization of First Level Filters �24
Representation for 3D: Volumetric Deformation [Yumer and Mitra 2016] �25
Representation for 3D: Volumetric Deformation [Yumer and Mitra 2016] �25
Efficient Volumetric Datastructures [Wang et al. 2017] �26
Data Structure and CNN Operations shuffled keys labels downsampling example (encode position in space) (parent label → child indices) ( “where there is an octant, there is CNN computation” ) faster neighbor access �27
Data Structure and CNN Operations shuffled keys labels downsampling example (encode position in space) (parent label → child indices) ( “where there is an octant, there is CNN computation” ) faster neighbor access �27
Data Structure and CNN Operations shuffled keys labels downsampling example (encode position in space) (parent label → child indices) ( “where there is an octant, there is CNN computation” ) faster neighbor access �27
Efficient Volumetric Datastructures Encoder Decoder/generator Wang et al. 2017 only generate non-empty voxels [Hane et al. 2018] �28
Efficient Volumetric Datastructures [Hane et al. 2018] �29
Lower Memory Footprint �30
Adaptive O-CNN [Wang et al. 2018] image to planar patch-based shapes
First-order Patches OCNN Adaptive OCNN �32
Field Probing Neural Networks for 3D Data [Li et al. 2016] �33
Field Probing Neural Networks for 3D Data [Li et al. 2016] �33
Spatial Probes �34
Spatial Probes �34
Spatial Probes �34
Method Details �35
Method Details �35
Method Details �35
Method Details �35
Representation for 3D • Image-based • Volumetric • PROS: adaptations of image networks • CONS: special layers for hierarchical datastructures, still too coarse • Surface-based • Point-based �36
Representation for 3D • Image-based • Volumetric • Surface-based • Point-based �37
Local/Global Parameterizations �38
Local/Global Parameterizations Geometry Image [Sinha et al. 2016] �38
Local/Global Parameterizations Geometry Image Metric Alignment (GWCNN) [Sinha et al. 2016] [Ezuz et al. 2017] �38
Shape Surfaces using Geometry Images �39
Shape Surfaces using Geometry Images �39
Shape Surfaces using Geometry Images �39
Using Geodesic Patches: GCNN [Masci et al. 2015] �40
<latexit sha1_base64="TquvtCTwYBxPgA56TL/7lpYdxY0=">ACO3icbZDLSxBEMZ7NA8z5rHRYy6FizBDRGYkEAkIknjwaIKrws6y1PTWuI09D7prxGXY/yuX/BO5eIlh0jINf07s4hPgoafnxfFdX1pZVWlqPoh7e0/Ojxk6crz/zV5y9evuq8XjuxZW0k9WSpS3OWoiWtCuqxYk1nlSHMU02n6cWnmX96ScaqsjmSUWDHM8LlSmJ7KRh50uQWIZDWAIwVUIH/bAT2ydD5uEx8S4ZaAwYLhLSQHpB20HpjQDw7cWBYGZmshsNON9qO5gX3IW6hK9o6Gna+J6NS1jkVLDVa24+jigcNGlZS09RPaksVygs8p7DAnOyg2Z+xQ2nTKCrDTuFQxz9f+JBnNrJ3nqOnPksb3rzcSHvH7N2e6gUVMxVysSirNXAJsyBhpAxJ1hMHKI1yfwU5RoOSXdy+CyG+e/J9ONnZjh1/ftfd/9jGsSLeiA0RiFi8F/viUByJnpDiq7gWv8SN98376f32/ixal7x2Zl3cKu/vP2AxqYM=</latexit> <latexit sha1_base64="TquvtCTwYBxPgA56TL/7lpYdxY0=">ACO3icbZDLSxBEMZ7NA8z5rHRYy6FizBDRGYkEAkIknjwaIKrws6y1PTWuI09D7prxGXY/yuX/BO5eIlh0jINf07s4hPgoafnxfFdX1pZVWlqPoh7e0/Ojxk6crz/zV5y9evuq8XjuxZW0k9WSpS3OWoiWtCuqxYk1nlSHMU02n6cWnmX96ScaqsjmSUWDHM8LlSmJ7KRh50uQWIZDWAIwVUIH/bAT2ydD5uEx8S4ZaAwYLhLSQHpB20HpjQDw7cWBYGZmshsNON9qO5gX3IW6hK9o6Gna+J6NS1jkVLDVa24+jigcNGlZS09RPaksVygs8p7DAnOyg2Z+xQ2nTKCrDTuFQxz9f+JBnNrJ3nqOnPksb3rzcSHvH7N2e6gUVMxVysSirNXAJsyBhpAxJ1hMHKI1yfwU5RoOSXdy+CyG+e/J9ONnZjh1/ftfd/9jGsSLeiA0RiFi8F/viUByJnpDiq7gWv8SN98376f32/ixal7x2Zl3cKu/vP2AxqYM=</latexit> <latexit sha1_base64="TquvtCTwYBxPgA56TL/7lpYdxY0=">ACO3icbZDLSxBEMZ7NA8z5rHRYy6FizBDRGYkEAkIknjwaIKrws6y1PTWuI09D7prxGXY/yuX/BO5eIlh0jINf07s4hPgoafnxfFdX1pZVWlqPoh7e0/Ojxk6crz/zV5y9evuq8XjuxZW0k9WSpS3OWoiWtCuqxYk1nlSHMU02n6cWnmX96ScaqsjmSUWDHM8LlSmJ7KRh50uQWIZDWAIwVUIH/bAT2ydD5uEx8S4ZaAwYLhLSQHpB20HpjQDw7cWBYGZmshsNON9qO5gX3IW6hK9o6Gna+J6NS1jkVLDVa24+jigcNGlZS09RPaksVygs8p7DAnOyg2Z+xQ2nTKCrDTuFQxz9f+JBnNrJ3nqOnPksb3rzcSHvH7N2e6gUVMxVysSirNXAJsyBhpAxJ1hMHKI1yfwU5RoOSXdy+CyG+e/J9ONnZjh1/ftfd/9jGsSLeiA0RiFi8F/viUByJnpDiq7gWv8SN98376f32/ixal7x2Zl3cKu/vP2AxqYM=</latexit> <latexit sha1_base64="TquvtCTwYBxPgA56TL/7lpYdxY0=">ACO3icbZDLSxBEMZ7NA8z5rHRYy6FizBDRGYkEAkIknjwaIKrws6y1PTWuI09D7prxGXY/yuX/BO5eIlh0jINf07s4hPgoafnxfFdX1pZVWlqPoh7e0/Ojxk6crz/zV5y9evuq8XjuxZW0k9WSpS3OWoiWtCuqxYk1nlSHMU02n6cWnmX96ScaqsjmSUWDHM8LlSmJ7KRh50uQWIZDWAIwVUIH/bAT2ydD5uEx8S4ZaAwYLhLSQHpB20HpjQDw7cWBYGZmshsNON9qO5gX3IW6hK9o6Gna+J6NS1jkVLDVa24+jigcNGlZS09RPaksVygs8p7DAnOyg2Z+xQ2nTKCrDTuFQxz9f+JBnNrJ3nqOnPksb3rzcSHvH7N2e6gUVMxVysSirNXAJsyBhpAxJ1hMHKI1yfwU5RoOSXdy+CyG+e/J9ONnZjh1/ftfd/9jGsSLeiA0RiFi8F/viUByJnpDiq7gWv8SN98376f32/ixal7x2Zl3cKu/vP2AxqYM=</latexit> Using Geodesic Patches: GCNN X ( f ? a )( x ) := a ( ✓ + ∆ ✓ , r )( D ( x ) f )( r, ✓ ) [Masci et al. 2015] θ ,r �40
<latexit sha1_base64="TquvtCTwYBxPgA56TL/7lpYdxY0=">ACO3icbZDLSxBEMZ7NA8z5rHRYy6FizBDRGYkEAkIknjwaIKrws6y1PTWuI09D7prxGXY/yuX/BO5eIlh0jINf07s4hPgoafnxfFdX1pZVWlqPoh7e0/Ojxk6crz/zV5y9evuq8XjuxZW0k9WSpS3OWoiWtCuqxYk1nlSHMU02n6cWnmX96ScaqsjmSUWDHM8LlSmJ7KRh50uQWIZDWAIwVUIH/bAT2ydD5uEx8S4ZaAwYLhLSQHpB20HpjQDw7cWBYGZmshsNON9qO5gX3IW6hK9o6Gna+J6NS1jkVLDVa24+jigcNGlZS09RPaksVygs8p7DAnOyg2Z+xQ2nTKCrDTuFQxz9f+JBnNrJ3nqOnPksb3rzcSHvH7N2e6gUVMxVysSirNXAJsyBhpAxJ1hMHKI1yfwU5RoOSXdy+CyG+e/J9ONnZjh1/ftfd/9jGsSLeiA0RiFi8F/viUByJnpDiq7gWv8SN98376f32/ixal7x2Zl3cKu/vP2AxqYM=</latexit> <latexit sha1_base64="TquvtCTwYBxPgA56TL/7lpYdxY0=">ACO3icbZDLSxBEMZ7NA8z5rHRYy6FizBDRGYkEAkIknjwaIKrws6y1PTWuI09D7prxGXY/yuX/BO5eIlh0jINf07s4hPgoafnxfFdX1pZVWlqPoh7e0/Ojxk6crz/zV5y9evuq8XjuxZW0k9WSpS3OWoiWtCuqxYk1nlSHMU02n6cWnmX96ScaqsjmSUWDHM8LlSmJ7KRh50uQWIZDWAIwVUIH/bAT2ydD5uEx8S4ZaAwYLhLSQHpB20HpjQDw7cWBYGZmshsNON9qO5gX3IW6hK9o6Gna+J6NS1jkVLDVa24+jigcNGlZS09RPaksVygs8p7DAnOyg2Z+xQ2nTKCrDTuFQxz9f+JBnNrJ3nqOnPksb3rzcSHvH7N2e6gUVMxVysSirNXAJsyBhpAxJ1hMHKI1yfwU5RoOSXdy+CyG+e/J9ONnZjh1/ftfd/9jGsSLeiA0RiFi8F/viUByJnpDiq7gWv8SN98376f32/ixal7x2Zl3cKu/vP2AxqYM=</latexit> <latexit sha1_base64="TquvtCTwYBxPgA56TL/7lpYdxY0=">ACO3icbZDLSxBEMZ7NA8z5rHRYy6FizBDRGYkEAkIknjwaIKrws6y1PTWuI09D7prxGXY/yuX/BO5eIlh0jINf07s4hPgoafnxfFdX1pZVWlqPoh7e0/Ojxk6crz/zV5y9evuq8XjuxZW0k9WSpS3OWoiWtCuqxYk1nlSHMU02n6cWnmX96ScaqsjmSUWDHM8LlSmJ7KRh50uQWIZDWAIwVUIH/bAT2ydD5uEx8S4ZaAwYLhLSQHpB20HpjQDw7cWBYGZmshsNON9qO5gX3IW6hK9o6Gna+J6NS1jkVLDVa24+jigcNGlZS09RPaksVygs8p7DAnOyg2Z+xQ2nTKCrDTuFQxz9f+JBnNrJ3nqOnPksb3rzcSHvH7N2e6gUVMxVysSirNXAJsyBhpAxJ1hMHKI1yfwU5RoOSXdy+CyG+e/J9ONnZjh1/ftfd/9jGsSLeiA0RiFi8F/viUByJnpDiq7gWv8SN98376f32/ixal7x2Zl3cKu/vP2AxqYM=</latexit> <latexit sha1_base64="TquvtCTwYBxPgA56TL/7lpYdxY0=">ACO3icbZDLSxBEMZ7NA8z5rHRYy6FizBDRGYkEAkIknjwaIKrws6y1PTWuI09D7prxGXY/yuX/BO5eIlh0jINf07s4hPgoafnxfFdX1pZVWlqPoh7e0/Ojxk6crz/zV5y9evuq8XjuxZW0k9WSpS3OWoiWtCuqxYk1nlSHMU02n6cWnmX96ScaqsjmSUWDHM8LlSmJ7KRh50uQWIZDWAIwVUIH/bAT2ydD5uEx8S4ZaAwYLhLSQHpB20HpjQDw7cWBYGZmshsNON9qO5gX3IW6hK9o6Gna+J6NS1jkVLDVa24+jigcNGlZS09RPaksVygs8p7DAnOyg2Z+xQ2nTKCrDTuFQxz9f+JBnNrJ3nqOnPksb3rzcSHvH7N2e6gUVMxVysSirNXAJsyBhpAxJ1hMHKI1yfwU5RoOSXdy+CyG+e/J9ONnZjh1/ftfd/9jGsSLeiA0RiFi8F/viUByJnpDiq7gWv8SN98376f32/ixal7x2Zl3cKu/vP2AxqYM=</latexit> Using Geodesic Patches: GCNN X ( f ? a )( x ) := a ( ✓ + ∆ ✓ , r )( D ( x ) f )( r, ✓ ) [Masci et al. 2015] θ ,r �40
<latexit sha1_base64="TquvtCTwYBxPgA56TL/7lpYdxY0=">ACO3icbZDLSxBEMZ7NA8z5rHRYy6FizBDRGYkEAkIknjwaIKrws6y1PTWuI09D7prxGXY/yuX/BO5eIlh0jINf07s4hPgoafnxfFdX1pZVWlqPoh7e0/Ojxk6crz/zV5y9evuq8XjuxZW0k9WSpS3OWoiWtCuqxYk1nlSHMU02n6cWnmX96ScaqsjmSUWDHM8LlSmJ7KRh50uQWIZDWAIwVUIH/bAT2ydD5uEx8S4ZaAwYLhLSQHpB20HpjQDw7cWBYGZmshsNON9qO5gX3IW6hK9o6Gna+J6NS1jkVLDVa24+jigcNGlZS09RPaksVygs8p7DAnOyg2Z+xQ2nTKCrDTuFQxz9f+JBnNrJ3nqOnPksb3rzcSHvH7N2e6gUVMxVysSirNXAJsyBhpAxJ1hMHKI1yfwU5RoOSXdy+CyG+e/J9ONnZjh1/ftfd/9jGsSLeiA0RiFi8F/viUByJnpDiq7gWv8SN98376f32/ixal7x2Zl3cKu/vP2AxqYM=</latexit> <latexit sha1_base64="TquvtCTwYBxPgA56TL/7lpYdxY0=">ACO3icbZDLSxBEMZ7NA8z5rHRYy6FizBDRGYkEAkIknjwaIKrws6y1PTWuI09D7prxGXY/yuX/BO5eIlh0jINf07s4hPgoafnxfFdX1pZVWlqPoh7e0/Ojxk6crz/zV5y9evuq8XjuxZW0k9WSpS3OWoiWtCuqxYk1nlSHMU02n6cWnmX96ScaqsjmSUWDHM8LlSmJ7KRh50uQWIZDWAIwVUIH/bAT2ydD5uEx8S4ZaAwYLhLSQHpB20HpjQDw7cWBYGZmshsNON9qO5gX3IW6hK9o6Gna+J6NS1jkVLDVa24+jigcNGlZS09RPaksVygs8p7DAnOyg2Z+xQ2nTKCrDTuFQxz9f+JBnNrJ3nqOnPksb3rzcSHvH7N2e6gUVMxVysSirNXAJsyBhpAxJ1hMHKI1yfwU5RoOSXdy+CyG+e/J9ONnZjh1/ftfd/9jGsSLeiA0RiFi8F/viUByJnpDiq7gWv8SN98376f32/ixal7x2Zl3cKu/vP2AxqYM=</latexit> <latexit sha1_base64="TquvtCTwYBxPgA56TL/7lpYdxY0=">ACO3icbZDLSxBEMZ7NA8z5rHRYy6FizBDRGYkEAkIknjwaIKrws6y1PTWuI09D7prxGXY/yuX/BO5eIlh0jINf07s4hPgoafnxfFdX1pZVWlqPoh7e0/Ojxk6crz/zV5y9evuq8XjuxZW0k9WSpS3OWoiWtCuqxYk1nlSHMU02n6cWnmX96ScaqsjmSUWDHM8LlSmJ7KRh50uQWIZDWAIwVUIH/bAT2ydD5uEx8S4ZaAwYLhLSQHpB20HpjQDw7cWBYGZmshsNON9qO5gX3IW6hK9o6Gna+J6NS1jkVLDVa24+jigcNGlZS09RPaksVygs8p7DAnOyg2Z+xQ2nTKCrDTuFQxz9f+JBnNrJ3nqOnPksb3rzcSHvH7N2e6gUVMxVysSirNXAJsyBhpAxJ1hMHKI1yfwU5RoOSXdy+CyG+e/J9ONnZjh1/ftfd/9jGsSLeiA0RiFi8F/viUByJnpDiq7gWv8SN98376f32/ixal7x2Zl3cKu/vP2AxqYM=</latexit> <latexit sha1_base64="TquvtCTwYBxPgA56TL/7lpYdxY0=">ACO3icbZDLSxBEMZ7NA8z5rHRYy6FizBDRGYkEAkIknjwaIKrws6y1PTWuI09D7prxGXY/yuX/BO5eIlh0jINf07s4hPgoafnxfFdX1pZVWlqPoh7e0/Ojxk6crz/zV5y9evuq8XjuxZW0k9WSpS3OWoiWtCuqxYk1nlSHMU02n6cWnmX96ScaqsjmSUWDHM8LlSmJ7KRh50uQWIZDWAIwVUIH/bAT2ydD5uEx8S4ZaAwYLhLSQHpB20HpjQDw7cWBYGZmshsNON9qO5gX3IW6hK9o6Gna+J6NS1jkVLDVa24+jigcNGlZS09RPaksVygs8p7DAnOyg2Z+xQ2nTKCrDTuFQxz9f+JBnNrJ3nqOnPksb3rzcSHvH7N2e6gUVMxVysSirNXAJsyBhpAxJ1hMHKI1yfwU5RoOSXdy+CyG+e/J9ONnZjh1/ftfd/9jGsSLeiA0RiFi8F/viUByJnpDiq7gWv8SN98376f32/ixal7x2Zl3cKu/vP2AxqYM=</latexit> Using Geodesic Patches: GCNN X ( f ? a )( x ) := a ( ✓ + ∆ ✓ , r )( D ( x ) f )( r, ✓ ) [Masci et al. 2015] θ ,r �40
GCNN Architecture �41
Handling Rotational Ambiguity �42
Parameterization for Surface Analysis map 3D surface to 2D domain [Maron et al. 2017] �43
Parameterization for Surface Analysis map 3D surface to 2D domain [Maron et al. 2017] �43
Parameterization for Surface Analysis [Maron et al. 2017] �44
Parameterization for Surface Analysis [Maron et al. 2017] �44
Parameterization for Surface Analysis • Map 3D surface to 2D domain • One such mapping: flat torus (seamless => translation-invariant) • Many mappings exists: sample a few and average result • Which functions to map? XYZ, normals, curvature, … [Maron et al. 2017] �45
Parameterization for Surface Analysis [Maron et al. 2017] �46
Texture Transfer (Parameterization + Alignment) [Wang et al. 2016] �47
AtlasNet for Surface Generation condition decoded points on 2D patches [Groueix et al. 2018] �48
AtlasNet for Surface Generation Latent representation can be inferred from images or point clouds [Groueix et al. 2018] �49
AtlasNet for Surface Generation Quad Mesh is generated by mapping a regular grid in 2D domain to 3D points Latent representation can be inferred from images or point clouds [Groueix et al. 2018] �50
AtlasNet for Surface Generation texture coordinates come for free!! Latent representation can be inferred from images or point clouds �51
Representation for 3D • Image-based • Volumetric • Surface-based • PROS: parameterize + image networks (instrinsic representation) • CONS: suffers from parameterisation artefacts (local versus global distortion), requires good quality mesh • Point-based �52
Representation for 3D • Image-based • Volumetric • Surface-based • Point-based �53
Representation for 3D: Point-based • Common representation: native representation • Easy to obtain from meshes, depth scans, laser scans �54
In Original Representation • Common representation • Easy to obtain from meshes, depth scans, laser scans • Unstructured (e.g., any permutation of points gives same shape!) [Qi et al. 2017] �55
PointNet for Point Cloud Analysis permutation-invariant functions [Qi et al. 2017] �56
PointNet for Point Cloud Analysis Use MLPs (h) and max-pooling (g) as simple symmetric functions [Qi et al. 2017] �57
PointNet Architecture [Qi et al. 2017] �58
PointNet for Point Cloud Analysis �59
PointNet++ [Qi et al. 2018] �60
PCPNet for Local Point Cloud Analysis [Guerrero et al. 2018] �61
Recommend
More recommend