Projective geometry-aware anisotropic convolutional filters Poster @ Pacific Ballroom #147 Renata Khasanova, Pascal Frossard, LTS4, EPFL, Switzerland
Motivation Photo credits: https://www.ti.uni-bielefeld.de/html/research/equipment.html https://www.t3.com/features/best-drone � 2 https://www.zemax.com/blog/zemax-blog/october-2017/getting-to-the-finish-line-faster-optical-technolo
Omnidirectional camera representation ? Spherical surface Rectangular representation � 3
Geometric distortion of equirectangular images � 4
Geometric distortion of cube-map images � 5
Graphs for geometry modelling � 6
Graphs for geometry modelling pixel — node pixel’s intensity — node’s signal � 7
Geometry aware graph filters Idea: adapt filters depending on image location Example of a filter applied to equirectangular projection � 8
Geometry aware graph filters Multiple directed graphs for anisotropic filters Example of filters � 9
<latexit sha1_base64="3zIjD/w+WfbO8SbJCx8EBMPIVNo=">AB6nicbVBNS8NAEJ3Ur1q/oh69LBbBU0lE0GPRi8eK9gPaUDbTbt0swm7E6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvTKUw6HnfTmltfWNzq7xd2dnd2z9wD49aJsk0402WyER3Qmq4FIo3UaDknVRzGoeSt8Px7cxvP3FtRKIecZLyIKZDJSLBKFrpod3+27Vq3lzkFXiF6QKBRp96s3SFgWc4VMUmO6vpdikFONgk+rfQyw1PKxnTIu5YqGnMT5PNTp+TMKgMSJdqWQjJXf0/kNDZmEoe2M6Y4MsveTPzP62YXQe5UGmGXLHFoiTBMy+5sMhOYM5cQSyrSwtxI2opoytOlUbAj+8surpHVR872af39Zrd8UcZThBE7hHy4gjrcQOawGAIz/AKb450Xpx352PRWnKmWP4A+fzB9lbjX8=</latexit> <latexit sha1_base64="vMCmCBvrQqvMkcZe5c78yK19qu0=">AB63icbVBNSwMxEJ3Ur1q/qh69BIvgqeyKoMeiF48VbC20S8m2W5okl2SrFCW/gUvHhTx6h/y5r8x2+5BWx8MPN6bYWZemApurOd9o8ra+sbmVnW7trO7t39QPzqmiTlHVoIhLdC4lhgivWsdwK1ks1IzIU7DGc3Bb+4xPThifqwU5TFkgyVjzilNhCGqQxH9YbXtObA68SvyQNKNEe1r8Go4RmkilLBTGm73upDXKiLaeCzWqDzLCU0AkZs76jikhmgnx+6wyfOWEo0S7UhbP1d8TOZHGTGXoOiWxsVn2CvE/r5/Z6DrIuUozyxRdLIoygW2Ci8fxiGtGrZg6Qqjm7lZMY6IJtS6emgvBX35lXQvmr7X9O8vG62bMo4qnMApnIMPV9CO2hDByjE8Ayv8IYkekHv6GPRWkHlzDH8Afr8AROIj8=</latexit> <latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOsmY2ZlplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDbH7tlJ45pU8H2rhSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MBRThxh3Ah3K+UjZhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl8ckz/wPn8Ao/ePKA=</latexit> <latexit sha1_base64="vMCmCBvrQqvMkcZe5c78yK19qu0=">AB63icbVBNSwMxEJ3Ur1q/qh69BIvgqeyKoMeiF48VbC20S8m2W5okl2SrFCW/gUvHhTx6h/y5r8x2+5BWx8MPN6bYWZemApurOd9o8ra+sbmVnW7trO7t39QPzqmiTlHVoIhLdC4lhgivWsdwK1ks1IzIU7DGc3Bb+4xPThifqwU5TFkgyVjzilNhCGqQxH9YbXtObA68SvyQNKNEe1r8Go4RmkilLBTGm73upDXKiLaeCzWqDzLCU0AkZs76jikhmgnx+6wyfOWEo0S7UhbP1d8TOZHGTGXoOiWxsVn2CvE/r5/Z6DrIuUozyxRdLIoygW2Ci8fxiGtGrZg6Qqjm7lZMY6IJtS6emgvBX35lXQvmr7X9O8vG62bMo4qnMApnIMPV9CO2hDByjE8Ayv8IYkekHv6GPRWkHlzDH8Afr8AROIj8=</latexit> <latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOsmY2ZlplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDbH7tlJ45pU8H2rhSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MBRThxh3Ah3K+UjZhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl8ckz/wPn8Ao/ePKA=</latexit> Geometry aware graph filters Multiple directed graphs for anisotropic filters Example of filters Directed graph construction with adjacency matrix W 1 � 10
<latexit sha1_base64="vMCmCBvrQqvMkcZe5c78yK19qu0=">AB63icbVBNSwMxEJ3Ur1q/qh69BIvgqeyKoMeiF48VbC20S8m2W5okl2SrFCW/gUvHhTx6h/y5r8x2+5BWx8MPN6bYWZemApurOd9o8ra+sbmVnW7trO7t39QPzqmiTlHVoIhLdC4lhgivWsdwK1ks1IzIU7DGc3Bb+4xPThifqwU5TFkgyVjzilNhCGqQxH9YbXtObA68SvyQNKNEe1r8Go4RmkilLBTGm73upDXKiLaeCzWqDzLCU0AkZs76jikhmgnx+6wyfOWEo0S7UhbP1d8TOZHGTGXoOiWxsVn2CvE/r5/Z6DrIuUozyxRdLIoygW2Ci8fxiGtGrZg6Qqjm7lZMY6IJtS6emgvBX35lXQvmr7X9O8vG62bMo4qnMApnIMPV9CO2hDByjE8Ayv8IYkekHv6GPRWkHlzDH8Afr8AROIj8=</latexit> <latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOsmY2ZlplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDbH7tlJ45pU8H2rhSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MBRThxh3Ah3K+UjZhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl8ckz/wPn8Ao/ePKA=</latexit> <latexit sha1_base64="vMCmCBvrQqvMkcZe5c78yK19qu0=">AB63icbVBNSwMxEJ3Ur1q/qh69BIvgqeyKoMeiF48VbC20S8m2W5okl2SrFCW/gUvHhTx6h/y5r8x2+5BWx8MPN6bYWZemApurOd9o8ra+sbmVnW7trO7t39QPzqmiTlHVoIhLdC4lhgivWsdwK1ks1IzIU7DGc3Bb+4xPThifqwU5TFkgyVjzilNhCGqQxH9YbXtObA68SvyQNKNEe1r8Go4RmkilLBTGm73upDXKiLaeCzWqDzLCU0AkZs76jikhmgnx+6wyfOWEo0S7UhbP1d8TOZHGTGXoOiWxsVn2CvE/r5/Z6DrIuUozyxRdLIoygW2Ci8fxiGtGrZg6Qqjm7lZMY6IJtS6emgvBX35lXQvmr7X9O8vG62bMo4qnMApnIMPV9CO2hDByjE8Ayv8IYkekHv6GPRWkHlzDH8Afr8AROIj8=</latexit> <latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOsmY2ZlplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDbH7tlJ45pU8H2rhSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MBRThxh3Ah3K+UjZhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl8ckz/wPn8Ao/ePKA=</latexit> <latexit sha1_base64="vMCmCBvrQqvMkcZe5c78yK19qu0=">AB63icbVBNSwMxEJ3Ur1q/qh69BIvgqeyKoMeiF48VbC20S8m2W5okl2SrFCW/gUvHhTx6h/y5r8x2+5BWx8MPN6bYWZemApurOd9o8ra+sbmVnW7trO7t39QPzqmiTlHVoIhLdC4lhgivWsdwK1ks1IzIU7DGc3Bb+4xPThifqwU5TFkgyVjzilNhCGqQxH9YbXtObA68SvyQNKNEe1r8Go4RmkilLBTGm73upDXKiLaeCzWqDzLCU0AkZs76jikhmgnx+6wyfOWEo0S7UhbP1d8TOZHGTGXoOiWxsVn2CvE/r5/Z6DrIuUozyxRdLIoygW2Ci8fxiGtGrZg6Qqjm7lZMY6IJtS6emgvBX35lXQvmr7X9O8vG62bMo4qnMApnIMPV9CO2hDByjE8Ayv8IYkekHv6GPRWkHlzDH8Afr8AROIj8=</latexit> <latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOsmY2ZlplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDbH7tlJ45pU8H2rhSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MBRThxh3Ah3K+UjZhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl8ckz/wPn8Ao/ePKA=</latexit> <latexit sha1_base64="vMCmCBvrQqvMkcZe5c78yK19qu0=">AB63icbVBNSwMxEJ3Ur1q/qh69BIvgqeyKoMeiF48VbC20S8m2W5okl2SrFCW/gUvHhTx6h/y5r8x2+5BWx8MPN6bYWZemApurOd9o8ra+sbmVnW7trO7t39QPzqmiTlHVoIhLdC4lhgivWsdwK1ks1IzIU7DGc3Bb+4xPThifqwU5TFkgyVjzilNhCGqQxH9YbXtObA68SvyQNKNEe1r8Go4RmkilLBTGm73upDXKiLaeCzWqDzLCU0AkZs76jikhmgnx+6wyfOWEo0S7UhbP1d8TOZHGTGXoOiWxsVn2CvE/r5/Z6DrIuUozyxRdLIoygW2Ci8fxiGtGrZg6Qqjm7lZMY6IJtS6emgvBX35lXQvmr7X9O8vG62bMo4qnMApnIMPV9CO2hDByjE8Ayv8IYkekHv6GPRWkHlzDH8Afr8AROIj8=</latexit> <latexit sha1_base64="JqEnYvV6PtsKBJYmBVwEpjIMANw=">AB7XicbVDLSgNBEJyNrxhfUY9eBoPgKeyKoMegF48RzAOSJcxOsmY2ZlplcIS/7BiwdFvPo/3vwbJ8keNLGgoajqprsrSqSw6PvfXmFtfWNzq7hd2tnd2z8oHx41rU4NhwbXUpt2xCxIoaCBAiW0EwMsjiS0ovHtzG89gbFCqwecJBDGbKjEQHCGTmp2cQTIeuWKX/XnoKskyEmF5Kj3yl/dvuZpDAq5ZNZ2Aj/BMGMGBZcwLXVTCwnjYzaEjqOKxWDbH7tlJ45pU8H2rhSOfq74mMxdZO4sh1xgxHdtmbif95nRQH12EmVJIiKL5YNEglRU1nr9O+MBRThxh3Ah3K+UjZhHF1DJhRAsv7xKmhfVwK8G95eV2k0eR5GckFNyTgJyRWrkjtRJg3DySJ7JK3nztPfivXsfi9aCl8ckz/wPn8Ao/ePKA=</latexit> Geometry aware graph filters Multiple directed graphs for anisotropic filters Example of filters Directed graphs construction � 11
Classification: adaptation to various projective geometries Dataset — projected MNIST Spherical (S) Modified spherical (MS1, MS2, MS3) Fish-eye stereographic projection (F) Cube-map (CM) � 12
Classification results Method S MS1 MS2 MS3 F CM 69.4 64.3 64.1 62.8 71.8 40.0 regular graph (w=1) regular graph (w=1/d) 69.8 63.4 64.5 62.5 70.2 40.5 GA graph (w=1/d) 70.2 63.9 62.5 62.8 72.1 44.2 ConvNets 94.2 91.3 91.2 90.5 93.4 79.4 SphereNet 94.8 — — — — — SphericalCNN 95.2 84.5 83.3 80.9 94.9 — Ours 96.9 95.1 95.3 94.9 95.7 84.3 � 13
Compression Cube-map projection of SUN* dataset with 360-indoor images *https://groups.csail.mit.edu/vision/SUN/ � 14
Decompression challenges � 15
Visual decompression result Original Balle et. al (2017) Ours � 16
Summary � Novel graph construction approach to define a filter, which adapts to the specific geometry of the wide-angle images � Our filters are anisotropic, which permits richer representation � Our filters can be applied to a wider class of tasks compared with standard graph- based filters � Our approach reaches state-of-the-art performance on classification and compression tasks Thank you! Poster: @ Pacific Ballroom #147 Code: https://github.com/RenataKh/GAfilters � 17
Recommend
More recommend