Invariant-equivariant representation learning for multi-class data Ilya Feige Faculty ➔
Invariant-equivariant representation learning High-level introduction � 2
Separating content from style This work is about disentangling representations. We present a novel approach to an old problem. What? Want to represent the class and the data instance separately � 3
<latexit sha1_base64="Ys2D6lj3c1+uto6IwD1WQZF5vj0=">AB+nicbVDLSsNAFJ3UV42vqEs3g6XgQkpSBV0W3bisYB/QhjKZTtuhk0mYuSmW2D9xJSiIW/ElX/jNM1CWw9cOJxz78y9J4gF1+C631ZhbX1jc6u4be/s7u0fOIdHTR0lirIGjUSk2gHRTHDJGsBsHasGAkDwVrB+HbutyZMaR7JB5jGzA/JUPIBpwSM1HOcLrBHSKkgWp/jGVY9p+RW3Ax4lXg5KaEc9Z7z1e1HNAmZhOyVjufG4KdEAaeCzexyN9EsJnRMhqxjqCQh036arT7DZaP08SBSpiTgTLV/TaQk1HoaBqYzJDSy95c/M/rJDC49lMu4wSYpIuPBonAEOF5DrjPFaMgpoYQqrhZFtMRUYSCScs2KXjLN6+SZrXiXVSq95el2k2eRxGdoFN0hjx0hWroDtVRA1E0Qc/oFb1ZT9aL9W59LFoLVj5zjP7A+vwBm+KThw=</latexit> Separating content from style This work is about disentangling representations. We present a novel approach to an old problem. What? Want to represent the class and the data instance separately class, r � 4
<latexit sha1_base64="e3aF6vstTuwkI9LmeOhMNzmeTyM=">ACAnicbVDLSgNBEJz1GeNr1YvgZTAEIoSwGwU9Br14jGAekCxhdjKbDJmdXWZ6g2GJz/Fk6AgXv0LT/6Nk8dBEwsaiqpurv8WHANjvNtrayurW9sZray2zu7e/v2wWFdR4mirEYjEamTzQTXLIacBCsGStGQl+whj+4mfiNIVOaR/IeRjHzQtKTPOCUgJE69nEb2AOkXQIkjriEIh7jgioOzp2zik5U+Bl4s5JDs1R7dhf7W5Ek5BJoIJo3XKdGLyUKOBUsHE23040iwkdkB5rGSpJyLSXTl8Y47xRujiIlCkJeKpmf02kJNR6FPqmMyTQ14veRPzPayUQXHkpl3ECTNLZoiARGCI8yQN3uWIUxMgQhU3x2LaJ4pQMKlTQru4s/LpF4uel8t1FrnI9zyODTtApKiAXaIKukVEMUPaJn9IrerCfrxXq3PmatK9Z85gj9gfX5AwGTlng=</latexit> <latexit sha1_base64="Ys2D6lj3c1+uto6IwD1WQZF5vj0=">AB+nicbVDLSsNAFJ3UV42vqEs3g6XgQkpSBV0W3bisYB/QhjKZTtuhk0mYuSmW2D9xJSiIW/ElX/jNM1CWw9cOJxz78y9J4gF1+C631ZhbX1jc6u4be/s7u0fOIdHTR0lirIGjUSk2gHRTHDJGsBsHasGAkDwVrB+HbutyZMaR7JB5jGzA/JUPIBpwSM1HOcLrBHSKkgWp/jGVY9p+RW3Ax4lXg5KaEc9Z7z1e1HNAmZhOyVjufG4KdEAaeCzexyN9EsJnRMhqxjqCQh036arT7DZaP08SBSpiTgTLV/TaQk1HoaBqYzJDSy95c/M/rJDC49lMu4wSYpIuPBonAEOF5DrjPFaMgpoYQqrhZFtMRUYSCScs2KXjLN6+SZrXiXVSq95el2k2eRxGdoFN0hjx0hWroDtVRA1E0Qc/oFb1ZT9aL9W59LFoLVj5zjP7A+vwBm+KThw=</latexit> Separating content from style This work is about disentangling representations. We present a novel approach to an old problem. What? Want to represent the class and the data instance separately class, r datapoint, ( r, v ) � 5
<latexit sha1_base64="Ys2D6lj3c1+uto6IwD1WQZF5vj0=">AB+nicbVDLSsNAFJ3UV42vqEs3g6XgQkpSBV0W3bisYB/QhjKZTtuhk0mYuSmW2D9xJSiIW/ElX/jNM1CWw9cOJxz78y9J4gF1+C631ZhbX1jc6u4be/s7u0fOIdHTR0lirIGjUSk2gHRTHDJGsBsHasGAkDwVrB+HbutyZMaR7JB5jGzA/JUPIBpwSM1HOcLrBHSKkgWp/jGVY9p+RW3Ax4lXg5KaEc9Z7z1e1HNAmZhOyVjufG4KdEAaeCzexyN9EsJnRMhqxjqCQh036arT7DZaP08SBSpiTgTLV/TaQk1HoaBqYzJDSy95c/M/rJDC49lMu4wSYpIuPBonAEOF5DrjPFaMgpoYQqrhZFtMRUYSCScs2KXjLN6+SZrXiXVSq95el2k2eRxGdoFN0hjx0hWroDtVRA1E0Qc/oFb1ZT9aL9W59LFoLVj5zjP7A+vwBm+KThw=</latexit> <latexit sha1_base64="e3aF6vstTuwkI9LmeOhMNzmeTyM=">ACAnicbVDLSgNBEJz1GeNr1YvgZTAEIoSwGwU9Br14jGAekCxhdjKbDJmdXWZ6g2GJz/Fk6AgXv0LT/6Nk8dBEwsaiqpurv8WHANjvNtrayurW9sZray2zu7e/v2wWFdR4mirEYjEamTzQTXLIacBCsGStGQl+whj+4mfiNIVOaR/IeRjHzQtKTPOCUgJE69nEb2AOkXQIkjriEIh7jgioOzp2zik5U+Bl4s5JDs1R7dhf7W5Ek5BJoIJo3XKdGLyUKOBUsHE23040iwkdkB5rGSpJyLSXTl8Y47xRujiIlCkJeKpmf02kJNR6FPqmMyTQ14veRPzPayUQXHkpl3ECTNLZoiARGCI8yQN3uWIUxMgQhU3x2LaJ4pQMKlTQru4s/LpF4uel8t1FrnI9zyODTtApKiAXaIKukVEMUPaJn9IrerCfrxXq3PmatK9Z85gj9gfX5AwGTlng=</latexit> Separating content from style This work is about disentangling representations. We present a novel approach to an old problem. What? Why? Want to represent the class and • Classification the data instance separately • Interpretability • Object detection • Topic modelling • Style transfer class, r • Face swap datapoint, ( r, v ) • … � 6
<latexit sha1_base64="Ys2D6lj3c1+uto6IwD1WQZF5vj0=">AB+nicbVDLSsNAFJ3UV42vqEs3g6XgQkpSBV0W3bisYB/QhjKZTtuhk0mYuSmW2D9xJSiIW/ElX/jNM1CWw9cOJxz78y9J4gF1+C631ZhbX1jc6u4be/s7u0fOIdHTR0lirIGjUSk2gHRTHDJGsBsHasGAkDwVrB+HbutyZMaR7JB5jGzA/JUPIBpwSM1HOcLrBHSKkgWp/jGVY9p+RW3Ax4lXg5KaEc9Z7z1e1HNAmZhOyVjufG4KdEAaeCzexyN9EsJnRMhqxjqCQh036arT7DZaP08SBSpiTgTLV/TaQk1HoaBqYzJDSy95c/M/rJDC49lMu4wSYpIuPBonAEOF5DrjPFaMgpoYQqrhZFtMRUYSCScs2KXjLN6+SZrXiXVSq95el2k2eRxGdoFN0hjx0hWroDtVRA1E0Qc/oFb1ZT9aL9W59LFoLVj5zjP7A+vwBm+KThw=</latexit> <latexit sha1_base64="e3aF6vstTuwkI9LmeOhMNzmeTyM=">ACAnicbVDLSgNBEJz1GeNr1YvgZTAEIoSwGwU9Br14jGAekCxhdjKbDJmdXWZ6g2GJz/Fk6AgXv0LT/6Nk8dBEwsaiqpurv8WHANjvNtrayurW9sZray2zu7e/v2wWFdR4mirEYjEamTzQTXLIacBCsGStGQl+whj+4mfiNIVOaR/IeRjHzQtKTPOCUgJE69nEb2AOkXQIkjriEIh7jgioOzp2zik5U+Bl4s5JDs1R7dhf7W5Ek5BJoIJo3XKdGLyUKOBUsHE23040iwkdkB5rGSpJyLSXTl8Y47xRujiIlCkJeKpmf02kJNR6FPqmMyTQ14veRPzPayUQXHkpl3ECTNLZoiARGCI8yQN3uWIUxMgQhU3x2LaJ4pQMKlTQru4s/LpF4uel8t1FrnI9zyODTtApKiAXaIKukVEMUPaJn9IrerCfrxXq3PmatK9Z85gj9gfX5AwGTlng=</latexit> Separating content from style This work is about disentangling representations. We present a novel approach to an old problem. What? Why? What else? Want to represent the class and This is not a new topic… • Classification the data instance separately • Tenenbaum & Freeman. (2000) • Interpretability • Reed et al. (2014) • Object detection • Cheung et al. (2014) • Zhu et al. (2014) • Topic modelling • Radford et al. (2016) • Style transfer • Chen et al. (2016) class, r • Makhzani et al. (2016) • Face swap • Siddharth et al. (2017) datapoint, ( r, v ) • … • … � 7
<latexit sha1_base64="/R7rH7pCczoJ5nIklzQdq4MNOA=">ACFHicbZDNSgMxFIUz/lv/qi7dBIsgImWmCroU3bhUsLbQmQ6ZNG2DmWRI7mjL0Hdw5aO4EhTErRtXvo1pOwtPRD4OPdebu6JEsENuO63MzM7N7+wuLRcWFldW98obm7dGpVqyqpUCaXrETFMcMmqwEGweqIZiSPBatHdxbBeu2facCVvoJ+wICYdyducErBWDzws17TO+w1K4e+aCkwFmN/gH2hZEfzTheI1uoB67AfFktu2R0JT4OXQwnlugqLX35L0TRmEqgxjQ8N4EgIxo4FWxQ2PNTwxJC70iHNSxKEjMTZKOjBnjPOi3cVto+CXjkFn5NZCQ2ph9HtjMm0DWTtaH5X62RQvs0yLhMUmCSjhe1U4FB4WFCuMU1oyD6FgjV3H4W0y7RhILNsWBT8CZvnobStk7Kleuj0tn53keS2gH7aJ95KETdIYu0RWqIoe0TN6RW/Ok/PivDsf49YZJ5/ZRn/kfP4AZ1qeZQ=</latexit> The main idea Inferring the class latent using strategic data routing Invariant (class) latent is deterministically calculated from “complementary” same-class examples: { x 1 , x 2 , . . . , x m } − → r y � 8
<latexit sha1_base64="Olmoz2YQ8D4Apf2fJNJUrUJz1dU=">ACBnicbVDLSsNAFJ34rPEVdamLwVJwISWpgi6LblxWsA9oQphMp+3QyUyYmVRL6MaVn+JKUBC3/oMr/8Zpm4W2HrhwOde7r0nShV2nW/raXldW19cKGvbm1vbPr7O03lEglJnUsmJCtCnCKCd1TUjrUQSFEeMNKPB9cRvDolUVPA7PUpIEKMep12KkTZS6Bz5mQxHp/DBH0OfCd6TtNfXSEpxD4ehU3TL7hRwkXg5KYIctdD58jsCpzHhGjOkVNtzEx1kSGqKGRnbJT9VJEF4gHqkbShHMVFBNn1jDEtG6cCukKa4hlPV/jWRoVipURyZzhjpvpr3JuJ/XjvV3csgozxJNeF4tqibMqgFnGQCO1QSrNnIEIQlNcdC3EcSYW2Ss0K3vzPi6RKXtn5crtebF6ledRAIfgGJwAD1yAKrgBNVAHGDyCZ/AK3qwn68V6tz5mrUtWPnMA/sD6/AHqLJi7</latexit> <latexit sha1_base64="/R7rH7pCczoJ5nIklzQdq4MNOA=">ACFHicbZDNSgMxFIUz/lv/qi7dBIsgImWmCroU3bhUsLbQmQ6ZNG2DmWRI7mjL0Hdw5aO4EhTErRtXvo1pOwtPRD4OPdebu6JEsENuO63MzM7N7+wuLRcWFldW98obm7dGpVqyqpUCaXrETFMcMmqwEGweqIZiSPBatHdxbBeu2facCVvoJ+wICYdyducErBWDzws17TO+w1K4e+aCkwFmN/gH2hZEfzTheI1uoB67AfFktu2R0JT4OXQwnlugqLX35L0TRmEqgxjQ8N4EgIxo4FWxQ2PNTwxJC70iHNSxKEjMTZKOjBnjPOi3cVto+CXjkFn5NZCQ2ph9HtjMm0DWTtaH5X62RQvs0yLhMUmCSjhe1U4FB4WFCuMU1oyD6FgjV3H4W0y7RhILNsWBT8CZvnobStk7Kleuj0tn53keS2gH7aJ95KETdIYu0RWqIoe0TN6RW/Ok/PivDsf49YZJ5/ZRn/kfP4AZ1qeZQ=</latexit> The main idea Inferring the class latent using strategic data routing Invariant (class) latent is deterministically calculated from “complementary” same-class examples: { x 1 , x 2 , . . . , x m } − → r y Equivariant (instance) latent is stochastically inferred from both class and datapoint information: { r y , x } − → v � 9
<latexit sha1_base64="Olmoz2YQ8D4Apf2fJNJUrUJz1dU=">ACBnicbVDLSsNAFJ34rPEVdamLwVJwISWpgi6LblxWsA9oQphMp+3QyUyYmVRL6MaVn+JKUBC3/oMr/8Zpm4W2HrhwOde7r0nShV2nW/raXldW19cKGvbm1vbPr7O03lEglJnUsmJCtCnCKCd1TUjrUQSFEeMNKPB9cRvDolUVPA7PUpIEKMep12KkTZS6Bz5mQxHp/DBH0OfCd6TtNfXSEpxD4ehU3TL7hRwkXg5KYIctdD58jsCpzHhGjOkVNtzEx1kSGqKGRnbJT9VJEF4gHqkbShHMVFBNn1jDEtG6cCukKa4hlPV/jWRoVipURyZzhjpvpr3JuJ/XjvV3csgozxJNeF4tqibMqgFnGQCO1QSrNnIEIQlNcdC3EcSYW2Ss0K3vzPi6RKXtn5crtebF6ledRAIfgGJwAD1yAKrgBNVAHGDyCZ/AK3qwn68V6tz5mrUtWPnMA/sD6/AHqLJi7</latexit> <latexit sha1_base64="/R7rH7pCczoJ5nIklzQdq4MNOA=">ACFHicbZDNSgMxFIUz/lv/qi7dBIsgImWmCroU3bhUsLbQmQ6ZNG2DmWRI7mjL0Hdw5aO4EhTErRtXvo1pOwtPRD4OPdebu6JEsENuO63MzM7N7+wuLRcWFldW98obm7dGpVqyqpUCaXrETFMcMmqwEGweqIZiSPBatHdxbBeu2facCVvoJ+wICYdyducErBWDzws17TO+w1K4e+aCkwFmN/gH2hZEfzTheI1uoB67AfFktu2R0JT4OXQwnlugqLX35L0TRmEqgxjQ8N4EgIxo4FWxQ2PNTwxJC70iHNSxKEjMTZKOjBnjPOi3cVto+CXjkFn5NZCQ2ph9HtjMm0DWTtaH5X62RQvs0yLhMUmCSjhe1U4FB4WFCuMU1oyD6FgjV3H4W0y7RhILNsWBT8CZvnobStk7Kleuj0tn53keS2gH7aJ95KETdIYu0RWqIoe0TN6RW/Ok/PivDsf49YZJ5/ZRn/kfP4AZ1qeZQ=</latexit> The main idea Inferring the class latent using strategic data routing Invariant (class) latent is deterministically calculated from “complementary” same-class examples: { x 1 , x 2 , . . . , x m } − → r y Equivariant (instance) latent is stochastically inferred from both class and datapoint information: { r y , x } − → v Inspired by GQNs (Eslami et al., 2018) � 10
Invariant-equivariant representation learning Some detail � 11
Recommend
More recommend
