Lecture 14: Representation learning 1
Announcements • Project proposal due after spring break • We’ll post description on website • New idea? Turn in half-page description • Premade project idea? Just tell us which one 2
Today • What representations do neural nets learn? • Transfer learning • Unsupervised learning 3
Observed image Drawn from memory [Bartlett, 1932] [Intraub & Richardson, 1989] 4 Source: Isola, Freeman, Torralba
Observed image Drawn from memory [Bartlett, 1932] [Intraub & Richardson, 1989] 5 Source: Isola, Freeman, Torralba
"I stand at the window and see a house, trees, sky. Theoretically I might say there were 327 brightnesses and nuances of colour. Do I have "327"? No. I have sky, house, and trees.” — Max Wertheimer, 1923 6 Source: Isola, Freeman, Torralba
Representation learning X “Coral” “Fish” Image Compact mental representation 7 Source: Isola, Freeman, Torralba
Representation learning Good representations are: 1. Compact ( minimal ) “Coral” 2. Explanatory ( sufficient ) “Fish” 3. Disentangled ( independent factors ) 4. Interpretable 5. Make subsequent problem solving easy 8 [See “Representation Learning”, Bengio 2013, for more commentary] Source: Isola, Freeman, Torralba
Transfer learning “Generally speaking, a good representation is one that makes a subsequent learning task easier.” — Deep Learning , Goodfellow et al. 2016 ? 9 Source: Isola, Freeman, Torralba
<latexit sha1_base64="btHp+XGA9KMZBtcL1WFTP9DuJw=">ACdHicfVHLbtQwFPWkPEp4tbCERURUCSE0SrqBZUVZsEUqTMdaRxVN56bjFU/IvumZRTlJ7otP8aPsMaZDhK0iCNZOj73XPs+ykZJT1n2YxRt3bl7/72g/jho8dPnu7sPpt62zqBE2GVdbMSPCpcEKSFM4ah6BLhSfl2eEQPzlH56U1x7RqsNBQG1lJARSk2bEDaSpT3fSbJytkdwm+YakbIOj093RlC+saDUaEgq8n+dZQ0UHjqRQ2Me89diAOIMa54Ea0OiLbl1wn+wFZFU1oVjKFmrf2Z0oL1f6TI4NdDS34wN4r9i85aq90UnTdMSGnH9UdWqhGwydJ8spENBahUICdDrYlYgNBYUZxzD9iaMbh5/DwlwYdkHVvOg6u1vCtD83V/O3A/meU5rcxsJgbvBWazCLjhvrdD/Pi4rIirKTpKc+5kvSTuhlsfh03kN+d+m0z3x3k2zr/upwcfNjvZi/YK/a5ewdO2Cf2BGbMEUu2RX7PvoZ/QySqO9a2s02uQ8Z38hGv8CD+/BLg=</latexit> <latexit sha1_base64="btHp+XGA9KMZBtcL1WFTP9DuJw=">ACdHicfVHLbtQwFPWkPEp4tbCERURUCSE0SrqBZUVZsEUqTMdaRxVN56bjFU/IvumZRTlJ7otP8aPsMaZDhK0iCNZOj73XPs+ykZJT1n2YxRt3bl7/72g/jho8dPnu7sPpt62zqBE2GVdbMSPCpcEKSFM4ah6BLhSfl2eEQPzlH56U1x7RqsNBQG1lJARSk2bEDaSpT3fSbJytkdwm+YakbIOj093RlC+saDUaEgq8n+dZQ0UHjqRQ2Me89diAOIMa54Ea0OiLbl1wn+wFZFU1oVjKFmrf2Z0oL1f6TI4NdDS34wN4r9i85aq90UnTdMSGnH9UdWqhGwydJ8spENBahUICdDrYlYgNBYUZxzD9iaMbh5/DwlwYdkHVvOg6u1vCtD83V/O3A/meU5rcxsJgbvBWazCLjhvrdD/Pi4rIirKTpKc+5kvSTuhlsfh03kN+d+m0z3x3k2zr/upwcfNjvZi/YK/a5ewdO2Cf2BGbMEUu2RX7PvoZ/QySqO9a2s02uQ8Z38hGv8CD+/BLg=</latexit> <latexit sha1_base64="btHp+XGA9KMZBtcL1WFTP9DuJw=">ACdHicfVHLbtQwFPWkPEp4tbCERURUCSE0SrqBZUVZsEUqTMdaRxVN56bjFU/IvumZRTlJ7otP8aPsMaZDhK0iCNZOj73XPs+ykZJT1n2YxRt3bl7/72g/jho8dPnu7sPpt62zqBE2GVdbMSPCpcEKSFM4ah6BLhSfl2eEQPzlH56U1x7RqsNBQG1lJARSk2bEDaSpT3fSbJytkdwm+YakbIOj093RlC+saDUaEgq8n+dZQ0UHjqRQ2Me89diAOIMa54Ea0OiLbl1wn+wFZFU1oVjKFmrf2Z0oL1f6TI4NdDS34wN4r9i85aq90UnTdMSGnH9UdWqhGwydJ8spENBahUICdDrYlYgNBYUZxzD9iaMbh5/DwlwYdkHVvOg6u1vCtD83V/O3A/meU5rcxsJgbvBWazCLjhvrdD/Pi4rIirKTpKc+5kvSTuhlsfh03kN+d+m0z3x3k2zr/upwcfNjvZi/YK/a5ewdO2Cf2BGbMEUu2RX7PvoZ/QySqO9a2s02uQ8Z38hGv8CD+/BLg=</latexit> <latexit sha1_base64="btHp+XGA9KMZBtcL1WFTP9DuJw=">ACdHicfVHLbtQwFPWkPEp4tbCERURUCSE0SrqBZUVZsEUqTMdaRxVN56bjFU/IvumZRTlJ7otP8aPsMaZDhK0iCNZOj73XPs+ykZJT1n2YxRt3bl7/72g/jho8dPnu7sPpt62zqBE2GVdbMSPCpcEKSFM4ah6BLhSfl2eEQPzlH56U1x7RqsNBQG1lJARSk2bEDaSpT3fSbJytkdwm+YakbIOj093RlC+saDUaEgq8n+dZQ0UHjqRQ2Me89diAOIMa54Ea0OiLbl1wn+wFZFU1oVjKFmrf2Z0oL1f6TI4NdDS34wN4r9i85aq90UnTdMSGnH9UdWqhGwydJ8spENBahUICdDrYlYgNBYUZxzD9iaMbh5/DwlwYdkHVvOg6u1vCtD83V/O3A/meU5rcxsJgbvBWazCLjhvrdD/Pi4rIirKTpKc+5kvSTuhlsfh03kN+d+m0z3x3k2zr/upwcfNjvZi/YK/a5ewdO2Cf2BGbMEUu2RX7PvoZ/QySqO9a2s02uQ8Z38hGv8CD+/BLg=</latexit> <latexit sha1_base64="xu7yi64Jzp9qaGC2BEKYLlLVM18=">ACc3icfVHLbtNAFJ24PIp5tbDsxmpAQgiuxtYVtAFG0SRmqRSxquJ9fOKPOwZq5bIsf0W35Mj6EPeM0SNAijTSmXPnbmPolbSU5r+GERbd+7eu7/9IH746PGTpzu7zybeNk7gWFhl3WkBHpU0OCZJCk9rh6ALhdNi+bGPT8/ReWnNCa1qzDVURpZSAVpeoKepKnOdobpKF0juU2yDRmyDY7PdgcTPrei0WhIKPB+lqU15S04kJhF/PGYw1iCRXOAjWg0eftut4ueRmUeVJaF46hZK3+mdGC9n6li+DUQAt/M9aL/4rNGirf560dUNoxPVHZaMSsknfDKXDgWpVSAgnAy1JmIBDgSFEcUxP8LQjMP4eEvNTog6163HFyl4VsXmqv4m579zyjNb2NgMTd4IazWYOYtN9bpbpblLVdYElcTdDTMuJPVgrjrb10cNpHdnPtMjkYZeko+3owPyw2ck2P7BXL2Dt2yD6xYzZmgi3ZJbti3wc/o71oP3pxbY0Gm5zn7C9Eb38BKGvAxg=</latexit> <latexit sha1_base64="xu7yi64Jzp9qaGC2BEKYLlLVM18=">ACc3icfVHLbtNAFJ24PIp5tbDsxmpAQgiuxtYVtAFG0SRmqRSxquJ9fOKPOwZq5bIsf0W35Mj6EPeM0SNAijTSmXPnbmPolbSU5r+GERbd+7eu7/9IH746PGTpzu7zybeNk7gWFhl3WkBHpU0OCZJCk9rh6ALhdNi+bGPT8/ReWnNCa1qzDVURpZSAVpeoKepKnOdobpKF0juU2yDRmyDY7PdgcTPrei0WhIKPB+lqU15S04kJhF/PGYw1iCRXOAjWg0eftut4ueRmUeVJaF46hZK3+mdGC9n6li+DUQAt/M9aL/4rNGirf560dUNoxPVHZaMSsknfDKXDgWpVSAgnAy1JmIBDgSFEcUxP8LQjMP4eEvNTog6163HFyl4VsXmqv4m579zyjNb2NgMTd4IazWYOYtN9bpbpblLVdYElcTdDTMuJPVgrjrb10cNpHdnPtMjkYZeko+3owPyw2ck2P7BXL2Dt2yD6xYzZmgi3ZJbti3wc/o71oP3pxbY0Gm5zn7C9Eb38BKGvAxg=</latexit> <latexit sha1_base64="xu7yi64Jzp9qaGC2BEKYLlLVM18=">ACc3icfVHLbtNAFJ24PIp5tbDsxmpAQgiuxtYVtAFG0SRmqRSxquJ9fOKPOwZq5bIsf0W35Mj6EPeM0SNAijTSmXPnbmPolbSU5r+GERbd+7eu7/9IH746PGTpzu7zybeNk7gWFhl3WkBHpU0OCZJCk9rh6ALhdNi+bGPT8/ReWnNCa1qzDVURpZSAVpeoKepKnOdobpKF0juU2yDRmyDY7PdgcTPrei0WhIKPB+lqU15S04kJhF/PGYw1iCRXOAjWg0eftut4ueRmUeVJaF46hZK3+mdGC9n6li+DUQAt/M9aL/4rNGirf560dUNoxPVHZaMSsknfDKXDgWpVSAgnAy1JmIBDgSFEcUxP8LQjMP4eEvNTog6163HFyl4VsXmqv4m579zyjNb2NgMTd4IazWYOYtN9bpbpblLVdYElcTdDTMuJPVgrjrb10cNpHdnPtMjkYZeko+3owPyw2ck2P7BXL2Dt2yD6xYzZmgi3ZJbti3wc/o71oP3pxbY0Gm5zn7C9Eb38BKGvAxg=</latexit> <latexit sha1_base64="xu7yi64Jzp9qaGC2BEKYLlLVM18=">ACc3icfVHLbtNAFJ24PIp5tbDsxmpAQgiuxtYVtAFG0SRmqRSxquJ9fOKPOwZq5bIsf0W35Mj6EPeM0SNAijTSmXPnbmPolbSU5r+GERbd+7eu7/9IH746PGTpzu7zybeNk7gWFhl3WkBHpU0OCZJCk9rh6ALhdNi+bGPT8/ReWnNCa1qzDVURpZSAVpeoKepKnOdobpKF0juU2yDRmyDY7PdgcTPrei0WhIKPB+lqU15S04kJhF/PGYw1iCRXOAjWg0eftut4ueRmUeVJaF46hZK3+mdGC9n6li+DUQAt/M9aL/4rNGirf560dUNoxPVHZaMSsknfDKXDgWpVSAgnAy1JmIBDgSFEcUxP8LQjMP4eEvNTog6163HFyl4VsXmqv4m579zyjNb2NgMTd4IazWYOYtN9bpbpblLVdYElcTdDTMuJPVgrjrb10cNpHdnPtMjkYZeko+3owPyw2ck2P7BXL2Dt2yD6xYzZmgi3ZJbti3wc/o71oP3pxbY0Gm5zn7C9Eb38BKGvAxg=</latexit> Training Testing Object recognition Place recognition ? “Fish” Often, what we will be “tested” on is to learn to do a new thing. 10 Source: Isola, Freeman, Torralba
Recommend
More recommend
