unsupervised learning k means
play

Unsupervised Learning, K Means March 12, 2020 Data Science CSCI - PowerPoint PPT Presentation

Feb 08, 2024 •355 likes •1.46k views

Unsupervised Learning, K Means March 12, 2020 Data Science CSCI 1951A Brown University Instructor: Ellie Pavlick HTAs: Josh Levin, Diane Mutako, Sol Zitter 1 Announcements Here we go! Get cozy..PJs, coffee-in-hand, ready to talk ML :)

  1. <latexit sha1_base64="r/S0kziZA4N57u3eiAn1RfFZ4w=">ACDHicbVDLSsNAFJ3UV62vqks3g0WoC0tSBN0Uim5ctmBfNGmYTCft0MkzEyEvIBbvwVNy4UcesHuPNvnLZaOuBgcM53LnHi9iVCrT/DZya+sbm1v57cLO7t7+QfHwqC3DWGDSwiELRdDkjDKSUtRxUg3EgQFHiMdb3I78zsPREga8ns1jYgToBGnPsVIacktlpqwBm0ZB25Ca1Y6SHha7rkUXkB7jFTS8HVZ0yK+YcJVYGSmBDA23+GUPQxwHhCvMkJR9y4yUkyChKGYkLdixJBHCEzQifU05Coh0kvkxKTzTyhD6odCPKzhXf08kKJByGng6GSA1lsveTPzP68fKv3YSyqNYEY4Xi/yYQRXCWTNwSAXBik01QVhQ/VeIx0grHR/BV2CtXzyKmlXK5ZsZqXpfpNVkcenIBTUAYWuAJ1cAcaoAUweATP4BW8GU/Gi/FufCyiOSObOQZ/YHz+ADvfmdM=</latexit> <latexit sha1_base64="r/S0kziZA4N57u3eiAn1RfFZ4w=">ACDHicbVDLSsNAFJ3UV62vqks3g0WoC0tSBN0Uim5ctmBfNGmYTCft0MkzEyEvIBbvwVNy4UcesHuPNvnLZaOuBgcM53LnHi9iVCrT/DZya+sbm1v57cLO7t7+QfHwqC3DWGDSwiELRdDkjDKSUtRxUg3EgQFHiMdb3I78zsPREga8ns1jYgToBGnPsVIacktlpqwBm0ZB25Ca1Y6SHha7rkUXkB7jFTS8HVZ0yK+YcJVYGSmBDA23+GUPQxwHhCvMkJR9y4yUkyChKGYkLdixJBHCEzQifU05Coh0kvkxKTzTyhD6odCPKzhXf08kKJByGng6GSA1lsveTPzP68fKv3YSyqNYEY4Xi/yYQRXCWTNwSAXBik01QVhQ/VeIx0grHR/BV2CtXzyKmlXK5ZsZqXpfpNVkcenIBTUAYWuAJ1cAcaoAUweATP4BW8GU/Gi/FufCyiOSObOQZ/YHz+ADvfmdM=</latexit> <latexit sha1_base64="r/S0kziZA4N57u3eiAn1RfFZ4w=">ACDHicbVDLSsNAFJ3UV62vqks3g0WoC0tSBN0Uim5ctmBfNGmYTCft0MkzEyEvIBbvwVNy4UcesHuPNvnLZaOuBgcM53LnHi9iVCrT/DZya+sbm1v57cLO7t7+QfHwqC3DWGDSwiELRdDkjDKSUtRxUg3EgQFHiMdb3I78zsPREga8ns1jYgToBGnPsVIacktlpqwBm0ZB25Ca1Y6SHha7rkUXkB7jFTS8HVZ0yK+YcJVYGSmBDA23+GUPQxwHhCvMkJR9y4yUkyChKGYkLdixJBHCEzQifU05Coh0kvkxKTzTyhD6odCPKzhXf08kKJByGng6GSA1lsveTPzP68fKv3YSyqNYEY4Xi/yYQRXCWTNwSAXBik01QVhQ/VeIx0grHR/BV2CtXzyKmlXK5ZsZqXpfpNVkcenIBTUAYWuAJ1cAcaoAUweATP4BW8GU/Gi/FufCyiOSObOQZ/YHz+ADvfmdM=</latexit> <latexit sha1_base64="r/S0kziZA4N57u3eiAn1RfFZ4w=">ACDHicbVDLSsNAFJ3UV62vqks3g0WoC0tSBN0Uim5ctmBfNGmYTCft0MkzEyEvIBbvwVNy4UcesHuPNvnLZaOuBgcM53LnHi9iVCrT/DZya+sbm1v57cLO7t7+QfHwqC3DWGDSwiELRdDkjDKSUtRxUg3EgQFHiMdb3I78zsPREga8ns1jYgToBGnPsVIacktlpqwBm0ZB25Ca1Y6SHha7rkUXkB7jFTS8HVZ0yK+YcJVYGSmBDA23+GUPQxwHhCvMkJR9y4yUkyChKGYkLdixJBHCEzQifU05Coh0kvkxKTzTyhD6odCPKzhXf08kKJByGng6GSA1lsveTPzP68fKv3YSyqNYEY4Xi/yYQRXCWTNwSAXBik01QVhQ/VeIx0grHR/BV2CtXzyKmlXK5ZsZqXpfpNVkcenIBTUAYWuAJ1cAcaoAUweATP4BW8GU/Gi/FufCyiOSObOQZ/YHz+ADvfmdM=</latexit> <latexit sha1_base64="WPNm1YuVK0X591E2DeKZG6XKYA=">ACB3icbVBNS8NAEN3Urxq/oh4FWSxC1ISEfQiFHvxWMG2kTaUzXbTLt1swu6mUEJuXvwrXjwo4tW/4M1/47bNQVsfDzem2Fmnh8zKpVtfxuFldW19Y3iprm1vbO7Z+0ftGSUCEyaOGKRcH0kCaOcNBVjLixICj0GWn7o/rUb4+JkDTi92oSEy9EA04DipHSUs869uE17AYC4dSsR+OyewYfKlqtpAou5WsZ5Xsqj0DXCZOTkogR6NnfX7EU5CwhVmSMqOY8fKS5FQFDOSmd1EkhjhERqQjqYchUR6eyPDJ5qpQ+DSOjiCs7U3xMpCqWchL7uDJEaykVvKv7ndRIVXHkp5XGiCMfzRUHCoIrgNBTYp4JgxSaICyovhXiIdKhKB2dqUNwFl9eJq3zqmNXnbuLUu0mj6MIjsAJKAMHXIauAUN0AQYPIJn8ArejCfjxXg3PuatBSOfOQR/YHz+ADg1lvo=</latexit> <latexit sha1_base64="WPNm1YuVK0X591E2DeKZG6XKYA=">ACB3icbVBNS8NAEN3Urxq/oh4FWSxC1ISEfQiFHvxWMG2kTaUzXbTLt1swu6mUEJuXvwrXjwo4tW/4M1/47bNQVsfDzem2Fmnh8zKpVtfxuFldW19Y3iprm1vbO7Z+0ftGSUCEyaOGKRcH0kCaOcNBVjLixICj0GWn7o/rUb4+JkDTi92oSEy9EA04DipHSUs869uE17AYC4dSsR+OyewYfKlqtpAou5WsZ5Xsqj0DXCZOTkogR6NnfX7EU5CwhVmSMqOY8fKS5FQFDOSmd1EkhjhERqQjqYchUR6eyPDJ5qpQ+DSOjiCs7U3xMpCqWchL7uDJEaykVvKv7ndRIVXHkp5XGiCMfzRUHCoIrgNBTYp4JgxSaICyovhXiIdKhKB2dqUNwFl9eJq3zqmNXnbuLUu0mj6MIjsAJKAMHXIauAUN0AQYPIJn8ArejCfjxXg3PuatBSOfOQR/YHz+ADg1lvo=</latexit> <latexit sha1_base64="WPNm1YuVK0X591E2DeKZG6XKYA=">ACB3icbVBNS8NAEN3Urxq/oh4FWSxC1ISEfQiFHvxWMG2kTaUzXbTLt1swu6mUEJuXvwrXjwo4tW/4M1/47bNQVsfDzem2Fmnh8zKpVtfxuFldW19Y3iprm1vbO7Z+0ftGSUCEyaOGKRcH0kCaOcNBVjLixICj0GWn7o/rUb4+JkDTi92oSEy9EA04DipHSUs869uE17AYC4dSsR+OyewYfKlqtpAou5WsZ5Xsqj0DXCZOTkogR6NnfX7EU5CwhVmSMqOY8fKS5FQFDOSmd1EkhjhERqQjqYchUR6eyPDJ5qpQ+DSOjiCs7U3xMpCqWchL7uDJEaykVvKv7ndRIVXHkp5XGiCMfzRUHCoIrgNBTYp4JgxSaICyovhXiIdKhKB2dqUNwFl9eJq3zqmNXnbuLUu0mj6MIjsAJKAMHXIauAUN0AQYPIJn8ArejCfjxXg3PuatBSOfOQR/YHz+ADg1lvo=</latexit> <latexit sha1_base64="WPNm1YuVK0X591E2DeKZG6XKYA=">ACB3icbVBNS8NAEN3Urxq/oh4FWSxC1ISEfQiFHvxWMG2kTaUzXbTLt1swu6mUEJuXvwrXjwo4tW/4M1/47bNQVsfDzem2Fmnh8zKpVtfxuFldW19Y3iprm1vbO7Z+0ftGSUCEyaOGKRcH0kCaOcNBVjLixICj0GWn7o/rUb4+JkDTi92oSEy9EA04DipHSUs869uE17AYC4dSsR+OyewYfKlqtpAou5WsZ5Xsqj0DXCZOTkogR6NnfX7EU5CwhVmSMqOY8fKS5FQFDOSmd1EkhjhERqQjqYchUR6eyPDJ5qpQ+DSOjiCs7U3xMpCqWchL7uDJEaykVvKv7ndRIVXHkp5XGiCMfzRUHCoIrgNBTYp4JgxSaICyovhXiIdKhKB2dqUNwFl9eJq3zqmNXnbuLUu0mj6MIjsAJKAMHXIauAUN0AQYPIJn8ArejCfjxXg3PuatBSOfOQR/YHz+ADg1lvo=</latexit> <latexit sha1_base64="hwHBCHxlXmNAb1i+1nKjhGPDZ7s=">ACAHicbZDLSgMxFIbP1Fut1EXLtwEi+DGMiOCboSiG5cV7EXaoWTSTBuaZIYkI5RhNr6KGxeKuPUx3Pk2peFtv4Q+PjPOZycP0w408bzvp3C0vLK6lpxvbSxubW94+7uNXScKkLrJOaxaoVYU84krRtmOG0limIRctoMhzfjevORKs1ieW9GCQ0E7ksWMYKNtbruQYiuUCfEKnvI0SkSU27lXbfsVbyJ0CL4MyjDTLWu+9XpxSQVBrCsdZt30tMkGFlGOE0L3VSTRNMhrhP2xYlFlQH2eSAHB1bp4eiWNknDZq4vycyLQeidB2CmwGer42Nv+rtVMTXQYZk0lqCTRVHKkYnROA3UY4oSw0cWMFHM/hWRAVaYGJtZyYbgz5+8CI2ziu9V/LvzcvV6FkcRDuEITsCHC6jCLdSgDgRyeIZXeHOenBfn3fmYthac2cw+/JHz+QOthJUt</latexit> <latexit sha1_base64="hwHBCHxlXmNAb1i+1nKjhGPDZ7s=">ACAHicbZDLSgMxFIbP1Fut1EXLtwEi+DGMiOCboSiG5cV7EXaoWTSTBuaZIYkI5RhNr6KGxeKuPUx3Pk2peFtv4Q+PjPOZycP0w408bzvp3C0vLK6lpxvbSxubW94+7uNXScKkLrJOaxaoVYU84krRtmOG0limIRctoMhzfjevORKs1ieW9GCQ0E7ksWMYKNtbruQYiuUCfEKnvI0SkSU27lXbfsVbyJ0CL4MyjDTLWu+9XpxSQVBrCsdZt30tMkGFlGOE0L3VSTRNMhrhP2xYlFlQH2eSAHB1bp4eiWNknDZq4vycyLQeidB2CmwGer42Nv+rtVMTXQYZk0lqCTRVHKkYnROA3UY4oSw0cWMFHM/hWRAVaYGJtZyYbgz5+8CI2ziu9V/LvzcvV6FkcRDuEITsCHC6jCLdSgDgRyeIZXeHOenBfn3fmYthac2cw+/JHz+QOthJUt</latexit> <latexit sha1_base64="hwHBCHxlXmNAb1i+1nKjhGPDZ7s=">ACAHicbZDLSgMxFIbP1Fut1EXLtwEi+DGMiOCboSiG5cV7EXaoWTSTBuaZIYkI5RhNr6KGxeKuPUx3Pk2peFtv4Q+PjPOZycP0w408bzvp3C0vLK6lpxvbSxubW94+7uNXScKkLrJOaxaoVYU84krRtmOG0limIRctoMhzfjevORKs1ieW9GCQ0E7ksWMYKNtbruQYiuUCfEKnvI0SkSU27lXbfsVbyJ0CL4MyjDTLWu+9XpxSQVBrCsdZt30tMkGFlGOE0L3VSTRNMhrhP2xYlFlQH2eSAHB1bp4eiWNknDZq4vycyLQeidB2CmwGer42Nv+rtVMTXQYZk0lqCTRVHKkYnROA3UY4oSw0cWMFHM/hWRAVaYGJtZyYbgz5+8CI2ziu9V/LvzcvV6FkcRDuEITsCHC6jCLdSgDgRyeIZXeHOenBfn3fmYthac2cw+/JHz+QOthJUt</latexit> <latexit sha1_base64="hwHBCHxlXmNAb1i+1nKjhGPDZ7s=">ACAHicbZDLSgMxFIbP1Fut1EXLtwEi+DGMiOCboSiG5cV7EXaoWTSTBuaZIYkI5RhNr6KGxeKuPUx3Pk2peFtv4Q+PjPOZycP0w408bzvp3C0vLK6lpxvbSxubW94+7uNXScKkLrJOaxaoVYU84krRtmOG0limIRctoMhzfjevORKs1ieW9GCQ0E7ksWMYKNtbruQYiuUCfEKnvI0SkSU27lXbfsVbyJ0CL4MyjDTLWu+9XpxSQVBrCsdZt30tMkGFlGOE0L3VSTRNMhrhP2xYlFlQH2eSAHB1bp4eiWNknDZq4vycyLQeidB2CmwGer42Nv+rtVMTXQYZk0lqCTRVHKkYnROA3UY4oSw0cWMFHM/hWRAVaYGJtZyYbgz5+8CI2ziu9V/LvzcvV6FkcRDuEITsCHC6jCLdSgDgRyeIZXeHOenBfn3fmYthac2cw+/JHz+QOthJUt</latexit> Training with Gradient Descent n ( Y i − ˆ X Y ) 2 = minimize i =1 = ¯ Y − m ¯ b = X Cov ( X, Y ) m = V ar ( X ) 8

  2. <latexit sha1_base64="r/S0kziZA4N57u3eiAn1RfFZ4w=">ACDHicbVDLSsNAFJ3UV62vqks3g0WoC0tSBN0Uim5ctmBfNGmYTCft0MkzEyEvIBbvwVNy4UcesHuPNvnLZaOuBgcM53LnHi9iVCrT/DZya+sbm1v57cLO7t7+QfHwqC3DWGDSwiELRdDkjDKSUtRxUg3EgQFHiMdb3I78zsPREga8ns1jYgToBGnPsVIacktlpqwBm0ZB25Ca1Y6SHha7rkUXkB7jFTS8HVZ0yK+YcJVYGSmBDA23+GUPQxwHhCvMkJR9y4yUkyChKGYkLdixJBHCEzQifU05Coh0kvkxKTzTyhD6odCPKzhXf08kKJByGng6GSA1lsveTPzP68fKv3YSyqNYEY4Xi/yYQRXCWTNwSAXBik01QVhQ/VeIx0grHR/BV2CtXzyKmlXK5ZsZqXpfpNVkcenIBTUAYWuAJ1cAcaoAUweATP4BW8GU/Gi/FufCyiOSObOQZ/YHz+ADvfmdM=</latexit> <latexit sha1_base64="r/S0kziZA4N57u3eiAn1RfFZ4w=">ACDHicbVDLSsNAFJ3UV62vqks3g0WoC0tSBN0Uim5ctmBfNGmYTCft0MkzEyEvIBbvwVNy4UcesHuPNvnLZaOuBgcM53LnHi9iVCrT/DZya+sbm1v57cLO7t7+QfHwqC3DWGDSwiELRdDkjDKSUtRxUg3EgQFHiMdb3I78zsPREga8ns1jYgToBGnPsVIacktlpqwBm0ZB25Ca1Y6SHha7rkUXkB7jFTS8HVZ0yK+YcJVYGSmBDA23+GUPQxwHhCvMkJR9y4yUkyChKGYkLdixJBHCEzQifU05Coh0kvkxKTzTyhD6odCPKzhXf08kKJByGng6GSA1lsveTPzP68fKv3YSyqNYEY4Xi/yYQRXCWTNwSAXBik01QVhQ/VeIx0grHR/BV2CtXzyKmlXK5ZsZqXpfpNVkcenIBTUAYWuAJ1cAcaoAUweATP4BW8GU/Gi/FufCyiOSObOQZ/YHz+ADvfmdM=</latexit> <latexit sha1_base64="r/S0kziZA4N57u3eiAn1RfFZ4w=">ACDHicbVDLSsNAFJ3UV62vqks3g0WoC0tSBN0Uim5ctmBfNGmYTCft0MkzEyEvIBbvwVNy4UcesHuPNvnLZaOuBgcM53LnHi9iVCrT/DZya+sbm1v57cLO7t7+QfHwqC3DWGDSwiELRdDkjDKSUtRxUg3EgQFHiMdb3I78zsPREga8ns1jYgToBGnPsVIacktlpqwBm0ZB25Ca1Y6SHha7rkUXkB7jFTS8HVZ0yK+YcJVYGSmBDA23+GUPQxwHhCvMkJR9y4yUkyChKGYkLdixJBHCEzQifU05Coh0kvkxKTzTyhD6odCPKzhXf08kKJByGng6GSA1lsveTPzP68fKv3YSyqNYEY4Xi/yYQRXCWTNwSAXBik01QVhQ/VeIx0grHR/BV2CtXzyKmlXK5ZsZqXpfpNVkcenIBTUAYWuAJ1cAcaoAUweATP4BW8GU/Gi/FufCyiOSObOQZ/YHz+ADvfmdM=</latexit> <latexit sha1_base64="r/S0kziZA4N57u3eiAn1RfFZ4w=">ACDHicbVDLSsNAFJ3UV62vqks3g0WoC0tSBN0Uim5ctmBfNGmYTCft0MkzEyEvIBbvwVNy4UcesHuPNvnLZaOuBgcM53LnHi9iVCrT/DZya+sbm1v57cLO7t7+QfHwqC3DWGDSwiELRdDkjDKSUtRxUg3EgQFHiMdb3I78zsPREga8ns1jYgToBGnPsVIacktlpqwBm0ZB25Ca1Y6SHha7rkUXkB7jFTS8HVZ0yK+YcJVYGSmBDA23+GUPQxwHhCvMkJR9y4yUkyChKGYkLdixJBHCEzQifU05Coh0kvkxKTzTyhD6odCPKzhXf08kKJByGng6GSA1lsveTPzP68fKv3YSyqNYEY4Xi/yYQRXCWTNwSAXBik01QVhQ/VeIx0grHR/BV2CtXzyKmlXK5ZsZqXpfpNVkcenIBTUAYWuAJ1cAcaoAUweATP4BW8GU/Gi/FufCyiOSObOQZ/YHz+ADvfmdM=</latexit> Training with Gradient Descent n ( Y i − ˆ X Y ) 2 = minimize i =1 9

  3. <latexit sha1_base64="XMJx4vTyfUZ86d4j9mLjvojhdw=">ACMXicbVDLSgMxFM34rPVdekmWIS6aJkpgm4KRTdtmAf0qlDJs20oUlmSDJCGeaX3Pgn4qYLRdz6E6YPUFsPXDg515u7vEjRpW27Ym1tr6xubWd2cnu7u0fHOaOjlsqjCUmTRyUHZ8pAijgjQ1Yx0IkQ9xlp+6Pbqd9+JFLRUNzpcUR6HA0EDShG2kheruYGEuHEjZDUFDHYSH84T2EFuirmXkIrTvqQiBQWyx2PFu49CovQN8XN8K02V4ub5fsGeAqcRYkDxaoe7kXtx/imBOhMUNKdR070r1kuhszkmbdWJEI4REakK6hAnGiesns4hSeG6UPg1CaEhrO1N8TCeJKjblvOjnSQ7XsTcX/vG6sg+teQkUayLwfFEQM6hDOI0P9qkWLOxIQhLav4K8RCZCLUJOWtCcJZPXiWtcsmxS07jMl+9WcSRAafgDBSA65AFdRAHTQBk/gFbyBd+vZmlgf1ue8dc1azJyAP7C+vgE9U6ea</latexit> <latexit sha1_base64="XMJx4vTyfUZ86d4j9mLjvojhdw=">ACMXicbVDLSgMxFM34rPVdekmWIS6aJkpgm4KRTdtmAf0qlDJs20oUlmSDJCGeaX3Pgn4qYLRdz6E6YPUFsPXDg515u7vEjRpW27Ym1tr6xubWd2cnu7u0fHOaOjlsqjCUmTRyUHZ8pAijgjQ1Yx0IkQ9xlp+6Pbqd9+JFLRUNzpcUR6HA0EDShG2kheruYGEuHEjZDUFDHYSH84T2EFuirmXkIrTvqQiBQWyx2PFu49CovQN8XN8K02V4ub5fsGeAqcRYkDxaoe7kXtx/imBOhMUNKdR070r1kuhszkmbdWJEI4REakK6hAnGiesns4hSeG6UPg1CaEhrO1N8TCeJKjblvOjnSQ7XsTcX/vG6sg+teQkUayLwfFEQM6hDOI0P9qkWLOxIQhLav4K8RCZCLUJOWtCcJZPXiWtcsmxS07jMl+9WcSRAafgDBSA65AFdRAHTQBk/gFbyBd+vZmlgf1ue8dc1azJyAP7C+vgE9U6ea</latexit> <latexit sha1_base64="XMJx4vTyfUZ86d4j9mLjvojhdw=">ACMXicbVDLSgMxFM34rPVdekmWIS6aJkpgm4KRTdtmAf0qlDJs20oUlmSDJCGeaX3Pgn4qYLRdz6E6YPUFsPXDg515u7vEjRpW27Ym1tr6xubWd2cnu7u0fHOaOjlsqjCUmTRyUHZ8pAijgjQ1Yx0IkQ9xlp+6Pbqd9+JFLRUNzpcUR6HA0EDShG2kheruYGEuHEjZDUFDHYSH84T2EFuirmXkIrTvqQiBQWyx2PFu49CovQN8XN8K02V4ub5fsGeAqcRYkDxaoe7kXtx/imBOhMUNKdR070r1kuhszkmbdWJEI4REakK6hAnGiesns4hSeG6UPg1CaEhrO1N8TCeJKjblvOjnSQ7XsTcX/vG6sg+teQkUayLwfFEQM6hDOI0P9qkWLOxIQhLav4K8RCZCLUJOWtCcJZPXiWtcsmxS07jMl+9WcSRAafgDBSA65AFdRAHTQBk/gFbyBd+vZmlgf1ue8dc1azJyAP7C+vgE9U6ea</latexit> <latexit sha1_base64="XMJx4vTyfUZ86d4j9mLjvojhdw=">ACMXicbVDLSgMxFM34rPVdekmWIS6aJkpgm4KRTdtmAf0qlDJs20oUlmSDJCGeaX3Pgn4qYLRdz6E6YPUFsPXDg515u7vEjRpW27Ym1tr6xubWd2cnu7u0fHOaOjlsqjCUmTRyUHZ8pAijgjQ1Yx0IkQ9xlp+6Pbqd9+JFLRUNzpcUR6HA0EDShG2kheruYGEuHEjZDUFDHYSH84T2EFuirmXkIrTvqQiBQWyx2PFu49CovQN8XN8K02V4ub5fsGeAqcRYkDxaoe7kXtx/imBOhMUNKdR070r1kuhszkmbdWJEI4REakK6hAnGiesns4hSeG6UPg1CaEhrO1N8TCeJKjblvOjnSQ7XsTcX/vG6sg+teQkUayLwfFEQM6hDOI0P9qkWLOxIQhLav4K8RCZCLUJOWtCcJZPXiWtcsmxS07jMl+9WcSRAafgDBSA65AFdRAHTQBk/gFbyBd+vZmlgf1ue8dc1azJyAP7C+vgE9U6ea</latexit> <latexit sha1_base64="r/S0kziZA4N57u3eiAn1RfFZ4w=">ACDHicbVDLSsNAFJ3UV62vqks3g0WoC0tSBN0Uim5ctmBfNGmYTCft0MkzEyEvIBbvwVNy4UcesHuPNvnLZaOuBgcM53LnHi9iVCrT/DZya+sbm1v57cLO7t7+QfHwqC3DWGDSwiELRdDkjDKSUtRxUg3EgQFHiMdb3I78zsPREga8ns1jYgToBGnPsVIacktlpqwBm0ZB25Ca1Y6SHha7rkUXkB7jFTS8HVZ0yK+YcJVYGSmBDA23+GUPQxwHhCvMkJR9y4yUkyChKGYkLdixJBHCEzQifU05Coh0kvkxKTzTyhD6odCPKzhXf08kKJByGng6GSA1lsveTPzP68fKv3YSyqNYEY4Xi/yYQRXCWTNwSAXBik01QVhQ/VeIx0grHR/BV2CtXzyKmlXK5ZsZqXpfpNVkcenIBTUAYWuAJ1cAcaoAUweATP4BW8GU/Gi/FufCyiOSObOQZ/YHz+ADvfmdM=</latexit> <latexit sha1_base64="r/S0kziZA4N57u3eiAn1RfFZ4w=">ACDHicbVDLSsNAFJ3UV62vqks3g0WoC0tSBN0Uim5ctmBfNGmYTCft0MkzEyEvIBbvwVNy4UcesHuPNvnLZaOuBgcM53LnHi9iVCrT/DZya+sbm1v57cLO7t7+QfHwqC3DWGDSwiELRdDkjDKSUtRxUg3EgQFHiMdb3I78zsPREga8ns1jYgToBGnPsVIacktlpqwBm0ZB25Ca1Y6SHha7rkUXkB7jFTS8HVZ0yK+YcJVYGSmBDA23+GUPQxwHhCvMkJR9y4yUkyChKGYkLdixJBHCEzQifU05Coh0kvkxKTzTyhD6odCPKzhXf08kKJByGng6GSA1lsveTPzP68fKv3YSyqNYEY4Xi/yYQRXCWTNwSAXBik01QVhQ/VeIx0grHR/BV2CtXzyKmlXK5ZsZqXpfpNVkcenIBTUAYWuAJ1cAcaoAUweATP4BW8GU/Gi/FufCyiOSObOQZ/YHz+ADvfmdM=</latexit> <latexit sha1_base64="r/S0kziZA4N57u3eiAn1RfFZ4w=">ACDHicbVDLSsNAFJ3UV62vqks3g0WoC0tSBN0Uim5ctmBfNGmYTCft0MkzEyEvIBbvwVNy4UcesHuPNvnLZaOuBgcM53LnHi9iVCrT/DZya+sbm1v57cLO7t7+QfHwqC3DWGDSwiELRdDkjDKSUtRxUg3EgQFHiMdb3I78zsPREga8ns1jYgToBGnPsVIacktlpqwBm0ZB25Ca1Y6SHha7rkUXkB7jFTS8HVZ0yK+YcJVYGSmBDA23+GUPQxwHhCvMkJR9y4yUkyChKGYkLdixJBHCEzQifU05Coh0kvkxKTzTyhD6odCPKzhXf08kKJByGng6GSA1lsveTPzP68fKv3YSyqNYEY4Xi/yYQRXCWTNwSAXBik01QVhQ/VeIx0grHR/BV2CtXzyKmlXK5ZsZqXpfpNVkcenIBTUAYWuAJ1cAcaoAUweATP4BW8GU/Gi/FufCyiOSObOQZ/YHz+ADvfmdM=</latexit> <latexit sha1_base64="r/S0kziZA4N57u3eiAn1RfFZ4w=">ACDHicbVDLSsNAFJ3UV62vqks3g0WoC0tSBN0Uim5ctmBfNGmYTCft0MkzEyEvIBbvwVNy4UcesHuPNvnLZaOuBgcM53LnHi9iVCrT/DZya+sbm1v57cLO7t7+QfHwqC3DWGDSwiELRdDkjDKSUtRxUg3EgQFHiMdb3I78zsPREga8ns1jYgToBGnPsVIacktlpqwBm0ZB25Ca1Y6SHha7rkUXkB7jFTS8HVZ0yK+YcJVYGSmBDA23+GUPQxwHhCvMkJR9y4yUkyChKGYkLdixJBHCEzQifU05Coh0kvkxKTzTyhD6odCPKzhXf08kKJByGng6GSA1lsveTPzP68fKv3YSyqNYEY4Xi/yYQRXCWTNwSAXBik01QVhQ/VeIx0grHR/BV2CtXzyKmlXK5ZsZqXpfpNVkcenIBTUAYWuAJ1cAcaoAUweATP4BW8GU/Gi/FufCyiOSObOQZ/YHz+ADvfmdM=</latexit> Training with Gradient Descent n ( Y i − ˆ X Y ) 2 = minimize i =1 n ∂ Q X ∂ m = − 2 X i ( Y i − b − mX i ) = 0 i =1 10

  4. <latexit sha1_base64="XMJx4vTyfUZ86d4j9mLjvojhdw=">ACMXicbVDLSgMxFM34rPVdekmWIS6aJkpgm4KRTdtmAf0qlDJs20oUlmSDJCGeaX3Pgn4qYLRdz6E6YPUFsPXDg515u7vEjRpW27Ym1tr6xubWd2cnu7u0fHOaOjlsqjCUmTRyUHZ8pAijgjQ1Yx0IkQ9xlp+6Pbqd9+JFLRUNzpcUR6HA0EDShG2kheruYGEuHEjZDUFDHYSH84T2EFuirmXkIrTvqQiBQWyx2PFu49CovQN8XN8K02V4ub5fsGeAqcRYkDxaoe7kXtx/imBOhMUNKdR070r1kuhszkmbdWJEI4REakK6hAnGiesns4hSeG6UPg1CaEhrO1N8TCeJKjblvOjnSQ7XsTcX/vG6sg+teQkUayLwfFEQM6hDOI0P9qkWLOxIQhLav4K8RCZCLUJOWtCcJZPXiWtcsmxS07jMl+9WcSRAafgDBSA65AFdRAHTQBk/gFbyBd+vZmlgf1ue8dc1azJyAP7C+vgE9U6ea</latexit> <latexit sha1_base64="XMJx4vTyfUZ86d4j9mLjvojhdw=">ACMXicbVDLSgMxFM34rPVdekmWIS6aJkpgm4KRTdtmAf0qlDJs20oUlmSDJCGeaX3Pgn4qYLRdz6E6YPUFsPXDg515u7vEjRpW27Ym1tr6xubWd2cnu7u0fHOaOjlsqjCUmTRyUHZ8pAijgjQ1Yx0IkQ9xlp+6Pbqd9+JFLRUNzpcUR6HA0EDShG2kheruYGEuHEjZDUFDHYSH84T2EFuirmXkIrTvqQiBQWyx2PFu49CovQN8XN8K02V4ub5fsGeAqcRYkDxaoe7kXtx/imBOhMUNKdR070r1kuhszkmbdWJEI4REakK6hAnGiesns4hSeG6UPg1CaEhrO1N8TCeJKjblvOjnSQ7XsTcX/vG6sg+teQkUayLwfFEQM6hDOI0P9qkWLOxIQhLav4K8RCZCLUJOWtCcJZPXiWtcsmxS07jMl+9WcSRAafgDBSA65AFdRAHTQBk/gFbyBd+vZmlgf1ue8dc1azJyAP7C+vgE9U6ea</latexit> <latexit sha1_base64="XMJx4vTyfUZ86d4j9mLjvojhdw=">ACMXicbVDLSgMxFM34rPVdekmWIS6aJkpgm4KRTdtmAf0qlDJs20oUlmSDJCGeaX3Pgn4qYLRdz6E6YPUFsPXDg515u7vEjRpW27Ym1tr6xubWd2cnu7u0fHOaOjlsqjCUmTRyUHZ8pAijgjQ1Yx0IkQ9xlp+6Pbqd9+JFLRUNzpcUR6HA0EDShG2kheruYGEuHEjZDUFDHYSH84T2EFuirmXkIrTvqQiBQWyx2PFu49CovQN8XN8K02V4ub5fsGeAqcRYkDxaoe7kXtx/imBOhMUNKdR070r1kuhszkmbdWJEI4REakK6hAnGiesns4hSeG6UPg1CaEhrO1N8TCeJKjblvOjnSQ7XsTcX/vG6sg+teQkUayLwfFEQM6hDOI0P9qkWLOxIQhLav4K8RCZCLUJOWtCcJZPXiWtcsmxS07jMl+9WcSRAafgDBSA65AFdRAHTQBk/gFbyBd+vZmlgf1ue8dc1azJyAP7C+vgE9U6ea</latexit> <latexit sha1_base64="XMJx4vTyfUZ86d4j9mLjvojhdw=">ACMXicbVDLSgMxFM34rPVdekmWIS6aJkpgm4KRTdtmAf0qlDJs20oUlmSDJCGeaX3Pgn4qYLRdz6E6YPUFsPXDg515u7vEjRpW27Ym1tr6xubWd2cnu7u0fHOaOjlsqjCUmTRyUHZ8pAijgjQ1Yx0IkQ9xlp+6Pbqd9+JFLRUNzpcUR6HA0EDShG2kheruYGEuHEjZDUFDHYSH84T2EFuirmXkIrTvqQiBQWyx2PFu49CovQN8XN8K02V4ub5fsGeAqcRYkDxaoe7kXtx/imBOhMUNKdR070r1kuhszkmbdWJEI4REakK6hAnGiesns4hSeG6UPg1CaEhrO1N8TCeJKjblvOjnSQ7XsTcX/vG6sg+teQkUayLwfFEQM6hDOI0P9qkWLOxIQhLav4K8RCZCLUJOWtCcJZPXiWtcsmxS07jMl+9WcSRAafgDBSA65AFdRAHTQBk/gFbyBd+vZmlgf1ue8dc1azJyAP7C+vgE9U6ea</latexit> <latexit sha1_base64="r/S0kziZA4N57u3eiAn1RfFZ4w=">ACDHicbVDLSsNAFJ3UV62vqks3g0WoC0tSBN0Uim5ctmBfNGmYTCft0MkzEyEvIBbvwVNy4UcesHuPNvnLZaOuBgcM53LnHi9iVCrT/DZya+sbm1v57cLO7t7+QfHwqC3DWGDSwiELRdDkjDKSUtRxUg3EgQFHiMdb3I78zsPREga8ns1jYgToBGnPsVIacktlpqwBm0ZB25Ca1Y6SHha7rkUXkB7jFTS8HVZ0yK+YcJVYGSmBDA23+GUPQxwHhCvMkJR9y4yUkyChKGYkLdixJBHCEzQifU05Coh0kvkxKTzTyhD6odCPKzhXf08kKJByGng6GSA1lsveTPzP68fKv3YSyqNYEY4Xi/yYQRXCWTNwSAXBik01QVhQ/VeIx0grHR/BV2CtXzyKmlXK5ZsZqXpfpNVkcenIBTUAYWuAJ1cAcaoAUweATP4BW8GU/Gi/FufCyiOSObOQZ/YHz+ADvfmdM=</latexit> <latexit sha1_base64="r/S0kziZA4N57u3eiAn1RfFZ4w=">ACDHicbVDLSsNAFJ3UV62vqks3g0WoC0tSBN0Uim5ctmBfNGmYTCft0MkzEyEvIBbvwVNy4UcesHuPNvnLZaOuBgcM53LnHi9iVCrT/DZya+sbm1v57cLO7t7+QfHwqC3DWGDSwiELRdDkjDKSUtRxUg3EgQFHiMdb3I78zsPREga8ns1jYgToBGnPsVIacktlpqwBm0ZB25Ca1Y6SHha7rkUXkB7jFTS8HVZ0yK+YcJVYGSmBDA23+GUPQxwHhCvMkJR9y4yUkyChKGYkLdixJBHCEzQifU05Coh0kvkxKTzTyhD6odCPKzhXf08kKJByGng6GSA1lsveTPzP68fKv3YSyqNYEY4Xi/yYQRXCWTNwSAXBik01QVhQ/VeIx0grHR/BV2CtXzyKmlXK5ZsZqXpfpNVkcenIBTUAYWuAJ1cAcaoAUweATP4BW8GU/Gi/FufCyiOSObOQZ/YHz+ADvfmdM=</latexit> <latexit sha1_base64="r/S0kziZA4N57u3eiAn1RfFZ4w=">ACDHicbVDLSsNAFJ3UV62vqks3g0WoC0tSBN0Uim5ctmBfNGmYTCft0MkzEyEvIBbvwVNy4UcesHuPNvnLZaOuBgcM53LnHi9iVCrT/DZya+sbm1v57cLO7t7+QfHwqC3DWGDSwiELRdDkjDKSUtRxUg3EgQFHiMdb3I78zsPREga8ns1jYgToBGnPsVIacktlpqwBm0ZB25Ca1Y6SHha7rkUXkB7jFTS8HVZ0yK+YcJVYGSmBDA23+GUPQxwHhCvMkJR9y4yUkyChKGYkLdixJBHCEzQifU05Coh0kvkxKTzTyhD6odCPKzhXf08kKJByGng6GSA1lsveTPzP68fKv3YSyqNYEY4Xi/yYQRXCWTNwSAXBik01QVhQ/VeIx0grHR/BV2CtXzyKmlXK5ZsZqXpfpNVkcenIBTUAYWuAJ1cAcaoAUweATP4BW8GU/Gi/FufCyiOSObOQZ/YHz+ADvfmdM=</latexit> <latexit sha1_base64="r/S0kziZA4N57u3eiAn1RfFZ4w=">ACDHicbVDLSsNAFJ3UV62vqks3g0WoC0tSBN0Uim5ctmBfNGmYTCft0MkzEyEvIBbvwVNy4UcesHuPNvnLZaOuBgcM53LnHi9iVCrT/DZya+sbm1v57cLO7t7+QfHwqC3DWGDSwiELRdDkjDKSUtRxUg3EgQFHiMdb3I78zsPREga8ns1jYgToBGnPsVIacktlpqwBm0ZB25Ca1Y6SHha7rkUXkB7jFTS8HVZ0yK+YcJVYGSmBDA23+GUPQxwHhCvMkJR9y4yUkyChKGYkLdixJBHCEzQifU05Coh0kvkxKTzTyhD6odCPKzhXf08kKJByGng6GSA1lsveTPzP68fKv3YSyqNYEY4Xi/yYQRXCWTNwSAXBik01QVhQ/VeIx0grHR/BV2CtXzyKmlXK5ZsZqXpfpNVkcenIBTUAYWuAJ1cAcaoAUweATP4BW8GU/Gi/FufCyiOSObOQZ/YHz+ADvfmdM=</latexit> Training with Gradient Descent n ( Y i − ˆ X Y ) 2 = minimize i =1 n ∂ Q X ∂ m = − 2 X i ( Y i − b − mX i ) = 0 i =1 11

  5. Training with Gradient Descent 12 https://independentseminarblog.com/2018/01/12/moving-below-the-surface-3-gradient-descent-william/

  6. <latexit sha1_base64="XMJx4vTyfUZ86d4j9mLjvojhdw=">ACMXicbVDLSgMxFM34rPVdekmWIS6aJkpgm4KRTdtmAf0qlDJs20oUlmSDJCGeaX3Pgn4qYLRdz6E6YPUFsPXDg515u7vEjRpW27Ym1tr6xubWd2cnu7u0fHOaOjlsqjCUmTRyUHZ8pAijgjQ1Yx0IkQ9xlp+6Pbqd9+JFLRUNzpcUR6HA0EDShG2kheruYGEuHEjZDUFDHYSH84T2EFuirmXkIrTvqQiBQWyx2PFu49CovQN8XN8K02V4ub5fsGeAqcRYkDxaoe7kXtx/imBOhMUNKdR070r1kuhszkmbdWJEI4REakK6hAnGiesns4hSeG6UPg1CaEhrO1N8TCeJKjblvOjnSQ7XsTcX/vG6sg+teQkUayLwfFEQM6hDOI0P9qkWLOxIQhLav4K8RCZCLUJOWtCcJZPXiWtcsmxS07jMl+9WcSRAafgDBSA65AFdRAHTQBk/gFbyBd+vZmlgf1ue8dc1azJyAP7C+vgE9U6ea</latexit> <latexit sha1_base64="XMJx4vTyfUZ86d4j9mLjvojhdw=">ACMXicbVDLSgMxFM34rPVdekmWIS6aJkpgm4KRTdtmAf0qlDJs20oUlmSDJCGeaX3Pgn4qYLRdz6E6YPUFsPXDg515u7vEjRpW27Ym1tr6xubWd2cnu7u0fHOaOjlsqjCUmTRyUHZ8pAijgjQ1Yx0IkQ9xlp+6Pbqd9+JFLRUNzpcUR6HA0EDShG2kheruYGEuHEjZDUFDHYSH84T2EFuirmXkIrTvqQiBQWyx2PFu49CovQN8XN8K02V4ub5fsGeAqcRYkDxaoe7kXtx/imBOhMUNKdR070r1kuhszkmbdWJEI4REakK6hAnGiesns4hSeG6UPg1CaEhrO1N8TCeJKjblvOjnSQ7XsTcX/vG6sg+teQkUayLwfFEQM6hDOI0P9qkWLOxIQhLav4K8RCZCLUJOWtCcJZPXiWtcsmxS07jMl+9WcSRAafgDBSA65AFdRAHTQBk/gFbyBd+vZmlgf1ue8dc1azJyAP7C+vgE9U6ea</latexit> <latexit sha1_base64="XMJx4vTyfUZ86d4j9mLjvojhdw=">ACMXicbVDLSgMxFM34rPVdekmWIS6aJkpgm4KRTdtmAf0qlDJs20oUlmSDJCGeaX3Pgn4qYLRdz6E6YPUFsPXDg515u7vEjRpW27Ym1tr6xubWd2cnu7u0fHOaOjlsqjCUmTRyUHZ8pAijgjQ1Yx0IkQ9xlp+6Pbqd9+JFLRUNzpcUR6HA0EDShG2kheruYGEuHEjZDUFDHYSH84T2EFuirmXkIrTvqQiBQWyx2PFu49CovQN8XN8K02V4ub5fsGeAqcRYkDxaoe7kXtx/imBOhMUNKdR070r1kuhszkmbdWJEI4REakK6hAnGiesns4hSeG6UPg1CaEhrO1N8TCeJKjblvOjnSQ7XsTcX/vG6sg+teQkUayLwfFEQM6hDOI0P9qkWLOxIQhLav4K8RCZCLUJOWtCcJZPXiWtcsmxS07jMl+9WcSRAafgDBSA65AFdRAHTQBk/gFbyBd+vZmlgf1ue8dc1azJyAP7C+vgE9U6ea</latexit> <latexit sha1_base64="XMJx4vTyfUZ86d4j9mLjvojhdw=">ACMXicbVDLSgMxFM34rPVdekmWIS6aJkpgm4KRTdtmAf0qlDJs20oUlmSDJCGeaX3Pgn4qYLRdz6E6YPUFsPXDg515u7vEjRpW27Ym1tr6xubWd2cnu7u0fHOaOjlsqjCUmTRyUHZ8pAijgjQ1Yx0IkQ9xlp+6Pbqd9+JFLRUNzpcUR6HA0EDShG2kheruYGEuHEjZDUFDHYSH84T2EFuirmXkIrTvqQiBQWyx2PFu49CovQN8XN8K02V4ub5fsGeAqcRYkDxaoe7kXtx/imBOhMUNKdR070r1kuhszkmbdWJEI4REakK6hAnGiesns4hSeG6UPg1CaEhrO1N8TCeJKjblvOjnSQ7XsTcX/vG6sg+teQkUayLwfFEQM6hDOI0P9qkWLOxIQhLav4K8RCZCLUJOWtCcJZPXiWtcsmxS07jMl+9WcSRAafgDBSA65AFdRAHTQBk/gFbyBd+vZmlgf1ue8dc1azJyAP7C+vgE9U6ea</latexit> <latexit sha1_base64="9cIJCpfvS+XsZ9OkPHz9MbmKXfE=">ACLnicbVBdS8MwFE3n15xfVR9CQ5hPjaIeiLIrg4wZuTtZa0ix1YUlaklQYpb/IF/+KPgq4qs/w3QO1OmBkM593LvPWHCqNKO82yVZmbn5hfKi5Wl5ZXVNXt9o6PiVGLSxjGLZTdEijAqSFtTzUg3kQTxkJHLcHha+Je3RCoaiws9SojP0Y2gEcVIGymwz7xIpx5CZKaIgZb+TcPc3gEPZXyIKNHbn6diRzuNWpXAYV7kHfHX7hrapzArjp1Zwz4l7gTUgUTNAP70evHOVEaMyQUj3XSbSfFYMxI3nFSxVJEB6iG9IzVCBOlJ+Nz83hjlH6MIqleULDsfqzI0NcqRE36+9wpAdq2ivE/7xeqNDP6MiSTUR+GtQlDKoY1hkB/tUEqzZyBCEJTW7QjxAJj9tEq6YENzpk/+STqPuOnW3tV89PpnEUQZbYBvUgAsOwDE4B03QBhjcgQfwAl6te+vJerPev0pL1qRnE/yC9fEJufumUQ=</latexit> <latexit sha1_base64="9cIJCpfvS+XsZ9OkPHz9MbmKXfE=">ACLnicbVBdS8MwFE3n15xfVR9CQ5hPjaIeiLIrg4wZuTtZa0ix1YUlaklQYpb/IF/+KPgq4qs/w3QO1OmBkM593LvPWHCqNKO82yVZmbn5hfKi5Wl5ZXVNXt9o6PiVGLSxjGLZTdEijAqSFtTzUg3kQTxkJHLcHha+Je3RCoaiws9SojP0Y2gEcVIGymwz7xIpx5CZKaIgZb+TcPc3gEPZXyIKNHbn6diRzuNWpXAYV7kHfHX7hrapzArjp1Zwz4l7gTUgUTNAP70evHOVEaMyQUj3XSbSfFYMxI3nFSxVJEB6iG9IzVCBOlJ+Nz83hjlH6MIqleULDsfqzI0NcqRE36+9wpAdq2ivE/7xeqNDP6MiSTUR+GtQlDKoY1hkB/tUEqzZyBCEJTW7QjxAJj9tEq6YENzpk/+STqPuOnW3tV89PpnEUQZbYBvUgAsOwDE4B03QBhjcgQfwAl6te+vJerPev0pL1qRnE/yC9fEJufumUQ=</latexit> <latexit sha1_base64="9cIJCpfvS+XsZ9OkPHz9MbmKXfE=">ACLnicbVBdS8MwFE3n15xfVR9CQ5hPjaIeiLIrg4wZuTtZa0ix1YUlaklQYpb/IF/+KPgq4qs/w3QO1OmBkM593LvPWHCqNKO82yVZmbn5hfKi5Wl5ZXVNXt9o6PiVGLSxjGLZTdEijAqSFtTzUg3kQTxkJHLcHha+Je3RCoaiws9SojP0Y2gEcVIGymwz7xIpx5CZKaIgZb+TcPc3gEPZXyIKNHbn6diRzuNWpXAYV7kHfHX7hrapzArjp1Zwz4l7gTUgUTNAP70evHOVEaMyQUj3XSbSfFYMxI3nFSxVJEB6iG9IzVCBOlJ+Nz83hjlH6MIqleULDsfqzI0NcqRE36+9wpAdq2ivE/7xeqNDP6MiSTUR+GtQlDKoY1hkB/tUEqzZyBCEJTW7QjxAJj9tEq6YENzpk/+STqPuOnW3tV89PpnEUQZbYBvUgAsOwDE4B03QBhjcgQfwAl6te+vJerPev0pL1qRnE/yC9fEJufumUQ=</latexit> <latexit sha1_base64="9cIJCpfvS+XsZ9OkPHz9MbmKXfE=">ACLnicbVBdS8MwFE3n15xfVR9CQ5hPjaIeiLIrg4wZuTtZa0ix1YUlaklQYpb/IF/+KPgq4qs/w3QO1OmBkM593LvPWHCqNKO82yVZmbn5hfKi5Wl5ZXVNXt9o6PiVGLSxjGLZTdEijAqSFtTzUg3kQTxkJHLcHha+Je3RCoaiws9SojP0Y2gEcVIGymwz7xIpx5CZKaIgZb+TcPc3gEPZXyIKNHbn6diRzuNWpXAYV7kHfHX7hrapzArjp1Zwz4l7gTUgUTNAP70evHOVEaMyQUj3XSbSfFYMxI3nFSxVJEB6iG9IzVCBOlJ+Nz83hjlH6MIqleULDsfqzI0NcqRE36+9wpAdq2ivE/7xeqNDP6MiSTUR+GtQlDKoY1hkB/tUEqzZyBCEJTW7QjxAJj9tEq6YENzpk/+STqPuOnW3tV89PpnEUQZbYBvUgAsOwDE4B03QBhjcgQfwAl6te+vJerPev0pL1qRnE/yC9fEJufumUQ=</latexit> <latexit sha1_base64="WPNm1YuVK0X591E2DeKZG6XKYA=">ACB3icbVBNS8NAEN3Urxq/oh4FWSxC1ISEfQiFHvxWMG2kTaUzXbTLt1swu6mUEJuXvwrXjwo4tW/4M1/47bNQVsfDzem2Fmnh8zKpVtfxuFldW19Y3iprm1vbO7Z+0ftGSUCEyaOGKRcH0kCaOcNBVjLixICj0GWn7o/rUb4+JkDTi92oSEy9EA04DipHSUs869uE17AYC4dSsR+OyewYfKlqtpAou5WsZ5Xsqj0DXCZOTkogR6NnfX7EU5CwhVmSMqOY8fKS5FQFDOSmd1EkhjhERqQjqYchUR6eyPDJ5qpQ+DSOjiCs7U3xMpCqWchL7uDJEaykVvKv7ndRIVXHkp5XGiCMfzRUHCoIrgNBTYp4JgxSaICyovhXiIdKhKB2dqUNwFl9eJq3zqmNXnbuLUu0mj6MIjsAJKAMHXIauAUN0AQYPIJn8ArejCfjxXg3PuatBSOfOQR/YHz+ADg1lvo=</latexit> <latexit sha1_base64="WPNm1YuVK0X591E2DeKZG6XKYA=">ACB3icbVBNS8NAEN3Urxq/oh4FWSxC1ISEfQiFHvxWMG2kTaUzXbTLt1swu6mUEJuXvwrXjwo4tW/4M1/47bNQVsfDzem2Fmnh8zKpVtfxuFldW19Y3iprm1vbO7Z+0ftGSUCEyaOGKRcH0kCaOcNBVjLixICj0GWn7o/rUb4+JkDTi92oSEy9EA04DipHSUs869uE17AYC4dSsR+OyewYfKlqtpAou5WsZ5Xsqj0DXCZOTkogR6NnfX7EU5CwhVmSMqOY8fKS5FQFDOSmd1EkhjhERqQjqYchUR6eyPDJ5qpQ+DSOjiCs7U3xMpCqWchL7uDJEaykVvKv7ndRIVXHkp5XGiCMfzRUHCoIrgNBTYp4JgxSaICyovhXiIdKhKB2dqUNwFl9eJq3zqmNXnbuLUu0mj6MIjsAJKAMHXIauAUN0AQYPIJn8ArejCfjxXg3PuatBSOfOQR/YHz+ADg1lvo=</latexit> <latexit sha1_base64="WPNm1YuVK0X591E2DeKZG6XKYA=">ACB3icbVBNS8NAEN3Urxq/oh4FWSxC1ISEfQiFHvxWMG2kTaUzXbTLt1swu6mUEJuXvwrXjwo4tW/4M1/47bNQVsfDzem2Fmnh8zKpVtfxuFldW19Y3iprm1vbO7Z+0ftGSUCEyaOGKRcH0kCaOcNBVjLixICj0GWn7o/rUb4+JkDTi92oSEy9EA04DipHSUs869uE17AYC4dSsR+OyewYfKlqtpAou5WsZ5Xsqj0DXCZOTkogR6NnfX7EU5CwhVmSMqOY8fKS5FQFDOSmd1EkhjhERqQjqYchUR6eyPDJ5qpQ+DSOjiCs7U3xMpCqWchL7uDJEaykVvKv7ndRIVXHkp5XGiCMfzRUHCoIrgNBTYp4JgxSaICyovhXiIdKhKB2dqUNwFl9eJq3zqmNXnbuLUu0mj6MIjsAJKAMHXIauAUN0AQYPIJn8ArejCfjxXg3PuatBSOfOQR/YHz+ADg1lvo=</latexit> <latexit sha1_base64="WPNm1YuVK0X591E2DeKZG6XKYA=">ACB3icbVBNS8NAEN3Urxq/oh4FWSxC1ISEfQiFHvxWMG2kTaUzXbTLt1swu6mUEJuXvwrXjwo4tW/4M1/47bNQVsfDzem2Fmnh8zKpVtfxuFldW19Y3iprm1vbO7Z+0ftGSUCEyaOGKRcH0kCaOcNBVjLixICj0GWn7o/rUb4+JkDTi92oSEy9EA04DipHSUs869uE17AYC4dSsR+OyewYfKlqtpAou5WsZ5Xsqj0DXCZOTkogR6NnfX7EU5CwhVmSMqOY8fKS5FQFDOSmd1EkhjhERqQjqYchUR6eyPDJ5qpQ+DSOjiCs7U3xMpCqWchL7uDJEaykVvKv7ndRIVXHkp5XGiCMfzRUHCoIrgNBTYp4JgxSaICyovhXiIdKhKB2dqUNwFl9eJq3zqmNXnbuLUu0mj6MIjsAJKAMHXIauAUN0AQYPIJn8ArejCfjxXg3PuatBSOfOQR/YHz+ADg1lvo=</latexit> <latexit sha1_base64="mxdaQB8GmfgkBu9RKA5ZbS6fR8=">ACkHicdVHdSsMwGE3r/ybeulNcAjbxUY7Bb1wOPVGvJrgdLJuJc1SDSZpSVJhlD6P7+Odb2O6daCbfhA4Od/5TpKTIGZUacf5suyl5ZXVtfWN0ubW9s5ueW/UWJxKSLIxbJXoAUYVSQrqakV4sCeIBI0/B203ef3onUtFIPOhxTAYcvQgaUoy0ofzyB4ct6IUS4dRTCfdT2nKzYSoyWH32ac+nsA69AMn0OTObWjavMuSwOdP0ck32j2E1NzOetToUhbowXjCtTlxr9Zlu2Mz8csVpOJOCi8AtQAU1fHLn94owgknQmOGlOq7TqwHKZKaYkaykpcoEiP8hl5I30CBOFGDdBJoBo8NM4JhJM0SGk7YnxMp4kqNeWCUHOlXNd/Lyb96/USH54OUijRODpQWHCoI5g/jtwRCXBmo0NQFhSc1eIX5EJU5s/LJkQ3PknL4LHZsN1Gu79aV9XcSxDg7BEagCF5yBNrgFHdAF2Nq2TqwLq2Xv2+f2pX01ldpWMXMAfpV9w3+Zca5</latexit> <latexit sha1_base64="mxdaQB8GmfgkBu9RKA5ZbS6fR8=">ACkHicdVHdSsMwGE3r/ybeulNcAjbxUY7Bb1wOPVGvJrgdLJuJc1SDSZpSVJhlD6P7+Odb2O6daCbfhA4Od/5TpKTIGZUacf5suyl5ZXVtfWN0ubW9s5ueW/UWJxKSLIxbJXoAUYVSQrqakV4sCeIBI0/B203ef3onUtFIPOhxTAYcvQgaUoy0ofzyB4ct6IUS4dRTCfdT2nKzYSoyWH32ac+nsA69AMn0OTObWjavMuSwOdP0ck32j2E1NzOetToUhbowXjCtTlxr9Zlu2Mz8csVpOJOCi8AtQAU1fHLn94owgknQmOGlOq7TqwHKZKaYkaykpcoEiP8hl5I30CBOFGDdBJoBo8NM4JhJM0SGk7YnxMp4kqNeWCUHOlXNd/Lyb96/USH54OUijRODpQWHCoI5g/jtwRCXBmo0NQFhSc1eIX5EJU5s/LJkQ3PknL4LHZsN1Gu79aV9XcSxDg7BEagCF5yBNrgFHdAF2Nq2TqwLq2Xv2+f2pX01ldpWMXMAfpV9w3+Zca5</latexit> <latexit sha1_base64="mxdaQB8GmfgkBu9RKA5ZbS6fR8=">ACkHicdVHdSsMwGE3r/ybeulNcAjbxUY7Bb1wOPVGvJrgdLJuJc1SDSZpSVJhlD6P7+Odb2O6daCbfhA4Od/5TpKTIGZUacf5suyl5ZXVtfWN0ubW9s5ueW/UWJxKSLIxbJXoAUYVSQrqakV4sCeIBI0/B203ef3onUtFIPOhxTAYcvQgaUoy0ofzyB4ct6IUS4dRTCfdT2nKzYSoyWH32ac+nsA69AMn0OTObWjavMuSwOdP0ck32j2E1NzOetToUhbowXjCtTlxr9Zlu2Mz8csVpOJOCi8AtQAU1fHLn94owgknQmOGlOq7TqwHKZKaYkaykpcoEiP8hl5I30CBOFGDdBJoBo8NM4JhJM0SGk7YnxMp4kqNeWCUHOlXNd/Lyb96/USH54OUijRODpQWHCoI5g/jtwRCXBmo0NQFhSc1eIX5EJU5s/LJkQ3PknL4LHZsN1Gu79aV9XcSxDg7BEagCF5yBNrgFHdAF2Nq2TqwLq2Xv2+f2pX01ldpWMXMAfpV9w3+Zca5</latexit> <latexit sha1_base64="mxdaQB8GmfgkBu9RKA5ZbS6fR8=">ACkHicdVHdSsMwGE3r/ybeulNcAjbxUY7Bb1wOPVGvJrgdLJuJc1SDSZpSVJhlD6P7+Odb2O6daCbfhA4Od/5TpKTIGZUacf5suyl5ZXVtfWN0ubW9s5ueW/UWJxKSLIxbJXoAUYVSQrqakV4sCeIBI0/B203ef3onUtFIPOhxTAYcvQgaUoy0ofzyB4ct6IUS4dRTCfdT2nKzYSoyWH32ac+nsA69AMn0OTObWjavMuSwOdP0ck32j2E1NzOetToUhbowXjCtTlxr9Zlu2Mz8csVpOJOCi8AtQAU1fHLn94owgknQmOGlOq7TqwHKZKaYkaykpcoEiP8hl5I30CBOFGDdBJoBo8NM4JhJM0SGk7YnxMp4kqNeWCUHOlXNd/Lyb96/USH54OUijRODpQWHCoI5g/jtwRCXBmo0NQFhSc1eIX5EJU5s/LJkQ3PknL4LHZsN1Gu79aV9XcSxDg7BEagCF5yBNrgFHdAF2Nq2TqwLq2Xv2+f2pX01ldpWMXMAfpV9w3+Zca5</latexit> <latexit sha1_base64="hwHBCHxlXmNAb1i+1nKjhGPDZ7s=">ACAHicbZDLSgMxFIbP1Fut1EXLtwEi+DGMiOCboSiG5cV7EXaoWTSTBuaZIYkI5RhNr6KGxeKuPUx3Pk2peFtv4Q+PjPOZycP0w408bzvp3C0vLK6lpxvbSxubW94+7uNXScKkLrJOaxaoVYU84krRtmOG0limIRctoMhzfjevORKs1ieW9GCQ0E7ksWMYKNtbruQYiuUCfEKnvI0SkSU27lXbfsVbyJ0CL4MyjDTLWu+9XpxSQVBrCsdZt30tMkGFlGOE0L3VSTRNMhrhP2xYlFlQH2eSAHB1bp4eiWNknDZq4vycyLQeidB2CmwGer42Nv+rtVMTXQYZk0lqCTRVHKkYnROA3UY4oSw0cWMFHM/hWRAVaYGJtZyYbgz5+8CI2ziu9V/LvzcvV6FkcRDuEITsCHC6jCLdSgDgRyeIZXeHOenBfn3fmYthac2cw+/JHz+QOthJUt</latexit> <latexit sha1_base64="hwHBCHxlXmNAb1i+1nKjhGPDZ7s=">ACAHicbZDLSgMxFIbP1Fut1EXLtwEi+DGMiOCboSiG5cV7EXaoWTSTBuaZIYkI5RhNr6KGxeKuPUx3Pk2peFtv4Q+PjPOZycP0w408bzvp3C0vLK6lpxvbSxubW94+7uNXScKkLrJOaxaoVYU84krRtmOG0limIRctoMhzfjevORKs1ieW9GCQ0E7ksWMYKNtbruQYiuUCfEKnvI0SkSU27lXbfsVbyJ0CL4MyjDTLWu+9XpxSQVBrCsdZt30tMkGFlGOE0L3VSTRNMhrhP2xYlFlQH2eSAHB1bp4eiWNknDZq4vycyLQeidB2CmwGer42Nv+rtVMTXQYZk0lqCTRVHKkYnROA3UY4oSw0cWMFHM/hWRAVaYGJtZyYbgz5+8CI2ziu9V/LvzcvV6FkcRDuEITsCHC6jCLdSgDgRyeIZXeHOenBfn3fmYthac2cw+/JHz+QOthJUt</latexit> <latexit sha1_base64="hwHBCHxlXmNAb1i+1nKjhGPDZ7s=">ACAHicbZDLSgMxFIbP1Fut1EXLtwEi+DGMiOCboSiG5cV7EXaoWTSTBuaZIYkI5RhNr6KGxeKuPUx3Pk2peFtv4Q+PjPOZycP0w408bzvp3C0vLK6lpxvbSxubW94+7uNXScKkLrJOaxaoVYU84krRtmOG0limIRctoMhzfjevORKs1ieW9GCQ0E7ksWMYKNtbruQYiuUCfEKnvI0SkSU27lXbfsVbyJ0CL4MyjDTLWu+9XpxSQVBrCsdZt30tMkGFlGOE0L3VSTRNMhrhP2xYlFlQH2eSAHB1bp4eiWNknDZq4vycyLQeidB2CmwGer42Nv+rtVMTXQYZk0lqCTRVHKkYnROA3UY4oSw0cWMFHM/hWRAVaYGJtZyYbgz5+8CI2ziu9V/LvzcvV6FkcRDuEITsCHC6jCLdSgDgRyeIZXeHOenBfn3fmYthac2cw+/JHz+QOthJUt</latexit> <latexit sha1_base64="hwHBCHxlXmNAb1i+1nKjhGPDZ7s=">ACAHicbZDLSgMxFIbP1Fut1EXLtwEi+DGMiOCboSiG5cV7EXaoWTSTBuaZIYkI5RhNr6KGxeKuPUx3Pk2peFtv4Q+PjPOZycP0w408bzvp3C0vLK6lpxvbSxubW94+7uNXScKkLrJOaxaoVYU84krRtmOG0limIRctoMhzfjevORKs1ieW9GCQ0E7ksWMYKNtbruQYiuUCfEKnvI0SkSU27lXbfsVbyJ0CL4MyjDTLWu+9XpxSQVBrCsdZt30tMkGFlGOE0L3VSTRNMhrhP2xYlFlQH2eSAHB1bp4eiWNknDZq4vycyLQeidB2CmwGer42Nv+rtVMTXQYZk0lqCTRVHKkYnROA3UY4oSw0cWMFHM/hWRAVaYGJtZyYbgz5+8CI2ziu9V/LvzcvV6FkcRDuEITsCHC6jCLdSgDgRyeIZXeHOenBfn3fmYthac2cw+/JHz+QOthJUt</latexit> <latexit sha1_base64="q3o/vVOTz7GUx+RiBjGcRGq1f0o=">ACD3icbVDLSgMxFM3UV62vUZdugkVpEctMEXRTKLpx2YJ9SB9DJk3b0CQzJBmhDP0DN/6KGxeKuHXrzr8xbWehrQcu93DOvST3+CGjSjvOt5VaWV1b30hvZra2d3b37P2DugoiUkNByQTR8pwqgNU01I81QEsR9Rhr+6GbqNx6IVDQd3ockg5HA0H7FCNtJM8+rcISbKuIezEtuZNuLCa5e4/Cc5jTdPoJ/Pd4uenXUKzgxwmbgJyYIEFc/+avcCHEiNGZIqZbrhLoTI6kpZmSaUeKhAiP0IC0DBWIE9WJZ/dM4IlRerAfSFNCw5n6eyNGXKkx980kR3qoFr2p+J/XinT/qhNTEUaCDx/qB8xqAM4DQf2qCRYs7EhCEtq/grxEmEtYkwY0JwF09eJvViwXUKbvUiW75O4kiDI3AMcsAFl6AMbkEF1AGj+AZvI368l6sd6tj/loykp2DsEfWJ8/aLmZsg=</latexit> <latexit sha1_base64="q3o/vVOTz7GUx+RiBjGcRGq1f0o=">ACD3icbVDLSgMxFM3UV62vUZdugkVpEctMEXRTKLpx2YJ9SB9DJk3b0CQzJBmhDP0DN/6KGxeKuHXrzr8xbWehrQcu93DOvST3+CGjSjvOt5VaWV1b30hvZra2d3b37P2DugoiUkNByQTR8pwqgNU01I81QEsR9Rhr+6GbqNx6IVDQd3ockg5HA0H7FCNtJM8+rcISbKuIezEtuZNuLCa5e4/Cc5jTdPoJ/Pd4uenXUKzgxwmbgJyYIEFc/+avcCHEiNGZIqZbrhLoTI6kpZmSaUeKhAiP0IC0DBWIE9WJZ/dM4IlRerAfSFNCw5n6eyNGXKkx980kR3qoFr2p+J/XinT/qhNTEUaCDx/qB8xqAM4DQf2qCRYs7EhCEtq/grxEmEtYkwY0JwF09eJvViwXUKbvUiW75O4kiDI3AMcsAFl6AMbkEF1AGj+AZvI368l6sd6tj/loykp2DsEfWJ8/aLmZsg=</latexit> <latexit sha1_base64="q3o/vVOTz7GUx+RiBjGcRGq1f0o=">ACD3icbVDLSgMxFM3UV62vUZdugkVpEctMEXRTKLpx2YJ9SB9DJk3b0CQzJBmhDP0DN/6KGxeKuHXrzr8xbWehrQcu93DOvST3+CGjSjvOt5VaWV1b30hvZra2d3b37P2DugoiUkNByQTR8pwqgNU01I81QEsR9Rhr+6GbqNx6IVDQd3ockg5HA0H7FCNtJM8+rcISbKuIezEtuZNuLCa5e4/Cc5jTdPoJ/Pd4uenXUKzgxwmbgJyYIEFc/+avcCHEiNGZIqZbrhLoTI6kpZmSaUeKhAiP0IC0DBWIE9WJZ/dM4IlRerAfSFNCw5n6eyNGXKkx980kR3qoFr2p+J/XinT/qhNTEUaCDx/qB8xqAM4DQf2qCRYs7EhCEtq/grxEmEtYkwY0JwF09eJvViwXUKbvUiW75O4kiDI3AMcsAFl6AMbkEF1AGj+AZvI368l6sd6tj/loykp2DsEfWJ8/aLmZsg=</latexit> <latexit sha1_base64="q3o/vVOTz7GUx+RiBjGcRGq1f0o=">ACD3icbVDLSgMxFM3UV62vUZdugkVpEctMEXRTKLpx2YJ9SB9DJk3b0CQzJBmhDP0DN/6KGxeKuHXrzr8xbWehrQcu93DOvST3+CGjSjvOt5VaWV1b30hvZra2d3b37P2DugoiUkNByQTR8pwqgNU01I81QEsR9Rhr+6GbqNx6IVDQd3ockg5HA0H7FCNtJM8+rcISbKuIezEtuZNuLCa5e4/Cc5jTdPoJ/Pd4uenXUKzgxwmbgJyYIEFc/+avcCHEiNGZIqZbrhLoTI6kpZmSaUeKhAiP0IC0DBWIE9WJZ/dM4IlRerAfSFNCw5n6eyNGXKkx980kR3qoFr2p+J/XinT/qhNTEUaCDx/qB8xqAM4DQf2qCRYs7EhCEtq/grxEmEtYkwY0JwF09eJvViwXUKbvUiW75O4kiDI3AMcsAFl6AMbkEF1AGj+AZvI368l6sd6tj/loykp2DsEfWJ8/aLmZsg=</latexit> Training with Gradient Descent Helpful equations for following along in the jupyter notebook n X ( Y i − ( mX i + b )) 2 Q = i =1 n ∂ Q X ∂ b = − 2( Y i − mX i − b ) = 0 i =1 n ∂ Q X ∂ m = − 2 X i ( Y i − b − mX i ) = 0 i =1 = Cov ( X, Y ) b = ¯ Y − m ¯ m = X V ar ( X ) 13 https://independentseminarblog.com/2018/01/12/moving-below-the-surface-3-gradient-descent-william/

  7. Supervised vs. Unsupervised Learning • Supervised: Explicit data labels • Sentiment analysis—review text -> star ratings • Image tagging—image -> caption • Unsupervised: No explicit labels • Clustering—find groups similar customers • Dimensionality Reduction—find features that differentiate individuals 14

  8. Supervised vs. Unsupervised Learning • Supervised: Explicit data labels • Sentiment analysis—review text -> star ratings • Image tagging—image -> caption • Unsupervised: No explicit labels • Clustering—find groups similar customers • Dimensionality Reduction—find features that differentiate individuals 15

  9. Supervised vs. Unsupervised Learning • Supervised: Explicit data labels • Sentiment analysis—review text -> star ratings • Image tagging—image -> caption • Unsupervised: No explicit labels • Clustering—find groups similar customers • Dimensionality Reduction—find features that differentiate individuals 16

  10. Supervised vs. Unsupervised Learning • Supervised: Explicit data labels • Sentiment analysis—review text -> star ratings • Image tagging—image -> caption • Unsupervised: No explicit labels • Clustering—find groups similar customers • Dimensionality Reduction—find features that differentiate individuals 17

  11. Supervised vs. Unsupervised Learning • Supervised: Explicit data labels • Sentiment analysis—review text -> star ratings • Image tagging—image -> caption • Unsupervised: No explicit labels • Clustering—find groups similar customers • Dimensionality Reduction—find features that differentiate individuals 18

  12. Supervised vs. Unsupervised Learning • Supervised: Explicit data labels • Sentiment analysis—review text -> star ratings • Image tagging—image -> caption • Unsupervised: No explicit labels • Clustering—find groups similar customers • Dimensionality Reduction—find features that differentiate individuals 19

  13. Supervised vs. Unsupervised Learning • Supervised: Explicit data labels • Sentiment analysis—review text -> star ratings • Image tagging—image -> caption • Unsupervised: No explicit labels • Clustering—find groups similar customers • Dimensionality Reduction—find features that differentiate individuals 20

  14. Supervised vs. Unsupervised Learning • Supervised: Explicit data labels • Sentiment analysis—review text -> star ratings • Image tagging—image -> caption Today • Unsupervised: No explicit labels • Clustering—find groups similar customers • Dimensionality Reduction—find features that differentiate individuals 21

  15. Supervised vs. Unsupervised Learning • Supervised: Explicit data labels • Sentiment analysis—review text -> star ratings • Image tagging—image -> caption Today • Unsupervised: No explicit labels • Clustering—find groups similar customers • Dimensionality Reduction—find features that differentiate individuals 22

  16. Supervised vs. Unsupervised Learning • Supervised: Explicit data labels • Sentiment analysis—review text -> star ratings • Image tagging—image -> caption Tuesday • Unsupervised: No explicit labels • Clustering—find groups similar customers • Dimensionality Reduction—find features that differentiate individuals 23

  17. Oh you thought it was that simple? How cute… • Semi Supervised—Combining large amounts of unlabelled with smaller amounts of labelled (pretraining) • Weakly/Distantly Supervised—using noisy labels or partial labels (bootstrapping, automatically-labeled data) • Reinforcement Learning—label on the result of a sequence of actions, but not on each action (games, robotics) • “Found” Data… (?) 24

  18. Oh you thought it was that simple? How cute… • Semi Supervised—Combining large amounts of unlabelled with smaller amounts of labelled (pretraining) • Weakly/Distantly Supervised—using noisy labels or partial labels (bootstrapping, automatically-labeled data) • Reinforcement Learning—label on the result of a sequence of actions, but not on each action (games, robotics) • “Found” Data… (?) 25

  19. Oh you thought it was that simple? How cute… • Semi Supervised—Combining large amounts of unlabelled with smaller amounts of labelled (pretraining) • Weakly/Distantly Supervised—using noisy labels or partial labels (bootstrapping, automatically-labeled data) • Reinforcement Learning—label on the result of a sequence of actions, but not on each action (games, robotics) • “Found” Data… (?) 26

  20. Oh you thought it was that simple? How cute… • Semi Supervised—Combining large amounts of unlabelled with smaller amounts of labelled (pretraining) • Weakly/Distantly Supervised—using noisy labels or partial labels (bootstrapping, automatically-labeled data) • Reinforcement Learning—label on the result of a sequence of actions, but not on each action (games, robotics) • “Found” Data… (?) 27

  21. Oh you thought it was that simple? How cute… • Semi Supervised—Combining large amounts of unlabelled with smaller amounts of labelled (pretraining) • Weakly/Distantly Supervised—using noisy labels or partial labels (bootstrapping, automatically-labeled data) • Reinforcement Learning—label on the result of a sequence of actions, but not on each action (games, robotics) • “Found” Data… (?) 28

  22. Unsupervised Learning • “Finding structure in data” • In data science, this is typically for “exploratory analysis”. “What the $@%! is this data even?! Enlighten me.” • Or for preprocessing/featurizing—e.g. so you can use article “topics” to predict clicks. • In ML, right now, used extensively for “pretraining” (e.g. autoencoding, dimensionality reduction, language modeling*) 29

  23. Unsupervised Learning • “Finding structure in data” (vs. predicting labels) • In data science, this is typically for “exploratory analysis”. “What the $@%! is this data even?! Enlighten me.” • Or for preprocessing/featurizing—e.g. so you can use article “topics” to predict clicks. • In ML, right now, used extensively for “pretraining” (e.g. autoencoding, dimensionality reduction, language modeling*) 30

  24. Unsupervised Learning • “Finding structure in data” (vs. predicting labels) • In data science, this is typically for “exploratory analysis”. “What the $@%! is this data even?! Enlighten me.” • Or for preprocessing/featurizing—e.g. so you can use article “topics” to predict clicks. • In ML, right now, used extensively for “pretraining” (e.g. autoencoding, dimensionality reduction, language modeling*) 31

  25. Unsupervised Learning • “Finding structure in data” (vs. predicting labels) • In data science, this is typically for “exploratory analysis”. “What the $@%! is this data even?! Enlighten me.” • Or for preprocessing/featurizing—e.g. so you can use article “topics” to predict clicks. • In ML, right now, used extensively for “pretraining” (e.g. autoencoding, dimensionality reduction, language modeling*) 32

  26. Unsupervised Learning • “Finding structure in data” (vs. predicting labels) • In data science, this is typically for “exploratory analysis”. “What the $@%! is this data even?! Enlighten me.” • Or for preprocessing/featurizing—e.g. so you can use article “topics” to predict clicks. • In ML, right now, used extensively for “pretraining” (e.g. autoencoding, dimensionality reduction, language modeling*) 33

  27. Clicker Question! 34

  28. Clustering Discussion Question! What is it good for…? (…because those free-form answers were enlightening last time…) 35

  29. Clustering • Find groups of customers with similar tastes • Find topics within a set of news articles • Find genres within a library of music • Extrapolating—make predictions about your new business based on behavior of similar old businesses 36

  30. harmonic complexity Clustering tempo 37

  31. harmonic complexity Clustering tempo 38

  32. K Means define parameters: K, max_iter, min_diff iter = 0 change = inf means = [random() for _ in range(K)] while iter < max_iter and change > min_diff: update_assignments() compute_new_means() change = max_i(dist(new_mean_i, old_mean_i)) iter += 1 39

  33. K Means “Hyperparameters” (i.e. not model parameters) define parameters: K, max_iter, min_diff iter = 0 change = inf means = [random() for _ in range(K)] while iter < max_iter and change > min_diff: update_assignments() compute_new_means() change = max_i(dist(new_mean_i, old_mean_i)) iter += 1 40

  34. K Means How many clusters we want to find define parameters: K, max_iter, min_diff iter = 0 change = inf means = [random() for _ in range(K)] while iter < max_iter and change > min_diff: update_assignments() compute_new_means() change = max_i(dist(new_mean_i, old_mean_i)) iter += 1 41

  35. K Means When to quit. Things aren’ t changing, or we have gotten bored. define parameters: K, max_iter, min_diff iter = 0 change = inf means = [random() for _ in range(K)] while iter < max_iter and change > min_diff: update_assignments() compute_new_means() change = max_i(dist(new_mean_i, old_mean_i)) iter += 1 42

  36. K Means Randomly guess what the means are (lots of ways to do this) define parameters: K, max_iter, min_diff iter = 0 change = inf means = [random() for _ in range(K)] while iter < max_iter and change > min_diff: update_assignments() compute_new_means() change = max_i(dist(new_mean_i, old_mean_i)) iter += 1 43

  37. K Means Repeat until your hyperparameters say to stop define parameters: K, max_iter, min_diff iter = 0 change = inf means = [random() for _ in range(K)] while iter < max_iter and change > min_diff: update_assignments() compute_new_means() change = max_i(dist(new_mean_i, old_mean_i)) iter += 1 44

  38. K Means Assign each point to its closest mean define parameters: K, max_iter, min_diff iter = 0 change = inf means = [random() for _ in range(K)] while iter < max_iter and change > min_diff: update_assignments() compute_new_means() change = max_i(dist(new_mean_i, old_mean_i)) iter += 1 45

  39. K Means Recompute the means to be the mean of the points assigned to each cluster define parameters: K, max_iter, min_diff iter = 0 change = inf means = [random() for _ in range(K)] while iter < max_iter and change > min_diff: update_assignments() compute_new_means() change = max_i(dist(new_mean_i, old_mean_i)) iter += 1 46

  40. harmonic complexity K Means tempo 47

  41. K Means guess what the means are harmonic complexity X X tempo 48

  42. K Means Assign each point to harmonic complexity closest mean X X tempo 49

  43. K Means re-compute means to be center of harmonic complexity clusters X X tempo 50

  44. K Means Assign each point to harmonic complexity closest mean X X tempo 51

  45. K Means re-compute means to be center of harmonic complexity clusters X X tempo 52

  46. K Means Assign each point to harmonic complexity closest mean X X tempo 53

  47. K Means re-compute means to be center of harmonic complexity clusters X X tempo 54

  48. K Means harmonic complexity Converged! X X tempo 55

  49. Clicker Question! 56 56

  50. Clicker Question! What is the “loss” that we are trying to minimize here? (a)Number of clusters (b)Distance of points to their respective clusters (c) Distance between clusters (d)Probability of observed data 57 57

  51. Clicker Question! What is the “loss” that we are trying to minimize here? (a)Number of clusters (b)Distance of points to their respective clusters (c) Distance between clusters (d)Probability of observed data 58 58

  52. Clicker Question! What is the “loss” that we are trying to minimize here? (a)Number of clusters (b)Distance of points to their respective clusters (c) Distance between clusters (d)Probability of observed data This in just a few slides! 59 59

  53. Clicker(/Discussion) Question! Is this a good objective? (a)Yes (b)No (c) Sure, why not. 60 60

  54. Clicker(/Discussion) Question! Is this a good objective? (a)Yes (b)No (c) Sure, why not. Potential problems? (Hint: hyperparameters, generalization…) 61 61

  55. How many clusters? mean dist to center X X 1 2 3 4 …….. N 62

  56. How many clusters? mean dist to center X X 1 2 3 4 …….. N 63

  57. How many clusters? mean dist to center X 1 2 3 4 …….. N 64

  58. How many clusters? mean dist to center X X X X X X X X X X X X X X 1 2 3 4 …….. N 65

  59. How many clusters? mean dist to center X “Elbow Point” X 1 2 3 4 …….. N 66

  60. Other techniques: How many clusters? • Silhouette • Intuition/Divine Intervention • LGTM mean dist to center X “Elbow Point” X 1 2 3 4 …….. N 67

  61. Other techniques: How many clusters? • Silhouette • Intuition/Divine Intervention • LGTM mean dist to center X “Elbow Point” distance to own cluster X distance to next best cluster 1 2 3 4 …….. N 68

  62. 69

  63. Expectation Maximization (EM) 70

  64. Expectation Maximization (EM) define parameters: K, max_iter, min_diff iter = 0 change = inf means = [random() for _ in range(K)] while iter < max_iter and change > min_diff: update_assignments() compute_new_means() change = max_i(dist(new_mean_i, old_mean_i)) iter += 1 71

  65. Expectation Maximization (EM) define parameters: K, max_iter, min_diff iter = 0 change = inf randomly initialize params while not converged: data = estimate_likelihood(params) params = maximize_likelihood(data) change = max_i(dist(new_mean_i, old_mean_i)) iter += 1 72

  66. Expectation Maximization (EM) define parameters: K, max_iter, min_diff E Step: estimate the likelihood of data iter = 0 under current parameter setting change = inf randomly initialize params while not converged: data = estimate_likelihood(params) params = maximize_likelihood(data) change = max_i(dist(new_mean_i, old_mean_i)) iter += 1 73

  67. Expectation Maximization (EM) define parameters: K, max_iter, min_diff M Step: adjust the the parameters so as to iter = 0 maximize the expectation of the data change = inf randomly initialize params while not converged: data = estimate_likelihood(params) params = maximize_likelihood(data) change = max_i(dist(new_mean_i, old_mean_i)) iter += 1 74

  68. Expectation Maximization (EM) E step: Assign each point to harmonic complexity closest mean X X tempo http://www.dirkhovy.com/portfolio/papers/download/an_evening_with_EM.pdf 75

  69. Expectation Maximization (EM) M step: Compute harmonic complexity means to be X center of clusters X tempo http://www.dirkhovy.com/portfolio/papers/download/an_evening_with_EM.pdf 76

  70. Expectation Maximization (EM) harmonic complexity 0.4 X 0.6 X EM -> “soft” K-Means, where points belong to a probability distribution over clusters… tempo http://www.dirkhovy.com/portfolio/papers/download/an_evening_with_EM.pdf 77

  71. 
 #tbt Slide from crowdsourcing lecture Quality Control Was arrested actress Heather • Why was Heather Heather Locklear Arrested for Locklear because of the driving under driving under the influence of drugs Locklear arrested? 
 the effect of an unknown medicine The actress Heather Locklear, The actress Heather Locklear that Driving while medicated is known to the Amanda through Amanda of the popular series Melrose Place, was arrested this the role from the series "Melrose • Why did the bystander weekend in Santa Barbara Place" was arrested at this Incentive Pay (California) after driving under the weekend in Santa Barbara (Californium) because of the influence of drugs. A witness call emergency driving under the effect of an viewed her performing inappropriate maneuvers while unknown medicine. A female services? trying to take her car out from a witness observed she attempted parking in Montecito, as revealed in quite strange way how to go from their parking space in to People magazine by a There was a lot of noise Montecito, speaker of the traffic spokesman for the Californian police of californium told the Highway Police. The witness • Where did the witness stated that around 4.30pm Ms. warehouse `People'. The female witness told in detail, that Locklear Locklear "hit the accelerator very 'pressed `after 16:30 clock violently, making excessive noise see her acting while trying to take her car out accelerator and a lot of noise did from the parking with abrupt back when she attempted to move their abnormally? 
 and forth maneuvers. While car towards behind or forward from the parking space, and when reversing, she passed several 
 it went backwards, she pulled times in front of his sunglasses." itself together unites Male at their Shortly after, the witness, who, in In a parking lot a first time, apparently had not sunglasses'. A little later the recognized the actress, saw Ms. female witness that did probably Second-Pass HIT Statistical Models 78

  72. 
 #tbt Slide from crowdsourcing lecture Quality Control Was arrested actress Heather • Why was Heather Heather Locklear Arrested for Locklear because of the driving under driving under the influence of drugs Locklear arrested? 
 the effect of an unknown medicine The actress Heather Locklear, The actress Heather Locklear that Driving while medicated is known to the Amanda through Amanda of the popular series the role from the series "Melrose Melrose Place, was arrested this • Why did the bystander Place" was arrested at this weekend in Santa Barbara Incentive Pay weekend in Santa Barbara (California) after driving under the (Californium) because of the influence of drugs. A witness call emergency driving under the effect of an viewed her performing inappropriate maneuvers while unknown medicine. A female services? witness observed she attempted trying to take her car out from a in quite strange way how to go parking in Montecito, as revealed from their parking space in to People magazine by a There was a lot of noise spokesman for the Californian Montecito, speaker of the traffic police of californium told the Highway Police. The witness • Where did the witness warehouse `People'. The female stated that around 4.30pm Ms. Locklear "hit the accelerator very witness told in detail, that Locklear 'pressed `after 16:30 clock violently, making excessive noise see her acting while trying to take her car out accelerator and a lot of noise did when she attempted to move their from the parking with abrupt back abnormally? 
 car towards behind or forward and forth maneuvers. While from the parking space, and when reversing, she passed several 
 it went backwards, she pulled times in front of his sunglasses." Shortly after, the witness, who, in itself together unites Male at their In a parking lot sunglasses'. A little later the a first time, apparently had not female witness that did probably recognized the actress, saw Ms. Second-Pass HIT Statistical Models (That I don’ t think we actually covered, but its cool its fine…) 79

  73. Goal: Find “true” labels despite noisy annotations from workers… worker1 worker2 worker3 worker4 worker5 email1 spam not not not spam email2 spam spam spam spam spam email3 not spam not not spam email4 spam spam spam spam not email5 spam not not not spam 80

  74. Goal: Find “true” labels despite noisy annotations from workers… worker1 worker2 worker3 worker4 worker5 email1 spam not not not spam Easy! If you tell me how much to trust each worker, I can trivially email2 spam spam spam spam spam compute labels email3 not spam not not spam email4 spam spam spam spam not email5 spam not not not spam 81

  75. Goal: Find “true” labels despite noisy annotations from workers… worker1 worker2 worker3 worker4 worker5 email1 spam not not not spam Easy! If you tell me how much to trust each worker, I can trivially email2 spam spam spam spam spam compute labels Sure, just tell me email3 not spam not not spam the labels and I can easily figure out which email4 spam spam spam spam not workers to trust. email5 spam not not not spam 82

  76. Goal: Find “true” labels despite EM EVERYTHING!!!! noisy annotations from workers… worker1 worker2 worker3 worker4 worker5 email1 spam not not not spam Easy! If you tell me how much to trust each worker, I can trivially email2 spam spam spam spam spam compute labels Sure, just tell me email3 not spam not not spam the labels and I can easily figure out which email4 spam spam spam spam not workers to trust. email5 spam not not not spam 83

  77. spam not w1 w2 w3 w4 w5 w1 spam ? ? email1 spam not not not spam not ? ? email2 spam spam spam spam spam spam not w2 email3 not spam not not spam spam ? ? email4 spam spam spam spam not not ? ? email5 spam not not not spam spam not w3 spam not spam ? ? email1 ? ? not ? ? spam not w4 email2 ? ? spam ? ? email3 ? ? not ? ? email4 ? ? spam not w5 spam ? ? email5 ? ? not ? ? 84

  78. spam not w1 w2 w3 w4 w5 w1 spam ? ? email1 spam not not not spam not ? ? email2 spam spam spam spam spam spam not w2 email3 not spam not not spam spam ? ? email4 spam spam spam spam not not ? ? email5 spam not not not spam P(email1 is spam) spam not w3 spam not spam ? ? email1 ? ? not ? ? spam not w4 email2 ? ? spam ? ? email3 ? ? not ? ? email4 ? ? spam not w5 spam ? ? email5 ? ? not ? ? 85

  79. spam not w1 w2 w3 w4 w5 w1 spam ? ? email1 spam not not not spam not ? ? email2 spam spam spam spam spam spam not w2 email3 not spam not not spam P(w1 says spam | not spam) spam ? ? email4 spam spam spam spam not not ? ? email5 spam not not not spam spam not w3 spam not spam ? ? email1 ? ? not ? ? spam not w4 email2 ? ? spam ? ? email3 ? ? not ? ? email4 ? ? spam not w5 spam ? ? email5 ? ? not ? ? 86

  80. spam not w1 w2 w3 w4 w5 w1 spam 1 0 email1 spam not not not spam not 0 1 email2 spam spam spam spam spam Assume spam not w2 email3 not spam not not spam spam 1 0 email4 spam spam spam spam not all workers not 0 1 email5 spam not not not spam are spam not w3 perfect spam not spam 1 0 email1 ? ? not 0 1 spam not w4 email2 ? ? spam 1 0 email3 ? ? not 0 1 email4 ? ? spam not w5 spam 1 0 email5 ? ? not 0 1 87

  81. spam not w1 w2 w3 w4 w5 w1 spam 1 0 email1 spam not not not spam not 0 1 email2 spam spam spam spam spam Compute spam not w2 email3 not spam not not spam labels spam 1 0 email4 spam spam spam spam not using not 0 1 email5 spam not not not spam majority spam not w3 vote spam not spam 1 0 email1 ? ? not 0 1 spam not w4 email2 ? ? spam 1 0 email3 ? ? not 0 1 email4 ? ? spam not w5 spam 1 0 email5 ? ? not 0 1 88

  82. Clicker Question! 89 89

  83. spam not w1 w2 w3 w4 w5 w1 spam 1 0 email1 spam not not not spam not 0 1 email2 spam spam spam spam spam Compute spam not w2 email3 not spam not not spam labels spam 1 0 email4 spam spam spam spam not using not 0 1 email5 spam not not not spam majority spam not w3 vote spam not spam 1 0 email1 ? ? not 0 1 Clicker Question! spam not w4 email2 ? ? spam 1 0 (a)0.4, 0.6 email3 ? ? not 0 1 (b)0.6, 0.4 email4 ? ? spam not w5 (c) 0.8, 0.2 spam 1 0 (d)1.0, 0.0 email5 ? ? not 0 1 90

  84. spam not w1 w2 w3 w4 w5 w1 spam 1 0 email1 spam not not not spam not 0 1 email2 spam spam spam spam spam Compute spam not w2 email3 not spam not not spam labels spam 1 0 email4 spam spam spam spam not using not 0 1 email5 spam not not not spam majority spam not w3 vote spam not spam 1 0 email1 ? ? not 0 1 Clicker Question! spam not w4 email2 ? ? spam 1 0 (a)0.4, 0.6 email3 ? ? not 0 1 (b)0.6, 0.4 email4 ? ? spam not w5 (c) 0.8, 0.2 spam 1 0 (d)1.0, 0.0 email5 ? ? not 0 1 91

  85. w1 w5 spam not w2 w3 w4 w1 spam 1 0 email1 spam not not not spam not 0 1 email2 spam spam spam spam spam Compute spam not w2 email3 not spam not not spam labels spam 1 0 email4 spam spam spam spam not using not 0 1 email5 spam not not not spam majority spam not w3 vote spam not spam 1 0 email1 0.4 ? not 0 1 spam not w4 email2 ? ? spam 1 0 email3 ? ? not 0 1 email4 ? ? spam not w5 spam 1 0 email5 ? ? not 0 1 92

  86. w2 w3 w4 spam not w1 w5 w1 spam 1 0 email1 spam not not not spam not 0 1 email2 spam spam spam spam spam Compute spam not w2 email3 not spam not not spam labels spam 1 0 email4 spam spam spam spam not using not 0 1 email5 spam not not not spam majority spam not w3 vote spam not spam 1 0 email1 0.4 0.6 not 0 1 spam not w4 email2 ? ? spam 1 0 email3 ? ? not 0 1 email4 ? ? spam not w5 spam 1 0 email5 ? ? not 0 1 93

  87. spam not w1 w2 w3 w4 w5 w1 spam 1 0 email1 spam not not not spam not 0 1 email2 spam spam spam spam spam Compute spam not w2 email3 not spam not not spam labels spam 1 0 email4 spam spam spam spam not using not 0 1 email5 spam not not not spam majority spam not w3 vote spam not spam 1 0 email1 0.4 0.6 not 0 1 spam not w4 email2 1 0 spam 1 0 email3 0.4 0.6 not 0 1 email4 0.8 0.2 spam not w5 spam 1 0 email5 0.4 0.6 not 0 1 94

  88. spam not w1 w2 w3 w4 w5 w1 spam 1 0 email1 spam not not not spam not 0 1 email2 spam spam spam spam spam Assume spam not w2 email3 not spam not not spam these labels, and spam 1 0 email4 spam spam spam spam not recompute not 0 1 email5 spam not not not spam confusion spam not w3 matrices spam not spam 1 0 email1 0.4 0.6 not 0 1 spam not w4 email2 1 0 spam 1 0 email3 0.4 0.6 not 0 1 email4 0.8 0.2 spam not w5 spam 1 0 email5 0.4 0.6 not 0 1 95

  89. spam not w1 w2 w3 w4 w5 w1 spam email1 spam not not not spam not email2 spam spam spam spam spam Assume spam not w2 email3 not spam not not spam these labels, and spam email4 spam spam spam spam not recompute not email5 spam not not not spam confusion spam not w3 matrices spam not spam email1 0.4 0.6 not spam not w4 email2 1 0 spam email3 0.4 0.6 not email4 0.8 0.2 spam not w5 spam email5 0.4 0.6 not 96

  90. Clicker Question! 97 97

  91. spam not w1 w2 w3 w4 w5 w1 spam ? ? email1 spam not not not spam not email2 spam spam spam spam spam Assume spam not w2 email3 not spam not not spam these labels, and spam email4 spam spam spam spam not Clicker Question! recompute not email5 spam not not not spam confusion (a)0.4, 0.6 spam not w3 matrices spam not (b)0.6, 0.4 spam (c) 0.8, 0.2 email1 0.4 0.6 not (d)1.0, 0.0 spam not w4 email2 1 0 spam email3 0.4 0.6 not email4 0.8 0.2 spam not w5 spam email5 0.4 0.6 not 98

  92. spam not w1 w2 w3 w4 w5 w1 spam ? ? email1 spam not not not spam not email2 spam spam spam spam spam Assume spam not w2 email3 not spam not not spam these labels, and spam email4 spam spam spam spam not Clicker Question! recompute not email5 spam not not not spam confusion (a)0.4, 0.6 spam not w3 matrices spam not (b)0.6, 0.4 spam (c) 0.8, 0.2 email1 0.4 0.6 not (d)1.0, 0.0 spam not w4 email2 1 0 spam email3 0.4 0.6 not email4 0.8 0.2 spam not w5 spam email5 0.4 0.6 not 99

  93. spam not w1 w2 w3 w4 w5 w1 spam 1 email1 spam not not not spam not email2 spam spam spam spam spam Assume spam not w2 email3 not spam not not spam these labels, and spam 1 0 email4 spam spam spam spam not recompute not 0 1 email5 spam not not not spam confusion spam not w3 matrices spam not spam 1 0 email1 0.4 0.6 not 0 1 spam not w4 email2 1 0 spam 1 0 email3 0.4 0.6 not 0 1 email4 0.8 0.2 spam not w5 spam 1 0 email5 0.4 0.6 not 0 1 100

Recommend

Learning From Data Lecture 19 A Peek At Unsupervised Learning k -Means Clustering Probability

Learning From Data Lecture 19 A Peek At Unsupervised Learning k -Means Clustering Probability

Learning From Data Lecture 19 A Peek At Unsupervised Learning k -Means Clustering Probability Density Estimation Gaussian Mixture Models M. Magdon-Ismail CSCI 4100/6100 recap: Radial Basis Functions Nonparametric RBF Parametric k -RBF-Network

253 views • 23 slides
UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised

UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised

UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised learning: X - y pairs, f(x) function approximation Unsupervised learning: only X, no y Exploring the space of X measurements,

440 views • 14 slides
Introduction to PCA Unsupervised Learning in R Unsupervised learning Two methods of

Introduction to PCA Unsupervised Learning in R Unsupervised learning Two methods of

UNSUPERVISED LEARNING IN R Introduction to PCA Unsupervised Learning in R Unsupervised learning Two methods of clustering finding groups of homogeneous items Next up, dimensionality reduction Find structure in features

904 views • 29 slides
Unsupervised Learning Marco Chiarandini Department of Mathematics &amp; Computer Science

Unsupervised Learning Marco Chiarandini Department of Mathematics &amp; Computer Science

DM825 Introduction to Machine Learning Lecture 13 Unsupervised Learning Marco Chiarandini Department of Mathematics &amp; Computer Science University of Southern Denmark k -means Outline EM Algorithm 1. k -means 2. Expectation Maximization

712 views • 16 slides
Machine Learning Fall 2017 Unsupervised Learning (Clustering: k -means, EM, mixture models)

Machine Learning Fall 2017 Unsupervised Learning (Clustering: k -means, EM, mixture models)

Machine Learning Fall 2017 Unsupervised Learning (Clustering: k -means, EM, mixture models) Professor Liang Huang (Chaps. 15-16 of CIML) Roadmap CIML Chaps. 3, 4,5,7,11,17 ,18 so far: (large-margin) supervised learning online learning:

815 views • 54 slides
K-Means an example of unsupervised learning CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu When

K-Means an example of unsupervised learning CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu When

K-Means an example of unsupervised learning CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu When applying a learning algorithm, some things are properties of the problem you are trying to solve, and some things are up to you to choose as the ML

581 views • 19 slides
Unsupervised Learning About this class Build a model for your data. Which datapoints

Unsupervised Learning About this class Build a model for your data. Which datapoints

Unsupervised Learning About this class Build a model for your data. Which datapoints Unsupervised learning are similar? k -means Clustering Nowadays lots of work on using unlabeled data to improve the performance of supervised learn-

450 views • 6 slides
Unsupervised Learning Principal Component Analysis CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu

Unsupervised Learning Principal Component Analysis CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu

Unsupervised Learning Principal Component Analysis CMSC 422 M ARINE C ARPUAT marine@cs.umd.edu Slides credit: Maria-Florina Balcan Unsupervised Learning Discovering hidden structure in data Last time: K-Means Clustering What is the

502 views • 15 slides
Machine Learning Supervised Learning Unsupervised Learning CSE 446: Expectation Maximization

Machine Learning Supervised Learning Unsupervised Learning CSE 446: Expectation Maximization

Machine Learning Supervised Learning Unsupervised Learning CSE 446: Expectation Maximization Reinforcement Learning (EM) Parametric Non-parametric Winter 2012 Fri K means &amp; Agglomerative Clustering Daniel Weld Mon Expectation

329 views • 10 slides
K-Means Clustering 3/3/17 Unsupervised Learning We have a collection of unlabeled data

K-Means Clustering 3/3/17 Unsupervised Learning We have a collection of unlabeled data

K-Means Clustering 3/3/17 Unsupervised Learning We have a collection of unlabeled data points. We want to find underlying structure in the data. Examples: Identify groups of similar data points. Clustering Find a better basis

831 views • 14 slides
Unsupervised Learning and Clustering l In unsupervised learning you are given a data set with no

Unsupervised Learning and Clustering l In unsupervised learning you are given a data set with no

Unsupervised Learning and Clustering l In unsupervised learning you are given a data set with no output classifications (labels) l Clustering is an important type of unsupervised learning PCA was another type of unsupervised learning l The

858 views • 38 slides
Unsupervised Learning Unsupervised vs Supervised Learning: Most of this course focuses on

Unsupervised Learning Unsupervised vs Supervised Learning: Most of this course focuses on

Unsupervised Learning Unsupervised vs Supervised Learning: Most of this course focuses on supervised learning methods such as regression and classification. In that setting we observe both a set of features X 1 , X 2 , . . . , X p for each

934 views • 50 slides
Unsupervised Learning Unsupervised vs Supervised Learning: Most of this course focuses on

Unsupervised Learning Unsupervised vs Supervised Learning: Most of this course focuses on

Unsupervised Learning Unsupervised vs Supervised Learning: Most of this course focuses on supervised learning methods such as regression and classification. In that setting we observe both a set of features X 1 , X 2 , . . . , X p for each

721 views • 60 slides
Unsupervised learning D. Dubhashi D. Dubhashi Introduction Introduction Everything weve

Unsupervised learning D. Dubhashi D. Dubhashi Introduction Introduction Everything weve

Introduction Introduction Unsupervised learning D. Dubhashi D. Dubhashi Introduction Introduction Everything weve seen so far has been supervised K-means K-means TDA231 Kernel K-means Kernel K-means We were given a set of x n

550 views • 13 slides
Unsupervised Learning Andrea Passerini passerini@disi.unitn.it Machine Learning Unsupervised

Unsupervised Learning Andrea Passerini passerini@disi.unitn.it Machine Learning Unsupervised

Unsupervised Learning Andrea Passerini passerini@disi.unitn.it Machine Learning Unsupervised Learning Unsupervised Learning Setting Supervised learning requires the availability of labelled examples Labelling examples can be an extremely

752 views • 27 slides
12. Unsupervised Deep Learning CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from

12. Unsupervised Deep Learning CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from

12. Unsupervised Deep Learning CS 535 Deep Learning, Winter 2018 Fuxin Li With materials from Wanli Ouyang, Zsolt Kira, Lawrence Neal, Raymond Yeh, Junting Lou and Teck-Yian Lim Unsupervised Learning in General Unsupervised learning is

1.48k views • 115 slides
Machine Learning for NLP Unsupervised Learning Aurlie Herbelot 2019 Centre for Mind/Brain

Machine Learning for NLP Unsupervised Learning Aurlie Herbelot 2019 Centre for Mind/Brain

Machine Learning for NLP Unsupervised Learning Aurlie Herbelot 2019 Centre for Mind/Brain Sciences University of Trento 1 Unsupervised learning In unsupervised learning, we learn without training data. The idea is to find a

998 views • 67 slides
Unsupervised Learning Introduction Nakul Verma Unsupervised Learning What can we learn from

Unsupervised Learning Introduction Nakul Verma Unsupervised Learning What can we learn from

Unsupervised Learning Introduction Nakul Verma Unsupervised Learning What can we learn from data when label information is not available? Supervised learning framework Supervised learning Data: Assumption: there is a (relatively simple)

442 views • 16 slides
Practical Unsupervised Learning INFO/CS 4300, Spring 2016 Jack Hessel Unsupervised Learning is

Practical Unsupervised Learning INFO/CS 4300, Spring 2016 Jack Hessel Unsupervised Learning is

Practical Unsupervised Learning INFO/CS 4300, Spring 2016 Jack Hessel Unsupervised Learning is Cool! But how can we use this in our projects? 1. 2. But first, lets look at our dataset... Data Dimensionality! Probability of Lung Cancer

999 views • 55 slides
Automatic Learning of a Morphological Model Theory and Unsupervised Approaches Unsupervised

Automatic Learning of a Morphological Model Theory and Unsupervised Approaches Unsupervised

Automatic Learning of a Morphological Model Theory and Unsupervised Approaches Unsupervised Learning - ToC Foundations Problem description General architecture Papers: Goldsmith 2001 and 2006 Goldsmith and Hu 2004

1.23k views • 43 slides
9.54 Class 13 Unsupervised learning Clustering Shimon Ullman + Tomaso Poggio Danny Harari +

9.54 Class 13 Unsupervised learning Clustering Shimon Ullman + Tomaso Poggio Danny Harari +

9.54 Class 13 Unsupervised learning Clustering Shimon Ullman + Tomaso Poggio Danny Harari + Daneil Zysman + Darren Seibert Outline Introduction to clustering K-means Bag of words (dictionary learning) Hierarchical clustering

895 views • 54 slides
Unsupervised learning introduction October 7, 2019 Unsupervised learning introduction

Unsupervised learning introduction October 7, 2019 Unsupervised learning introduction

Unsupervised learning introduction October 7, 2019 Unsupervised learning introduction October 7, 2019 1 / 39 Intro General statement of the problem One has a set of N observations ( x 1 , x 2 , ..., x N ) of a random p -vector X having

495 views • 32 slides
Unsupervised learning General introduction to unsupervised learning PCA Special directions

Unsupervised learning General introduction to unsupervised learning PCA Special directions

Unsupervised learning General introduction to unsupervised learning PCA Special directions These are special directions we will try to find. Best direction u : |u| 2 = 1 2 1. Minimize : d i X i T u is the projection length x i u d i T

1.01k views • 29 slides
When Dictionary Learning Meets Classification Bufford, Teresa 1 Chen, Yuxin 2 Horning, Mitchell 3

When Dictionary Learning Meets Classification Bufford, Teresa 1 Chen, Yuxin 2 Horning, Mitchell 3

The Method Supervised Unsupervised - spectral Unsupervised - k-means Gaussian Noise Experiments Additional Material When Dictionary Learning Meets Classification Bufford, Teresa 1 Chen, Yuxin 2 Horning, Mitchell 3 Shee, Liberty 1 Mentor:

532 views • 34 slides

More recommend