lecture 5
play

Lecture 5 Jan-Willem van de Meent Conjugate Priors <latexit - PowerPoint PPT Presentation

Unsupervised Machine Learning and Data Mining DS 5230 / DS 4420 - Fall 2018 Lecture 5 Jan-Willem van de Meent Conjugate Priors <latexit


  1. Unsupervised Machine Learning 
 and Data Mining DS 5230 / DS 4420 - Fall 2018 Lecture 5 Jan-Willem van de Meent

  2. Conjugate Priors

  3. <latexit sha1_base64="EgFw9vO2o+NpxsqpE8DCaQWRAo=">AGzHicjZTLbtQwFIYzpQwlXNrCk3ESKgDoyopVdsNUlVUwQqVqjdpPB05zpnEqnPBcdpLbNC7OCJeBHehCXJIhJHFAdKXJ8v9c7BO7CaOpsO2fnYU7i3e795bumw8ePnq8vL65CSNM07gmMQs5mcuToHRCI4FQzOEg4dBmcuhdvC/vpJfCUxtGRuE5gFGI/ohNKsMiXxiu/kAhA4POXlvnijWUizP0QT8flqvXZQiz2rWSt+kYh9axp30Qop/8BIwYTMdQ0ydq0byFO/UCMbqefltLS2v/j73ZOmrVb2Svjle6dnr9mxY+sSpJj2jGgfj1cUfyItJFkIkCMNpOnTsRIwk5oISBspEWQoJhfYh2E+jXAI6UjOjkhZNeuRM5KTOBIQkZpM4jANsQi0xQJO6skyAMDr4etFkey8OJBSv2ornJDZrIg0neLrPMpOeyDJQ8fLenpD3Yej1wNrZVA+HgVYSzYw/ypwn4HCqkJ3NgbO1ozNJxhMGfyG7wIpsOERwReIwxJEn0SUQNcz3B0GUZhyKQiRyQ9lzlFIaXK5ZmY30bxqREtQTRpZyqJnY9hxWF5tC1Bt20+brRsE9tWNlxLdmLdhpnLWymsZkOcQ3izQyhNSYkKWVxpNUzmaNnfTLRg7I5pjrjwiXL7yAPax6ToB1PAqxh42TOVRFu8wT5b+vJIoT4FjEvPjprqgIGA2pSGVlV7qKRv9X5fZmsH1Vb8ri7bpyX2kcdmsMet7p3co4V6dK6pswXxex8qDawGTBlhtcEHm953TvN30ycnGumOvOx83e7t71c23ZDwznhtrhmNsG7vGe+PAODZI57zpfOt8737oSu6sqtKdKFTaZ4atdH9+hs1eHTv</latexit> <latexit sha1_base64="EgFw9vO2o+NpxsqpE8DCaQWRAo=">AGzHicjZTLbtQwFIYzpQwlXNrCk3ESKgDoyopVdsNUlVUwQqVqjdpPB05zpnEqnPBcdpLbNC7OCJeBHehCXJIhJHFAdKXJ8v9c7BO7CaOpsO2fnYU7i3e795bumw8ePnq8vL65CSNM07gmMQs5mcuToHRCI4FQzOEg4dBmcuhdvC/vpJfCUxtGRuE5gFGI/ohNKsMiXxiu/kAhA4POXlvnijWUizP0QT8flqvXZQiz2rWSt+kYh9axp30Qop/8BIwYTMdQ0ydq0byFO/UCMbqefltLS2v/j73ZOmrVb2Svjle6dnr9mxY+sSpJj2jGgfj1cUfyItJFkIkCMNpOnTsRIwk5oISBspEWQoJhfYh2E+jXAI6UjOjkhZNeuRM5KTOBIQkZpM4jANsQi0xQJO6skyAMDr4etFkey8OJBSv2ornJDZrIg0neLrPMpOeyDJQ8fLenpD3Yej1wNrZVA+HgVYSzYw/ypwn4HCqkJ3NgbO1ozNJxhMGfyG7wIpsOERwReIwxJEn0SUQNcz3B0GUZhyKQiRyQ9lzlFIaXK5ZmY30bxqREtQTRpZyqJnY9hxWF5tC1Bt20+brRsE9tWNlxLdmLdhpnLWymsZkOcQ3izQyhNSYkKWVxpNUzmaNnfTLRg7I5pjrjwiXL7yAPax6ToB1PAqxh42TOVRFu8wT5b+vJIoT4FjEvPjprqgIGA2pSGVlV7qKRv9X5fZmsH1Vb8ri7bpyX2kcdmsMet7p3co4V6dK6pswXxex8qDawGTBlhtcEHm953TvN30ycnGumOvOx83e7t71c23ZDwznhtrhmNsG7vGe+PAODZI57zpfOt8737oSu6sqtKdKFTaZ4atdH9+hs1eHTv</latexit> <latexit sha1_base64="2bpafo5d0NT+MPGghH2LVwOajeA=">AGQ3icfZRLb9QwEIC9pYESHm3hyCViJVTQqkpK1faCVBVcEKl6kuql5XjzO5adR7YTrutZX4ZV34CB34DN8QNIZHsRmITR3WkaDTzcseO8w4k8r3f3QW7iw6d+8t3XcfPHz0eHl9cmJTHNB4ZimPBVnIZHAWQLHikOZ5kAEocTsOLt6X9BKEZGlypK4z6MdklLAho0QVqsFKitUYFPn0ynNfvPEwEaOYTAYzpfFypxbTIeh7mUapkz5sMPrx0MW71wDwd3eI2WOn6/50ebYQVEIXVetgsLr4FUcpzWNIFOVEyvPAz1RfE6EY5WBcnEvICL0gIzgvxITEIPt6ujPGq1mPgr4epomChNbcNIlTNTYUpawrGvpuEgMop62UvZ1GSUCyUZJ3SuMjeviCIbFKU0r01HIczD68N2e0X5v63Uv2Ng2DURAVBHBjt8rviYwEgBJhexs9oKtHZvJcpFx+A/5JVZWIyCBK5rGMUkijS+BmvNifzAkMhdQNqJxGOtuYIyx4Bla+EztLp43TozWuFYgvtQT08Su57Cy0QK6tqCbtlg3Fva5DZtNYEv1qp0meQubW2xuQ8KCRLNCaM0JmWQ8Tax+hnP0dE6GdlI+x1RnXIbkxdWPiBUxG7fj2ZhZ7GHjZA5NOS7zxOzOG43TDARqSgv3RVTY85ipqSu7Mb2YsntXoW9mWzf1Iey/Ieh3jcWSUM+Hcz63tkTSkVU58ouW7CRqGOzg2sBswZYbXBJFu9d0HzdbOFkYz3w14OPm93dverlW0LP0HO0hgK0jXbRe3SAjhF39GfzmLHcb45P51fzu8ZutCpfJ6i2nL+/gMAGkC</latexit> <latexit sha1_base64="2bpafo5d0NT+MPGghH2LVwOajeA=">AGQ3icfZRLb9QwEIC9pYESHm3hyCViJVTQqkpK1faCVBVcEKl6kuql5XjzO5adR7YTrutZX4ZV34CB34DN8QNIZHsRmITR3WkaDTzcseO8w4k8r3f3QW7iw6d+8t3XcfPHz0eHl9cmJTHNB4ZimPBVnIZHAWQLHikOZ5kAEocTsOLt6X9BKEZGlypK4z6MdklLAho0QVqsFKitUYFPn0ynNfvPEwEaOYTAYzpfFypxbTIeh7mUapkz5sMPrx0MW71wDwd3eI2WOn6/50ebYQVEIXVetgsLr4FUcpzWNIFOVEyvPAz1RfE6EY5WBcnEvICL0gIzgvxITEIPt6ujPGq1mPgr4epomChNbcNIlTNTYUpawrGvpuEgMop62UvZ1GSUCyUZJ3SuMjeviCIbFKU0r01HIczD68N2e0X5v63Uv2Ng2DURAVBHBjt8rviYwEgBJhexs9oKtHZvJcpFx+A/5JVZWIyCBK5rGMUkijS+BmvNifzAkMhdQNqJxGOtuYIyx4Bla+EztLp43TozWuFYgvtQT08Su57Cy0QK6tqCbtlg3Fva5DZtNYEv1qp0meQubW2xuQ8KCRLNCaM0JmWQ8Tax+hnP0dE6GdlI+x1RnXIbkxdWPiBUxG7fj2ZhZ7GHjZA5NOS7zxOzOG43TDARqSgv3RVTY85ipqSu7Mb2YsntXoW9mWzf1Iey/Ieh3jcWSUM+Hcz63tkTSkVU58ouW7CRqGOzg2sBswZYbXBJFu9d0HzdbOFkYz3w14OPm93dverlW0LP0HO0hgK0jXbRe3SAjhF39GfzmLHcb45P51fzu8ZutCpfJ6i2nL+/gMAGkC</latexit> ML and MAP estimation Maximum Likelihood Estimation Maximum A Posterior Estimation ML Objective Regularization (same as minimizing a loss function)

  4. <latexit sha1_base64="E59eI3vLjhVQUj5ANDoMnc+Po=">AGCHicfZTdbtMwFIC9scIofxtchPRm1WqRjKmbTdI09AEl6Pan7RUk+OctacxNjO1s7yC/AIPAV3CIkLxB28AW9D3AbRxBGOFJ2c850/+8QRZ1Qq3/+9tHxnpX3ur9oOHjx4/WVt/eiqzXBA4IRnLxHmEJTCawomisE5F4CTiMFZdPXG2s+uQUiapcdqymGQ4FKh5RgVagu17b5RqjGoLAXJjT2Jl3vtc3JvOvuaXr/W63ktr7F6udfxNf7Y8VwhKoYPKdXS5vItjDOSJ5AqwrCUF4HP1UBjoShYNphLoFjcoVHcFGIKU5ADvSsPeNVrMfBQA+zVEFKm4aJzLBauwoLSyrWjIuEoOopi2VA2jxCDpK16RYlpt8MYhsVWzyrTcRyMLr/9sBov7fzqhds7ZoaIiAuiWDP7xVPHRgJgLRE9rZ7wc6ey/BcAb/IN9ithoBKdyQLElwGuvwGoi5KPYnhFTmAmwjOowS3QmMQ48Rwufmb0dLhonRuwUmB4rSemjk0XMNtoAU0d6LYp1q2DfWjC5nPXUL1qpnHewOYOm7uQcCBRrxAacwKXlGWp089wgZ7NydBNyhaY8oxtSFb8vzF2IvJxM87H1GH7tZPpGzsuiwQWowQX5xmHARWmbA/3Q1VY0YTqQu7cb1oun/vQp7PdmhqQ6lfUeRPjQOSI2G8zq3rkTSkRc5WyXDdhIVLH5wTWAvAaWG2zJ4r4L6rebK5xubQb+ZvB+u7N/UN58q+g5eoE2UIB20T56h47QCSLoE/qOfqJfrY+tz60vra9zdHmp9HmGKqv14w+e1jPw</latexit> <latexit sha1_base64="L8j+V7PAJMvZrRHurXhl1Y7ETYE=">AF7nicfZTdbtMwFIC9scIofx3cwU1Fb4ZUTcmYtl1Omya4HNP+pKWaHOe0teY4xna2dpYRT8Ed4g5xBw/Ai/A2xG0kmjCkaIjn+/4/NqxYFTpIPiztHxvpX/werD9qPHT54+6w9P1NZLgmckoxl8iLGChjlcKqpZnAhJOA0ZnAeXx84/fkNSEUzfqKnAgYpHnE6pATrYuq8zLSY9C4G2mYaPNp/SCjvLtPsXpjrzq9YCOYra4vhKXQ+U6ulpb+R0lGclT4JowrNRlGAg9MFhqShjYdpQrEJhc4xFcFiLHKaiBmSVhuxXtSTgw4xr4KRiZnCqUqzH3qaDVXWXjAvHIKtuy82BcackoOiIV63i1LbUQLDoqCzyEwSsxysOX63b03Q37bDzd3bA2RkJREuBv0i68OjCQAL5HdrX64veszIpeCwT8ocJiLRgKHW5KlKeaJiW6A2MuiPhFwlUtwiZgoTk0vtNZ68BwtbGb6drSonFhjokqA0Y2Z2Do2XcBcogU09aC7prPuPOxjEzafwYbodTON8wY29jch6QHyXqE0OgThKIs414+wV6NidD3ylbYMoeuyNZcUsT7J0oxs24GFOPa515ti6cVksByluOhzlAmQWGfSXbpbqseMplQrU+qtb0X5/60Kfd3Zoa0OpfvHsTm0HkliNhvMau38CSUyqXIuywZsJKvYvHENoKiBZYEdWbx3Yf184WzY0w2Ag/bPX29suXbxW9Qq/ROgrRDtpD79EROkUEfUbf0U/0qyVaX1pfW9/m6PJSafMCVbrx18GySwU</latexit> <latexit sha1_base64="hq5cQZqmFVe52rldOGLUfIrapF4=">AF5HicfZRPb9MwFMC9scIo/zo4cqnoZUjVlIxp63EamuA4pnWbtFST47y21mwn2E63zvI34Ia4IW7wKfgifBviNhJNHOFI0dN7v/fPfnacMap0EPxZW3+w0Xr4aPNx+8nTZ89fdLZenqs0lwSGJGWpvIyxAkYFDXVDC4zCZjHDC7im/fOfjEDqWgqzvQ8gxHE0HlGBdqK47nWw70lPQuBtxmnTv3l53esFOsFhdXwhLoYfKdXK9tfE7SlKScxCaMKzUVRhkemSw1JQwsO0oV5BhcoMncFWIAnNQI7Mo3XYr1rNwZMap0CBIxc1grjWU0/pYFXVkmRGQ1bakcGRclAUnouoVc9tuRwmMi21cVGaSmOVgzemHI2uC/v67frh7YGuIhKQkwkHQL746MJEAokQGe/1wf+AzWS4zBv+gwGuGgkCbknKORaJiWZA7FWxPxEIlUtwjZgo5qYXWms9eIkWPgt7O1o13ljokqB0czc2To2X8FcowU096D7plj3Hva5CVvOXUP1upnGeQObe2zuQ9KDZL1CaMwJmaIsFV4/4xV6MSdjPylbYcozdiFZcTcT7EXMps14NqUe1o7mVPrxmWVwHLCcXHOUZqBxDqV7tLdUj1lFOtTGm3vhcV/cq7PVkx7Y6lO4fx+bYeiSJ2WIwq3vnTyiRSZVzXTZgE1nFlgfXAGY1sNxgRxbvXVh/3XzhfHcnDHbCT3u9w6Py5dtEr9EbtI1CdIAO0Ud0goaIoBn6jn6iX61x60vra+vbEl1fK31eocpq/fgLqoAnjw=</latexit> Intuition: Bayesian Posterior Posterior Likelihood Prior MAP estimate after 50 trials

  5. <latexit sha1_base64="z9jbK0C2B0bymVtEWNsdOrtQecg=">AF/XicfZTdbtMwFIC9scIofx1clNRCW1SqeJurOECaRqa4HJM+5OWqnKc09Za4gTb2bpZFo/AU3CHuEPcIHgO3oakDdDUAUeKnHO+8+sT+0nIpHKcn0vLN1ZqN2+t3q7fuXv/oPG2sNjGaeCwhGNw1ic+kRCyDgcKaZCOE0EkMgP4cQ/f5XrTy5ASBbzQ3WVQD8iI86GjBKViQaNjcmg23z6srmOn0GeMPz6p4agyJ/pMUn3hg0Wk5nu/cu27T6XRd7OKtbO82MS9bhN3nOlqoWLtD9ZWvnlBTNMIuKIhkfIMO4nqayIUoyGYupdKSAg9JyM4y7acRCD7elqTaZa0h7ivhzFXwGnJTJNIRkSNLWEOy7KUjrPAIMphC2Ff514CkGzEy1Z+ZOp1L4Bh1t9pZjrwxSMPni9a7T3t5s427PLCACgoLArtPOnkVgJAB4gbhbzt2kySiSEv5CTY3k2Ajhc0jiKCA+0dwHUnGX98YDLVEBeiPb8SLewMcaCZ2hmM9XvXnlxGjtlRL0LvTELGJXc1heaAZdWdB1la9rC3tXhc1GriJ7VU2TtIJNLTa1IWFBYjFDqIwJiWRhzK16hnP0dE6GdtBwjinOHcZj9tQCyPybgaT8bMYg8WTubA5OMyTxAxikh2zl6cgCAqFvlPd8nUOGQRU1IXemNbMf5/q0y/GzPlIcyf/u+3jMWSf1wOpjl3tkTSkVQ5vIqK7CRKGOzg6sAkwWwaHBOZvfd70ut+e/NcbeDnQ5+u9Xa2S1uvlX0GD1B6wijHtpBb9A+OkIUfUBf0Hf0o/a+9rH2qfZ5hi4vFTaPUGnVv4ClWsu1Q=</latexit> <latexit sha1_base64="m49vKTjt/uVv8k07CRg8QZKlH4=">AGnicfZTPb9MwFMe9scIovzY4cqnohUNVxd1Yw20aGnAc035JTakc5zW1ljBcfbL8n/Cn8CfwIkb4oa4IPhjSNJUNHAkSLb7/N979l+thsHLJGW9XNl9dZa6/ad9bvte/cfPHy0sfn4JIlSQeGYRkEkzlySQMA4HEsmAziLBZDQDeDUPX+V208vQCQs4kfyOoZxSHzOpowSmU1Nl4rp3AyEr47VlZ/Z/gC23bP6g9sbOPtrGO93MLDgXbkDCSZ4PfqaoJ1pxwO8uFA68lGd6HtLSdhbaD+1bRuqhsB5PNtc+OF9E0BC5pQJkhK1YjhURktEAdNtJE4gJPSc+jLIuJyEkY1XkqjsV6xEeq2nEJXBakSkSJiGRM2Myh5PqLJ1lgUFUw5aTY5V78SBhPq+q3FC324H02zi8yU5wYpaHX4Zk8rq7ez1cODoa4hArySwLbVy7464AsAXiL2dg/v2CYTpyIO4C9k5ViejQAOlzQKQ8I95VwA1aNsfxzgSogX4hy3FB1sdbagOdopinsbWfZeKVWSqLBJ0LdaXr2PUSli80g64N6KbJ142BfWjC5oXkL1spknawKYGm5qQMCBRzxAaY0KcsCDixnqmS3RJ1MzaLDElGecuwyG+0Rw2M8a8bjGTPYw9rJHOq8XJYJIvyQZOfsRDEIiORX7pLJmcBC5lMVGnXporx/6syez3Yvq4WZf53XbWvDZK6QVGY1b0zK5QKr8rlq2zAfFHF5gfXAMY1sNzgnMzeu8Wj1vl352TQx1Yfv9vu7u6VL986eoqeoecIoyHaRW/RATpGFH1C39Ev9Lv1sfWl9bX1bY6urpSaJ6jSWj/+AI5xOt4=</latexit> Likelihood: Bernoulli (a discrete distribution with outcomes x=0 and x=1) µ x (1 − µ ) 1 − x Bern( x | µ ) = E [ x ] = µ var[ x ] = µ (1 − µ ) � 1 if µ � 0 . 5 , mode[ x ] = 0 otherwise H[ ] = ln (1 ) ln µ ∈ [0 , 1] that ariable x ∈ { 0 , 1 } by a single continuous

Recommend


More recommend