Maximum Likelihood Jonathan Pillow Mathematical Tools for - PowerPoint PPT Presentation

Jan 07, 2024 •123 likes •330 views

Maximum Likelihood Jonathan Pillow Mathematical Tools for Neuroscience (NEU 314) Spring, 2016 lecture 16 Estimation model measured dataset parameter (sample) An e stimator is a function often we will write or just

Maximum Likelihood Jonathan Pillow Mathematical Tools for Neuroscience (NEU 314) Spring, 2016 lecture 16
Estimation model measured dataset   parameter (“sample”) An e stimator is a function • often we will write or just
Properties of an estimator “expected” value   (average over draws of m) bias: • “unbiased” if bias=0 variance: • “consistent” if bias and variance both go   to zero asymptotically mean squared error (MSE)
Example 1: linear Poisson neuron spike count spike rate parameter stimulus encoding model:
60 p(y|x) conditional distribution (spike count) 40 20 0 0 20 40 0 20 40 60 (contrast)
60 p(y|x) conditional distribution (spike count) 40 20 0 0 20 40 0 20 40 60 (contrast)
60 p(y|x) conditional distribution (spike count) 40 20 0 0 20 40 0 20 40 60 (contrast)
Maximum Likelihood Estimation: • given observed data , find that maximizes 60 p(y|x) (spike count) 40 20 0 0 20 40 (contrast)
Maximum Likelihood Estimation: • given observed data , find that maximizes 60 p(y|x) (spike count) 40 20 0 0 20 40 (contrast)
Maximum Likelihood Estimation: • given observed data , find that maximizes 60 p(y|x) (spike count) 40 20 0 0 20 40 (contrast)
Likelihood function: as a function of Because data are independent: likelihood 0 1 2
Likelihood function: as a function of Because data are independent: likelihood 0 1 2 log log-likelihood 0 1 2
log-likelihood 0 1 2 • Closed-form solution (exists for “exponential family” models)
Properties of the MLE (maximum likelihood estimator) • consistent (converges to true in limit of infinite data) • e ffi cient   (converges as quickly as possible,   i.e., achieves minimum possible asymptotic error)
Example 2: linear Gaussian neuron spike count spike rate parameter stimulus encoding model:
60 encoding distribution 40 (spike count) 20 0 0 20 40 0 20 40 60 (contrast) All slices have same width
Log-Likelihood Differentiate and set to zero: Maximum-Likelihood Estimator: (“Least squares regression” solution) (Recall that for Poisson, )
Example 3: unknown neuron 100 75 (spike count) 50 25 0 -25 0 25 (contrast) What model would you use to fit this neuron?
Example 3: unknown neuron 100 75 (spike count) 50 25 0 -25 0 25 (contrast) More general setup: for some nonlinear function f

Recommend

Maximum Likelihood properties Maximum parsimony Maximum likelihood Experimental design

N. Salamin c Sept 2007 Lecture outline Maximum likelihood in phylogenetics Definition Phylogenetics and bioinformatics for evolution Maximum likelihood and models Likelihood of a tree Computational complexity Statistical Maximum

697 views • 30 slides

Outline n Maximum likelihood (ML) n Priors, and maximum a posteriori (MAP) n

Maximum Likelihood (ML), Expectation Maximization (EM) Pieter Abbeel UC Berkeley EECS Many slides adapted from Thrun, Burgard and Fox, Probabilistic Robotics Outline n Maximum likelihood (ML) n Priors, and maximum a posteriori (MAP) n

649 views • 23 slides

Maximum Likelihood Estimation CS 446 Maximum likelihood: abstract formulation Weve had one

Maximum Likelihood Estimation CS 446 Maximum likelihood: abstract formulation Weve had one main meta-algorithm this semester: (Regularized) ERM principle: pick the model that minimizes an average loss over training data. 1 / 70

599 views • 42 slides

Maximum Likelihood Estimation CS 446 Maximum likelihood: abstract formulation Weve had one

Maximum Likelihood Estimation CS 446 Maximum likelihood: abstract formulation Weve had one main meta-algorithm) this semester: (Regularized) ERM principle: pick the model that minimizes an average loss over training data. 1 / 76

643 views • 45 slides

1 Being Normal, Simultaneously Maximizing Likelihood with Uniform Now have two equations, two

Likelihood of Data Maximum Likelihood Estimator The Maximum Likelihood Estimator (MLE) of , Consider n I.I.D. random variables X 1 , X 2 , ..., X n is the value of that maximizes L ( ) X i a sample from density function f (X

419 views • 3 slides

Binary choice 3.3 Maximum likelihood estimation Michel Bierlaire Maximum likelihood

Binary choice 3.3 Maximum likelihood estimation Michel Bierlaire Maximum likelihood estimation We now estimate the values of the unknown parameters 1 ,. . . , K from a sample of observations drawn at random from the population. Each

758 views • 8 slides

Maximum likelihood models Tues. Feb. 27, 2018 1 Overview of today Informal notion of

COMP 546 Lecture 14 Maximum likelihood models Tues. Feb. 27, 2018 1 Overview of today Informal notion of likelihood Formal definition of likelihood as conditional probability Maximum likelihood problems (sketch) 2 Scene

960 views • 40 slides

Maximum Likelihood (ML), Expectation Maximization (EM) Pieter Abbeel UC Berkeley EECS Many

630 views • 46 slides

Binary choice 3.3 Maximum likelihood estimation Michel Bierlaire Output of the estimation

Binary choice 3.3 Maximum likelihood estimation Michel Bierlaire Output of the estimation We explain here the various outputs from the maximum likelihood estimation procedure. Solution of the maximum likelihood estimation The main

170 views • 5 slides

Chapter 8: Estimation In this chapter we will cover: 1. The likelihood and maximum likelihood

Chapter 8: Estimation In this chapter we will cover: 1. The likelihood and maximum likelihood estimation ( 8.3, 8.5 Rice) 2. Introduction to decision theory( 15.115.3) The likelihood function We have seen many cases where we have

413 views • 10 slides

15-388/688 - Practical Data Science: Maximum likelihood estimation, nave Bayes J. Zico Kolter

15-388/688 - Practical Data Science: Maximum likelihood estimation, nave Bayes J. Zico Kolter Carnegie Mellon University Spring 2018 1 Outline Maximum likelihood estimation Naive Bayes Machine learning and maximum likelihood 2 Outline

352 views • 22 slides

PARALLELIZATION OF MAXIMUM LIKELIHOOD MOTIVATION To analyze large amount of data using

MAYANK KALE PARALLELIZATION OF MAXIMUM LIKELIHOOD MOTIVATION To analyze large amount of data using computationally intensive, model-based optimality criterion such as maximum likelihood (ML), there must be development in parallel and

273 views • 5 slides

Lecture 7: Maximum Likelihood Estimation (MLE) Maximum a Posteriori (MAP) Aykut Erdem

Lecture 7: Maximum Likelihood Estimation (MLE) Maximum a Posteriori (MAP) Aykut Erdem October 2016 Hacettepe University Administrative Assignment 2 will be out on Thursday It is due November 10 (i.e. in 2 weeks) You will

831 views • 70 slides

Lecture 8: Maximum Likelihood Estimation (MLE) (contd.) Maximum a posteriori (MAP)

Lecture 8: Maximum Likelihood Estimation (MLE) (contd.) Maximum a posteriori (MAP) estimation Nave Bayes Classifier Aykut Erdem March 2016 Hacettepe University Last time Flipping a Coin I have a coin, if I flip it, whats the

932 views • 82 slides

Maximum likelihood and EM algorithm (after the Chapter 8) Pasha Zusmanovich, deCODE Statistics

1 Maximum likelihood and EM algorithm (after the Chapter 8) Pasha Zusmanovich, deCODE Statistics Colloquium March 30, 2007 2 What is likelihood and what it is good for? Likelihood is just a conditional probability. Formal definition Given

775 views • 42 slides

ECON 626: Applied Microeconomics Lecture 11: Maximum Likelihood Estimation Professors: Pamela

ECON 626: Applied Microeconomics Lecture 11: Maximum Likelihood Estimation Professors: Pamela Jakiela and Owen Ozier Maximum Likelihood: Motivation So far, weve been thinking about average treatment effects, but the ATE may or may not be

564 views • 23 slides

Lecture 3. Inadmissibility of Maximum Likelihood Estimate and James-Stein Estimator Yuan Yao

Lecture 3. Inadmissibility of Maximum Likelihood Estimate and James-Stein Estimator Yuan Yao Hong Kong University of Science and Technology March 4, 2020 Outline Recall: PCA in Noise Maximum Likelihood Estimate Example: Multivariate Normal

496 views • 46 slides

Maximum Likelihood Setting parameters Chris Williams, School of Informatics We choose a

Maximum Likelihood Setting parameters Chris Williams, School of Informatics We choose a parametric model p ( x | ) University of Edinburgh Overview We are given data x 1 , . . . , x n Maximum likelihood parameter estimation

46 views • 3 slides

CS 287 Advanced Robotics (Fall 2019) Lecture 13: Kalman Smoother, Maximum A Posteriori, Maximum

CS 287 Advanced Robotics (Fall 2019) Lecture 13: Kalman Smoother, Maximum A Posteriori, Maximum Likelihood, Expectation Maximization Pieter Abbeel UC Berkeley EECS Outline n Kalman smoothing n Maximum a posteriori sequence n Maximum likelihood

865 views • 68 slides

Density Estimation Parametric techniques Maximum Likelihood Maximum A Posteriori

1 Density Estimation Parametric techniques Maximum Likelihood Maximum A Posteriori Bayesian Inference G aussian M ixture M odels (GMM) EM-Algorithm Non-parametric techniques Histogram Parzen Windows

168 views • 16 slides

Density Estimation Parametric techniques Maximum Likelihood Maximum A Posteriori

1 Density Estimation Parametric techniques Maximum Likelihood Maximum A Posteriori Bayesian Inference G aussian M ixture M odels (GMM) EM-Algorithm Non-parametric techniques Histogram Parzen Windows

496 views • 11 slides

Simulated maximum likelihood for time series models with nonlinear non-Gaussian observation

Simulated maximum likelihood for time series models with nonlinear non-Gaussian observation equations: new results and an application Siem Jan Koopman Vrije Universiteit Amsterdam Tinbergen Institute Simulated maximum likelihood for time

602 views • 32 slides

Curve Fitting Re-visited, Bishop1.2.5 Maximum Likelihood Bishop 1.2.5 Model Likelihood

Curve Fitting Re-visited, Bishop1.2.5 Maximum Likelihood Bishop 1.2.5 Model Likelihood differentiation Maximum Likelihood N N t n | y ( x n , w ) , 1 p ( t | x , w , ) = . (1.61) n =1 As we did in the

597 views • 49 slides

Maximum likelihood parameter estimation Maximum likelihood parameter estimation For an HMM

Maximum likelihood parameter estimation Maximum likelihood parameter estimation For an HMM with observed state data X , and s states, For some observed data O = o 1 o n , and a model, we do the same: here a bigram model,

270 views • 3 slides