Maximum Likelihood Estimator Lecture 10 Biostatistics 602 - - PowerPoint PPT Presentation

. . February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang February 12th, 2013 Hyun Min Kang Maximum Likelihood Estimator Lecture 10 Biostatistics 602 - Statistical Inference . Summary . . MLE Recap . . . . . . . . . 1 / 20 . . . . . . . . . . . . .

. . February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang 5 How can you find an MLE? . . 4 What is a maximum likelihood estimator (MLE)? . . estimator? 3 What are advantages and disadvantages of method of moment . . 2 What is a method of moment estimator? . . 1 What is a point estimator, and a point estimate? . . Last Lecture Summary . MLE Recap . . . . . . . . . 2 / 20 . . . . . . . . . . . . .

. n February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang X k n . . . n . 3 / 20 MLE . Recap - Method of Moment Estimator . Summary . . . . . . . . Recap . . . . . . . . . . . . . • Point Estimation - Estimate θ or τ ( θ ) . • Method of Moment m 1 = 1 ∑ X i = E X = µ 1 m 2 = 1 i = E X 2 = µ 2 ∑ X 2 m k = 1 i = E X k = µ k ∑

• Easy to implement • Easy to understand • Estimators can be improved; use as initial value to get other • No guarantee that the estimator will fall into the range of valid . . February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang parameter space. estimators i n n 4 / 20 . . . i.i.d. Recap - Example of Method of Moment Estimator . . . . . . . Summary Recap MLE . . . . . . . . . . . . . ∼ N ( µ, σ 2 ) X 1 , · · · , X n µ = X ˆ σ 2 = E X 2 = 1 µ 2 + ˆ ∑ X 2 ˆ i =1 σ 2 = ∑ ( X i − X ) 2 / n ˆ

• Easy to understand • Estimators can be improved; use as initial value to get other • No guarantee that the estimator will fall into the range of valid . . February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang parameter space. estimators i n n 4 / 20 i.i.d. MLE . . Recap - Example of Method of Moment Estimator Summary . . . . . . . . Recap . . . . . . . . . . . . . ∼ N ( µ, σ 2 ) X 1 , · · · , X n µ = X ˆ σ 2 = E X 2 = 1 µ 2 + ˆ ∑ X 2 ˆ i =1 σ 2 = ∑ ( X i − X ) 2 / n ˆ • Easy to implement

• Estimators can be improved; use as initial value to get other • No guarantee that the estimator will fall into the range of valid . i.i.d. February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang parameter space. estimators i n n . 4 / 20 . . Recap - Example of Method of Moment Estimator Summary . . . . . . . . MLE Recap . . . . . . . . . . . . . ∼ N ( µ, σ 2 ) X 1 , · · · , X n µ = X ˆ σ 2 = E X 2 = 1 µ 2 + ˆ ∑ X 2 ˆ i =1 σ 2 = ∑ ( X i − X ) 2 / n ˆ • Easy to implement • Easy to understand

• No guarantee that the estimator will fall into the range of valid . i.i.d. February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang parameter space. estimators i n n . 4 / 20 Recap . . Summary . MLE . . . . . . . Recap - Example of Method of Moment Estimator . . . . . . . . . . . . . ∼ N ( µ, σ 2 ) X 1 , · · · , X n µ = X ˆ σ 2 = E X 2 = 1 µ 2 + ˆ ∑ X 2 ˆ i =1 σ 2 = ∑ ( X i − X ) 2 / n ˆ • Easy to implement • Easy to understand • Estimators can be improved; use as initial value to get other

. i.i.d. February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang parameter space. estimators i n n . 4 / 20 Recap - Example of Method of Moment Estimator MLE . . . . . . . . . Recap . Summary . . . . . . . . . . . . . ∼ N ( µ, σ 2 ) X 1 , · · · , X n µ = X ˆ σ 2 = E X 2 = 1 µ 2 + ˆ ∑ X 2 ˆ i =1 σ 2 = ∑ ( X i − X ) 2 / n ˆ • Easy to implement • Easy to understand • Estimators can be improved; use as initial value to get other • No guarantee that the estimator will fall into the range of valid

. . February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang is called the likelihood function. f x x defined by L x is observed, the function of Given that X n i.i.d. . 5 / 20 . Definition . . . . . . . . . Recap MLE . Summary Recap - Likelihood Function . . . . . . . . . . . . . . X 1 , · · · , X n ∼ f X ( x | θ ) . The join distribution of X = ( X 1 , · · · , X n ) is ∏ f X ( x | θ ) = f X ( x i | θ ) i =1

. . February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang is called the likelihood function. n i.i.d. . . Definition . Recap - Likelihood Function Summary . . . . . . . . . Recap MLE . 5 / 20 . . . . . . . . . . . . . X 1 , · · · , X n ∼ f X ( x | θ ) . The join distribution of X = ( X 1 , · · · , X n ) is ∏ f X ( x | θ ) = f X ( x i | θ ) i =1 Given that X = x is observed, the function of θ defined by L ( θ | x ) = f ( x | θ )

. . February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang i.i.d. . Summary Recap - Example Likelihood Function MLE Recap . . . . . . . . . 6 / 20 . . . . . . . . . . . . . • X 1 , X 2 , X 3 , X 4 ∼ Bernoulli ( p ) , 0 < p < 1 . • x = (1 , 1 , 1 , 1) T • Intuitively, it is more likely that p is larger than smaller. • L ( p | x ) = f ( x | p ) = ∏ 4 i =1 p x i (1 − p ) 1 − x i = p 4 .

• For one-dimensional parameter, negative second order derivative • For two-dimensional parameter, suppose L • Check boundary points to see whether boundary gives global maximum. • Use numerical methods • Or perform direct maximization, using inequalities, or properties of L is the likelihood function. Then we need to show (a) L or February 12th, 2013 . Biostatistics 602 - Lecture 10 implies local maximum. If the function is NOT differentiable with respect to . the function. Hyun Min Kang (b) Determinant of second-order derivative is positive . . 2 Check second-order derivative to check local maximum. . . . . . . . . . Recap MLE . Summary How do we find MLE? . . 1 Find candidates that makes first order derivative to be zero . . 7 / 20 . . . . . . . . . . . . . If the function is differentiable with respect to θ ,

Maximum Likelihood Estimator Lecture 10 Biostatistics 602 - - PowerPoint PPT Presentation

. . February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang February 12th, 2013 Hyun Min Kang Maximum Likelihood Estimator Lecture 10 Biostatistics 602 - Statistical Inference . Summary . . MLE Recap . . . . . . . . . 1

1 Being Normal, Simultaneously Maximizing Likelihood with Uniform Now have two equations, two

Lecture 3. Inadmissibility of Maximum Likelihood Estimate and James-Stein Estimator Yuan Yao

Biostatistics Preparatory Course: Methods and Computing Lecture 9 Maximum Likelihood & the

Maximum Likelihood properties Maximum parsimony Maximum likelihood Experimental design

k-Maximum Likelihood Estimator for mixtures of generalized Gaussians ICPR 2012, Tokyo, Japan

Profile Maximum Likelihood: An Optimal, Universal, Plug-and-Play Functional Estimator Yi Hao and

A Maximum Likelihood estimator of a Markov model for disease activity in chronic diseases that

Lecture 7: Maximum Likelihood Estimation (MLE) Maximum a Posteriori (MAP) Aykut Erdem

Lecture 8: Maximum Likelihood Estimation (MLE) (contd.) Maximum a posteriori (MAP)

Maximum likelihood models Tues. Feb. 27, 2018 1 Overview of today Informal notion of

Outline n Maximum likelihood (ML) n Priors, and maximum a posteriori (MAP) n

Maximum Likelihood Estimation CS 446 Maximum likelihood: abstract formulation Weve had one

Maximum Likelihood Estimation CS 446 Maximum likelihood: abstract formulation Weve had one

ECON 626: Applied Microeconomics Lecture 11: Maximum Likelihood Estimation Professors: Pamela

Bayes Estimator Lecture 15 Biostatistics 602 - Statistical Inference . . Summary Conjugate

CS 287 Advanced Robotics (Fall 2019) Lecture 13: Kalman Smoother, Maximum A Posteriori, Maximum

Likelihood and Point Estimation Lecture 09 Biostatistics 602 - Statistical Inference . . . .

Binary choice 3.3 Maximum likelihood estimation Michel Bierlaire Maximum likelihood

Lecture 4. Maximum Likelihood Estimation - confidence intervals. Igor Rychlik Chalmers

Applied Statistics Lecturer: Serena Arima Likelihood ML estimator Summaries ML properties LR

Maximum Likelihood (ML), Expectation Maximization (EM) Pieter Abbeel UC Berkeley EECS Many

Binary choice 3.3 Maximum likelihood estimation Michel Bierlaire Output of the estimation

Lecture 3, Estimation and model validation Erik Lindstrmm Maximum likelihood, recap (1)

Week 2: Maximum Likelihood Estimation Instructor: Sergey Levine 1 Recap: MLE for the binomial

Maximum Likelihood Estimator Lecture 10 Biostatistics 602 - - PowerPoint PPT Presentation

. . February 12th, 2013 Biostatistics 602 - Lecture 10 Hyun Min Kang February 12th, 2013 Hyun Min Kang Maximum Likelihood Estimator Lecture 10 Biostatistics 602 - Statistical Inference . Summary . . MLE Recap . . . . . . . . . 1

1 Being Normal, Simultaneously Maximizing Likelihood with Uniform Now have two equations, two

Lecture 3. Inadmissibility of Maximum Likelihood Estimate and James-Stein Estimator Yuan Yao

Biostatistics Preparatory Course: Methods and Computing Lecture 9 Maximum Likelihood &amp; the

Maximum Likelihood properties Maximum parsimony Maximum likelihood Experimental design

k-Maximum Likelihood Estimator for mixtures of generalized Gaussians ICPR 2012, Tokyo, Japan

Profile Maximum Likelihood: An Optimal, Universal, Plug-and-Play Functional Estimator Yi Hao and

A Maximum Likelihood estimator of a Markov model for disease activity in chronic diseases that

Lecture 7: Maximum Likelihood Estimation (MLE) Maximum a Posteriori (MAP) Aykut Erdem

Lecture 8: Maximum Likelihood Estimation (MLE) (contd.) Maximum a posteriori (MAP)

Maximum likelihood models Tues. Feb. 27, 2018 1 Overview of today Informal notion of

Outline n Maximum likelihood (ML) n Priors, and maximum a posteriori (MAP) n

Maximum Likelihood Estimation CS 446 Maximum likelihood: abstract formulation Weve had one

Maximum Likelihood Estimation CS 446 Maximum likelihood: abstract formulation Weve had one

ECON 626: Applied Microeconomics Lecture 11: Maximum Likelihood Estimation Professors: Pamela

Bayes Estimator Lecture 15 Biostatistics 602 - Statistical Inference . . Summary Conjugate

CS 287 Advanced Robotics (Fall 2019) Lecture 13: Kalman Smoother, Maximum A Posteriori, Maximum

Likelihood and Point Estimation Lecture 09 Biostatistics 602 - Statistical Inference . . . .

Binary choice 3.3 Maximum likelihood estimation Michel Bierlaire Maximum likelihood

Lecture 4. Maximum Likelihood Estimation - confidence intervals. Igor Rychlik Chalmers

Applied Statistics Lecturer: Serena Arima Likelihood ML estimator Summaries ML properties LR

Maximum Likelihood (ML), Expectation Maximization (EM) Pieter Abbeel UC Berkeley EECS Many

Binary choice 3.3 Maximum likelihood estimation Michel Bierlaire Output of the estimation

Lecture 3, Estimation and model validation Erik Lindstrmm Maximum likelihood, recap (1)

Week 2: Maximum Likelihood Estimation Instructor: Sergey Levine 1 Recap: MLE for the binomial

Biostatistics Preparatory Course: Methods and Computing Lecture 9 Maximum Likelihood & the