DISTRIBUTION ECE565: ESTIMATION DETECTION and FILTERING YOUSEF - PowerPoint PPT Presentation

PARAMETERS ESTIMATION OF POLYA DISTRIBUTION ECE565: ESTIMATION DETECTION and FILTERING YOUSEF QASSIM, ARASH ABBASI DECEMBER 7 TH ,2011

CONTENTS  Introduction  FIM and CRLB  Maximum Likelihood (MLE)  Method of Moments (MoM)

INTRODUCTION

MULTIVARIATE POLYA DISTRIBUTION  x is vector of counts of each category. x=(n 1 ,n 2 ,...,n K )  p k is the probability to draw from category k p=(p 1 ,p 2 ,...,p K )  Dirichlet distribution with parameter vector

BETA BINOMIAL DISTRIBUTION  This project focuses on estimating the parameters of beta binomial distribution, with the parameters α 1 and α 2 . The distribution is given by  Γ( α ) m n(x )! Γ(n (x )+α ))   k p(x ,x ,...,x )= ( i k k i k ),   1 1 m n (x )! Γ(n(x )+ α )) Γ(α ) i=1 k k i i k k k k where n(x i ) is the length of x ,

BETA BINOMIAL DISTRIBUTION  One dimensional version of the multivariate Polya distribution  A family of discrete probability distribution on finite support arising, where probability of success is random  probability of success is drawn randomly from the Beta distribution with the parameters α 1 and α 2

BETA BINOMIAL DISTRIBUTION  Polya urn, where two colored balls green and blue placed with a probability p, and then a ball is drawn randomly  Coins with different probability p(m) , and then toss n times

THE PROBLEM  Compute FIM and CRLB  Estimate the parameters α 1 and α 2 using maximum likelihood and method of moment  Compare MSE of MLE and MoM with CRLB values

FIM AND CRLB

FIM AND CRLB  Find the log likelihood function

FIM AND CRLB  Take expectation

FIM AND CRLB  No close form solution d logp(x| α) = Ψ ( 2       α )-Ψ (n + α )+Ψ (n +α )-Ψ (α ) k i k ik k k 2 d α k k k d logp(x| α) = Ψ ( 2      α )-Ψ (n + α ) (k j) k i k d α α k k k j  Compute them numerically

SINGLE OBSERVATION VECTOR CASE

SINGLE OBSERVATION VECTOR CASE

MULTI OBSERVATION VECTORS CASE

MULTI OBSERVATION VECTORS CASE

MAXIMUM LIKELIHOOD (MLE)

MAXIMUM LIKELIHOOD (MLE)  Log likelihood function

MAXIMUM LIKELIHOOD (MLE)  To find the maximum differentiate and set to zero  No close form sloution

MAXIMUM LIKELIHOOD (MLE)  Estimate the parameters using Minka’s fixed point iteration  Ψ(n +α )-Ψ(α ) ik k k new α =α i    k k Ψ(n + α )-Ψ( α ) i k k k k i  Find MSE

SINGLE OBSERVATION VECTOR

SINGLE OBSERVATION VECTOR

MULTI OBSERVATION VECTORS CASE

MULTI OBSERVATION VECTORS CASE

METHOD OF MOMENTS (MOM)

METHOD OF MOMENTS (MOM)  Use the moments, and sample moments to estimate the parameters

METHOD OF MOMENTS (MOM)  Equating the above equations  Find MSE

RESULTS

RESULTS