The Geometry of Imprecise Inference Mik elis Bickis University of - PowerPoint PPT Presentation

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples The Geometry of Imprecise Inference Mik ¸elis Bickis University of Saskatchewan ISIPTA’15 20 July 2015 Pescara, Italy Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples ◮ I am Professor Emeritus in the Department of Mathematics and Statistics at the in Saskatoon ◮ My recent graduates: ◮ Osama Bataineh, PhD 2012 ◮ Chel Hee Lee, PhD 2014 ◮ Two current students ◮ Research supported by the Natural Sciences and Engineering Research Council of Canada Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples Where is Saskatoon? ��������� ������� Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples Bayes’ theorem: P ( A | B ) = P ( B | A ) P ( A ) P ( B ) Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples Bayes’ theorem: P ( A | B ) = P ( B | A ) P ( A ) P ( B ) log P ( B | A ) = log P ( B | A ) − log P ( B ) P ( A ) Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples Bayes’ theorem: P ( A | B ) = P ( B | A ) P ( A ) P ( B ) log P ( B | A ) = log P ( B | A ) − log P ( B ) P ( A ) log P ( A | B ) P ( A c | B ) = log P ( B | A ) P ( B | A c ) + log P ( A ) P ( A c ) Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples Bayes’ theorem: P ( A | B ) = P ( B | A ) P ( A ) P ( B ) log P ( B | A ) = log P ( B | A ) − log P ( B ) P ( A ) log P ( A | B ) P ( A c | B ) = log P ( B | A ) P ( B | A c ) + log P ( A ) P ( A c ) P ( B | A c ) + log P ( A c | B ) log P ( A | B ) = log P ( B | A ) P ( A c ) P ( A ) Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples Bayes’ theorem: P ( B | A c ) + log P ( A c | B ) log P ( A | B ) = log P ( B | A ) P ( A c ) P ( A ) Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples Bayes’ theorem: P ( B | A c ) + log P ( A c | B ) log P ( A | B ) = log P ( B | A ) P ( A c ) P ( A ) log d Π y ( θ ) = log dP θ ( y ) + something else d Π 0 dP 0 Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples Bayes’ theorem: P ( B | A c ) + log P ( A c | B ) log P ( A | B ) = log P ( B | A ) P ( A c ) P ( A ) log d Π y ( θ ) = log dP θ ( y ) + something else d Π 0 dP 0 log d Π y ( θ ) = θ T v ( y ) − I ( P 0 | P θ ) − ψ ( y ) d Π 0 Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples An exponential family of distributions can be represented in terms of the vector space of minimal sufficient statistics (i.e. functions on the observation space.) The manifold of distributions M maps uniquely unto a tangent space L . spanned by the minimal sufficient statistics. L P 0 3 v I ( P 0 j P 3 ) log dP θ P 3 = θ v − I ( P 0 | P θ ) M dP 0 Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples A prior distribution is a distribution on L . An exponential family of priors can be expressed in terms of the space of linear functions on L . This dual space will include the evaluation functional v �→ v ( y ), and thus will include all possible posteriors from any prior in the family. log d Π y ( v ) = v ( y ) − I ( P 0 | P v ) − ψ ( y ) d Π 0 Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples The lower envelope theorem says that a lower prevision can be expressed as the infimum over a set of linear previsions. This means that an imprecise prior can represented by a set of precise priors. In the expression log d Π y ( θ ) = θ T v ( y ) − I ( P 0 | P θ ) − ψ ( y ) d Π 0 the first two terms on the right do not depend on the prior, and thus would translate all priors in the same way. The third term does not depend on the model parameter θ . Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples Precise infrence Imprecise infrence posterior posterior η 1 prior η 1 prior 0 0 −2 0 2 −2 0 2 η 0 η 0 Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples Normal family with known variance. 10 8 6 4 2 2 1 0 -2 -4 -6 -8 -10 -1 0 1 2 3 4 5 6 7 8 2 0 Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples Binomial model with normal prior of logit expectation contours of logit-normal 0.2 0.15 0.1 0.05 0 2 1.5 -2 1 -1 0 0.5 1 2 0 0 2 2 1 Bickis U of S The Geometry of Imprecise Inference

Bayes’ theorem Exponential family Dual manifold Lower envelope theorem Linear updating Examples Censored exponential model with gamma prior 5 t=1 prior censored at 2 4 death at 3 t=2 3 2 2 t=3 2 1 0 -1 -1 0 1 2 3 4 5 2 1 Bickis U of S The Geometry of Imprecise Inference