4csll5 parameter estimation supervised and unsupervised
play

4CSLL5 Parameter Estimation (Supervised and Unsupervised) Martin - PowerPoint PPT Presentation

Aug 14, 2023 •103 likes •903 views

4CSLL5 Parameter Estimation (Supervised and Unsupervised) 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Martin Emms September 20, 2019 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Outline Supervised Maximum Likelihood

  1. 4CSLL5 Parameter Estimation (Supervised and Unsupervised) 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Martin Emms September 20, 2019

  2. 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Outline Supervised Maximum Likelihood Estimation(MLE) First scenario: (toss a ’coin’ Z) D 2nd scenario: (toss Z; (then A or B) 10 ) D

  3. 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Outline Parameter Estimation

  4. 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Supervised Maximum Likelihood Estimation(MLE) First scenario: (toss a ’coin’ Z) D Outline Supervised Maximum Likelihood Estimation(MLE) First scenario: (toss a ’coin’ Z) D 2nd scenario: (toss Z; (then A or B) 10 ) D

  5. 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Supervised Maximum Likelihood Estimation(MLE) First scenario: (toss a ’coin’ Z) D Common-sense and relative frequency Suppose a 2-sided ’coin’ Z , one side labelled ’a’, other side labelled ’b’

  6. 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Supervised Maximum Likelihood Estimation(MLE) First scenario: (toss a ’coin’ Z) D Common-sense and relative frequency Suppose a 2-sided ’coin’ Z , one side labelled ’a’, other side labelled ’b’ P ( Z = a ): probability of giving ’a’ when tossed – currently not known

  7. 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Supervised Maximum Likelihood Estimation(MLE) First scenario: (toss a ’coin’ Z) D Common-sense and relative frequency Suppose a 2-sided ’coin’ Z , one side labelled ’a’, other side labelled ’b’ P ( Z = a ): probability of giving ’a’ when tossed – currently not known P ( Z = b ): probability of giving ’b’ when tossed – currently not known

  8. 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Supervised Maximum Likelihood Estimation(MLE) First scenario: (toss a ’coin’ Z) D Common-sense and relative frequency Suppose a 2-sided ’coin’ Z , one side labelled ’a’, other side labelled ’b’ P ( Z = a ): probability of giving ’a’ when tossed – currently not known P ( Z = b ): probability of giving ’b’ when tossed – currently not known Suppose you have data d recording 100 tosses of Z

  9. 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Supervised Maximum Likelihood Estimation(MLE) First scenario: (toss a ’coin’ Z) D Common-sense and relative frequency Suppose a 2-sided ’coin’ Z , one side labelled ’a’, other side labelled ’b’ P ( Z = a ): probability of giving ’a’ when tossed – currently not known P ( Z = b ): probability of giving ’b’ when tossed – currently not known Suppose you have data d recording 100 tosses of Z if there were (50 a, 50 b) in d , ’common-sense’ says P ( Z = a ) = 50 / 100

  10. 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Supervised Maximum Likelihood Estimation(MLE) First scenario: (toss a ’coin’ Z) D Common-sense and relative frequency Suppose a 2-sided ’coin’ Z , one side labelled ’a’, other side labelled ’b’ P ( Z = a ): probability of giving ’a’ when tossed – currently not known P ( Z = b ): probability of giving ’b’ when tossed – currently not known Suppose you have data d recording 100 tosses of Z if there were (50 a, 50 b) in d , ’common-sense’ says P ( Z = a ) = 50 / 100 if there were (30 a, 70 b) in d , ’common-sense’ says P ( Z = a ) = 30 / 100

  11. 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Supervised Maximum Likelihood Estimation(MLE) First scenario: (toss a ’coin’ Z) D Common-sense and relative frequency Suppose a 2-sided ’coin’ Z , one side labelled ’a’, other side labelled ’b’ P ( Z = a ): probability of giving ’a’ when tossed – currently not known P ( Z = b ): probability of giving ’b’ when tossed – currently not known Suppose you have data d recording 100 tosses of Z if there were (50 a, 50 b) in d , ’common-sense’ says P ( Z = a ) = 50 / 100 if there were (30 a, 70 b) in d , ’common-sense’ says P ( Z = a ) = 30 / 100 ie. you ’define’ or ’estimate’ the probability by the relative frequency

  12. 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Supervised Maximum Likelihood Estimation(MLE) First scenario: (toss a ’coin’ Z) D Data likelihood assuming the tosses of Z are all independent, can work out the probability of the observed data d if Z ’s probabilities had particular values.

  13. 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Supervised Maximum Likelihood Estimation(MLE) First scenario: (toss a ’coin’ Z) D Data likelihood assuming the tosses of Z are all independent, can work out the probability of the observed data d if Z ’s probabilities had particular values. let θ a and θ b stand for P ( Z = a ) and P ( Z = b )

  14. 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Supervised Maximum Likelihood Estimation(MLE) First scenario: (toss a ’coin’ Z) D Data likelihood assuming the tosses of Z are all independent, can work out the probability of the observed data d if Z ’s probabilities had particular values. let θ a and θ b stand for P ( Z = a ) and P ( Z = b ) let #( a ) be the number of ’a’ outcomes in the sequence d

  15. 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Supervised Maximum Likelihood Estimation(MLE) First scenario: (toss a ’coin’ Z) D Data likelihood assuming the tosses of Z are all independent, can work out the probability of the observed data d if Z ’s probabilities had particular values. let θ a and θ b stand for P ( Z = a ) and P ( Z = b ) let #( a ) be the number of ’a’ outcomes in the sequence d let #( b ) be the number of ’b’ outcomes in the sequence d

  16. 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Supervised Maximum Likelihood Estimation(MLE) First scenario: (toss a ’coin’ Z) D Data likelihood assuming the tosses of Z are all independent, can work out the probability of the observed data d if Z ’s probabilities had particular values. let θ a and θ b stand for P ( Z = a ) and P ( Z = b ) let #( a ) be the number of ’a’ outcomes in the sequence d let #( b ) be the number of ’b’ outcomes in the sequence d the probability of d , assuming the probability settings θ a and θ b is

  17. 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Supervised Maximum Likelihood Estimation(MLE) First scenario: (toss a ’coin’ Z) D Data likelihood assuming the tosses of Z are all independent, can work out the probability of the observed data d if Z ’s probabilities had particular values. let θ a and θ b stand for P ( Z = a ) and P ( Z = b ) let #( a ) be the number of ’a’ outcomes in the sequence d let #( b ) be the number of ’b’ outcomes in the sequence d the probability of d , assuming the probability settings θ a and θ b is × θ #( b ) p ( d ) = θ #( a ) (1) a b

  18. 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Supervised Maximum Likelihood Estimation(MLE) First scenario: (toss a ’coin’ Z) D Data likelihood assuming the tosses of Z are all independent, can work out the probability of the observed data d if Z ’s probabilities had particular values. let θ a and θ b stand for P ( Z = a ) and P ( Z = b ) let #( a ) be the number of ’a’ outcomes in the sequence d let #( b ) be the number of ’b’ outcomes in the sequence d the probability of d , assuming the probability settings θ a and θ b is × θ #( b ) p ( d ) = θ #( a ) (1) a b different settings of θ a and θ b will give different values for p ( d ) following slides investigate this empirically

  19. 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Supervised Maximum Likelihood Estimation(MLE) First scenario: (toss a ’coin’ Z) D p ( d ) for 50 a, 50 b X 1.2e−21 as θ a is varied, data prob p ( d ) varies 8.0e−22 4.0e−22 0.0e+00 0.0 0.2 0.4 0.6 0.8 1.0

  20. 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Supervised Maximum Likelihood Estimation(MLE) First scenario: (toss a ’coin’ Z) D p ( d ) for 50 a, 50 b X 1.2e−21 as θ a is varied, data prob p ( d ) varies 8.0e−22 max occurs at θ a = 0 . 5 4.0e−22 0.0e+00 0.0 0.2 0.4 0.6 0.8 1.0

  21. 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Supervised Maximum Likelihood Estimation(MLE) First scenario: (toss a ’coin’ Z) D p ( d ) for 50 a, 50 b X 1.2e−21 as θ a is varied, data prob p ( d ) varies 8.0e−22 max occurs at θ a = 0 . 5 4.0e−22 50 which is 50 + 50 0.0e+00 0.0 0.2 0.4 0.6 0.8 1.0

  22. 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Supervised Maximum Likelihood Estimation(MLE) First scenario: (toss a ’coin’ Z) D p ( d ) for 30 a, 70 b 4e−19 X 3e−19 as θ a is varied, data prob p ( d ; θ a , θ b ) varies 2e−19 1e−19 0e+00 0.0 0.2 0.4 0.6 0.8 1.0

  23. 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Supervised Maximum Likelihood Estimation(MLE) First scenario: (toss a ’coin’ Z) D p ( d ) for 30 a, 70 b 4e−19 X 3e−19 as θ a is varied, data prob p ( d ; θ a , θ b ) varies 2e−19 max occurs at θ a = 0 . 3 1e−19 0e+00 0.0 0.2 0.4 0.6 0.8 1.0

  24. 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Supervised Maximum Likelihood Estimation(MLE) First scenario: (toss a ’coin’ Z) D p ( d ) for 30 a, 70 b 4e−19 X 3e−19 as θ a is varied, data prob p ( d ; θ a , θ b ) varies 2e−19 max occurs at θ a = 0 . 3 1e−19 30 which is 30 + 70 0e+00 0.0 0.2 0.4 0.6 0.8 1.0

Recommend

4CSLL5 Parameter Estimation (Supervised and Unsupervised) Unsupervised Maximum Likelihood

4CSLL5 Parameter Estimation (Supervised and Unsupervised) Unsupervised Maximum Likelihood

4CSLL5 Parameter Estimation (Supervised and Unsupervised) 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Outline 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Unsupervised Maximum Likelihood (re-)Estimation Hidden

310 views • 8 slides
4CSLL5 Parameter Estimation (Supervised and Unsupervised) Supervised Maximum Likelihood

4CSLL5 Parameter Estimation (Supervised and Unsupervised) Supervised Maximum Likelihood

4CSLL5 Parameter Estimation (Supervised and Unsupervised) 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Outline 4CSLL5 Parameter Estimation (Supervised and Unsupervised) Supervised Maximum Likelihood Estimation(MLE) First scenario:

332 views • 5 slides
4CSLL5 IBM Translation Models Martin Emms October 29, 2020 4CSLL5 IBM Translation Models

4CSLL5 IBM Translation Models Martin Emms October 29, 2020 4CSLL5 IBM Translation Models

4CSLL5 IBM Translation Models 4CSLL5 IBM Translation Models Martin Emms October 29, 2020 4CSLL5 IBM Translation Models Parameter learning (efficient) How to sum alignments efficiently Efficient EM via p (( j , i ) a | o , s ) 4CSLL5 IBM

658 views • 52 slides
4CSLL5 IBM Translation Models Martin Emms October 23, 2020 4CSLL5 IBM Translation Models

4CSLL5 IBM Translation Models Martin Emms October 23, 2020 4CSLL5 IBM Translation Models

4CSLL5 IBM Translation Models 4CSLL5 IBM Translation Models Martin Emms October 23, 2020 4CSLL5 IBM Translation Models Parameter learning (brute force) Introduction The brute force EM algorithm defined A formula for p ( a | o , s ) Examples

759 views • 50 slides
UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised

UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised

UNSUPERVISED LEARNING, CLUSTERING UNSUPERVISED LEARNING UNSUPERVISED LEARNING Supervised learning: X - y pairs, f(x) function approximation Unsupervised learning: only X, no y Exploring the space of X measurements,

440 views • 14 slides
Unsupervised Learning Unsupervised vs Supervised Learning: Most of this course focuses on

Unsupervised Learning Unsupervised vs Supervised Learning: Most of this course focuses on

Unsupervised Learning Unsupervised vs Supervised Learning: Most of this course focuses on supervised learning methods such as regression and classification. In that setting we observe both a set of features X 1 , X 2 , . . . , X p for each

934 views • 50 slides
Unsupervised Learning Unsupervised vs Supervised Learning: Most of this course focuses on

Unsupervised Learning Unsupervised vs Supervised Learning: Most of this course focuses on

Unsupervised Learning Unsupervised vs Supervised Learning: Most of this course focuses on supervised learning methods such as regression and classification. In that setting we observe both a set of features X 1 , X 2 , . . . , X p for each

721 views • 60 slides
Unsupervised and Semi-supervised Learning of Structure Graham Neubig Site

Unsupervised and Semi-supervised Learning of Structure Graham Neubig Site

CS11-747 Neural Networks for NLP Unsupervised and Semi-supervised Learning of Structure Graham Neubig Site https://phontron.com/class/nn4nlp2018/ Supervised, Unsupervised, Semi-supervised Most models handled here are supervised learning

670 views • 32 slides
Unsupervised and Semi-supervised Learning of Structure Graham Neubig Site

Unsupervised and Semi-supervised Learning of Structure Graham Neubig Site

CS11-747 Neural Networks for NLP Unsupervised and Semi-supervised Learning of Structure Graham Neubig Site https://phontron.com/class/nn4nlp2020/ Supervised, Unsupervised, Semi-supervised Most models handled here are supervised learning

580 views • 39 slides
Mutation models: probabilistic study and parameter estimation Adrien Mazoyer, supervised by

Mutation models: probabilistic study and parameter estimation Adrien Mazoyer, supervised by

Mutation models: probabilistic study and parameter estimation Adrien Mazoyer, supervised by Bernard Ycart Laboratoire Jean Kuntzmann, UGA GRENOBLE JPS 2016 Adrien Mazoyer (LJK) Mutation models JPS 2016 1 / 17 Example mutation rate = 0.05 ,

508 views • 32 slides
Augmenting Supervised Neural Networks with Unsupervised Objectives for Large-scale Image

Augmenting Supervised Neural Networks with Unsupervised Objectives for Large-scale Image

Augmenting Supervised Neural Networks with Unsupervised Objectives for Large-scale Image Classification Yu Yuting Zh Zhang, Ki Kibok Le Lee, Ho Honglak Le Lee University of Michigan, Ann Arbor Unsupervised and supervised deep learning

753 views • 52 slides
I 4 - Bayesian parameter estimation in a normal model STAT 587 (Engineering) Iowa State

I 4 - Bayesian parameter estimation in a normal model STAT 587 (Engineering) Iowa State

I 4 - Bayesian parameter estimation in a normal model STAT 587 (Engineering) Iowa State University September 18, 2020 Bayesian parameter estimation Bayesian parameter estimation Recall that Bayesian parameter estimation involves p ( | y ) =

439 views • 20 slides
Outline Introduction Knowledge Structures Parameter Estimation Maximum Likelihood Estimation

Outline Introduction Knowledge Structures Parameter Estimation Maximum Likelihood Estimation

Faculty of Science Introduction Knowledge Structures Parameter Estimation Implementation in R Concluding Remarks Department of Psychology Outline Introduction Knowledge Structures Parameter Estimation Maximum Likelihood Estimation

647 views • 9 slides
Metti 5 Optimization for nonlinear parameter estimation and function estimation Lecture 7

Metti 5 Optimization for nonlinear parameter estimation and function estimation Lecture 7

Metti 5 Optimization for nonlinear parameter estimation and function estimation Lecture 7 Roscoff, June 13-18, 2011 Objectives Direct problem input BC Model State solution IC Parameters

869 views • 69 slides
Machine Learning Lecture 6 Unsupervised Learning with a bit of Supervised Learning Clustering

Machine Learning Lecture 6 Unsupervised Learning with a bit of Supervised Learning Clustering

Machine Learning Lecture 6 Unsupervised Learning with a bit of Supervised Learning Clustering and nearest neighbour classifiers. Justin Pearson 1 2020 1 http://user.it.uu.se/~justin/Teaching/MachineLearning/index.html 1 / 27 Supervised

813 views • 27 slides
4CSLL5 IBM Translation Models IBM models Probabilities and Translation Alignments Martin Emms

4CSLL5 IBM Translation Models IBM models Probabilities and Translation Alignments Martin Emms

4CSLL5 IBM Translation Models 4CSLL5 IBM Translation Models 4CSLL5 IBM Translation Models IBM models Probabilities and Translation Alignments Martin Emms IBM Model 1 definitions October 22, 2020 4CSLL5 IBM Translation Models 4CSLL5 IBM

571 views • 7 slides
Maximum-likelihood and Bayesian parameter estimation Andrea Passerini passerini@disi.unitn.it

Maximum-likelihood and Bayesian parameter estimation Andrea Passerini passerini@disi.unitn.it

Maximum-likelihood and Bayesian parameter estimation Andrea Passerini passerini@disi.unitn.it Machine Learning Maximum-likelihood and Bayesian parameter estimation Parameter estimation Setting Data are sampled from a probability distribution

856 views • 52 slides
4CSLL5 IBM Translation Models Martin Emms October 22, 2020 4CSLL5 IBM Translation Models IBM

4CSLL5 IBM Translation Models Martin Emms October 22, 2020 4CSLL5 IBM Translation Models IBM

4CSLL5 IBM Translation Models 4CSLL5 IBM Translation Models Martin Emms October 22, 2020 4CSLL5 IBM Translation Models IBM models Probabilities and Translation Alignments IBM Model 1 definitions 4CSLL5 IBM Translation Models IBM models

1.23k views • 103 slides
Unsupervised Learning Introduction Nakul Verma Unsupervised Learning What can we learn from

Unsupervised Learning Introduction Nakul Verma Unsupervised Learning What can we learn from

Unsupervised Learning Introduction Nakul Verma Unsupervised Learning What can we learn from data when label information is not available? Supervised learning framework Supervised learning Data: Assumption: there is a (relatively simple)

442 views • 16 slides
Combining Unsupervised and Supervised Parser Mar$n Riedl, Irina

Combining Unsupervised and Supervised Parser Mar$n Riedl, Irina

Combining Unsupervised and Supervised Parser Mar$n Riedl, Irina Alles and Chris Biemann Language Technology Technische Universitt Darmstadt, Germany COLING

493 views • 21 slides
Maximum Likelihood Estimation MLE tool for parameter estimation good approach for cases

Maximum Likelihood Estimation MLE tool for parameter estimation good approach for cases

Maximum Likelihood Estimation MLE tool for parameter estimation good approach for cases when OLS (ordinary least squares) assumptions are violated e.g. for non-linear models with non-normal data in MLE, we estimate the parameters

1.01k views • 66 slides
Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning

Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning

Hierarchical Clustering 4-4-16 Hierarchical clustering: the setting Unsupervised learning no labels/output, only x/input Clustering Group similar points together Machine learning taxonomy Supervised Semi-Supervised Unsupervised

366 views • 15 slides
Current State of Unsupervised Deep Learning William Falcon, PhD Student AGENDA AGENDA

Current State of Unsupervised Deep Learning William Falcon, PhD Student AGENDA AGENDA

Current State of Unsupervised Deep Learning William Falcon, PhD Student AGENDA AGENDA Unsupervised vs self-supervised vs supervised learning Why we don't like supervised learning Cost of supervised learning Theoretical approaches to

846 views • 46 slides
OUTLINE Introduction Supervised ,Unsupervised And Semi-Supervised Learning.

OUTLINE Introduction Supervised ,Unsupervised And Semi-Supervised Learning.

Presentation in A Bayesian Approach To Satellite Aircraft Image Identification Using Invariant Moments . Dickson Gichaga Wambaa F56/76676/2009 Supervised By Professor Elijah Mwangi University Of Nairobi Electrical And Information Engineering

322 views • 29 slides

More recommend