Strong Asymptotic Assertions for Discrete MDL in Regression and - PowerPoint PPT Presentation

Strong Asymptotic Assertions for Discrete MDL in Regression and Classification or A Strange Way of Proving Consistency of MDL Learning Jan Poland and Marcus Hutter IDSIA � Lugano � Switzerland

2 Focus of this Talk Regression Classification this talk this paper Sequence Prediction COLT‘04

3 Why Consistency? � ��

4 Setup � �� !� �� x �� n , y �� n � � �� x i � � �� y i � � � , � � �� i � n � �� x � � " �� # �� y � � $�� %��& �� # �� y � x � � � � � '��& �� '($" � �� " )))

5 Bayesian Framework � * �� ν ��! � �� # �� !�� , � � � �� !�� +�� ν � � �� w ν > � � ,�� %��& � ν ∈C w ν � � � +��!��& � ��

6 Proper/Online Learning .�� !��& � �� x � � �� !� �� !��! � �� y �� # �� -�� & �� x �� t , y <t � �� x <t , y <t �

7 Bayes Mixture � ��" �� x �� n , y �� n � " �� n �� t �� ν � y t � x t � ν w ν � � n ξ � y n �� x �� n �� , y �� n � � t �� ν � y t � x t � ν w ν � �� /�� !�� /�� !��" ��

8 Static MDL ��" �� !�� $*.�& ̺ �� y n �� x �� n �� , y �� n � � ν ∗ � x �� ,y �� y n �� x n �� ν ∗ ν ∈C � w ν ν � y �� n � x �� n � � � x �� ,y �� 0��!�� ! � )

9 Dynamic MDL �� ! �� $0� �� #��!��1�� !��1�� " �� & ̺ � y n � , y <n � � ̺ � y �� n � x �� n � ̺ � y <n � x <n � ̺ � y �� n � x �� n � � ̺ � y n � , y <n � � � y � ̺ � y �� n � x �� n � �� ̺ � y �� n � x �� n � � �� ν ∈C � w ν ν � y �� n � x �� n � � . �� !��& ��!�� !�� y n ) �� !�� $0� �� !�� !�� !�� 2�)

10 Distance and Convergence �� & �� h � � � y t � x �� t , y <t � � ψ � y t � x �� t , y <t � t � �, ψ � � . y � ∈{ � , � } �� ψ #�� #�� ∞ H � � � h � t � �, ψ �� < � . x � ∞ � �, ψ � � t �� !�� h � t � � �� )

11 Other Distance Measures � � � � � � y t � x �� t , y <t � � ψ � y t � x �� t , y <t � s t � �, ψ � � y � ∈{ � , � } �%�� y t � x �� t , y <t � � ψ � y t � x �� t , y <t � a t � �, ψ � � � y � ∈{ � , � } �� y t � x �� t , y <t � � �� y t � x �� t , y <t � d t � �, ψ � � ψ � y t � x �� t , y <t � y � ∈{ � , � } ,��3#��

12 Distance Measures: Properties � � �� %�� a t � 4�� h t & � d t � � � �!�� %�� a t � 5�� s t & � d t � � � �!�� %�� *�� a t & � d t � �� %�� ,��3#�� d t & � a t

13 Convergence Theorem 6�� " �� w � �� " �� ̺ �� ̺ ̺ � � � � � t a t � � w − � t a t � � w − � t d t � � w − � � � � � H � � �,̺ �� w − � �

14 Properties of the Proof � .�� -#��!� � �� '��!��7� �� # ��

15 Loss bounds � *��!� �� ℓ � y, � y � x � � �� x " �� y " �� # �� y � �� ) �� /��#��!�� L �� !�� & � L � ̺ � � L � � � � �� w − � �� w − � � � � � L � � � � � �� #�� 1�� # !��

Strong Asymptotic Assertions for Discrete MDL in Regression and - PowerPoint PPT Presentation

Strong Asymptotic Assertions for Discrete MDL in Regression and Classification or A Strange Way of Proving Consistency of MDL Learning Jan Poland and Marcus Hutter IDSIA Lugano Switzerland 2 Focus of this Talk Regression

Asymptotic approximations for options with discrete sampling Sam Howison Mathematical Institute

Program Analysis discrete math refresher READING: Textbook chapter 2 (p. 13-26)

Zero topological entropy and asymptotic pairs Czech-Slovak-Spanish-Polish Workshop on Discrete

High Order Asymptotic Preserving Schemes for Some Discrete-Velocity Kinetic Equations Fengyan Li

Program Analysis 1 Summary Summary analysis of algorithms asymptotic analysis

1 Java assert statement Java assert (cont.) an assert statement can be placed anywhere you

Assertions, pre/post- conditions Assertions: Section 4.2 in Savitch (p. 239) Programming as a

Assertions and Triggers Rose-Hulman Institute of Technology Curt Clifton Assertions Like

The Discovery of Asymptotic Freedom & The Emergence of QCD David Gross Nobel Lecture

The strong asymptotic freeness of large random and deterministic matrices Camille Male

Types of Formal Specifications for Assertions Concurrent and Reactive Systems Assertions

ASYNCHRONOUS ASSERTIONS What are assertions? public class ATM { public void

On the Asymptotic Variance of the Estimator of Kendalls Tau Barbara Dengler, Uwe Schmock

Asymptotic behavior of series of multiple integrals. K. K. Kozlowski CNRS, Institut de

10/6/2016 Give Asymptotic Analyses for the Following g( n ) = 45 n log n + 2 n 2 + 65 1. 2. g( n

1 Should debug code be left in shipped version Step through your code Pro: Maguire

Assertions, Denials Questions, Answers & the Common Ground Greg Restall dianoia institute (

the DNA of asymptotic safety Daniel F Litim based on work with Andrew Bond and Tugba Buyukbese

Plan Discrete paths as Heyting algebras Discrete paths as categories Discrete paths as quantales

Lattice-Based Cryptography: Trapdoors, Discrete Gaussians, and Applications Chris Peikert

FUNCTORS IN THE ASYMPTOTIC CATEGORIES Mykhailo Zarichnyi Lviv National University and

An Introduction to Asymptotic Theory Ping Yu School of Economics and Finance The University of

Cut-points in asymptotic cones of groups Mark Sapir With J. Behrstock, C. Drut u, S. Mozes,

On-time diagnosis of discrete event systems Aditya Mahajan and Demosthenis Teneketzis Dept. of

Strong Asymptotic Assertions for Discrete MDL in Regression and - PowerPoint PPT Presentation

Strong Asymptotic Assertions for Discrete MDL in Regression and Classification or A Strange Way of Proving Consistency of MDL Learning Jan Poland and Marcus Hutter IDSIA Lugano Switzerland 2 Focus of this Talk Regression

Asymptotic approximations for options with discrete sampling Sam Howison Mathematical Institute

Program Analysis discrete math refresher READING: Textbook chapter 2 (p. 13-26)

Zero topological entropy and asymptotic pairs Czech-Slovak-Spanish-Polish Workshop on Discrete

High Order Asymptotic Preserving Schemes for Some Discrete-Velocity Kinetic Equations Fengyan Li

Program Analysis 1 Summary Summary analysis of algorithms asymptotic analysis

1 Java assert statement Java assert (cont.) an assert statement can be placed anywhere you

Assertions, pre/post- conditions Assertions: Section 4.2 in Savitch (p. 239) Programming as a

Assertions and Triggers Rose-Hulman Institute of Technology Curt Clifton Assertions Like

The Discovery of Asymptotic Freedom &amp; The Emergence of QCD David Gross Nobel Lecture

The strong asymptotic freeness of large random and deterministic matrices Camille Male

Types of Formal Specifications for Assertions Concurrent and Reactive Systems Assertions

ASYNCHRONOUS ASSERTIONS What are assertions? public class ATM { public void

On the Asymptotic Variance of the Estimator of Kendalls Tau Barbara Dengler, Uwe Schmock

Asymptotic behavior of series of multiple integrals. K. K. Kozlowski CNRS, Institut de

10/6/2016 Give Asymptotic Analyses for the Following g( n ) = 45 n log n + 2 n 2 + 65 1. 2. g( n

1 Should debug code be left in shipped version Step through your code Pro: Maguire

Assertions, Denials Questions, Answers &amp; the Common Ground Greg Restall dianoia institute (

the DNA of asymptotic safety Daniel F Litim based on work with Andrew Bond and Tugba Buyukbese

Plan Discrete paths as Heyting algebras Discrete paths as categories Discrete paths as quantales

Lattice-Based Cryptography: Trapdoors, Discrete Gaussians, and Applications Chris Peikert

FUNCTORS IN THE ASYMPTOTIC CATEGORIES Mykhailo Zarichnyi Lviv National University and

An Introduction to Asymptotic Theory Ping Yu School of Economics and Finance The University of

Cut-points in asymptotic cones of groups Mark Sapir With J. Behrstock, C. Drut u, S. Mozes,

On-time diagnosis of discrete event systems Aditya Mahajan and Demosthenis Teneketzis Dept. of

The Discovery of Asymptotic Freedom & The Emergence of QCD David Gross Nobel Lecture

Assertions, Denials Questions, Answers & the Common Ground Greg Restall dianoia institute (