Generalized characterizations of semicom- putable semicomputable - PowerPoint PPT Presentation

Generalized characteriza- tions of Generalized characterizations of semicom- putable semicomputable semimeasures semimeasures Tom Sterkenburg Motivation Tom Sterkenburg The semicom- putable semimeasures The universal semimeasures Conclusion CCR 2015 June 24, 2015 Heidelberg University Tom Sterkenburg Generalized characterizations of semicomputable semimeasures

Solomonoff’s theory of prediction Generalized characteriza- ◮ How to predict the continuation of a given finite string of bits? tions of semicom- putable semimeasures Tom Sterkenburg Motivation The semicom- putable semimeasures The universal semimeasures Conclusion Tom Sterkenburg Generalized characterizations of semicomputable semimeasures

Solomonoff’s theory of prediction Generalized characteriza- ◮ How to predict the continuation of a given finite string of bits? tions of semicom- ⊲ Devise an “a priori” probability distribution on 2 <ω , and predict by putable semimeasures conditionalization. Tom Sterkenburg Motivation The semicom- putable semimeasures The universal semimeasures Conclusion Tom Sterkenburg Generalized characterizations of semicomputable semimeasures

Solomonoff’s theory of prediction Generalized characteriza- ◮ How to predict the continuation of a given finite string of bits? tions of semicom- ⊲ Devise an “a priori” probability distribution on 2 <ω , and predict by putable semimeasures conditionalization. Tom A priori probabilities are assigned to strings of symbols Sterkenburg by examining the manner in which these strings might Motivation be produced by a universal Turing machine. Strings The semicom- with short (...) “descriptions” (...) are assigned high a putable semimeasures priori probabilities. (Solomonoff, A formal theory of inductive The universal inference , Inform. Contr. 7, 1964) semimeasures Conclusion Tom Sterkenburg Generalized characterizations of semicomputable semimeasures

Solomonoff’s theory of prediction Generalized characteriza- ◮ How to predict the continuation of a given finite string of bits? tions of semicom- ⊲ Devise an “a priori” probability distribution on 2 <ω , and predict by putable semimeasures conditionalization. Tom A priori probabilities are assigned to strings of symbols Sterkenburg by examining the manner in which these strings might Motivation be produced by a universal Turing machine. Strings The semicom- with short (...) “descriptions” (...) are assigned high a putable semimeasures priori probabilities. (Solomonoff, A formal theory of inductive The universal inference , Inform. Contr. 7, 1964) semimeasures ◮ The algorithmic probability distribution Q U via universal machine U Conclusion is given by � 2 −| ρ | , Q U ( σ ) := ρ ∈ D U ,σ with D U ,σ the set of minimal U -descriptions of σ . Tom Sterkenburg Generalized characterizations of semicomputable semimeasures

More precisely Generalized Definition (Levin, 1973) characteriza- tions of A monotone machine is a c.e. set M ⊆ 2 <ω × 2 <ω of pairs of strings such semicom- putable that if ( ρ 1 , σ 1 ) , ( ρ 2 , σ 2 ) ∈ M and ρ 1 � ρ 2 then σ 1 ∼ σ 2 . semimeasures Tom Sterkenburg Motivation The semicom- putable semimeasures The universal semimeasures Conclusion Tom Sterkenburg Generalized characterizations of semicomputable semimeasures

More precisely Generalized Definition (Levin, 1973) characteriza- tions of A monotone machine is a c.e. set M ⊆ 2 <ω × 2 <ω of pairs of strings such semicom- putable that if ( ρ 1 , σ 1 ) , ( ρ 2 , σ 2 ) ∈ M and ρ 1 � ρ 2 then σ 1 ∼ σ 2 . semimeasures Tom Sterkenburg ⊲ Monotone machine U is universal if Motivation ( ρ e ρ, σ ) ∈ U ⇔ ( ρ, σ ) ∈ M e The semicom- putable semimeasures for some encoding { ρ e } e ∈ N of the class { M e } e ∈ N of all monotone machines. The universal semimeasures Conclusion Tom Sterkenburg Generalized characterizations of semicomputable semimeasures

More precisely Generalized Definition (Levin, 1973) characteriza- tions of A monotone machine is a c.e. set M ⊆ 2 <ω × 2 <ω of pairs of strings such semicom- putable that if ( ρ 1 , σ 1 ) , ( ρ 2 , σ 2 ) ∈ M and ρ 1 � ρ 2 then σ 1 ∼ σ 2 . semimeasures Tom Sterkenburg ⊲ Monotone machine U is universal if Motivation ( ρ e ρ, σ ) ∈ U ⇔ ( ρ, σ ) ∈ M e The semicom- putable semimeasures for some encoding { ρ e } e ∈ N of the class { M e } e ∈ N of all monotone machines. The universal semimeasures Conclusion Definition A continuous a priori semimeasure is defined by λ U ( σ ) = λ ( � { ρ : ∃ σ ′ � σ (( ρ, σ ′ ) ∈ U ) } � ) for universal monotone machine U . Tom Sterkenburg Generalized characterizations of semicomputable semimeasures

Foundational principles Generalized characteriza- tions of semicom- putable semimeasures Tom Sterkenburg Motivation The semicom- putable semimeasures The universal semimeasures Conclusion Tom Sterkenburg Generalized characterizations of semicomputable semimeasures

Foundational principles Generalized characteriza- tions of ◮ The principle of Occam’s razor . semicom- putable That [this model] might be valid is suggested by “Occam’s semimeasures razor,” (...) that the more “simple” or “economical” of Tom several hypotheses is the more likely. Turing machines are Sterkenburg then used to explicate the concepts of “simplicity” or “economy”—the more “simple” hypothesis being that Motivation with the shortest “description.” (Solomonoff, 1964, p. 3) The semicom- putable semimeasures The universal semimeasures Conclusion Tom Sterkenburg Generalized characterizations of semicomputable semimeasures

Foundational principles Generalized characteriza- tions of ◮ The principle of Occam’s razor . semicom- putable That [this model] might be valid is suggested by “Occam’s semimeasures razor,” (...) that the more “simple” or “economical” of Tom several hypotheses is the more likely. Turing machines are Sterkenburg then used to explicate the concepts of “simplicity” or “economy”—the more “simple” hypothesis being that Motivation with the shortest “description.” (Solomonoff, 1964, p. 3) The semicom- putable semimeasures ◮ The principle of indifference . The universal Another suggested point of support is the principle of semimeasures indifference. If all inputs to a Turing machine that are of Conclusion a given fixed length, are assigned “indifferently equal a priori” likelihoods, then the probability distribution on the output strings is equivalent to that imposed by the (...) model described. (Solomonoff, 1964, p. 4) Tom Sterkenburg Generalized characterizations of semicomputable semimeasures

The question Generalized characteriza- ◮ Both associations rely on unique features of the uniform measure λ . tions of semicom- ⊲ But is λ really essential in the definition? Couldn’t we equivalently putable semimeasures define µ U ( σ ) := µ ( � { ρ : ∃ σ ′ � σ (( ρ, σ ′ ) ∈ U ) } � ) , Tom Sterkenburg for some other computable measure µ ? Motivation ⊲ If the choice for λ is only circumstantial, this might be thought to The semicom- undermine both associations. putable semimeasures The universal semimeasures Conclusion Tom Sterkenburg Generalized characterizations of semicomputable semimeasures

The question Generalized characteriza- ◮ Both associations rely on unique features of the uniform measure λ . tions of semicom- ⊲ But is λ really essential in the definition? Couldn’t we equivalently putable semimeasures define µ U ( σ ) := µ ( � { ρ : ∃ σ ′ � σ (( ρ, σ ′ ) ∈ U ) } � ) , Tom Sterkenburg for some other computable measure µ ? Motivation ⊲ If the choice for λ is only circumstantial, this might be thought to The semicom- undermine both associations. putable semimeasures Question The universal semimeasures Can we replace the uniform measure λ in the definition by an element µ Conclusion from some wider class of computable measures and still obtain the same class of a priori semimeasures? That is, does it hold for some wider class (and what class?) of computable µ that { µ U } U = { λ U } U ? Tom Sterkenburg Generalized characterizations of semicomputable semimeasures

Overview Generalized characteriza- tions of semicom- putable semimeasures ◮ Machine characterizations of the semicomputable semimeasures Tom Sterkenburg ⊲ The continuous semimeasures and monotone machines ⊲ The discrete semimeasures and prefix-free machines Motivation The semicom- ◮ The a priori semicomputable semimeasures putable semimeasures ⊲ Bayesian mixtures over all semicomputable semimeasures The universal ⊲ The (positive) answer semimeasures ◮ Wrapping up Conclusion Tom Sterkenburg Generalized characterizations of semicomputable semimeasures

The class M and its characterization Generalized characteriza- Definition tions of semicom- A continuous semimeasure ν : 2 ω → R ≥ 0 satisfies putable semimeasures ν ( � ǫ � ) ≤ 1; 1 Tom Sterkenburg ν ( � σ 0 � ) + ν ( � σ 1 � ) ≤ ν ( � σ � ) for all σ ∈ 2 <ω . 2 Motivation The semicom- putable semimeasures The universal semimeasures Conclusion Tom Sterkenburg Generalized characterizations of semicomputable semimeasures