On Computability and Learnability of the Pumping Lemma Function - PowerPoint PPT Presentation

On Computability and Learnability of the Pumping Lemma Function Dariusz Kaloci´ nski University of Warsaw, Poland March 11, 2014 8 th International Conference, LATA 2014

Structure ◮ what is the pumping lemma function? ◮ how complex is it? ◮ computable? ◮ learnable? ◮ exact placement of the function in the arithmetical hierarchy

Structure ◮ what is the pumping lemma function? ◮ how complex is it? ◮ computable? ◮ learnable? ◮ exact placement of the function in the arithmetical hierarchy ◮ on the way: we get a ,,natural” Π 0 2 -complete problem ◮ final remarks

Pumping Lemma (for Regular Languages) For regular L ⊆ Σ ∗ ( ∃ c > 0 )

Pumping Lemma (for Regular Languages) For regular L ⊆ Σ ∗ ( ∃ c > 0 ) ( ∀ ω ∈ L , | ω | ≥ c ) ( ∃ αβγ ): ◮ αβγ = ω ◮ | αβ | ≤ c ◮ β � = ε ◮ ( ∀ i ∈ N ) αβ i γ ∈ L

Pumping Lemma (for Regular Languages) For regular L ⊆ Σ ∗ ( ∃ c > 0 ) ( ∀ ω ∈ L , | ω | ≥ c ) ( ∃ αβγ ): ◮ αβγ = ω ◮ | αβ | ≤ c ◮ β � = ε ◮ ( ∀ i ∈ N ) αβ i γ ∈ L φ ( L , c ) - formula in yellow box ◮ ◮ φ ( L , c ) means: for given L , c is the witness for ∃ c ◮ c satisfying φ ( L , c ) is called a pumping constant for L

Problem Input: arbitrary L Output: the least pumping constant for L (if exists) ◮ we focus on r.e. languages ◮ W e = the domain of the e th algorithm ◮ L is r.e. ⇔ ∃ e ( L = W e ) ◮ R ( e , c ) ⇔ df c is a pumping constant for W e Pumping Lemma Function � the least c st. R ( e , c ) if ∃ cR ( e , c ) f ( e ) = undefined otherwise

Questions R ( e , c ) ⇔ df c is a pumping constant for W e � the least c st. R ( e , c ) if ∃ cR ( e , c ) f ( e ) = undefined otherwise Graph ( f ) = the graph of f = { ( x , y ) : f ( x ) = y } How complex are f and R ? ◮ is f computable? ◮ is Graph ( f ) r.e.? ◮ is f algorithmically learnable? ◮ if not, how strong oracle we need to make f learnable? ◮ how exactly does Graph ( f ) fit in arithmetical hierarchy? ◮ how exactly does R fit in arithmetical hierarchy?

Is f computable? We need ◮ EMPTY = { e ∈ N : W e = ∅} ◮ EMPTY is Π 0 1 -complete ◮ ≤ rec - reducibility via recursive function ◮ R ( e , c ) ⇔ df c is a pumping constant for W e Lemmas ◮ EMPTY ≤ rec R ◮ If R ( e , c ) then ( ∀ d > c ) R ( e , d ). Theorem f is not computable Proof. Suppose the contrary. Then R is Σ 0 1 . Let A ∈ Π 0 1 . A ≤ rec EMPTY ≤ rec R ∈ Σ 0 1 . Hence, Π 0 1 ⊆ Σ 0 1 . �

Is Graph ( f ) r.e.? We need ◮ EMPTY = { e ∈ N : W e = ∅} ◮ EMPTY is Π 0 1 -complete ◮ ≤ rec - reducibility via recursive function ◮ R ( e , c ) ⇔ df c is a pumping constant for W e Lemmas ◮ Graph ( f ) ∈ Σ 0 1 ⇒ R ∈ Σ 0 1 ◮ EMPTY ≤ rec R Theorem Graph ( f ) is not r.e. Proof. Suppose the contrary. By lemma R ∈ Σ 0 1 . Since EMPTY ≤ rec R , then EMPTY ≤ rec R . Hence, Π 0 1 ⊆ Σ 0 1 . �

Learnability Definition f : N k → N (possibly partial) is learnable if there is a total computable function g t ( x ) st. for all x ∈ N k : lim t →∞ g t ( x ) = f ( x ) , (1) which means that one of the two conditions hold: ◮ neither f ( x ) nor lim t →∞ g t ( x ) exist ◮ both f ( x ) and lim t →∞ g t ( x ) exist and are equal Example f ( x ) = 5 0 1 2 1 7 5 5 5 || || || || || || || || g 0 ( x ) g 1 ( x ) g 2 ( x ) g 3 ( x ) . . . g 1487 ( x ) g 1488 ( x ) g 1489 ( x ) . . .

Is f learnable? We need ◮ TOT = { e : W e = Σ ∗ } ◮ TOT is Π 0 2 -complete ◮ Gold’s lemma: f is learnable ⇔ Graph ( f ) ∈ Σ 0 2 ◮ R ( e , c ) ⇔ df c is a pumping constant for W e Lemma TOT ≤ rec R Theorem f is not learnable Proof. Suppose the contrary. Then Graph ( f ) ∈ Σ 0 2 . We have: R ( x , y ) ⇔ ∃ c (( x , c ) ∈ Graph ( f ) ∧ c ≤ y ) ⇔ ∃ ( ∃∀ . . . ∧ . . . ). So R ∈ Σ 0 2 . But by lemma TOT ≤ rec R . Hence, TOT ∈ Σ 0 2 . �

How complex oracle does make f learnable? We need ◮ HALT = the halting problem = { ( e , x ) : x ∈ W e } ◮ ≤ bl - bounded lexicographical order on words ◮ Gold’s lemma: f is learnable ⇔ Graph ( f ) ∈ Σ 0 2 Theorem f is learnable in HALT . Proof. R ( e , x ) ⇔ rec. in HALT rec. in HALT rec. � �� � � �� � αβ i γ ∈ W e )] } ( ∀ ω ) { [ ω ∈ W e ∧ . . . ] ⇒ ( ∃ α, β, γ ≤ bl ω )[ ���� . . . ∧ ( ∀ i ) R ( e , x ) ⇔ ∀ [ . . . ⇒ ∀ . . . ], so R ∈ Π 0 1 in HALT. ( e , x ) ∈ Graph ( f ) ⇔ R ( e , x ) ∧ ( ∀ y < x ) ¬ R ( e , y ) � �� � � �� � 1 in HALT 1 in HALT Π 0 Σ 0 Hence, Graph ( f ) ∈ Σ 0 2 in HALT and f is learnable in HALT.

How complex is R ? We need ◮ HALT = the halting problem = { ( e , x ) : x ∈ W e } ◮ TOT = { e : W e = N } ◮ TOT is Π 0 2 -complete ◮ R ( e , c ) ⇔ df c is a pumping constant for W e Lemma TOT ≤ rec R Theorem R is Π 0 2 -complete Proof. R is Π 0 2 -hard, since TOT ≤ rec R x ∈ W e ⇔ ∃ c T ( e , x , c ), T - Kleene predicate R ( e , x ) ⇔ ∀ [ ∃ . . . ⇒ ∃ ≤ bl ω ( . . . ∧ ∀∃ . . . )] Hence, R ∈ Π 0 2 .

f - exact place in arithmetical hierarchy Lemmas ◮ Graph ( f ) ∈ ∆ 0 (see paper) 3 ∈ Σ 0 ◮ Graph ( f ) / (proved) 2 ◮ R ( e , c ) ⇔ df c is a pumping constant for W e ◮ R is Π 0 2 -complete (proved) Theorem Graph ( f ) ∈ ∆ 0 3 − (Σ 0 2 ∪ Π 0 2 ) Proof. ∈ Π 0 We show Graph ( f ) / 2 . Suppose the contrary. Now show R ≤ T Graph ( f ). Algorithm with oracle Graph ( f ) that computes χ R : on input ( e , x ) output YES iff ( e , y ) ∈ Graph ( f ) holds for some y ≤ x . Hence, R ≤ T Graph ( f ). Since R is Π 0 2 -complete, R is Σ 0 2 -complete. Let A ∈ Σ 0 2 . We have A ≤ T R ≤ T Graph ( f ). Then Σ 0 2 ⊆ Π 0 2 . �

Final remarks ◮ what about other input representations? ◮ CFGs: f learnable ◮ oracle for characteristic function ◮ f learnable ◮ use in language identification? ◮ time bounded Turing machines ◮ f learnable

Thanks for your attention!