The Core Method Steffen H olldobler International Center for - PowerPoint PPT Presentation

The Core Method Steffen H¨ olldobler International Center for Computational Logic Technische Universit¨ at Dresden Germany ◮ The Very Idea ◮ The Propositional CORE Method ◮ Human Reasoning Steffen H¨ olldobler The Core Method 1

The Very Idea ◮ Various semantics for logic programs coincide with fixed points of associated immediate consequence operators (Apt, van Emden: Contributions to the Theory of Logic Programming. Journal of the ACM 29, 841-862: 1982). ◮ Banach Contraction Mapping Theorem A contraction mapping f defined on a complete metric space ( X , d ) has a unique fixed point. The sequence y , f ( y ) , f ( f ( y )) , . . . converges to this fixed point for any y ∈ X . ⊲ Consider programs whose immediate consequence operator is a contraction. (Fitting: Metric Methods – Three Examples and a Theorem. Journal of Logic Programming 21, 113-127: 1994). ◮ Every continuous function on the reals can be uniformly approximated by feed- forward connectionist networks (Funahashi: On the Approximate Realization of Continuous Mappings by Neural Networks. Neural Networks 2, 183-192: 1989). ⊲ Consider programs whose immediate consequence operator is continuous. (H., Kalinke, St¨ orr: Approximating the Semantics of Logic Programs by Recurrent Neural Networks. Applied Intelligence 11, 45-59: 1999). Steffen H¨ olldobler The Core Method 2

First Ideas ◮ H., Kalinke: Towards a New Massively Parallel Computational Model for Logic Programming In: Proceedings of the ECAI94 Workshop on Combining Symbolic and Connectionist Processing, 68-77: 1994. Steffen H¨ olldobler The Core Method 3

Interpretations ◮ Let L be a propositional language and {⊤ , ⊥} the set of truth values. ◮ An interpretation I is a mapping L → {⊤ , ⊥} . ◮ For a given program P , an interpretation I can be represented by the set of atoms occurring in P which are mapped to ⊤ under I , i.e. I ⊆ R P . ◮ 2 R P is the set of all interpretations for P . ◮ ( 2 R P , ⊆ ) is a complete lattice. ◮ An interpretation I for P is a model for P iff I ( P ) = ⊤ . Steffen H¨ olldobler The Core Method 4

Immediate Consequence Operator P : 2 R P → 2 R P : ◮ Immediate consequence operator T P ( I ) = { A | there is a clause A ← L 1 ∧ . . . ∧ L n ∈ P T such that I | = L 1 ∧ . . . ∧ L n } . ◮ I is a supported model iff T P ( I ) = I . ◮ Let lfp T P be the least fixed point of T P if it exists. Steffen H¨ olldobler The Core Method 5

The Propositional CORE Method ◮ Let L be a propositional logic language. ◮ Given a logic program P together with immediate consequence operator T P . ◮ Let |R P | = m and 2 R P be the set of interpretations for P . ◮ Find a mapping rep : 2 R P → R m . ◮ Construct a feed-forward network computing f P : R m → R m , called the core, such that the following holds: P ( I ) = J then f P ( rep ( I )) = rep ( J ) , where I , J ∈ 2 R P . ⊲ If T P ( rep − 1 ( s )) = rep − 1 ( t ) , where s , t ∈ R m . ⊲ If f P ( s ) = t then T ◮ Connect the units in the output layer recursively to the units in the input layer. ◮ Show that the following holds ⊲ I = lfp T P iff the recurrent network converges to rep ( I ) , i.e. it reaches a stable state with input and output layer representing rep ( I ) . ◮ Connectionist model generation using recurrent networks with feed-forward core. Steffen H¨ olldobler The Core Method 6

3-Layer Recurrent Networks core 1 . . . . . . . . . 1 input layer hidden layer output layer ◮ At each point in time all units do: ⊲ apply activation function to obtain potential, ⊲ apply output function to obtain output. Steffen H¨ olldobler The Core Method 7

Propositional CORE Method using Binary Threshold Units ◮ Let L be the language of propositional logic. ◮ Let P be a propositional logic program, e.g., P = { p , r ← p ∧ ∼ q , r ← ∼ p ∧ q } . ◮ T P ( I ) = { A | A ← L 1 ∧ . . . ∧ L m ∈ P such that I | = L 1 ∧ . . . ∧ L m } . T P ( ∅ ) = { p } P ( { p } ) { p , r } T = P ( { p , r } ) { p , r } T = = lfp T P Steffen H¨ olldobler The Core Method 8

Representing Interpretations ◮ 2 R P ◮ Let m = |R P | and identify R P with { 1 , . . . , m } . ◮ Define rep : 2 R P → R m such that for all 1 ≤ j ≤ m we find: � 1 if j ∈ I , rep ( I )[ j ] = 0 if j �∈ I . E.g., if R P = { p , q , r } = { 1 , 2 , 3 } and I = { p , r } then rep ( I ) = ( 1 , 0 , 1 ) . ◮ Other encodings are possible, e.g., � 1 if j ∈ I , rep ′ ( I )[ j ] = − 1 if j �∈ I . ◮ We can represent interpretations by arrays of binary or bipolar threshold units. Steffen H¨ olldobler The Core Method 9

Computing the Core ◮ Theorem For each program P , there exists a core of logical threshold units computing T P . ◮ Proof Let P be a program, m = |R P | , and ω ∈ R + . Wlog we assume that all occurrences of ⊤ in P have been eliminated. ⊲ Translation Algorithm 1 Input and output layer: vector of length m of binary threshold units with threshold 0 . 5 and ω/ 2 in the input and output layer, respectively. 2 For each clause of the form A ← L 1 ∧ . . . ∧ L k ∈ P , k ≥ 0, do: 2.1 Add a binary threshold unit u h to the hidden layer. 2.2 Connect u h to the unit representing A in the output layer with weight ω . 2.3 For each literal L j , 1 ≤ j ≤ k , connect the unit representing L j in the input layer to u h and, if L j is an atom then set the weight to ω otherwise set the weight to − ω . 2.4 Set the threshold of u h to ( p − 0 . 5 ) ω , where p is the number of positive literals occurring in L 1 ∧ . . . ∧ L k . Steffen H¨ olldobler The Core Method 10

Computing the Core (Continued) ◮ Theorem For each program P , there exists a core of logical threshold units computing T P . ◮ Proof (Continued) Some observations: ⊲ u h becomes active at time t + 1 iff L 1 ∧ . . . ∧ L k is mapped to ⊤ by the interpretation represented by the state of the input layer at time t . ⊲ The unit representing A in the output layer becomes active at time t + 2 iff there is a rule of the form A ← L 1 ∧ . . . ∧ L k ∈ P and the unit u h in the hidden layer corresponding to this rule is active at time t + 1. ⊲ The result follows immediately from these observations. qed Steffen H¨ olldobler The Core Method 11

Computing the Core (Example) ◮ Consider again P = { p , r ← p ∧ ∼ q , r ← ∼ p ∧ q } . ◮ The translation algorithm yields: ω - ω ω 1 p p 2 2 2 ω − ω 1 ω ω q q 2 2 2 − ω ω ω ω 1 ω ω r r 2 2 2 Steffen H¨ olldobler The Core Method 12

Hidden Layers are Needed ◮ The XOR can be represented by the program { r ← p ∧ ∼ q , r ← ∼ p ∧ q } . ◮ Proposition 2-layer networks cannot compute T P for definite P . ◮ Proof Suppose there exist 2-layer networks for definite P computing T P . ⊲ Consider P = { p ← q , p ← r ∧ s , p ← t ∧ u } . p 1 θ 7 7 p . 5 w 72 ω q 2 8 q . 5 2 ω . 5 r 3 9 r 2 ω s 4 . 5 10 s w 76 2 ω t 5 . 5 11 t 2 ω u 6 . 5 12 u 2 ⊲ Let v be the state of the input layer; v is an interpretation. Steffen H¨ olldobler The Core Method 13

Hidden Layers are Needed (Continued) ◮ Proposition 2-layer networks cannot compute T P for definite P . ◮ Proof (Continued) Consider P = { p ← q , p ← r ∧ s , p ← t ∧ u } . ⊲ We have to find θ 7 and w 7 j , 2 ≤ j ≤ 6, such that p ∈ T P ( v ) iff w 72 v 2 + w 73 v 3 + w 74 v 4 + w 75 v 5 + w 76 v 6 ≥ θ 7 . ⊲ Because conjunction is commutative we find p ∈ T P ( v ) iff w 72 v 2 + w 74 v 3 + w 73 v 4 + w 76 v 5 + w 75 v 6 ≥ θ 7 . ⊲ Consequently, p ∈ T P ( v ) iff w 72 v 2 + w 1 ( v 3 + v 4 ) + w 2 ( v 5 + v 6 ) ≥ θ 7 , where w 1 = 1 2 ( w 73 + w 74 ) and w 2 = 1 2 ( w 75 + w 76 ) . Steffen H¨ olldobler The Core Method 14

Hidden Layers are Needed (Continued) ◮ Proposition 2-layer networks cannot compute T P for definite P . ◮ Proof (Continued) Consider P = { p ← q , p ← r ∧ s , p ← t ∧ u } . ⊲ Likewise, because disjunction is commutative we find p ∈ T P ( v ) iff w · x ≥ θ 7 , 3 ( w 72 + w 1 + w 2 ) and x = � 7 where w = 1 j = 2 v i . ⊲ For the network to compute T P the following must hold: ◮ If x = 0 ( v 2 = . . . = v 6 = 0) then w · x − θ 7 < 0. ◮ ◮ If x = 1 ( v 2 = 1 , v 3 = . . . = v 6 = 0) then w · x − θ 7 ≥ 0. ◮ ◮ If x = 2 ( v 2 = v 4 = v 6 = 0 , v 3 = v 5 = 1) then w · x − θ 7 < 0. ◮ ⊲ However, d ( w · x − θ 7 ) = w cannot change its sign; contradiction. dx ⊲ Consequently, 2-layer feed-forward networks cannot compute T qed P Steffen H¨ olldobler The Core Method 15

Adding Recurrent Connections ◮ Recall P = { p , r ← p ∧ ∼ q , r ← ∼ p ∧ q } . - ω - ω ω ω . 5 . 5 2 2 2 2 ω ω ω . 5 2 2 2 ω ω ω . 5 . 5 2 2 2 Steffen H¨ olldobler The Core Method 16