Polynomial threshold functions and Boolean threshold circuits - PowerPoint PPT Presentation

Polynomial threshold functions and Boolean threshold circuits Kristoffer Arnsfelt Hansen 1 Vladimir V. Podolskii 2 1 Aarhus University 2 Steklov Mathematical Institute MFCS 2013 1 / 27

Boolean Threshold Functions Boolean function f : { a , b } n → {− 1 , 1 } . Polynomial threshold gate computing f is a polynomial p ∈ R [ x 1 , . . . , x n ] such that for all x ∈ { a , b } n we have f ( x ) = sign p ( x ) . Complexity measures: The degree of p is the degree of the polynomial. The length of p is the number of its monomials. 2 / 27

Example { a , b } = { 1 , 2 } . p ( x , y ) = 16 − 15 xy + 3 x 2 y 2 . 3 / 27

Example { a , b } = { 1 , 2 } . p ( x , y ) = 16 − 15 xy + 3 x 2 y 2 . x = y = 1 16 − 15 + 3 > 0 x = 2 , y = 1 16 − 30 + 12 < 0 x = y = 2 16 − 60 + 48 > 0 3 / 27

Example { a , b } = { 1 , 2 } . p ( x , y ) = 16 − 15 xy + 3 x 2 y 2 . x = y = 1 16 − 15 + 3 > 0 x = 2 , y = 1 16 − 30 + 12 < 0 x = y = 2 16 − 60 + 48 > 0 p ( x , y ) computes PARITY function: p ( x , y ) > 0 iff x + y is odd. 3 / 27

The domain The most studied cases are { a , b } = { 0 , 1 } and { a , b } = {− 1 , 1 } . In these cases we can assume that deg p ≤ n . Indeed, x 2 = x , if x ∈ { 0 , 1 } and x 2 = 1, if x ∈ {− 1 , 1 } . 4 / 27

The domain The most studied cases are { a , b } = { 0 , 1 } and { a , b } = {− 1 , 1 } . In these cases we can assume that deg p ≤ n . Indeed, x 2 = x , if x ∈ { 0 , 1 } and x 2 = 1, if x ∈ {− 1 , 1 } . For general { a , b } this is not the case, in principle degree greater than n can help to reduce the length. 4 / 27

Degree vs. length Indeed, large degree can help. Theorem (Basu et. al, 2004) PARITY over { 1 , 2 } requires length 2 n when the degree is bounded by n, but is computable by degree n 2 and length n + 1 threshold gate. Our example: p ( x , y ) = 16 − 15 xy + 3 x 2 y 2 n = 2, length is n + 1 = 3 degree is n 2 = 4. 5 / 27

The PTF complexity class Given l ( n ) and d ( n ), we denote by PTF a , b ( l ( n ) , d ( n )) the class of Boolean functions over { a , b } n computable by polynomial threshold functions of length l ( n ) and degree d ( n ). PTF a , b ( l ( n ) , ∞ ) — no bound on the degree. PTF a , b ( d ( n )) = PTF a , b (poly( n ) , d ( n )). 6 / 27

The PTF complexity class Given l ( n ) and d ( n ), we denote by PTF a , b ( l ( n ) , d ( n )) the class of Boolean functions over { a , b } n computable by polynomial threshold functions of length l ( n ) and degree d ( n ). PTF a , b ( l ( n ) , ∞ ) — no bound on the degree. PTF a , b ( d ( n )) = PTF a , b (poly( n ) , d ( n )). Below we concentrate on { 1 , 2 } -domain. Our results also hold for all { a , b } -domains, which are essentially different from { 0 , 1 } and {− 1 , 1 } . 6 / 27

Circuit Classes Notation We consider classes AND, OR, XOR, AC 0 . THR: f ( x ) = sign( � i w i x i + w 0 ). MAJ: f ( x ) = sign( � i w i x i + w 0 ), where all w i are integers bounded by polynomial in n . Let C 1 and C 2 be two classes of Boolean circuits. By C 1 ◦ C 2 we denote the class of polynomial size circuits consisting of circuit from C 1 with circuits from C 2 as inputs. 7 / 27

Exponential form of PTFs For a variable y ∈ { 1 , 2 } consider x = log 2 y ∈ { 0 , 1 } . Then y = 2 x . For monomials we have y a 1 n = 2 a 1 x 1 + ... + a n x n 1 . . . y a n and for polynomials l n l l � n � n y a ij i =1 a ij x i = � � � � i =1 a ij x i +log 2 c j P ( y ) = = c j 2 2 c j i j =1 i =1 j =1 j =1 8 / 27

Initial results Lemma PTF 1 , 2 (2 , ∞ ) = THR and PTF 1 , 2 (2 , poly ( n )) = MAJ . 9 / 27

Initial results Lemma PTF 1 , 2 (2 , ∞ ) = THR and PTF 1 , 2 (2 , poly ( n )) = MAJ . Proof. Consider THR gate: � n i =1 w i x i − w 0 ≥ 0. Raise each side to the power of 2. In the other direction, consider � n � n i =1 a i x i + c 2 2 i =1 b i x i ≥ 0 . c 1 2 Interesting case: sign c 1 � = sign c 2 . Move one summand to the other side and take a logarithm. 9 / 27

Bounded degree PTFs Theorem PTF 1 , 2 ( poly ( n )) = THR ◦ MAJ 10 / 27

Bounded degree PTFs Theorem PTF 1 , 2 ( poly ( n )) = THR ◦ MAJ Note that PTF 0 , 1 (poly( n )) = THR ◦ AND and PTF − 1 , 1 (poly( n )) = THR ◦ XOR . Thus, threshold gates over { 1 , 2 } are strictly stronger. 10 / 27

Depth 2 Threshold Circuits THR ◦ THR ? THR ◦ MAJ Goldman et al., 92 MAJ ◦ MAJ Theorem (Goldman, H˚ astad, Razborov, 92) MAJ ◦ THR = MAJ ◦ MAJ . 11 / 27

Bounded degree PTFs Theorem (restated) PTF 1 , 2 ( poly ( n )) = THR ◦ MAJ Main observation: linear form in each MAJ gate can obtain only polynomially many values. We can precisely compute each MAJ gate by polynomial length { 1 , 2 } -polynomial. 12 / 27

Byproduct Lemma Any polynomial size circuit in THR ◦ MAJ is equivalent to a polynomial size circuit of the same form such that all majority gates on the bottom level are monotone. The same is true for MAJ ◦ MAJ. 13 / 27

Lower bounds Let x , y ∈ { 0 , 1 } n . Inner product function: � IP ( x , y ) = x i ∧ y i . i 14 / 27

Lower bounds Let x , y ∈ { 0 , 1 } n . Inner product function: � IP ( x , y ) = x i ∧ y i . i Theorem (restated) PTF 1 , 2 ( poly ( n )) = THR ◦ MAJ Corollary IP / ∈ PTF 1 , 2 ( poly ( n )) , AND ◦ OR ◦ AND 2 / ∈ PTF 1 , 2 ( poly ( n )) . 14 / 27

Lower bounds Let x , y ∈ { 0 , 1 } n . Inner product function: � IP ( x , y ) = x i ∧ y i . i Theorem (restated) PTF 1 , 2 ( poly ( n )) = THR ◦ MAJ Corollary IP / ∈ PTF 1 , 2 ( poly ( n )) , AND ◦ OR ◦ AND 2 / ∈ PTF 1 , 2 ( poly ( n )) . What about PTF 1 , 2 ( ∞ )? 14 / 27

Sign rank Let A = ( a ij ) be a real matrix with nonzero elements. Sign rank of A is the minimal rank of the real matrix B = ( b ij ) such that sign b ij = sign a ij for all i , j . For the Boolean function f ( x , y ) consider the matrix M f = ( f ( x , y )) x , y of size 2 n × 2 n . The sign rank of f ( x , y ) is the sign rank of M f . Theorem (Forster, 2002) The sign rank of IP ( x , y ) is 2 Ω( n ) . Theorem (Razborov, Sherstov, 2010) The sign rank of AND ◦ OR ◦ AND 2 is 2 Ω( n 1 / 3 ) . From this: IP and AND ◦ OR ◦ AND 2 require exponential size THR ◦ MAJ circuits. Why: MAJ gates compute low rank matrices. Rank is subadditive. 15 / 27

Lower bounds for PTF 1 , 2 ( ∞ ) Lemma Assume f : { 0 , 1 } n × { 0 , 1 } n → {− 1 , 1 } is computed by a PTF of length s on the domain { 1 , 2 } n × { 1 , 2 } n . Then the matrix M f has sign rank at most s. Proof. Consider one monomial �� � �� � � x a i i y b i x a i y b i = · . i i i i i i It defines rank 1 matrix. 16 / 27

Lower bounds for PTF 1 , 2 ( ∞ ) Lemma Assume f : { 0 , 1 } n × { 0 , 1 } n → {− 1 , 1 } is computed by a PTF of length s on the domain { 1 , 2 } n × { 1 , 2 } n . Then the matrix M f has sign rank at most s. Proof. Consider one monomial �� � �� � � x a i i y b i x a i y b i = · . i i i i i i It defines rank 1 matrix. Corollary Any PTF on the domain { 1 , 2 } n × { 1 , 2 } n computing IP 2 requires length 2 Ω( n ) . Any PTF on the domain { 1 , 2 } n × { 1 , 2 } n computing AND ◦ OR ◦ AND 2 requires length 2 Ω( n 1 / 3 ) . 16 / 27

Bounded Weight vs. Unbounded Weight Is it true that PTF 1 , 2 (poly( n )) = PTF 1 , 2 ( ∞ )? 17 / 27

Bounded Weight vs. Unbounded Weight Is it true that PTF 1 , 2 (poly( n )) = PTF 1 , 2 ( ∞ )? Open problem! 17 / 27

Bounded Weight vs. Unbounded Weight Is it true that PTF 1 , 2 (poly( n )) = PTF 1 , 2 ( ∞ )? Open problem! Theorem If THR ◦ THR � THR ◦ MAJ ◦ AND 2 then PTF 1 , 2 ( ∞ ) � PTF 1 , 2 ( poly ( n )) . 17 / 27

Bounded Weight vs. Unbounded Weight Is it true that PTF 1 , 2 (poly( n )) = PTF 1 , 2 ( ∞ )? Open problem! Theorem If THR ◦ THR � THR ◦ MAJ ◦ AND 2 then PTF 1 , 2 ( ∞ ) � PTF 1 , 2 ( poly ( n )) . To prove this we need the following lemma. Lemma THR ◦ THR ⊆ PTF 1 , 2 ( ∞ ) ◦ AND 2 . 17 / 27

Proof of the lemma Lemma (restated) THR ◦ THR ⊆ PTF 1 , 2 ( ∞ ) ◦ AND 2 . 18 / 27

Proof of the lemma Lemma (restated) THR ◦ THR ⊆ PTF 1 , 2 ( ∞ ) ◦ AND 2 . Proof of the lemma. Definition. ETHR: f ( x ) = 1 iff � i w i x i + w 0 = 0. It is known that THR ◦ THR = THR ◦ ETHR (Hansen, P., 2010). 18 / 27

Proof of the lemma Lemma (restated) THR ◦ THR ⊆ PTF 1 , 2 ( ∞ ) ◦ AND 2 . Proof of the lemma. Definition. ETHR: f ( x ) = 1 iff � i w i x i + w 0 = 0. It is known that THR ◦ THR = THR ◦ ETHR (Hansen, P., 2010). Note that ETHR -gate defined by L ( x ) = 0 can be approximated by 2 − c · L ( x ) 2 , where c is positive constant. Thus we can rewrite THR ◦ ETHR in the form �� � 2 − c · L i ( x ) 2 sign , i where L i ( x ) are linear forms. Opening the brackets in the exponent we get the circuit of the form PTF 1 , 2 ( ∞ ) ◦ AND 2 . 18 / 27

Bounded Weight vs. Unbounded Weight Theorem (restated) If THR ◦ THR � THR ◦ MAJ ◦ AND 2 then PTF 1 , 2 ( ∞ ) � PTF 1 , 2 ( poly ( n )) . Proof. Assume PTF 1 , 2 (poly( n )) = PTF 1 , 2 ( ∞ ). Then THR ◦ THR ⊆ PTF 1 , 2 ( ∞ ) ◦ AND 2 = PTF 1 , 2 (poly( n )) ◦ AND 2 = THR ◦ MAJ ◦ AND 2 , Note that THR ◦ THR ⊆ THR ◦ MAJ ◦ AND 2 implies THR ◦ THR ◦ AND = THR ◦ MAJ ◦ AND. 19 / 27