Making Polynomials Robust to Noise Alexander Sherstov U C L A Noise - PowerPoint PPT Presentation

Reduction to homogeneous case p : { − 1 , +1 } n → [ − 1 , +1] ● Decompose into homogeneous parts: p = p 0 + p 1 + · · · + p d ● Robustly approximate each : p i p i ( x + ✏ ) | < c − d k p i k ∞ | p i ( x ) � ˜ deg ˜ p i = O ( d ) ● Set p = ˜ ˜ p 0 + ˜ p 1 + · · · + ˜ p d 19

Reduction to homogeneous case d X | p ( x ) − ˜ p ( x + ✏ ) | ≤ | p i ( x ) − ˜ p i ( x + ✏ ) | i =0 20

Reduction to homogeneous case d X | p ( x ) − ˜ p ( x + ✏ ) | ≤ | p i ( x ) − ˜ p i ( x + ✏ ) | i =0 d X c − d k p i k ∞  i =0 20

20 Reduction to homogeneous case p i ( x + ✏ ) | } | p i ( x ) − ˜ c − d k p i k ∞ ≤ 4 d h X X i =0 i =0 d d p ( x + ✏ ) | ≤  | p ( x ) − ˜

20 Reduction to homogeneous case p i ( x + ✏ ) | } | p i ( x ) − ˜ c − d k p i k ∞ ≤ 4 d h ≤ 2 − Ω ( d ) X X i =0 i =0 d d p ( x + ✏ ) | ≤  | p ( x ) − ˜

20 Reduction to homogeneous case p i ( x + ✏ ) | } | p i ( x ) − ˜ c − d k p i k ∞ ≤ 4 d h ≤ 2 − Ω ( d ) X X i =0 i =0 d d p ( x + ✏ ) | ≤  | p ( x ) − ˜

Reduction to homogeneous case � � d � � X For any p i ( x ) � ≤ 1 x ∈ { − 1 , +1 } n , � � � � � i =0 21

Reduction to homogeneous case � � d � � X For any p i ( x ) � ≤ 1 x ∈ { − 1 , +1 } n , � � � � � i =0 � � d � � X p i ( x ) t i max � ≤ 1 � � ⇒ = � � − 1 ≤ t ≤ 1 � i =0 21

Reduction to homogeneous case � � d � � X For any p i ( x ) � ≤ 1 x ∈ { − 1 , +1 } n , � � � � � i =0 � � d � � X p i ( x ) t i max � ≤ 1 � � ⇒ = � � − 1 ≤ t ≤ 1 � i =0 " d # X = E p i ( x ) i =0 21

Reduction to homogeneous case � � d � � X For any p i ( x ) � ≤ 1 x ∈ { − 1 , +1 } n , � � � � � i =0 � � d � � X p i ( x ) t i max � ≤ 1 � � ⇒ = � � − 1 ≤ t ≤ 1 � i =0 " d # X = E p i ( x ) i =0 are coefficients of a polynomial p 0 ( x ) , . . . , p d ( x ) ⇒ = [ � 1 , +1] 7! [ � 1 , +1] 21

Our solution ✔ 1. Robust approximation of a monomial n Y p ( x ) = x i i =1 2. Robust approximation of homogeneous p X Y p ( x ) = a S x i i ∈ S | S | = d ✔ 3. Robust approximation of arbitrary p 22

Our solution ✔ 1. Robust approximation of a monomial n Y p ( x ) = x i i =1 2. Robust approximation of homogeneous p X Y p ( x ) = a S x i i ∈ S | S | = d ✔ 3. Robust approximation of arbitrary p 22

Homogeneous case X Given: s.t. x ∈ { − 1 , +1 } n | p ( x ) | ≤ 1 max p = a S χ S | S | = d 23

Homogeneous case X Given: s.t. x ∈ { − 1 , +1 } n | p ( x ) | ≤ 1 max p = a S χ S | S | = d X Define p = ˜ a S ˜ χ S | S | = d 23

Homogeneous case X Given: s.t. x ∈ { − 1 , +1 } n | p ( x ) | ≤ 1 max p = a S χ S | S | = d X Define p = ˜ a S ˜ χ S | S | = d Seems crazy! X | p ( x ) − ˜ p ( x + ✏ ) | ≤ | a S || � S ( x ) − ˜ � S ( x + ✏ ) | | S | = d 23

Homogeneous case X Given: s.t. x ∈ { − 1 , +1 } n | p ( x ) | ≤ 1 max p = a S χ S | S | = d X Define p = ˜ a S ˜ χ S | S | = d Seems crazy! X | p ( x ) − ˜ p ( x + ✏ ) | ≤ | a S || � S ( x ) − ˜ � S ( x + ✏ ) | | S | = d X | a S | · c − d ≤ | S | = d 23

23 } x ∈ { − 1 , +1 } n | p ( x ) | ≤ 1 � 1 · c − d � S ( x + ✏ ) | h ◆ 1 / 2 Homogeneous case ✓ n d | a S || � S ( x ) − ˜ max ≤ | a S | · c − d s.t. | S | = d | S | = d X X a S χ S χ S ≤ p ( x + ✏ ) | ≤ a S ˜ | S | = d | S | = d X X Seems crazy! p = | p ( x ) − ˜ p = ˜ Define Given:

Homogeneous case X Given: s.t. x ∈ { − 1 , +1 } n | p ( x ) | ≤ 1 max p = a S χ S | S | = d X Define p = ˜ a S ˜ χ S | S | = d Seems crazy! X | p ( x ) − ˜ p ( x + ✏ ) | ≤ | a S || � S ( x ) − ˜ � S ( x + ✏ ) | | S | = d 23

Homogeneous case X Given: s.t. x ∈ { − 1 , +1 } n | p ( x ) | ≤ 1 max p = a S χ S | S | = d X Define p = ˜ a S ˜ χ S | S | = d 24

Homogeneous case X Given: s.t. x ∈ { − 1 , +1 } n | p ( x ) | ≤ 1 max p = a S χ S | S | = d use this directly X Define p = ˜ a S ˜ χ S | S | = d 24

Homogeneous case X Given: s.t. x ∈ { − 1 , +1 } n | p ( x ) | ≤ 1 max p = a S χ S | S | = d use this directly X Define p = ˜ a S ˜ χ S | S | = d z 1 , z 2 , . . . , z i , . . . ∈ [0 , 1] n s.t. Find ∞ X p ( x ) − ˜ p ( x + ✏ ) = ⇠ i p ( z i ) , i =1 ∞ X | ξ i | < 2 − Ω ( d ) i =1 24

Homogeneous case X Given: s.t. x ∈ { − 1 , +1 } n | p ( x ) | ≤ 1 max p = a S χ S | S | = d use this directly X Define p = ˜ a S ˜ χ S | S | = d } z 1 , z 2 , . . . , z i , . . . ∈ [0 , 1] n s.t. Find inverting h ∞ X p ( x ) − ˜ p ( x + ✏ ) = ⇠ i p ( z i ) , infinite matrix i =1 ∞ X | ξ i | < 2 − Ω ( d ) i =1 24

Warmup: Boolean inputs Theorem. φ : { − 1 , +1 } n → R homogeneous of degree d δ : { − 1 , +1 } d → R symmetric Then: � �  d d � � X ˆ d ! k φ k ∞ k ˆ � � max φ ( S ) δ ( x | S ) δ k 1 � � x ∈ { − 1 , +1 } n � � | S | = d � � 25

Warmup: Boolean inputs what we want Theorem. to robustly approximate φ : { − 1 , +1 } n → R homogeneous of degree d δ : { − 1 , +1 } d → R symmetric Then: � �  d d � � X ˆ d ! k φ k ∞ k ˆ � � max φ ( S ) δ ( x | S ) δ k 1 � � x ∈ { − 1 , +1 } n � � | S | = d � � 25

Warmup: Boolean inputs what we want Theorem. to robustly approximate φ : { − 1 , +1 } n → R homogeneous of degree d error for δ : { − 1 , +1 } d → R symmetric a single monomial Then: � �  d d � � X ˆ d ! k φ k ∞ k ˆ � � max φ ( S ) δ ( x | S ) δ k 1 � � x ∈ { − 1 , +1 } n � � | S | = d � � 25

Warmup: Boolean inputs what we want Theorem. to robustly approximate φ : { − 1 , +1 } n → R homogeneous of degree d error for δ : { − 1 , +1 } d → R symmetric a single monomial Then: � �  d d � � X ˆ d ! k φ k ∞ k ˆ � � max φ ( S ) δ ( x | S ) δ k 1 � � x ∈ { − 1 , +1 } n � � | S | = d � � cumulative error 25

Warmup: Boolean inputs what we want Theorem. to robustly approximate φ : { − 1 , +1 } n → R homogeneous of degree d error for δ : { − 1 , +1 } d → R symmetric a single monomial Then: � �  d d � � X ˆ d ! k φ k ∞ k ˆ � � max φ ( S ) δ ( x | S ) δ k 1 � � x ∈ { − 1 , +1 } n � � | S | = d � � } cumulative error independent of n h 25

Warmup: Boolean inputs what we want Theorem. to robustly approximate φ : { − 1 , +1 } n → R homogeneous of degree d error for δ : { − 1 , +1 } d → R symmetric a single monomial Then: � �  d d � � X ˆ d ! k φ k ∞ k ˆ � � max φ ( S ) δ ( x | S ) δ k 1 � � x ∈ { − 1 , +1 } n � � | S | = d � � } cumulative error independent of n h 25

Warmup: Boolean inputs Idea: express error as linear combination of φ ( x ) , x ∈ { − 1 , +1 } n 26

Warmup: Boolean inputs Idea: express error as linear combination of φ ( x ) , x ∈ { − 1 , +1 } n Key: operator A v : R { +1 , − 1 } n → R { − 1 , +1 } n 26

Warmup: Boolean inputs Idea: express error as linear combination of φ ( x ) , x ∈ { − 1 , +1 } n Key: operator A v : R { +1 , − 1 } n → R { − 1 , +1 } n h j -th coordinate } ( A v f )( x ) = + · · · + z d x v d ! z 1 x v 1 + z 2 x v 2 j j j E z 1 z 2 . . . z d f . . . , , . . . d z ∈ { − 1 , +1 } d 26

Warmup: Boolean inputs evaluate on non-Boolean Idea: express error as linear combination of inputs by identifying f φ ( x ) , x ∈ { − 1 , +1 } n with its multilinear extension to R n Key: operator A v : R { +1 , − 1 } n → R { − 1 , +1 } n h j -th coordinate } ( A v f )( x ) = + · · · + z d x v d ! z 1 x v 1 + z 2 x v 2 j j j E z 1 z 2 . . . z d f . . . , , . . . d z ∈ { − 1 , +1 } d 26

Warmup: Boolean inputs ✔ linear 27

Warmup: Boolean inputs ✔ linear ✔ bounded: k A v k ∞→∞ = 1 27

Warmup: Boolean inputs ✔ linear ✔ bounded: k A v k ∞→∞ = 1 ✔ symmetric 27

Warmup: Boolean inputs ✔ linear ✔ bounded: k A v k ∞→∞ = 1 ✔ symmetric ✔ A v χ { 1 , 2 ,...,d } = d ! E χ T d d { 1 , 2 ,...,d } T ∈ ( v 1+ ··· + vd ) 27

Warmup: Boolean inputs A v χ S = d ! ( | S | = d ) E χ T d d S T ∈ ( v 1+ ··· + vd ) 28

Warmup: Boolean inputs A v χ S = d ! ( | S | = d ) E χ T d d S T ∈ ( v 1+ ··· + vd ) ◆ d d ✓ d X χ T ⇒ d ! A 1 k 0 d − k χ S = = k T ∈ ( S k ) 28

Warmup: Boolean inputs X χ T T ∈ ( S k ) ◆ d d ✓ d d ! A 1 k 0 d − k χ S = k 29

Warmup: Boolean inputs d X ˆ X δ ( { 1 , . . . , k } ) χ T k =0 T ∈ ( S k ) d ◆ d d ✓ d X ˆ δ ( { 1 , . . . , k } ) d ! A 1 k 0 d − k χ S = k k =0 30

Warmup: Boolean inputs = δ ( x | S ) d X ˆ X δ ( { 1 , . . . , k } ) χ T k =0 T ∈ ( S k ) d ◆ d d ✓ d X ˆ δ ( { 1 , . . . , k } ) d ! A 1 k 0 d − k χ S = k k =0 30

Warmup: Boolean inputs = δ ( x | S ) d X ˆ X ˆ X φ ( S ) δ ( { 1 , . . . , k } ) χ T | S | = d k =0 T ∈ ( S k ) d ◆ d d ✓ d X ˆ X ˆ φ ( S ) δ ( { 1 , . . . , k } ) d ! A 1 k 0 d − k χ S = k | S | = d k =0 31

Warmup: Boolean inputs = cumulative error = δ ( x | S ) d X ˆ X ˆ X φ ( S ) δ ( { 1 , . . . , k } ) χ T | S | = d k =0 T ∈ ( S k ) d ◆ d d ✓ d X ˆ X ˆ φ ( S ) δ ( { 1 , . . . , k } ) d ! A 1 k 0 d − k χ S = k | S | = d k =0 31

Warmup: Boolean inputs = cumulative error = δ ( x | S ) d X ˆ X ˆ X φ ( S ) δ ( { 1 , . . . , k } ) χ T | S | = d k =0 T ∈ ( S k ) d ◆ d d ✓ d X ˆ δ ( { 1 , . . . , k } ) d ! A 1 k 0 d − k φ = k k =0 32

32 = cumulative error } bounded by 1 Warmup: Boolean inputs d ! A 1 k 0 d − k φ h = δ ( x | S ) ◆ d d χ T ✓ d k k ) X S δ ( { 1 , . . . , k } ) T ∈ ( δ ( { 1 , . . . , k } ) ˆ X k =0 d ˆ X k =0 = d φ ( S ) ˆ | S | = d X

32 = cumulative error } bounded by 1 Warmup: Boolean inputs d ! A 1 k 0 d − k φ h = δ ( x | S ) } ◆ d d χ T ✓ d k δ k 1 k ) X S δ ( { 1 , . . . , k } ) k ˆ T ∈ ( h bounded by δ ( { 1 , . . . , k } ) ˆ X k =0 d ˆ X k =0 = d φ ( S ) ˆ | S | = d X

32 ⇤ = cumulative error } bounded by 1 Warmup: Boolean inputs d ! A 1 k 0 d − k φ h = δ ( x | S ) } ◆ d d χ T ✓ d k δ k 1 k ) X S δ ( { 1 , . . . , k } ) k ˆ T ∈ ( h bounded by δ ( { 1 , . . . , k } ) ˆ X k =0 d ˆ X k =0 = d φ ( S ) ˆ | S | = d X

Just proved: ✔ Theorem. φ : { − 1 , +1 } n → R homogeneous of degree d δ : { − 1 , +1 } d → R symmetric Then: � �  d d � � X ˆ d ! k φ k ∞ k ˆ � � max φ ( S ) δ ( x | S ) δ k 1 � � x ∈ { − 1 , +1 } n � � | S | = d � � 33