CSC2412: The Projection Mechanism Sasho Nikolov 1 Motivation - PowerPoint PPT Presentation

CSC2412: The Projection Mechanism Sasho Nikolov 1

Motivation

Reminder: Query Release qi :D → do I } , Recall the query release problem: gilt ) - tu j Yik ;) • Workload Q = { q 1 , . . . , q k } of k counting queries 0 q 1 ( X ) 1 . . A 2 [0 , 1] k . Q ( X ) = B C . @ q k ( X ) • Compute, with ( " , � )-DP, some Y 2 R k so that k max i =1 | Y i � q i ( X ) |  ↵ , with probability � 1 � � . 2

Private MW: running time l - way marginal query ( over 2--40 , Bd ) is an specified by e attributes for d values of the and them a data point prescribed values asks has whether the and Recall: PMW answers ` -way marginals on X = { 0 , 1 } d with error ↵ when Both bounds p n � ` d log( d ) ` d log( d ) log(1 / � ) n � at ↵ 3 " ↵ 2 " are t linear f most ( E , f) - DP - DP in ol E What about running time? exponential ol Polynomial in in u t 201 values mistake maintain ↳ bounded learner 3

Gaussian noise: running time Recall: The Gaussian noise mechanism answers ` -way marginals on X = { 0 , 1 } d with error ↵ when Much than It more n � d ` / 2 p ` log( d ) log(1 / � ) with ↵" PM w Olden l ) What about running time? 4

Open problem penumbra queries Find an ( " , � )-DP mechanism running in time polynomial in d ` and n which answers ` -way marginals with error ↵ when p ` d log( d ) log(1 / � ) n � ↵ 2 " ↳ what need you for PNW the Gaussian noise week time I. e get running of we can . the but PM w guarantees of error ? ? 123 e as raw Il I - t : Effingham ÷ " 'm a. well . 5

The Projection Mechanism

⇒ Feasible query answers Yet of all possible What can actual query answers look like? query answers I.e., what is the set { Q ( X ) : X 2 X n } ? set geometrically → e will think of this When n = 1: S Q = { Q ( { x } ) : x 2 X} ✓ { 0 , 1 } k all possible vectors of answers → - a single data point on General n : - I ¥ For any C- Q gift ) , qicxjl qi , = In IZ Q ( f x ; } ) of QCX ) average an → of collection some in Sq points 6

Example sa -71%161,41 's + sedition :B X = { 0 , 1 } 2 , Q = { q 1 , q 2 } . * Yolk :B So q 1 ( x ) = 1 { x 1 = 1 } , q 2 ( x ) = 1 { x 1 = 0 OR x 2 = 1 } . it ! .EE# !! sete:fiae , " it is :÷÷ ! :* tix ,%: :* .ae condition is true a * stoning . - fist 's ah Hitomi 's ) 7

Feasible polytope ° :qo hull quiver K Q = conv ( S Q ) = { � 1 y 1 + . . . + � m y m : y 1 , . . . , y m 2 S Q , � 1 , . . . , � m � 0 , � 1 + . . . + � m = 1 } . Key observation: for any dataset X , Q ( X ) 2 K Q . e tha t toys Htt = th and 'm ) ' ( Itis ) Qlt ) + . . - c- So tf i Qcsxi } ) , 8

↳ Post-processing? • ftp.y-QLH-Z • QH ) Ka • Projection mechanism: • Run the Gaussian noise mechanism to get ˜ k Y = Q ( X ) + Z , Z ⇠ N (0 , ⇢ n 2 I ). - • Project ˜ ee , di Y onto K Q : - DP go , 5- Y = arg min { k y � ˜ ˆ Y k 2 : y 2 K Q } . if bogus ) Output ˆ Y . feasible vector of the closest , output So me eh output of the Gauss to the , Wis answers . 9

Privacy analysis the output B g of post processing is mechanism the of noise Gaussian theorem 5 composition Privacy of t I - Dp , 51 CE that is mean 10

Accuracy analysis? We can show that projection decreases the error on average . mean squared → Root error The average error of a mechanism X is - v k u r t E 1 E 1 u X ( M ( X ) i � q i ( X )) 2 = max k k M ( X ) � Q ( X ) k 2 max 2 . k X ∈ X n X ∈ X n i =1 ↳ we respect to randomness of µ case error guarantees Usually avg error guarantees be turned into worst - can using private boosting Mw ) ( related to The projection mechanism has average error at most ↵ as long as marginals l - wise For - q 197 ¥ 39 p log 1 log |X| · n . rr � n � ↵ 2 · " d ' e ' d E 11

Mean width ÷÷÷÷ii÷÷ Support function : h K Q ( z ) = max { h y , z i : y 2 K Q } = max { h y , z i : y 2 S Q } Mean width : ` ∗ ( K Q ) = E h K Q ( G ), where G ⇠ N (0 , I ). . • Geometric lemma : q log 1 ` ∗ ( K Q ) · δ The Projection Mechanism has average error ↵ if n � . √ ↵ 2 · " · k Hk ) e El 12

Running time Running time bottleneck is solving arg min { k y � ˜ Y k 2 : y 2 K Q } . Su ffi cient (more or less): have an e ffi cient algorithm to decide if y 2 K Q . } { Can we solve this problem if Q = 2-way marginals? • No : would allow solving NP-hard problems like Max-Cut 13

A work-around - { 2- way marginals } For Q - c. hotel A -1 ⇒ l*k)t ① l"k answers / = ion Saving grace: there exists an L so that a. www.iz ) hazy tz - → 1 • 10 L ✓ K Q ✓ L feasible query b- L all contains • can e ffi ciently solve arg min { k y � ˜ Y k 2 : y 2 L } shall get Cau So project onto L when , d error than Ka avg u x trolley rather He 14