LightDP: Towards Automating Differential Privacy Proofs Danfeng - PowerPoint PPT Presentation

LightDP: Towards Automating Differential Privacy Proofs Danfeng Zhang Daniel Kifer Penn State University

Database w/o Database w/ Alice’s data Alice’s data 𝜈 " 𝜈 $ Alice’s data remain private if 𝜈 " , 𝜈 $ are close 2

(Pure) Differential Privacy 𝜈 $ (𝑤) 𝜈 " (𝑤) 𝜈 " 𝜈 $ If for any adjacent databases and value 𝑤 , 𝜈 " (𝑤)/𝜈 $ (𝑤) ≤ 𝑓 + for some constant 𝜗 , then a computation is 𝜗 -private 3 Privacy Cost

Motivation DP has seen explosive growth since 2006 • U.S. Census Bureau LEHD OnTheMap tool [Machanavajjhala et al. 2008] • Google Chrome Browser [Erlingsson et al. 2014] • Apple’s new data collection efforts [Greenberg 2016] But also accompanied with flawed (paper-and- pencil) proofs • e.g., ones categorized in [Chen&Machanavajjhala’15, Lyu et al.’16] Rigorous methods are needed for differential privacy proofs 4

Related Work DP programming platforms (e.g., PINQ, Airavat) • Use (instead of verify) basic DP mechanisms • Cannot offer tight bounds for sophisticated algorithms Methods based on customized logics • Steep learning curve • Heavy annotation burden LightDP offers a better balance between expressiveness and usability 5

LightDP: Overview Target Program with distinguished variable Source Program Relational, Dependent Type System Main Theorem v + bounded by constant 𝜗 Source program type checks in the target program Source program is 𝜗 -private 6

Source Language: Syntax Random Expression Random (e.g., Laplace dist.) variable 7

Source Language: Semantics Memory: mapping from variables to values Initial memory Adjacent memory Relational Reasoning via Type System Final memory dist. Final memory dist. 8

Relational Types Base Type Distance Example Related Memories Γ 𝑦 : num 6 𝑦: u 𝑦: u Γ(𝑧): num " 𝑧 : v 𝑧 : v+1 e.g., int, real 9

Dependent Types Can be a program variable Example Related Memories Γ 𝑦 : num 6 𝑦: u 𝑦: u Γ(𝑧): num 8 𝑧 : v 𝑧 : v + u 10

Dependent Types Can be a non-prob. expression Example Related Memories Γ 𝑦 : num 6 𝑦: u 𝑦: u 𝑧 : v 𝑧 : 9v + 2, u ≥ 1 Γ(𝑧): num 8>"?$:6 v ,u < 1 Notation 𝑛 " Γ 𝑛 $ if 𝑛 " and 𝑛 $ are related by Γ 11

(for the non-probabilistic subset) Types form an invariant on two related program executions: Γ 𝑛 " If initial memories 𝑛 $ Then after executing a well-typed program, Γ A A 𝑛 $ 𝑛 " final memories Enforced by a type system 12

Type System Expression: e.g., + | − < | > | = | ≤ | ≥ 13

Type System Distance must Command: be identical e.g., Related executions take same branch 14

Relating Two Distributions 𝜈 " Γ 𝜈 $ w.r.t. privacy cost 𝝑 if ∀𝑛. 𝜈 " (𝑛)/𝜈 $ (Γ(𝑛)) ≤ 𝑓 + 𝜈 " 𝜈 $ Laplace dist. w/ mean 0 and a scale factor 𝑠 Program 𝜃 := Lap 𝑠 With no Γ 𝜃 = num 6 cost 15

Relating Two Distributions 𝜈 " Γ 𝜈 $ w.r.t. privacy cost 𝜗 if ∀𝑛. 𝜈 " (𝑛)/𝜈 $ (Γ(𝑛)) ≤ 𝑓 + 𝜈 " 𝜈 $ Laplace dist. w/ mean 0 Observation and a scale factor 𝑠 Program 𝜃 may have an arbitrary distance , 𝜃 := Lap 𝑠 which affects the added cost With cost Γ 𝜃 = num " 𝟐/𝒔 due to dist. property 16

Observation 𝜃 may have an arbitrary distance , which affects the added cost 𝜃 has a polymorphic type Non-deterministic operation target program, explicitly source program tracks added privacy cost Intuitively, target program computes the added cost for one sample from distribution 𝜈 17

In General Target program with Source program distinguished variable Type System source program target program 18

Target Language set x to arbitrary value Verification task in the target language: Proving is bounded by some constant 𝜗 in any execution (in a non-probabilistic program) A safety property. Can be verified using off-the-shelf tools (e.g., Hoare logic, model checking) 19

Putting Together The Sparse Vector Method [Dwork and Roth’14] Source Program • Correctness proof is subtle Incorrect variants categorized in [Chen&Machanavajjhala’15, Lyu et al.’16] • Formally verified very recently [Barthe et al. 2016] with heavy annotation burden 20

Required Types Distance depends on the value of 𝑗th query answer ( 𝑟[𝑗] ) Type Inference Types can be inferred by the inference algorithm of LightDP 21

Target Program 22

Completing the Proof Loop Invariant Main Theorem Source program type checks + bounded by constant 𝜗 Postcondition: = source program is 𝜗 -private 23

More in the Paper Type inference algorithm Searching for proof with minimum cost w/ MaxSMT Formal proof for the main theorem More verified examples (with little manual efforts) 24

Summary A safety property (verified by existing tools) Target Program with Automated by distinguished variable inference engine Source Program Relational, Dependent Type System Decomposing differential privacy into subtasks substantially simplifies language-based proof 25