Advanced Probabilistic Couplings for Differential Privacy Gilles - PowerPoint PPT Presentation

Advanced Probabilistic Couplings for Differential Privacy Gilles Barthe, Noémie Fong, Marco Gaboardi, Benjamin Grégoire, Justin Hsu, Pierre-Yves Strub October 25, 2016 1

A new approach to formulating privacy goals: the risk to one’s privacy, or in general, any type of risk . . . should not substantially increase as a result of participating in a statistical database. This is captured by differential privacy. — Cynthia Dwork 2

Increasing interest In research. . . 3

Increasing interest In research. . . . . . and beyond 3

4

Dwork, McSherry, Nissim, and Smith Let ǫ, δ ≥ 0 be parameters, and suppose there is a binary adjacency relation Adj on D . A randomized algorithm M : D → Distr ( R ) is ( ǫ, δ )-differentially private if for every set of outputs S ⊆ R and every pair of adjacent inputs d 1 , d 2 , we have Pr x ∼ M ( d 1 ) [ x ∈ S ] ≤ exp( ǫ ) · Pr x ∼ M ( d 2 ) [ x ∈ S ] + δ . 5

Dwork, McSherry, Nissim, and Smith Let ǫ, δ ≥ 0 be parameters, and suppose there is a binary adjacency relation Adj on D . A randomized algorithm M : D → Distr ( R ) is ( ǫ, δ )-differentially private if for every set of outputs S ⊆ R and every pair of adjacent inputs d 1 , d 2 , we have Pr x ∼ M ( d 1 ) [ x ∈ S ] ≤ exp( ǫ ) · Pr x ∼ M ( d 2 ) [ x ∈ S ] + δ . How to formally verify? 5

Differential privacy is a: relational property of probabilistic programs. 6

Composition properties Program is ( ǫ + ǫ ′ , δ + δ ′ )-private 7

Composition properties Program is ( ǫ + ǫ ′ , δ + δ ′ )-private Formally Consider randomized algorithms M : D → Distr ( R ) and M : R → D → Distr ( R ′ ). If M is ( ǫ, δ ) -private and for every r ∈ R , M ′ ( r ) is ( ǫ ′ , δ ′ ) -private, then the composition is ( ǫ + ǫ ′ , δ + δ ′ ) -private: ← M ( d ); res ← M ( r , d ); return( res ) r $ $ 7

When privacy follows from composition 8

When privacy follows from composition (Linear types, refinement types, self products, relational Hoare logics, . . . ) 8

When privacy doesn’t follow from composition 9

Complicated privacy proofs — Lyu, Su, Dong 10

Complicated privacy proofs — Lyu, Su, Dong How to verify these proofs? 10

Recent progress (2016) Differential privacy ≈ Approximate couplings 11

Recent progress (2016) Differential privacy ≈ Approximate couplings Approximate couplings ≈ Proofs in the logic apRHL 11

Recent progress (2016) Differential privacy ≈ Approximate couplings Approximate couplings ≈ Proofs in the logic apRHL Only proofs beyond composition for ( ǫ, 0)-privacy 11

Enhance the logic New coupling constructions ⇓ New proof rules ⇓ Richer formal proofs of privacy 12

Our work: formal privacy proofs with: Accuracy-dependent privacy Advanced composition Adaptive inputs 13

Our work: formal privacy proofs with: Accuracy-dependent privacy Advanced composition Adaptive inputs 13

A crash course: the program logic apRHL [BKOZB] Imperative language with random sampling ← L ǫ ( e ) x $ 14

A crash course: the program logic apRHL [BKOZB] Imperative language with random sampling ← L ǫ ( e ) x $ approximate probabilistic Relational Hoare Logic ⊢ { Φ } c 1 ∼ ( ǫ,δ ) c 2 { Ψ } 14

A crash course: the program logic apRHL [BKOZB] Imperative language with random sampling ← L ǫ ( e ) x $ approximate probabilistic Relational Hoare Logic ⊢ { Φ } c 1 ∼ ( ǫ,δ ) c 2 { Ψ } Non-probablistic, relational ( x 1 = x 2 ) 14

A crash course: the program logic apRHL [BKOZB] Imperative language with random sampling ← L ǫ ( e ) x $ approximate probabilistic Relational Hoare Logic ⊢ { Φ } c 1 ∼ ( ǫ,δ ) c 2 { Ψ } Numeric index 14

Approximate couplings [BKOZB, BO] Definition Let R ⊆ A × A be a relation and ǫ, δ ≥ 0. Two distributions µ 1 , µ 2 ∈ Distr ( A ) are related by an ( ǫ, δ )-approximate coupling with support R if there exists µ L , µ R ∈ Distr ( A × A ) with: 15

Approximate couplings [BKOZB, BO] Definition Let R ⊆ A × A be a relation and ǫ, δ ≥ 0. Two distributions µ 1 , µ 2 ∈ Distr ( A ) are related by an ( ǫ, δ )-approximate coupling with support R if there exists µ L , µ R ∈ Distr ( A × A ) with: ◮ support in R ; 15

Approximate couplings [BKOZB, BO] Definition Let R ⊆ A × A be a relation and ǫ, δ ≥ 0. Two distributions µ 1 , µ 2 ∈ Distr ( A ) are related by an ( ǫ, δ )-approximate coupling with support R if there exists µ L , µ R ∈ Distr ( A × A ) with: ◮ support in R ; ◮ π 1 ( µ L ) = µ 1 and π 2 ( µ R ) = µ 2 ; 15

Approximate couplings [BKOZB, BO] Definition Let R ⊆ A × A be a relation and ǫ, δ ≥ 0. Two distributions µ 1 , µ 2 ∈ Distr ( A ) are related by an ( ǫ, δ )-approximate coupling with support R if there exists µ L , µ R ∈ Distr ( A × A ) with: ◮ support in R ; ◮ π 1 ( µ L ) = µ 1 and π 2 ( µ R ) = µ 2 ; ◮ for every S ⊆ A × A , Pr z ∼ µ L [ z ∈ S ] ≤ exp( ǫ ) · Pr z ∼ µ R [ z ∈ S ] + δ 15

Approximate couplings [BKOZB, BO] Definition Let R ⊆ A × A be a relation and ǫ, δ ≥ 0. Two distributions µ 1 , µ 2 ∈ Distr ( A ) are related by an ( ǫ, δ )-approximate coupling with support R if there exists µ L , µ R ∈ Distr ( A × A ) with: ◮ support in R ; ◮ π 1 ( µ L ) = µ 1 and π 2 ( µ R ) = µ 2 ; ◮ for every S ⊆ A × A , Pr z ∼ µ L [ z ∈ S ] ≤ exp( ǫ ) · Pr z ∼ µ R [ z ∈ S ] + δ 15

Approximate couplings [BKOZB, BO] Definition Let R ⊆ A × A be a relation and ǫ, δ ≥ 0. Two distributions µ 1 , µ 2 ∈ Distr ( A ) are related by an ( ǫ, δ )-approximate coupling with support R if there exists µ L , µ R ∈ Distr ( A × A ) with: ◮ support in R ; ◮ π 1 ( µ L ) = µ 1 and π 2 ( µ R ) = µ 2 ; ◮ for every S ⊆ A × A , Pr z ∼ µ L [ z ∈ S ] ≤ exp( ǫ ) · Pr z ∼ µ R [ z ∈ S ] + δ R ♯ Write: µ 1 µ 2 ( ǫ,δ ) 15

Interpreting judgments ⊢ { Φ } c 1 ∼ ( ǫ,δ ) c 2 { Ψ } 16

Interpreting judgments ⊢ { Φ } c 1 ∼ ( ǫ,δ ) c 2 { Ψ } Two memories related by Φ 16

Interpreting judgments ⊢ { Φ } c 1 ∼ ( ǫ,δ ) c 2 { Ψ } Two memories related by Φ ⇓ Two distributions related by Ψ ♯ ( ǫ,δ ) 16

Differential privacy in apRHL ⊢ { Adj ( d 1 , d 2 ) } c ∼ ( ǫ,δ ) c { res 1 = res 2 } 17

Differential privacy in apRHL ⊢ { Adj ( d 1 , d 2 ) } c ∼ ( ǫ,δ ) c { res 1 = res 2 } ( ǫ, δ )-differential privacy 17

Proof rules Proof rule ≈ Recipe to combine couplings 18

Proof rules Proof rule ≈ Recipe to combine couplings Sequence rule ≈ standard composition of privacy ⊢ { Ψ } c ′ 1 ∼ ( ǫ ′ ,δ ′ ) c ′ Seq ⊢ { Φ } c 1 ∼ ( ǫ,δ ) c 2 { Ψ } 2 { Θ } ⊢ { Φ } c 1 ; c ′ 1 ∼ ( ǫ + ǫ ′ ,δ + δ ′ ) c 2 ; c ′ 2 { Θ } 18

Proof rules Proof rule ≈ Recipe to combine couplings Sequence rule ≈ standard composition of privacy ⊢ { Ψ } c ′ 1 ∼ ( ǫ ′ ,δ ′ ) c ′ Seq ⊢ { Φ } c 1 ∼ ( ǫ,δ ) c 2 { Ψ } 2 { Θ } ⊢ { Φ } c 1 ; c ′ 1 ∼ ( ǫ + ǫ ′ ,δ + δ ′ ) c 2 ; c ′ 2 { Θ } 18

Our work: formal privacy proofs with: Accuracy-dependent privacy Advanced composition Adaptive inputs 19

Accuracy-dependent privacy 20

Accuracy-dependent privacy Rough intuition ◮ Think of δ in ( ǫ, δ )-privacy as failure probability ◮ “Algorithm is private except with small probability δ ” ◮ “If the noise added is not too large, then . . . ” Similar to up-to-bad reasoning ◮ Common tool in crypto proofs ◮ “If bad event doesn’t happen, then protocol is safe” 21

In apRHL: up-to-bad rule ⊢ { Φ } c 1 ∼ ( ǫ,δ ) c 2 {¬ Ψ � 1 � → x 1 = x 2 } ]( m 1 ) [Ψ � 1 � ] < δ ′ | = m ∈ Θ = ⇒ Pr [ [ c 1 ] UtB ⊢ { Φ } c 1 ∼ ( ǫ,δ + δ ′ ) c 2 { x 1 = x 2 } 22

In apRHL: up-to-bad rule ⊢ { Φ } c 1 ∼ ( ǫ,δ ) c 2 {¬ Ψ � 1 � → x 1 = x 2 } ]( m 1 ) [Ψ � 1 � ] < δ ′ | = m ∈ Θ = ⇒ Pr [ [ c 1 ] UtB ⊢ { Φ } c 1 ∼ ( ǫ,δ + δ ′ ) c 2 { x 1 = x 2 } Notes ◮ Ψ � 1 � is “bad event”, only mentions c 1 22

In apRHL: up-to-bad rule ⊢ { Φ } c 1 ∼ ( ǫ,δ ) c 2 {¬ Ψ � 1 � → x 1 = x 2 } ]( m 1 ) [Ψ � 1 � ] < δ ′ | = m ∈ Θ = ⇒ Pr [ [ c 1 ] UtB ⊢ { Φ } c 1 ∼ ( ǫ,δ + δ ′ ) c 2 { x 1 = x 2 } Notes ◮ Ψ � 1 � is “bad event”, only mentions c 1 ◮ If bad event doesn’t happen, have privacy 22

In apRHL: up-to-bad rule ⊢ { Φ } c 1 ∼ ( ǫ,δ ) c 2 {¬ Ψ � 1 � → x 1 = x 2 } ]( m 1 ) [Ψ � 1 � ] < δ ′ | = m ∈ Θ = ⇒ Pr [ [ c 1 ] UtB ⊢ { Φ } c 1 ∼ ( ǫ,δ + δ ′ ) c 2 { x 1 = x 2 } Notes ◮ Ψ � 1 � is “bad event”, only mentions c 1 ◮ If bad event doesn’t happen, have privacy ◮ Bound probability of Ψ after c 1 22

23

Advanced composition theorem Compose n mechanisms, each ( ǫ, δ )-private ◮ Standard composition: ( n · ǫ, n · δ )-private ◮ Advanced composition: ( ǫ ∗ , δ ∗ )-private ǫ ∗ ≈ √ n · ǫ δ ∗ ≈ n · δ + δ ′ and 24