Metrics for Differential Privacy in Concurrent Systems Lili Xu 1 , 3 - PowerPoint PPT Presentation

Introduction Two Pseudometrics Non-expansive Process Operators An application to the Dining Cryptographers Protocol Summary Metrics for Differential Privacy in Concurrent Systems Lili Xu 1 , 3 , 4 , Konstantinos Chatzikokolakis 2 , 3 , Huimin Lin 4 1 INRIA 2 CNRS 3 Ecole Polytechnique, Paris, France 4 Institute of Software, Chinese Academy of Sciences, Beijing, China Berlin, Germany June 5th, FORTE 2014 Xu, Chatzikokolakis, Lin Metrics for Differential Privacy in Concurrent Systems

Introduction Two Pseudometrics Non-expansive Process Operators An application to the Dining Cryptographers Protocol Summary Background Sketch Geolocation Privacy Programming Languages Social Networks Probabilistic Process Calculus Strong Anonymity M ������������������� Probable ���������������� Innocence �������������������� �������������� Quantitative Information Flow Differential Privacy A. Narayanan M. Gaboardi This talk A. Machanavajjhala ������ �������� ������� G. Barthe ��������������� Xu, Chatzikokolakis, Lin Metrics for Differential Privacy in Concurrent Systems

Introduction Two Pseudometrics Non-expansive Process Operators An application to the Dining Cryptographers Protocol Summary How To Quantify the Amount of Privacy? Definition (Standard Definition of Differential Privacy) A query mechanism A is ǫ -differentially private if for any two adjacent databases u 1 and u 2 , i.e. which differ only for one individual, and any property Z , the probability distributions of A ( u 1 ) , A ( u 2 ) differ on Z at most by e ǫ , namely, Pr [ A ( u 1 ) ∈ Z ] ≤ e ǫ · Pr [ A ( u 2 ) ∈ Z ] . The lower the value ǫ is, the better the privacy is protected. Some Merits of Differential Privacy Strong notion of privacy. Independence from side knowledge. Robustness to attacks based on combining various sources of information. Looser restrictions between non-adjacent secrets. Xu, Chatzikokolakis, Lin Metrics for Differential Privacy in Concurrent Systems

Introduction Two Pseudometrics Non-expansive Process Operators An application to the Dining Cryptographers Protocol Summary Background Sketch Geolocation Privacy Programming Languages Social Networks Probabilistic Process Calculus Strong Anonymity M ������������������� Probable ���������������� Innocence �������������������� �������������� Quantitative Information Flow Differential Privacy A. Narayanan M. Gaboardi This talk A. Machanavajjhala ������ �������� ������� G. Barthe ��������������� Xu, Chatzikokolakis, Lin Metrics for Differential Privacy in Concurrent Systems

Introduction Two Pseudometrics Non-expansive Process Operators An application to the Dining Cryptographers Protocol Summary Outline Introduction 1 Concurrent Systems Differential Privacy The Verification Framework Two Pseudometrics 2 The Accumulative Bijection Pseudometric The Amortised Bijection Pseudometric Comparison Non-expansive Process Operators 3 A Probabilistic Process calculus: CCS p An application to the Dining Cryptographers Protocol 4 The Dining Cryptographers Protocol Xu, Chatzikokolakis, Lin Metrics for Differential Privacy in Concurrent Systems

Introduction Two Pseudometrics Concurrent Systems Non-expansive Process Operators Differential Privacy An application to the Dining Cryptographers Protocol The Verification Framework Summary Motivation The model: Concurrent systems modeled as probabilistic automata. The measure of the level of privacy: Differential privacy Xu, Chatzikokolakis, Lin Metrics for Differential Privacy in Concurrent Systems

Introduction Two Pseudometrics Concurrent Systems Non-expansive Process Operators Differential Privacy An application to the Dining Cryptographers Protocol The Verification Framework Summary Motivation The model: Concurrent systems modeled as probabilistic automata. The measure of the level of privacy: Differential privacy Goal: To verify differential privacy properties for concurrent systems Xu, Chatzikokolakis, Lin Metrics for Differential Privacy in Concurrent Systems

Introduction Two Pseudometrics Concurrent Systems Non-expansive Process Operators Differential Privacy An application to the Dining Cryptographers Protocol The Verification Framework Summary Outline Introduction 1 Concurrent Systems Differential Privacy The Verification Framework Two Pseudometrics 2 The Accumulative Bijection Pseudometric The Amortised Bijection Pseudometric Comparison Non-expansive Process Operators 3 A Probabilistic Process calculus: CCS p An application to the Dining Cryptographers Protocol 4 The Dining Cryptographers Protocol Xu, Chatzikokolakis, Lin Metrics for Differential Privacy in Concurrent Systems

Introduction Two Pseudometrics Concurrent Systems Non-expansive Process Operators Differential Privacy An application to the Dining Cryptographers Protocol The Verification Framework Summary Our Model A probabilistic automaton is a tuple ( S , s , A , D ) S : a finite set of states; s ∈ S : the start state; A : a finite set of action labels; a D ⊆ S × A × Disc ( S ) : a transition relation. We also write s − → µ . Definition (Concurrent Systems with Secret Information) Let U be a set of secrets. A concurrent system with secret information A is a mapping of secrets to probabilistic automata, where A ( u ) , u ∈ U is the automaton modelling the behavior of the system when running on u . Xu, Chatzikokolakis, Lin Metrics for Differential Privacy in Concurrent Systems

Introduction Two Pseudometrics Concurrent Systems Non-expansive Process Operators Differential Privacy An application to the Dining Cryptographers Protocol The Verification Framework Summary How to Reason about Probabilistic Observations? A scheduler ζ resolves the non-determinism based on the history of a computation, inducing a probability measure over traces. Xu, Chatzikokolakis, Lin Metrics for Differential Privacy in Concurrent Systems

Introduction Two Pseudometrics Concurrent Systems Non-expansive Process Operators Differential Privacy An application to the Dining Cryptographers Protocol The Verification Framework Summary How to Reason about Probabilistic Observations? A scheduler ζ resolves the non-determinism based on the history of a computation, inducing a probability measure over traces. Probabilities of finite traces Let α be the history up to the current state s . The probability of observing a finite trace � t starting from α , denoted by Pr ζ [ α ⊲ � t ] , is defined recursively as follows. if �  1 t is empty,   b ζ [ α ⊲ � if � t = a � � Pr t ] = 0 t ′ , ζ ( α ) = s − → µ and b � = a , t ′ and ζ ( α ) = s a s i µ ( s i ) Pr ζ [ α as i ⊲ � if � t = a � �  t ′ ] � − → µ .  Xu, Chatzikokolakis, Lin Metrics for Differential Privacy in Concurrent Systems

Introduction Two Pseudometrics Concurrent Systems Non-expansive Process Operators Differential Privacy An application to the Dining Cryptographers Protocol The Verification Framework Summary An example: A PIN-Checking System A ( u ) A ( u ) u 1 u 2 A ( u 1 ) A ( u 2 ) a 1 a 2 a 1 a 2 0 . 4 0 . 6 0 . 4 0 . 6 0 . 6 0 . 4 0 . 6 0 . 4 s 1 s 2 s 3 t 1 t 2 t 3 no no no no ok ok Example: The scheduler executes the a 1 -branch. Pr ζ [ A ( u 1 ) ⊲ a 1 ok ] = 0 . 6 Pr ζ [ A ( u 2 ) ⊲ a 1 ok ] = 0 . 4 Pr ζ [ A ( u 1 ) ⊲ a 1 no ] = 0 . 4 Pr ζ [ A ( u 2 ) ⊲ a 1 no ] = 0 . 6 Pr ζ [ A ( u 1 ) ⊲ a 2 ok ] = 0 Pr ζ [ A ( u 2 ) ⊲ a 2 ok ] = 0 Pr ζ [ A ( u 1 ) ⊲ a 2 no ] = 0 Pr ζ [ A ( u 2 ) ⊲ a 2 no ] = 0 Xu, Chatzikokolakis, Lin Metrics for Differential Privacy in Concurrent Systems

Introduction Two Pseudometrics Concurrent Systems Non-expansive Process Operators Differential Privacy An application to the Dining Cryptographers Protocol The Verification Framework Summary Outline Introduction 1 Concurrent Systems Differential Privacy The Verification Framework Two Pseudometrics 2 The Accumulative Bijection Pseudometric The Amortised Bijection Pseudometric Comparison Non-expansive Process Operators 3 A Probabilistic Process calculus: CCS p An application to the Dining Cryptographers Protocol 4 The Dining Cryptographers Protocol Xu, Chatzikokolakis, Lin Metrics for Differential Privacy in Concurrent Systems

Introduction Two Pseudometrics Concurrent Systems Non-expansive Process Operators Differential Privacy An application to the Dining Cryptographers Protocol The Verification Framework Summary Differential Privacy in the Context of Concurrent Systems The scheduler can easily break many security and privacy properties. We consider a restricted class of schedulers, called admissible schedulers. make them unable to distinguish between secrets in the histories. Definition (Differential Privacy in Our Setting) A concurrent system A satisfies ǫ - differential privacy (DP) iff for any two adjacent secrets u , u ′ , any finite trace � t and any admissible scheduler ζ : t ] ≤ e ǫ · Pr ζ [ A ( u ) ⊲ � ζ [ A ( u ′ ) ⊲ � Pr t ] Xu, Chatzikokolakis, Lin Metrics for Differential Privacy in Concurrent Systems