machine learning classification over encrypted data
play

Machine Learning Classification over Encrypted Data Raphael Bost, - PowerPoint PPT Presentation

Jan 20, 2024 •746 likes •1.29k views

Machine Learning Classification over Encrypted Data Raphael Bost, Raluca Ada Popa, Stephen Tu, Shafi Goldwasser Classification (Machine Learning) Supervised learning (training) Classification data classification training model

  1. Machine Learning Classification over Encrypted Data Raphael Bost, Raluca Ada Popa, Stephen Tu, Shafi Goldwasser

  2. Classification (Machine Learning) • Supervised learning (training) • Classification data classification training model data set phase phase prediction server client

  3. Secure Classification • The provider’s model is sensible financial model, genetic sequences, … • Client’s private data medical records, credit history, …

  4. Secure Classification • The provider’s model is sensible financial model, genetic sequences, … • Client’s private data medical records, credit history, … MPC / 2PC

  5. Using General 2PC ? + Works for every circuit + Constant number of interactions - Have to build circuits - Hard to ‘compose’ - Not easily reusable ➡ Ad Hoc protocols

  6. Scope of our work • Secure classification, no learning the model is already known • Differential privacy is out of scope can be treated separately • Classifiers as specialized 2PC, but not a specialized classifier

  7. Approach • Security model: passive (honest-but-curious) adversary • Identify and construct reusable building blocks • Practical performance as a primary goal • Choose the best fitted primitives Homomorphic Encryption, FHE, Garbled Circuits, …

  8. Building Blocks • Dot product • Encrypted Comparison • Encrypted (arg)max • Decision trees • Encryption scheme switching

  9. Argmax • Alice ( J a 1 K , . . . , J a n K , PK ) • Bob SK • The comparison pattern must not depend on the values

  10. Argmax • Alice ( J a 1 K , . . . , J a n K , PK ) • Bob SK • The comparison pattern must not depend on the values • Compare everything

  11. Argmax • Alice ( J a 1 K , . . . , J a n K , PK ) • Bob SK • The comparison pattern must not depend on the values • Compare everything ⇒ O ( n 2 )

  12. Argmax • Alice ( J a 1 K , . . . , J a n K , PK ) • Bob SK • The comparison pattern must not depend on the values • Compare everything ⇒ O ( n 2 )

  13. Argmax • Alice ( J a 1 K , . . . , J a n K , PK ) • Bob SK • The comparison pattern must not depend on the values • Compare everything ⇒ O ( n 2 ) • ‘Classical’ algorithm

  14. Argmax • Alice ( J a 1 K , . . . , J a n K , PK ) • Bob SK • The comparison pattern must not depend on the values • Compare everything ⇒ O ( n 2 ) • ‘Classical’ algorithm ⇒ O ( n )

  15. Compare & Swap Alice SK Bob ( PK , J v K , J w K ) J max( v, w ) K ( v < w )

  16. Compare & Swap Alice SK Bob ( PK , J v K , J w K ) Compare ( v < w ) ∅ J max( v, w ) K ( v < w )

  17. Compare & Swap Alice SK Bob ( PK , J v K , J w K ) Compare ( v < w ) ∅ Swap ∅ J max( v, w ) K ( v < w )

  18. Compare & Swap Alice SK Bob ( PK , J v K , J w K ) EncCompare b = ( v < w ) J max( v, w ) K ( v < w )

  19. Compare & Swap Alice SK Bob ( PK , J v K , J w K ) EncCompare b = ( v < w ) ( r, s ) ← M 2 J v 0 K = J v + r K J w 0 K = J w + s K J max( v, w ) K ( v < w )

  20. Compare & Swap Alice SK Bob ( PK , J v K , J w K ) EncCompare b = ( v < w ) ( r, s ) ← M 2 J v 0 K , J w 0 K J v 0 K = J v + r K J w 0 K = J w + s K J max( v, w ) K ( v < w )

  21. Compare & Swap Alice SK Bob ( PK , J v K , J w K ) EncCompare b = ( v < w ) ( r, s ) ← M 2 J v 0 K , J w 0 K J v 0 K = J v + r K J w 0 K = J w + s K ( J w 0 K if b J m 0 K ← J v 0 K o/w. J max( v, w ) K ( v < w )

  22. Compare & Swap Alice SK Bob ( PK , J v K , J w K ) EncCompare b = ( v < w ) ( r, s ) ← M 2 J v 0 K , J w 0 K J v 0 K = J v + r K J w 0 K = J w + s K ( J w 0 K if b J m 0 K ← J v 0 K o/w. ( J b K , J m 0 K ) J max( v, w ) K ( v < w )

  23. Compare & Swap Alice SK Bob ( PK , J v K , J w K ) EncCompare b = ( v < w ) ( r, s ) ← M 2 J v 0 K , J w 0 K J v 0 K = J v + r K J w 0 K = J w + s K ( J w 0 K if b J m 0 K ← J m K ← J m 0 K · ( g � 1 · J b K ) r J v 0 K o/w. ( J b K , J m 0 K ) · J b K � s J max( v, w ) K ( v < w )

  24. Compare & Swap Alice SK Bob ( PK , J v K , J w K ) EncCompare b = ( v < w ) ( r, s ) ← M 2 J v 0 K , J w 0 K J v 0 K = J v + r K J w 0 K = J w + s K ( J w 0 K if b J m 0 K ← J m K ← J m 0 K · ( g � 1 · J b K ) r J v 0 K o/w. ( J b K , J m 0 K ) · J b K � s J m K ← J m 0 − ¯ b.r − b.s K J max( v, w ) K ( v < w )

  25. Argmax • Protocol : n-1 Compare & Swap Alice Bob J m K ← J a 1 K

  26. Argmax • Protocol : n-1 Compare & Swap Alice Bob J m K ← J a 1 K C & S J m K ← J max( m, a 2 ) K ( m < a 2 )

  27. Argmax • Protocol : n-1 Compare & Swap Alice Bob J m K ← J a 1 K C & S J m K ← J max( m, a 2 ) K ( m < a 2 ) C & S J m K ← J max( m, a i ) K ( m < a i )

  28. Argmax • Protocol : n-1 Compare & Swap Alice Bob J m K ← J a 1 K C & S J m K ← J max( m, a 2 ) K ( m < a 2 ) C & S J m K ← J max( m, a i ) K ( m < a i ) C & S J m K ← J max( m, a n ) K ( m < a n )

  29. Argmax • Protocol : n-1 Compare & Swap Alice Bob J m K ← J a 1 K C & S ( m < a 2 ) J m K ← J max( a 1 , a 2 ) K s { C & S ( m < a i ) J m K ← max j ∈ [1 ,i ] a j s { C & S ( m < a n ) J m K ← j ∈ [1 ,n ] a j max

  30. Argmax • Protocol : n-1 Compare & Swap Alice Bob J m K ← J a 1 K C & S ( a 1 < a 2 ) J m K ← J max( a 1 , a 2 ) K s { C & S J m K ← ( m < a i ) ⇒ argmax a j max j ∈ [1 ,i ] a j j ∈ [1 ,i ] s { C & S J m K ← ( m < a n ) ⇒ argmax a j j ∈ [1 ,n ] a j max j ∈ [1 ,n ]

  31. Argmax • Protocol : n-1 Compare & Swap Alice Bob J m K ← J a π (1) K C & S ( a π (1) < a π (1) ) J m K ← J max( a π (1) , a π (2) ) K s { ( m < a π ( i ) ) C & S J m K ← j ∈ [1 ,i ] a π ( j ) max ⇒ argmax a π ( j ) j ∈ [1 ,i ] s { C & S ( m < a π ( n ) ) J m K ← j ∈ [1 ,n ] a π ( j ) max ⇒ argmax a π ( j ) j ∈ [1 ,n ] π (argmax a j ) max a j

  32. Argmax • Protocol : n-1 Compare & Swap

  33. Argmax • Protocol : n-1 Compare & Swap sequentially

  34. Argmax • Protocol : n-1 Compare & Swap sequentially or in parallel

  35. Argmax • Protocol : n-1 Compare & Swap sequentially or in parallel 7000 Party A Party B 6000 Communication Tree 5000 Time (ms) 4000 3000 2000 1000 0 4 5 6 7 8 9 1 1 1 1 1 1 1 1 1 1 2 2 3 3 5 0 1 2 3 4 5 6 7 8 9 0 5 0 5 0 Elements

  36. Decision Trees y x ≥ x 2 x < x 2 B D y 2 y < y 1 y > y 2 y 1 E D B A C x ≥ x 1 x < x 1 E C A x 1 x 2 x

  37. Decision Trees b 1 0 1 b 2 b 3 0 1 0 1 b 4 c 1 c 2 c 3 0 1 c 4 c 5 P ( b 1 , b 2 , b 3 , b 4 , c 1 , . . . , c 5 ) = b 1 · ( b 3 · ( b 4 · c 5 + (1 − b 4 ) · c 4 ) + (1 − b 3 ) · c 3 ) +(1 − b 1 ) · ( b 2 · c 2 + (1 − b 2 ) · c 1 )

  38. Decision Trees P ( b 1 , b 2 , b 3 , b 4 , c 1 , . . . , c 5 ) = b 1 · ( b 3 · ( b 4 · c 5 + (1 − b 4 ) · c 4 ) + (1 − b 3 ) · c 3 ) +(1 − b 1 ) · ( b 2 · c 2 + (1 − b 2 ) · c 1 ) • Polynomial evaluation Leveled Homomorphic Encryption • Binary Variables ) Efficient LHE • Binary Coefficients ! (SIMD)

  39. Classifiers In Practice • Linear Classifier • Naïve Bayes Classifier • Decision Trees

  40. Linear Classifier • Separate two sets of points • Very common classifier • Dot product + Encrypted compare

  41. Linear Classifier Computation Time / protocol Model Total Comm. Inter. Dot Enc. Size Client Server Product Comp. 30 46.4 ms 43.8 ms 194 ms 9.67 ms 204 ms 35.84 kB 7 47 55.5 ms 43.8 ms 194 ms 23.6 ms 217 ms 40.19 kB 7 Evaluation on UC Irvine ML databases 
 40 ms network latency 
 2,66 GHz Intel Core i7

  42. Naïve Bayes Classifier

  43. Naïve Bayes Classifier • Classification argmax p ( C = c i | X = x ) i ∈ [ k ]

  44. Naïve Bayes Classifier • Classification argmax p ( C = c i | X = x ) i ∈ [ k ] p ( C = c i , X = x ) • Bayes Formula argmax p ( X = x ) i ∈ [ k ]

  45. Naïve Bayes Classifier • Classification argmax p ( C = c i | X = x ) i ∈ [ k ] • Bayes Formula argmax p ( C = c i , X = x ) i ∈ [ k ]

  46. Naïve Bayes Classifier • Classification argmax p ( C = c i | X = x ) i ∈ [ k ] • Bayes Formula argmax p ( C = c i , X = x ) i ∈ [ k ] • Naïve Model argmax p ( C = c i , X 1 = x 1 , . . . , X d = x d ) i ∈ [ k ]

  47. Naïve Bayes Classifier • Classification argmax p ( C = c i | X = x ) i ∈ [ k ] • Bayes Formula argmax p ( C = c i , X = x ) i ∈ [ k ] d Y • Naïve Model argmax p ( C = c i ) p ( X j = x j | C = c i ) i ∈ [ k ] j =1

  48. Naïve Bayes Classifier • Classification argmax p ( C = c i | X = x ) i ∈ [ k ] • Bayes Formula argmax p ( C = c i , X = x ) i ∈ [ k ] d Y • Naïve Model argmax p ( C = c i ) p ( X j = x j | C = c i ) i ∈ [ k ] j =1

  49. Naïve Bayes Classifier • Classification argmax p ( C = c i | X = x ) i ∈ [ k ] • Bayes Formula argmax p ( C = c i , X = x ) i ∈ [ k ] d Y • Naïve Model argmax p ( C = c i ) p ( X j = x j | C = c i ) i ∈ [ k ] j =1 d X argmax log p ( C = c i ) log p ( X j = x j | C = c i ) i ∈ [ k ] j =1

Recommend

Machine Learning Classification over Encrypted Data Raphal Bost Raluca Ada Popa,

Machine Learning Classification over Encrypted Data Raphal Bost Raluca Ada Popa,

Machine Learning Classification over Encrypted Data Raphal Bost Raluca Ada Popa, Universit Rennes 1 ETH Zrich MIT MIT Stephen Tu Shafi Goldwasser MIT MIT Classification (Machine Learning) Supervised learning (training)

1.3k views • 29 slides
Introduction to machine learning COMS 4721 Learning from data Machine learning : the study

Introduction to machine learning COMS 4721 Learning from data Machine learning : the study

Introduction to machine learning COMS 4721 Learning from data Machine learning : the study of computational mechanisms that learn from data in order to make predictions and decisions. Example 1: image classification A birdwatcher

932 views • 14 slides
Decision Trees TJ Machine Learning Club Classification vs. Regression Classification

Decision Trees TJ Machine Learning Club Classification vs. Regression Classification

Decision Trees TJ Machine Learning Club Classification vs. Regression Classification Classifying photos of fruits Determining whether tumor is benign or malignant Regression Predicting COVID-19 cases given demographic data

505 views • 26 slides
Data Mining: Classification Jay Urbain, PhD Credits: Tom Mitchell, Machine Learning, CMU Nazli

Data Mining: Classification Jay Urbain, PhD Credits: Tom Mitchell, Machine Learning, CMU Nazli

Data Mining: Classification Jay Urbain, PhD Credits: Tom Mitchell, Machine Learning, CMU Nazli Goharian, IIT/Georgetown Jiawei Han, Micheline Kamber, and Jian Pei, Data Mining 1 1 2 Classification: Basic Concepts n Classifica(on:

1.39k views • 126 slides
x ? Machine Learning 5/4/20 Tim Althoff, UW CS547: Machine Learning for Big Data,

x ? Machine Learning 5/4/20 Tim Althoff, UW CS547: Machine Learning for Big Data,

? ? x ? Machine Learning 5/4/20 Tim Althoff, UW CS547: Machine Learning for Big Data, http://www.cs.washington.edu/cse547 2 ? ? ? ? Machine Learning ? Node classification 5/4/20 Tim Althoff, UW CS547: Machine Learning for Big Data,

755 views • 53 slides
1: Sentiment Classification Machine Learning and Real-world Data (MLRD) Ann Copestake (based on

1: Sentiment Classification Machine Learning and Real-world Data (MLRD) Ann Copestake (based on

1: Sentiment Classification Machine Learning and Real-world Data (MLRD) Ann Copestake (based on slides created by Simone Teufel) Lent 2018 This course: Machine Learning and Real-world Data (MLRD) Three Topics: Classification: sentiment

337 views • 23 slides
Overview CS 446 What is machine learning? Machine learning : study of computational

Overview CS 446 What is machine learning? Machine learning : study of computational

Overview CS 446 What is machine learning? Machine learning : study of computational mechanisms that learn from data to make predictions and decisions. 1 / 21 Application 1: image classification Birdwatcher takes photos of birds,

1.08k views • 83 slides
Classifiers: Support Vector Machine 1 MACHINE LEARNING What is Classification? Female Adult

Classifiers: Support Vector Machine 1 MACHINE LEARNING What is Classification? Female Adult

MACHINE LEARNING MACHINE LEARNING Classifiers: Support Vector Machine 1 MACHINE LEARNING What is Classification? Female Adult Children Detecting facial attributes He &amp; Zhang, Pattern Recognition, 2011 Sony (Make Believe) Training set

914 views • 80 slides
Introduction to Big Data and Machine Learning Classification Dr. Mihail September 19, 2019 (Dr.

Introduction to Big Data and Machine Learning Classification Dr. Mihail September 19, 2019 (Dr.

Introduction to Big Data and Machine Learning Classification Dr. Mihail September 19, 2019 (Dr. Mihail) Intro Big Data September 19, 2019 1 / 3 Linear models for classification Goal Goal of classification: take an input vector x and assign

511 views • 17 slides
MACHINE LEARNING Slide adapted from learning from data book and course, and Berkeley cs188 by Dan

MACHINE LEARNING Slide adapted from learning from data book and course, and Berkeley cs188 by Dan

MACHINE LEARNING Slide adapted from learning from data book and course, and Berkeley cs188 by Dan Klein, and Pieter Abbeel Machine Learning ?? Learning from data Tasks: Prediction Classification Recognition Focus on

691 views • 36 slides
1 Model-Based Classification Model-Based Classification Model-based approach Build a

1 Model-Based Classification Model-Based Classification Model-based approach Build a

Machine Learning CSE 473: Artificial Intelligence Nave Bayes Up until now: how use a model to make optimal decisions Machine learning: how to acquire a model from data / experience Learning parameters (e.g. probabilities)

628 views • 8 slides
15-388/688 - Practical Data Science: Linear classification J. Zico Kolter Carnegie Mellon

15-388/688 - Practical Data Science: Linear classification J. Zico Kolter Carnegie Mellon

15-388/688 - Practical Data Science: Linear classification J. Zico Kolter Carnegie Mellon University Fall 2019 1 Outline Example: classifying tumors Classification in machine learning Example classification algorithms Libraries for machine

629 views • 27 slides
Generative and discriminative classification techniques Machine Learning and Category

Generative and discriminative classification techniques Machine Learning and Category

Generative and discriminative classification techniques Machine Learning and Category Representation 2014-2015 Jakob Verbeek, November 28, 2014 Course website: http://lear.inrialpes.fr/~verbeek/MLCR.14.15 Classification Given training data

295 views • 28 slides
Traditional Machine Learning: Unsupervised Learning Juhan Nam Traditional Machine Learning

Traditional Machine Learning: Unsupervised Learning Juhan Nam Traditional Machine Learning

GCT634/AI613: Musical Applications of Machine Learning (Fall 2020) Traditional Machine Learning: Unsupervised Learning Juhan Nam Traditional Machine Learning Pipeline in Classification Tasks A set of hand-designed audio features are selected

398 views • 26 slides
2: Naive Bayes Classification Machine Learning and Real-world Data Simone Teufel and Ann

2: Naive Bayes Classification Machine Learning and Real-world Data Simone Teufel and Ann

2: Naive Bayes Classification Machine Learning and Real-world Data Simone Teufel and Ann Copestake Computer Laboratory University of Cambridge Lent 2017 Last session: an algorithmic solution to sentiment detection You built a symbolic

436 views • 16 slides
Multiclass Classification Machine Learning So far: Binary Classification We have seen linear

Multiclass Classification Machine Learning So far: Binary Classification We have seen linear

Multiclass Classification Machine Learning So far: Binary Classification We have seen linear models Learning algorithms for linear models Perceptron, Winnow, Adaboost, SVM We will see more soon: Nave Bayes, Logistic Regression

635 views • 24 slides
Introduction to Machine Learning Classification: Tasks Sonar Learning goals 0.20 Understand

Introduction to Machine Learning Classification: Tasks Sonar Learning goals 0.20 Understand

Introduction to Machine Learning Classification: Tasks Sonar Learning goals 0.20 Understand the main difference between regression and 0.15 classification Response V2 M R 0.10 Know that classification can be binary or multiclass 0.05

510 views • 8 slides
Machine learning strikes from below, a mining application: Material Classification by Drilling

Machine learning strikes from below, a mining application: Material Classification by Drilling

Machine learning strikes from below, a mining application: Material Classification by Drilling Machine Learning 2005 Johan Larsson Papers Material Classification by Drilling Diana LaBelle, John Bares, Illah Nourbakhsh Robotics Institute,

509 views • 29 slides
Partially Encrypted Machine Learning using Functional Encryption eo Ryffel 1,2 Edouard Dufour-Sans

Partially Encrypted Machine Learning using Functional Encryption eo Ryffel 1,2 Edouard Dufour-Sans

Partially Encrypted Machine Learning using Functional Encryption eo Ryffel 1,2 Edouard Dufour-Sans 1 Romain Gay 1,3 Th Francis Bach 2,1 David Pointcheval 1,2 1 Ecole Normale Sup erieure 2 INRIA 3 UC Berkeley August 18, 2019 Table of

418 views • 38 slides
1 Why Study Machine Learning? Why Study Machine Learning? Cognitive Science The Time is Ripe

1 Why Study Machine Learning? Why Study Machine Learning? Cognitive Science The Time is Ripe

What is Learning? Herbert Simon: Learning is any process by which a system improves performance from CS 391L: Machine Learning experience. Introduction What is the task? Classification Problem solving / planning /

603 views • 6 slides
Machine learning and event classification SOTARRIVA ALVAREZ ISAI ROBERTO Advisor Dr. Antonio

Machine learning and event classification SOTARRIVA ALVAREZ ISAI ROBERTO Advisor Dr. Antonio

Machine learning and event classification SOTARRIVA ALVAREZ ISAI ROBERTO Advisor Dr. Antonio Ortiz Velasquez 1 MACHINE LEARNING APPLIED TO EVENT CLASSIFICATION Motivation Jet like Sphere like OR Machine learning (low spherocity) (High

301 views • 14 slides
Generative and discriminative classification techniques Machine Learning and Object Recognition

Generative and discriminative classification techniques Machine Learning and Object Recognition

Generative and discriminative classification techniques Machine Learning and Object Recognition 2015-2016 Jakob Verbeek, December 11, 2015 Course website: http://lear.inrialpes.fr/~verbeek/MLOR.15.16 Classification Given training data

862 views • 61 slides
2. Naive Bayes Classification Machine Learning and Real-world Data (MLRD) Paula Buttery (based

2. Naive Bayes Classification Machine Learning and Real-world Data (MLRD) Paula Buttery (based

2. Naive Bayes Classification Machine Learning and Real-world Data (MLRD) Paula Buttery (based on slides created by Simone Teufel) Lent 2018 Last session: we used a sentiment lexicon for sentiment classification Movie review sentiment

446 views • 15 slides
CPSC 340: Machine Learning and Data Mining Non-Parametric Models Summer 2020 Course Map

CPSC 340: Machine Learning and Data Mining Non-Parametric Models Summer 2020 Course Map

CPSC 340: Machine Learning and Data Mining Non-Parametric Models Summer 2020 Course Map Machine Learning Approaches Supervised Semi-supervised Unsupervised Reinforcement Learning Learning Learning Learning Classification Regression

1.06k views • 57 slides

More recommend