with Constant Multiplicative Error Uri Stemmer Ben-Gurion - PowerPoint PPT Presentation

Differentially Private k-Means with Constant Multiplicative Error Uri Stemmer Ben-Gurion University joint work with Haim Kaplan

What is 𝒍 -Means Clustering? Given: Data points 𝑻 = 𝒚 𝟐 , … , 𝒚 𝒐 ∈ ℝ 𝒆 𝒐 and parameter 𝒍 Identify 𝒍 centers 𝑫 = 𝒗 𝟐 , … , 𝒗 𝒍 minimizing 𝐝𝐩𝐭𝐮 𝑫 = 𝐧𝐣𝐨 ℓ 𝒚 𝒋 − 𝒗 ℓ 𝟑 𝒋

What is 𝒍 -Means Clustering? Given: Data points 𝑻 = 𝒚 𝟐 , … , 𝒚 𝒐 ∈ ℝ 𝒆 𝒐 and parameter 𝒍 Identify 𝒍 centers 𝑫 = 𝒗 𝟐 , … , 𝒗 𝒍 minimizing 𝐝𝐩𝐭𝐮 𝑫 = 𝐧𝐣𝐨 ℓ 𝒚 𝒋 − 𝒗 ℓ 𝟑 𝒋 ✓ Probably the most well-studied clustering problem ✓ Tons of applications ✓ Super popular

What is 𝒍 -Means Clustering? Given: Data points 𝑻 = 𝒚 𝟐 , … , 𝒚 𝒐 ∈ ℝ 𝒆 𝒐 and parameter 𝒍 Identify 𝒍 centers 𝑫 = 𝒗 𝟐 , … , 𝒗 𝒍 minimizing 𝐝𝐩𝐭𝐮 𝑫 = 𝐧𝐣𝐨 ℓ 𝒚 𝒋 − 𝒗 ℓ 𝟑 𝒋 What is Differentially Private 𝒍 -Means? [Dwork, McSherry, Nissim, Smith 06] (informal)  Every data point 𝒚 𝒋 represents the (private) information of one individual  Goal: the output (the set of centers) does not reveal information that is specific to any single individual

What is 𝒍 -Means Clustering? Given: Data points 𝑻 = 𝒚 𝟐 , … , 𝒚 𝒐 ∈ ℝ 𝒆 𝒐 and parameter 𝒍 Identify 𝒍 centers 𝑫 = 𝒗 𝟐 , … , 𝒗 𝒍 minimizing 𝐝𝐩𝐭𝐮 𝑫 = 𝐧𝐣𝐨 ℓ 𝒚 𝒋 − 𝒗 ℓ 𝟑 𝒋 What is Differentially Private 𝒍 -Means? [Dwork, McSherry, Nissim, Smith 06] (informal)  Every data point 𝒚 𝒋 represents the (private) information of one individual  Goal: the output (the set of centers) does not reveal information that is specific to any single individual  Requirement: the output distribution is insensitive to any arbitrarily change of a single input point (an algorithm satisfying this requirement is differentially private )

What is 𝒍 -Means Clustering? Given: Data points 𝑻 = 𝒚 𝟐 , … , 𝒚 𝒐 ∈ ℝ 𝒆 𝒐 and parameter 𝒍 Why is that a good privacy definition? Even if an observer knows all other data point but mine, and now she sees the outcome Identify 𝒍 centers 𝑫 = 𝒗 𝟐 , … , 𝒗 𝒍 minimizing 𝐝𝐩𝐭𝐮 𝑫 = 𝐧𝐣𝐨 ℓ 𝒚 𝒋 − 𝒗 ℓ 𝟑 of the computation, then she still cannot learn “anything” on my data point 𝒋 What is Differentially Private 𝒍 -Means? [Dwork, McSherry, Nissim, Smith 06] (informal)  Every data point 𝒚 𝒋 represents the (private) information of one individual  Goal: the output (the set of centers) does not reveal information that is specific to any single individual  Requirement: the output distribution is insensitive to any arbitrarily change of a single input point (an algorithm satisfying this requirement is differentially private )

Differentially Private 𝒍 -Means Clustering Given: Data points 𝑻 = 𝒚 𝟐 , … , 𝒚 𝒐 ∈ ℝ 𝒆 𝒐 and parameter 𝒍 Identify 𝒍 centers 𝑫 = 𝒗 𝟐 , … , 𝒗 𝒍 minimizing 𝐝𝐩𝐭𝐮 𝑫 = 𝐧𝐣𝐨 ℓ 𝒚 𝒋 − 𝒗 ℓ 𝟑 𝒋 Requirement: the output distribution is insensitive to any arbitrarily change of a single input point

Differentially Private 𝒍 -Means Clustering Given: Data points 𝑻 = 𝒚 𝟐 , … , 𝒚 𝒐 ∈ ℝ 𝒆 𝒐 and parameter 𝒍 Identify 𝒍 centers 𝑫 = 𝒗 𝟐 , … , 𝒗 𝒍 minimizing 𝐝𝐩𝐭𝐮 𝑫 = 𝐧𝐣𝐨 ℓ 𝒚 𝒋 − 𝒗 ℓ 𝟑 𝒋 Requirement: the output distribution is insensitive to any arbitrarily change of a single input point Observe: With privacy we must have additive error • Assume 𝒍 = 𝒐 = 𝟒 • OPT’s cost = 0

Differentially Private 𝒍 -Means Clustering Given: Data points 𝑻 = 𝒚 𝟐 , … , 𝒚 𝒐 ∈ ℝ 𝒆 𝒐 and parameter 𝒍 Identify 𝒍 centers 𝑫 = 𝒗 𝟐 , … , 𝒗 𝒍 minimizing 𝐝𝐩𝐭𝐮 𝑫 = 𝐧𝐣𝐨 ℓ 𝒚 𝒋 − 𝒗 ℓ 𝟑 𝒋 Requirement: the output distribution is insensitive to any arbitrarily change of a single input point Observe: With privacy we must have additive error • Assume 𝒍 = 𝒐 = 𝟒 • OPT’s cost = 0 • Move one point • OPT’s cost = 0 𝚳

Differentially Private 𝒍 -Means Clustering Given: Data points 𝑻 = 𝒚 𝟐 , … , 𝒚 𝒐 ∈ ℝ 𝒆 𝒐 and parameter 𝒍 Identify 𝒍 centers 𝑫 = 𝒗 𝟐 , … , 𝒗 𝒍 minimizing 𝐝𝐩𝐭𝐮 𝑫 = 𝐧𝐣𝐨 ℓ 𝒚 𝒋 − 𝒗 ℓ 𝟑 𝒋 Requirement: the output distribution is insensitive to any arbitrarily change of a single input point Observe: With privacy we must have additive error • Assume 𝒍 = 𝒐 = 𝟒 • OPT’s cost = 0 • Move one point • OPT’s cost = 0 𝚳 • Each solution must remain approx. equally likely • On at least one of these inputs our cost is ≈ 𝚳 𝟑

Differentially Private 𝒍 -Means Clustering Given: Data points 𝑻 = 𝒚 𝟐 , … , 𝒚 𝒐 ∈ ℝ 𝒆 𝒐 and parameter 𝒍 Identify 𝒍 centers 𝑫 = 𝒗 𝟐 , … , 𝒗 𝒍 minimizing 𝐝𝐩𝐭𝐮 𝑫 = 𝐧𝐣𝐨 ℓ 𝒚 𝒋 − 𝒗 ℓ 𝟑 𝒋 Requirement: the output distribution is insensitive to any arbitrarily change of a single input point Observe: With privacy we must have additive error • Assume 𝒍 = 𝒐 = 𝟒 • OPT’s cost = 0 • Move one point • OPT’s cost = 0 𝚳 • Each solution must remain approx. equally likely • On at least one of these inputs our cost is ≈ 𝚳 𝟑 ⟹ We assume that input points come from the unit ball

Previous and New Bounds Ref Model Runtime Bounds 𝒍 𝟑 ⋅ 𝒆 𝒐 𝒆 GLMRT’10 𝐏 𝟐 ⋅𝐏𝐐𝐔 + 𝑷 differential privacy 𝒐 𝐪𝐩𝐦𝐳 NCBN’16 𝐏 𝐦𝐩𝐡 𝒍 ⋅𝐏𝐐𝐔 + 𝑷 differential privacy 𝒍 𝟒/𝟑 ⋅ FXZR’17 𝐪𝐩𝐦𝐳 𝑷 𝒍 𝐦𝐩𝐡 𝒐 ⋅𝐏𝐐𝐔 + 𝑷 𝒆 differential privacy 𝒍 𝟑 + 𝒆 𝑷 𝐦𝐩𝐡 𝟒 𝒐 ⋅𝐏𝐐𝐔 + 𝑷 𝐪𝐩𝐦𝐳 BDLMZ’17 differential privacy 𝒍 𝟐.𝟔𝟐 ⋅ 𝒆 𝟏.𝟔𝟐 NS’18 𝐪𝐩𝐦𝐳 𝑷 𝒍 ⋅𝐏𝐐𝐔 + 𝑷 differential privacy 𝒍 𝟐.𝟏𝟐 ⋅ 𝒆 𝟏.𝟔𝟐 + 𝒍 𝟒/𝟑 New 𝐪𝐩𝐦𝐳 𝑷 𝟐 ⋅𝐏𝐐𝐔 + 𝑷 differential privacy