bounding user contributions a bias variance trade off in
play

Bounding User Contributions: A Bias-Variance Trade-off in - PowerPoint PPT Presentation

Nov 11, 2023 •519 likes •669 views

Bounding User Contributions: A Bias-Variance Trade-off in Differential Privacy Kareem Amin, Alex Kulesza, Andrs Muoz Medina, Sergei Vassilvitskii Google Research NY Typical DP assumption: Reality: One user = one example Users contribute

  1. Bounding User Contributions: A Bias-Variance Trade-off in Differential Privacy Kareem Amin, Alex Kulesza, Andrés Muñoz Medina, Sergei Vassilvitskii Google Research NY

  2. Typical DP assumption: Reality: One user = one example Users contribute many times

  3. High cap = excessive noise Low cap = biased data We investigate this bias-variance trade-off using tools from learning theory

  4. Setting Infinite collection of users • Distribution P over users • Each user has a unique distribution over examples • I.i.d. data: first sample a user from P , then sample the user’s distribution

  5. Learning • Cap each user at a 𝞾 0 fraction of the dataset • Run a standard differentially private ERM algorithm

  6. <latexit sha1_base64="7uFYgDiClgiAivHmMF4hNngkNgc=">ACw3icbVFba9swFJbdXbrslnaPexELA4dBsEthewdg8AG7WBJC1FmZEWORWTZlY4LQehP9q37NZNzKb0dEPr0ne87ks7JaikMxPFNEO49e/7i5f6rzus3b9+97x4cTkzVaMbHrJKVvsio4VIoPgYBkl/UmtMyk/w8W35v8+dXBtRqT+wqvmspAslcsEoeCrt/iMlhYJRaX+5qEjXJ13aWosr18dE8ktMhMpTW/gNjxy+p+93vuBTL8ohwsRcarCY5Joyu6szodpFo76zBGiTxg7AlosCvBOgkIOef21D1VItmZsLpjI7g13t65U/78exRt1P1badrtxYN4HfgxSLagh7ZxlnavybxiTckVMEmNmSZxDTNLNQgmueuQxvCasiVd8KmHipbczOx6Bg5/9swc5X2SwFes3cdlpbGrMrMK9vemIe5lnwqN20g/zazQtUNcMU2F+WNxFDhdqB4LjRnIFceUKaFfytmBfV9AT/2jm9C8vDLj8HkaJDEg+T3cW94sm3HPvqIPqEIJegrGqIROkNjxIJhkAdVUIc/wmWoQ9hIw2Dr+YDuRej+A0h/2hY=</latexit> <latexit sha1_base64="7uFYgDiClgiAivHmMF4hNngkNgc=">ACw3icbVFba9swFJbdXbrslnaPexELA4dBsEthewdg8AG7WBJC1FmZEWORWTZlY4LQehP9q37NZNzKb0dEPr0ne87ks7JaikMxPFNEO49e/7i5f6rzus3b9+97x4cTkzVaMbHrJKVvsio4VIoPgYBkl/UmtMyk/w8W35v8+dXBtRqT+wqvmspAslcsEoeCrt/iMlhYJRaX+5qEjXJ13aWosr18dE8ktMhMpTW/gNjxy+p+93vuBTL8ohwsRcarCY5Joyu6szodpFo76zBGiTxg7AlosCvBOgkIOef21D1VItmZsLpjI7g13t65U/78exRt1P1badrtxYN4HfgxSLagh7ZxlnavybxiTckVMEmNmSZxDTNLNQgmueuQxvCasiVd8KmHipbczOx6Bg5/9swc5X2SwFes3cdlpbGrMrMK9vemIe5lnwqN20g/zazQtUNcMU2F+WNxFDhdqB4LjRnIFceUKaFfytmBfV9AT/2jm9C8vDLj8HkaJDEg+T3cW94sm3HPvqIPqEIJegrGqIROkNjxIJhkAdVUIc/wmWoQ9hIw2Dr+YDuRej+A0h/2hY=</latexit> <latexit sha1_base64="7uFYgDiClgiAivHmMF4hNngkNgc=">ACw3icbVFba9swFJbdXbrslnaPexELA4dBsEthewdg8AG7WBJC1FmZEWORWTZlY4LQehP9q37NZNzKb0dEPr0ne87ks7JaikMxPFNEO49e/7i5f6rzus3b9+97x4cTkzVaMbHrJKVvsio4VIoPgYBkl/UmtMyk/w8W35v8+dXBtRqT+wqvmspAslcsEoeCrt/iMlhYJRaX+5qEjXJ13aWosr18dE8ktMhMpTW/gNjxy+p+93vuBTL8ohwsRcarCY5Joyu6szodpFo76zBGiTxg7AlosCvBOgkIOef21D1VItmZsLpjI7g13t65U/78exRt1P1badrtxYN4HfgxSLagh7ZxlnavybxiTckVMEmNmSZxDTNLNQgmueuQxvCasiVd8KmHipbczOx6Bg5/9swc5X2SwFes3cdlpbGrMrMK9vemIe5lnwqN20g/zazQtUNcMU2F+WNxFDhdqB4LjRnIFceUKaFfytmBfV9AT/2jm9C8vDLj8HkaJDEg+T3cW94sm3HPvqIPqEIJegrGqIROkNjxIJhkAdVUIc/wmWoQ9hIw2Dr+YDuRej+A0h/2hY=</latexit> <latexit sha1_base64="7uFYgDiClgiAivHmMF4hNngkNgc=">ACw3icbVFba9swFJbdXbrslnaPexELA4dBsEthewdg8AG7WBJC1FmZEWORWTZlY4LQehP9q37NZNzKb0dEPr0ne87ks7JaikMxPFNEO49e/7i5f6rzus3b9+97x4cTkzVaMbHrJKVvsio4VIoPgYBkl/UmtMyk/w8W35v8+dXBtRqT+wqvmspAslcsEoeCrt/iMlhYJRaX+5qEjXJ13aWosr18dE8ktMhMpTW/gNjxy+p+93vuBTL8ohwsRcarCY5Joyu6szodpFo76zBGiTxg7AlosCvBOgkIOef21D1VItmZsLpjI7g13t65U/78exRt1P1badrtxYN4HfgxSLagh7ZxlnavybxiTckVMEmNmSZxDTNLNQgmueuQxvCasiVd8KmHipbczOx6Bg5/9swc5X2SwFes3cdlpbGrMrMK9vemIe5lnwqN20g/zazQtUNcMU2F+WNxFDhdqB4LjRnIFceUKaFfytmBfV9AT/2jm9C8vDLj8HkaJDEg+T3cW94sm3HPvqIPqEIJegrGqIROkNjxIJhkAdVUIc/wmWoQ9hIw2Dr+YDuRej+A0h/2hY=</latexit> Result Finite 0 1 s Bias due to Privacy noise ✓r ◆ ✓ ◆ Var( H ) 1 1 A + ˜ sample L ( h priv ) ≤ inf h ∈ H L ( h ) + O + O O @ capping variance K 2 ( τ 0 ) τ 0 n τ 0 variance

  7. <latexit sha1_base64="7uFYgDiClgiAivHmMF4hNngkNgc=">ACw3icbVFba9swFJbdXbrslnaPexELA4dBsEthewdg8AG7WBJC1FmZEWORWTZlY4LQehP9q37NZNzKb0dEPr0ne87ks7JaikMxPFNEO49e/7i5f6rzus3b9+97x4cTkzVaMbHrJKVvsio4VIoPgYBkl/UmtMyk/w8W35v8+dXBtRqT+wqvmspAslcsEoeCrt/iMlhYJRaX+5qEjXJ13aWosr18dE8ktMhMpTW/gNjxy+p+93vuBTL8ohwsRcarCY5Joyu6szodpFo76zBGiTxg7AlosCvBOgkIOef21D1VItmZsLpjI7g13t65U/78exRt1P1badrtxYN4HfgxSLagh7ZxlnavybxiTckVMEmNmSZxDTNLNQgmueuQxvCasiVd8KmHipbczOx6Bg5/9swc5X2SwFes3cdlpbGrMrMK9vemIe5lnwqN20g/zazQtUNcMU2F+WNxFDhdqB4LjRnIFceUKaFfytmBfV9AT/2jm9C8vDLj8HkaJDEg+T3cW94sm3HPvqIPqEIJegrGqIROkNjxIJhkAdVUIc/wmWoQ9hIw2Dr+YDuRej+A0h/2hY=</latexit> <latexit sha1_base64="7uFYgDiClgiAivHmMF4hNngkNgc=">ACw3icbVFba9swFJbdXbrslnaPexELA4dBsEthewdg8AG7WBJC1FmZEWORWTZlY4LQehP9q37NZNzKb0dEPr0ne87ks7JaikMxPFNEO49e/7i5f6rzus3b9+97x4cTkzVaMbHrJKVvsio4VIoPgYBkl/UmtMyk/w8W35v8+dXBtRqT+wqvmspAslcsEoeCrt/iMlhYJRaX+5qEjXJ13aWosr18dE8ktMhMpTW/gNjxy+p+93vuBTL8ohwsRcarCY5Joyu6szodpFo76zBGiTxg7AlosCvBOgkIOef21D1VItmZsLpjI7g13t65U/78exRt1P1badrtxYN4HfgxSLagh7ZxlnavybxiTckVMEmNmSZxDTNLNQgmueuQxvCasiVd8KmHipbczOx6Bg5/9swc5X2SwFes3cdlpbGrMrMK9vemIe5lnwqN20g/zazQtUNcMU2F+WNxFDhdqB4LjRnIFceUKaFfytmBfV9AT/2jm9C8vDLj8HkaJDEg+T3cW94sm3HPvqIPqEIJegrGqIROkNjxIJhkAdVUIc/wmWoQ9hIw2Dr+YDuRej+A0h/2hY=</latexit> <latexit sha1_base64="7uFYgDiClgiAivHmMF4hNngkNgc=">ACw3icbVFba9swFJbdXbrslnaPexELA4dBsEthewdg8AG7WBJC1FmZEWORWTZlY4LQehP9q37NZNzKb0dEPr0ne87ks7JaikMxPFNEO49e/7i5f6rzus3b9+97x4cTkzVaMbHrJKVvsio4VIoPgYBkl/UmtMyk/w8W35v8+dXBtRqT+wqvmspAslcsEoeCrt/iMlhYJRaX+5qEjXJ13aWosr18dE8ktMhMpTW/gNjxy+p+93vuBTL8ohwsRcarCY5Joyu6szodpFo76zBGiTxg7AlosCvBOgkIOef21D1VItmZsLpjI7g13t65U/78exRt1P1badrtxYN4HfgxSLagh7ZxlnavybxiTckVMEmNmSZxDTNLNQgmueuQxvCasiVd8KmHipbczOx6Bg5/9swc5X2SwFes3cdlpbGrMrMK9vemIe5lnwqN20g/zazQtUNcMU2F+WNxFDhdqB4LjRnIFceUKaFfytmBfV9AT/2jm9C8vDLj8HkaJDEg+T3cW94sm3HPvqIPqEIJegrGqIROkNjxIJhkAdVUIc/wmWoQ9hIw2Dr+YDuRej+A0h/2hY=</latexit> <latexit sha1_base64="7uFYgDiClgiAivHmMF4hNngkNgc=">ACw3icbVFba9swFJbdXbrslnaPexELA4dBsEthewdg8AG7WBJC1FmZEWORWTZlY4LQehP9q37NZNzKb0dEPr0ne87ks7JaikMxPFNEO49e/7i5f6rzus3b9+97x4cTkzVaMbHrJKVvsio4VIoPgYBkl/UmtMyk/w8W35v8+dXBtRqT+wqvmspAslcsEoeCrt/iMlhYJRaX+5qEjXJ13aWosr18dE8ktMhMpTW/gNjxy+p+93vuBTL8ohwsRcarCY5Joyu6szodpFo76zBGiTxg7AlosCvBOgkIOef21D1VItmZsLpjI7g13t65U/78exRt1P1badrtxYN4HfgxSLagh7ZxlnavybxiTckVMEmNmSZxDTNLNQgmueuQxvCasiVd8KmHipbczOx6Bg5/9swc5X2SwFes3cdlpbGrMrMK9vemIe5lnwqN20g/zazQtUNcMU2F+WNxFDhdqB4LjRnIFceUKaFfytmBfV9AT/2jm9C8vDLj8HkaJDEg+T3cW94sm3HPvqIPqEIJegrGqIROkNjxIJhkAdVUIc/wmWoQ9hIw2Dr+YDuRej+A0h/2hY=</latexit> Result Finite 0 1 s Privacy noise ✓r ◆ ✓ ◆ Var( H ) 1 1 A + ˜ sample L ( h priv ) ≤ inf h ∈ H L ( h ) + O + O O @ variance K 2 ( τ 0 ) τ 0 n τ 0 variance Bias due to capping

  8. <latexit sha1_base64="7uFYgDiClgiAivHmMF4hNngkNgc=">ACw3icbVFba9swFJbdXbrslnaPexELA4dBsEthewdg8AG7WBJC1FmZEWORWTZlY4LQehP9q37NZNzKb0dEPr0ne87ks7JaikMxPFNEO49e/7i5f6rzus3b9+97x4cTkzVaMbHrJKVvsio4VIoPgYBkl/UmtMyk/w8W35v8+dXBtRqT+wqvmspAslcsEoeCrt/iMlhYJRaX+5qEjXJ13aWosr18dE8ktMhMpTW/gNjxy+p+93vuBTL8ohwsRcarCY5Joyu6szodpFo76zBGiTxg7AlosCvBOgkIOef21D1VItmZsLpjI7g13t65U/78exRt1P1badrtxYN4HfgxSLagh7ZxlnavybxiTckVMEmNmSZxDTNLNQgmueuQxvCasiVd8KmHipbczOx6Bg5/9swc5X2SwFes3cdlpbGrMrMK9vemIe5lnwqN20g/zazQtUNcMU2F+WNxFDhdqB4LjRnIFceUKaFfytmBfV9AT/2jm9C8vDLj8HkaJDEg+T3cW94sm3HPvqIPqEIJegrGqIROkNjxIJhkAdVUIc/wmWoQ9hIw2Dr+YDuRej+A0h/2hY=</latexit> <latexit sha1_base64="7uFYgDiClgiAivHmMF4hNngkNgc=">ACw3icbVFba9swFJbdXbrslnaPexELA4dBsEthewdg8AG7WBJC1FmZEWORWTZlY4LQehP9q37NZNzKb0dEPr0ne87ks7JaikMxPFNEO49e/7i5f6rzus3b9+97x4cTkzVaMbHrJKVvsio4VIoPgYBkl/UmtMyk/w8W35v8+dXBtRqT+wqvmspAslcsEoeCrt/iMlhYJRaX+5qEjXJ13aWosr18dE8ktMhMpTW/gNjxy+p+93vuBTL8ohwsRcarCY5Joyu6szodpFo76zBGiTxg7AlosCvBOgkIOef21D1VItmZsLpjI7g13t65U/78exRt1P1badrtxYN4HfgxSLagh7ZxlnavybxiTckVMEmNmSZxDTNLNQgmueuQxvCasiVd8KmHipbczOx6Bg5/9swc5X2SwFes3cdlpbGrMrMK9vemIe5lnwqN20g/zazQtUNcMU2F+WNxFDhdqB4LjRnIFceUKaFfytmBfV9AT/2jm9C8vDLj8HkaJDEg+T3cW94sm3HPvqIPqEIJegrGqIROkNjxIJhkAdVUIc/wmWoQ9hIw2Dr+YDuRej+A0h/2hY=</latexit> <latexit sha1_base64="7uFYgDiClgiAivHmMF4hNngkNgc=">ACw3icbVFba9swFJbdXbrslnaPexELA4dBsEthewdg8AG7WBJC1FmZEWORWTZlY4LQehP9q37NZNzKb0dEPr0ne87ks7JaikMxPFNEO49e/7i5f6rzus3b9+97x4cTkzVaMbHrJKVvsio4VIoPgYBkl/UmtMyk/w8W35v8+dXBtRqT+wqvmspAslcsEoeCrt/iMlhYJRaX+5qEjXJ13aWosr18dE8ktMhMpTW/gNjxy+p+93vuBTL8ohwsRcarCY5Joyu6szodpFo76zBGiTxg7AlosCvBOgkIOef21D1VItmZsLpjI7g13t65U/78exRt1P1badrtxYN4HfgxSLagh7ZxlnavybxiTckVMEmNmSZxDTNLNQgmueuQxvCasiVd8KmHipbczOx6Bg5/9swc5X2SwFes3cdlpbGrMrMK9vemIe5lnwqN20g/zazQtUNcMU2F+WNxFDhdqB4LjRnIFceUKaFfytmBfV9AT/2jm9C8vDLj8HkaJDEg+T3cW94sm3HPvqIPqEIJegrGqIROkNjxIJhkAdVUIc/wmWoQ9hIw2Dr+YDuRej+A0h/2hY=</latexit> <latexit sha1_base64="7uFYgDiClgiAivHmMF4hNngkNgc=">ACw3icbVFba9swFJbdXbrslnaPexELA4dBsEthewdg8AG7WBJC1FmZEWORWTZlY4LQehP9q37NZNzKb0dEPr0ne87ks7JaikMxPFNEO49e/7i5f6rzus3b9+97x4cTkzVaMbHrJKVvsio4VIoPgYBkl/UmtMyk/w8W35v8+dXBtRqT+wqvmspAslcsEoeCrt/iMlhYJRaX+5qEjXJ13aWosr18dE8ktMhMpTW/gNjxy+p+93vuBTL8ohwsRcarCY5Joyu6szodpFo76zBGiTxg7AlosCvBOgkIOef21D1VItmZsLpjI7g13t65U/78exRt1P1badrtxYN4HfgxSLagh7ZxlnavybxiTckVMEmNmSZxDTNLNQgmueuQxvCasiVd8KmHipbczOx6Bg5/9swc5X2SwFes3cdlpbGrMrMK9vemIe5lnwqN20g/zazQtUNcMU2F+WNxFDhdqB4LjRnIFceUKaFfytmBfV9AT/2jm9C8vDLj8HkaJDEg+T3cW94sm3HPvqIPqEIJegrGqIROkNjxIJhkAdVUIc/wmWoQ9hIw2Dr+YDuRej+A0h/2hY=</latexit> Result 0 1 s Privacy noise ✓r ◆ ✓ ◆ Var( H ) 1 1 A + ˜ L ( h priv ) ≤ inf h ∈ H L ( h ) + O + O O @ variance K 2 ( τ 0 ) τ 0 n τ 0 Bias due to Finite capping sample variance

Recommend

Bias-variance trade-off. Crossvalidation. Regularization. Petr Po s k P. Po s k

Bias-variance trade-off. Crossvalidation. Regularization. Petr Po s k P. Po s k

CZECH TECHNICAL UNIVERSITY IN PRAGUE Faculty of Electrical Engineering Department of Cybernetics Bias-variance trade-off. Crossvalidation. Regularization. Petr Po s k P. Po s k c 2015 Artificial Intelligence 1 / 13

274 views • 26 slides
MLCC 2019 Local Methods and Bias Variance Trade-Off Lorenzo Rosasco UNIGE-MIT-IIT About this

MLCC 2019 Local Methods and Bias Variance Trade-Off Lorenzo Rosasco UNIGE-MIT-IIT About this

MLCC 2019 Local Methods and Bias Variance Trade-Off Lorenzo Rosasco UNIGE-MIT-IIT About this class 1. Introduce a basic class of learning methods, namely local methods . 2. Discuss the fundamental concept of bias-variance trade-off to

1.06k views • 79 slides
Review Selection bias, overfitting Bias v. variance v. residual Bias-variance tradeoff

Review Selection bias, overfitting Bias v. variance v. residual Bias-variance tradeoff

Review Selection bias, overfitting Bias v. variance v. residual Bias-variance tradeoff 1 n=1 Cramr-Rao bound n=4 0.8 n=30 0.6 CDF of max of n samples of 0.4 N( =2, 2 =1) 0.2 [representing error estimates for n

517 views • 28 slides
Bias, Variance and Error Bias and Variance given algorithm that outputs estimate for , we

Bias, Variance and Error Bias and Variance given algorithm that outputs estimate for , we

Bias, Variance and Error Bias and Variance given algorithm that outputs estimate for , we define: the bias of the estimator: the variance of estimator: e.g., estimator for probability of heads, based on n independent coin

606 views • 25 slides
GMM &amp; EM Last time summary Normalization Bias-Variance trade-off Overfitting and

GMM &amp; EM Last time summary Normalization Bias-Variance trade-off Overfitting and

GMM &amp; EM Last time summary Normalization Bias-Variance trade-off Overfitting and underfitting MLE vs MAP estimate How to use the prior LRT (Bayes Classifier) Nave Bayes A simple decision rule If we can know

471 views • 35 slides
Bias Variance Trade-off Intuition: If the model is too simple, the solution is biased and

Bias Variance Trade-off Intuition: If the model is too simple, the solution is biased and

Carnegie Mellon University 10-701 Machine Learning Spring 2013 Bias Variance Trade-off Intuition: If the model is too simple, the solution is biased and does not fit the data If the model is too complex then it is very sensitive to

361 views • 14 slides
MLCC 2017 Local Methods and Bias Variance Trade-Off Lorenzo Rosasco UNIGE-MIT-IIT June 25, 2017

MLCC 2017 Local Methods and Bias Variance Trade-Off Lorenzo Rosasco UNIGE-MIT-IIT June 25, 2017

MLCC 2017 Local Methods and Bias Variance Trade-Off Lorenzo Rosasco UNIGE-MIT-IIT June 25, 2017 About this class 1. Introduce a basic class of learning methods, namely local methods . 2. Discuss the fundamental concept of bias-variance

826 views • 78 slides
Bias-Variance Tradeoff Machine Learning 1 Bias and variance Every learning algorithm requires

Bias-Variance Tradeoff Machine Learning 1 Bias and variance Every learning algorithm requires

Bias-Variance Tradeoff Machine Learning 1 Bias and variance Every learning algorithm requires assumptions about the hypothesis space. Eg: My hypothesis space is linear decision trees with 5 nodes a three layer

189 views • 17 slides
Bias-Variance Tradeoff Matthieu R. Bloch h in a given set H that minimizes the true risk R ( h ) .

Bias-Variance Tradeoff Matthieu R. Bloch h in a given set H that minimizes the true risk R ( h ) .

1 Regularization plays a similar role by biasing answer away from complex functions. Tiis is particu- ; We formalize the bias-variance tradeoff assuming the following: which takes the form We now develop an alternative method to quantify the

454 views • 3 slides
Lecture 20: AdaBoost Aykut Erdem December 2017 Hacettepe University Last time Bias/Variance

Lecture 20: AdaBoost Aykut Erdem December 2017 Hacettepe University Last time Bias/Variance

Lecture 20: AdaBoost Aykut Erdem December 2017 Hacettepe University Last time Bias/Variance Tradeo ff slide by David Sontag Graphical illustration of bias and variance. http://scott.fortmann-roe.com/docs/BiasVariance.html 2 Last time

641 views • 37 slides
Self-bounding functions and concentration of variance Andreas Maurer Advances in stochastic

Self-bounding functions and concentration of variance Andreas Maurer Advances in stochastic

Self-bounding functions and concentration of variance Andreas Maurer Advances in stochastic inequalities and their applications, BIRS 2009 Notation and denitions := Q n k =1 k is some product space with product probability = n

622 views • 15 slides
Bias- -Variance Theory Variance Theory Bias Decompose Error Rate into components, some

Bias- -Variance Theory Variance Theory Bias Decompose Error Rate into components, some

Bias- -Variance Theory Variance Theory Bias Decompose Error Rate into components, some Decompose Error Rate into components, some of which can be measured on unlabeled data of which can be measured on unlabeled data Bias- -Variance

942 views • 77 slides
10701 Machine Learning Boosting Fighting the bias-variance tradeoff Simple (a.k.a. weak)

10701 Machine Learning Boosting Fighting the bias-variance tradeoff Simple (a.k.a. weak)

10701 Machine Learning Boosting Fighting the bias-variance tradeoff Simple (a.k.a. weak) learners are good e.g., nave Bayes, logistic regression, decision stumps (or shallow decision trees) Low variance, dont usually overfit

686 views • 31 slides
The role of over-parametrisation in NNs The role of over-parametrisation in NNs Levent Sagun,

The role of over-parametrisation in NNs The role of over-parametrisation in NNs Levent Sagun,

The role of over-parametrisation in NNs The role of over-parametrisation in NNs Levent Sagun, EPFL Classical bias-variance dilemma Classical bias-variance dilemma Error Test Train Capacity Classical bias-variance dilemma, or? Classical

914 views • 60 slides
A Unified Bias-Variance Decomposition and its Applications Pedro Domingos

A Unified Bias-Variance Decomposition and its Applications Pedro Domingos

A Unified Bias-Variance Decomposition and its Applications Pedro Domingos pedrod@cs.washington.edu Department of Computer Science and Engineering, University of Washington, Seattle, WA 98195, U.S.A. Abstract predictions fluctuate in response to

476 views • 8 slides
CSC 311: Introduction to Machine Learning Lecture 5 - Decision Trees &amp; Bias-Variance

CSC 311: Introduction to Machine Learning Lecture 5 - Decision Trees &amp; Bias-Variance

CSC 311: Introduction to Machine Learning Lecture 5 - Decision Trees &amp; Bias-Variance Decomposition Roger Grosse Chris Maddison Juhan Bae Silviu Pitis University of Toronto, Fall 2020 Intro ML (UofT) CSC311-Lec5 1 / 49 Today Decision

1.49k views • 50 slides
Variance = E[I 2 ] 2pE[I] + p 2 = E[I] 2p p + p 2 = 2 2 = p-2p+ p pq variance.1

Variance = E[I 2 ] 2pE[I] + p 2 = E[I] 2p p + p 2 = 2 2 = p-2p+ p pq variance.1

Variance of an Indicator Mathematics for Computer Science MIT 6.042J/18.062J IanindicatorwithE[I]=p: Var[I]:: = E[(I p) 2 ] Variance = E[I 2 ] 2pE[I] + p 2 = E[I] 2p p + p 2 = 2 2 = p-2p+ p pq variance.1 variance.2

104 views • 6 slides
CS7015 (Deep Learning) : Lecture 8 Regularization: Bias Variance Tradeoff, l2 regularization,

CS7015 (Deep Learning) : Lecture 8 Regularization: Bias Variance Tradeoff, l2 regularization,

CS7015 (Deep Learning) : Lecture 8 Regularization: Bias Variance Tradeoff, l2 regularization, Early stopping, Dataset augmentation, Parameter sharing and tying, Injecting noise at input, Ensemble methods, Dropout Mitesh M. Khapra Department of

1.06k views • 86 slides
recap: Approximation Versus Generalization VC Analysis Bias-Variance Analysis E out E in +

recap: Approximation Versus Generalization VC Analysis Bias-Variance Analysis E out E in +

recap: Approximation Versus Generalization VC Analysis Bias-Variance Analysis E out E in + ( d vc ) E out = bias + var 1. Did you fit your data well enough ( E in )? 1. How well can you fit your data ( bias )? 2. Are you confident your E in

543 views • 6 slides
Machine Learning Lecture 05: The Bias-Variance decomposition Nevin L. Zhang lzhang@cse.ust.hk

Machine Learning Lecture 05: The Bias-Variance decomposition Nevin L. Zhang lzhang@cse.ust.hk

Machine Learning Lecture 05: The Bias-Variance decomposition Nevin L. Zhang lzhang@cse.ust.hk Department of Computer Science and Engineering The Hong Kong University of Science and Technology This set of notes is based on internet resources

620 views • 24 slides
Linear Regression, Regularization Bias-Variance Tradeoff Thanks to C Guestrin, T Dietterich, R

Linear Regression, Regularization Bias-Variance Tradeoff Thanks to C Guestrin, T Dietterich, R

HTF: Ch3, 7 B: Ch3 Linear Regression, Regularization Bias-Variance Tradeoff Thanks to C Guestrin, T Dietterich, R Parr, N Ray Outline Linear Regression MLE = Least Squares! Basis functions Evaluating Predictors

1.45k views • 58 slides
Lecture 4: Rule-based classification and regression Felix Held, Mathematical Sciences

Lecture 4: Rule-based classification and regression Felix Held, Mathematical Sciences

Lecture 4: Rule-based classification and regression Felix Held, Mathematical Sciences MSA220/MVE440 Statistical Learning for Big Data 1st April 2019 Amendment: Bias-Variance Tradeoff ] Variance Bias 2 Error Model complexity Overfit Underfit

620 views • 30 slides
Bias, Variance and Parsimony in Regression Analysis ECS 256 Winter 2014 Christopher Patton,

Bias, Variance and Parsimony in Regression Analysis ECS 256 Winter 2014 Christopher Patton,

Bias, Variance and Parsimony in Regression Analysis ECS 256 Winter 2014 Christopher Patton, cjpatton@ucdavis.edu Alex Rumbaugh, aprumbaugh@ucdavis.edu Thomas Provan, tcprovan@ucdavis.edu Olga Prilepova, prilepova@gmail.com John Chen,

585 views • 34 slides
Regression and the Bias-Variance Decomposition William Cohen 10-601 April 2008 Readings: Bishop

Regression and the Bias-Variance Decomposition William Cohen 10-601 April 2008 Readings: Bishop

Regression and the Bias-Variance Decomposition William Cohen 10-601 April 2008 Readings: Bishop 3.1,3.2 1 Regression Technically: learning a function f( x )=y where y is real-valued , rather than discrete . Replace

651 views • 31 slides

More recommend