Poster #20 Bayesian Nonparametric Federated Learning of Neural - PowerPoint PPT Presentation

Jan 20, 2023 •30 likes •120 views

IBM Research, MIT-IBM Watson AI Lab PFNM , Poster #20 ICML 2019 Poster #20 Bayesian Nonparametric Federated Learning of Neural Networks Mikhail Yurochkin Mayank Agarwal, Soumya Ghosh, Kristjan Greenewald, Nghia Hoang, Yasaman Khazaeni IBM

IBM Research, MIT-IBM Watson AI Lab PFNM , Poster #20 ICML 2019 Poster #20 Bayesian Nonparametric Federated Learning of Neural Networks Mikhail Yurochkin Mayank Agarwal, Soumya Ghosh, Kristjan Greenewald, Nghia Hoang, Yasaman Khazaeni IBM Research, MIT-IBM Watson AI Lab June 12th 1
IBM Research, MIT-IBM Watson AI Lab PFNM , Poster #20 Federated Learning
IBM Research, MIT-IBM Watson AI Lab PFNM , Poster #20 Model fusion perspective
IBM Research, MIT-IBM Watson AI Lab PFNM , Poster #20 Probabilistic Federated Neural Matching
IBM Research, MIT-IBM Watson AI Lab PFNM , Poster #20 Simulated heterogeneous Federated Learning on MNIST Client 1 Client 2
IBM Research, MIT-IBM Watson AI Lab PFNM , Poster #20 Examples of first layer weights Neuron 12 Neuron 21 Neuron 49 Client 1 Neuron 7 Neuron 8 Neuron 36 Client 2
IBM Research, MIT-IBM Watson AI Lab PFNM , Poster #20 PFNM discovers correspondences among weights Client 1 Client 2 Client 1 Client 2 Client 2 Client 1 Neuron 12 Neuron 8 Neuron 49 Neuron 7 Neuron 36 Neuron 21 Matched neuron 8 Matched neuron 33 Matched neuron 44 Matched neuron 58
IBM Research, MIT-IBM Watson AI Lab PFNM , Poster #20 Summary PFNM is a method for combining pre-trained fully-connected neural networks: • Can combine NNs trained on heterogeneous data without access to data • Can be further improved with few communication rounds (if data is available) • Outperforms Distributed SGD and Federated Averaging Technical contributions: • Indian Buffet Process based model to govern correspondences between weights of local neural networks. Applicable to multilayer networks • BNP allows for adaptive learning of global NN size • Fast MAP inference using iterative Hungarian algorithm
IBM Research, MIT-IBM Watson AI Lab PFNM , Poster #20 THANK YOU | Please come to poster #2 #20 9

Recommend

Outline Motivation and challenge Dirichlet Process and Infinite Mixture Formulation

Nonparametric Bayesian M Nonparametric Bayesian Models odels --Learning and Reasoning in Open Possible Worlds -- Learning and Reasoning in Open Possible Worlds Eric Xing epxing@cs.cmu.edu Machine Learning Dept./Language Technology

783 views • 40 slides

Synergies in learning syllables and words or Adaptor grammars: a class of nonparametric Bayesian

Synergies in learning syllables and words or Adaptor grammars: a class of nonparametric Bayesian models Mark Johnson Brown University Joint work with Sharon Goldwater and Tom Griffiths NECPHON, November, 2008 1 / 26 Research goals Most

470 views • 26 slides

Dirichlet Processes and Nonparametric Bayesian Modelling Volker Tresp 1 Motivation Infinite

Dirichlet Processes and Nonparametric Bayesian Modelling Volker Tresp 1 Motivation Infinite models have recently gained a lot of attention in Bayesian machine learning They offer great flexibility and, in many applications, allow a more

1.84k views • 144 slides

Toward Reliable Bayesian Nonparametric Learning Erik Sudderth Brown University Department of

Toward Reliable Bayesian Nonparametric Learning Erik Sudderth Brown University Department of Computer Science Donglai Wei & Michael Bryant (HDP topics) Joint work with Michael Hughes & Emily Fox (BP-HMM) Documents & Topic Models

649 views • 61 slides

Nonparametric Bayesian Models for Sparse Matrices and Covariances Zoubin Ghahramani Department

Nonparametric Bayesian Models for Sparse Matrices and Covariances Zoubin Ghahramani Department of Engineering University of Cambridge, UK zoubin@eng.cam.ac.uk http://learning.eng.cam.ac.uk/zoubin/ Bayes 250 Edinburgh 2011 Bayesian Machine

402 views • 36 slides

Federated Learning Min Du Postdoc, UC Berkeley Outline q Preliminary: deep learning and SGD q

Federated Learning Min Du Postdoc, UC Berkeley Outline q Preliminary: deep learning and SGD q Federated learning: FedSGD and FedAvg q Related research in federated learning q Open problems Outline q Preliminary: deep learning and SGD q Federated

616 views • 50 slides

Bayesian Nonparametric Models for Data Exploration Melanie F. Pradier Friday 15 th September,

Bayesian Nonparametric Models for Data Exploration Melanie F. Pradier Friday 15 th September, 2017 Intro BNPs ADDP for Marathon Modeling C-IBP for Clinical Trial PFAs for International Trade Conclusions Outline 1 Introduction 2 Bayesian

1.82k views • 118 slides

Bayesian Learning 1 Outline MLE, MAP vs. Bayesian Learning Bayesian Linear Regression

Bayesian Learning 1 Outline MLE, MAP vs. Bayesian Learning Bayesian Linear Regression Bayesian Gaussian Mixture Models Non-parametric Bayes 2 Take Away ... 1. Maximum Likelihood Estimate (MLE) = arg max p ( D| )

609 views • 27 slides

Analyzing Federated Learning through an Adversarial Lens Arjun Nitin Bhagoji 1 , Supriyo

Analyzing Federated Learning through an Adversarial Lens Arjun Nitin Bhagoji 1 , Supriyo Chakraborty 2 , Prateek Mittal 1 and Seraphin Calo 2 1 Princeton University 2 IBM Research ICML 2019 Federated learning (with a malicious agent) Federated

1.46k views • 59 slides

Bayesian Meta-Learning CS 330 1 Logistics Homework 2 due next Wednesday. Project proposal due in

Bayesian Meta-Learning CS 330 1 Logistics Homework 2 due next Wednesday. Project proposal due in two weeks . Poster presentation: Tues 12/3 at 1:30 pm . 2 Disclaimers Bayesian meta-learning is an ac#ve area of research (like most of the

622 views • 33 slides

Bayesian nonparametric inference for diffusion models with discrete sampling Delft University of

Bayesian nonparametric inference for diffusion models with discrete sampling Delft University of Technology Jakob S ohl joint work with Richard Nickl Van Dantzig Seminar, Leiden, 26 October 2016 Jakob S ohl (TU Delft) Bayesian

514 views • 29 slides

Dr. Nonparametric Bayes Or: How I Learned to Stop Worrying and Love the Dirichlet Process Kurt

Dr. Nonparametric Bayes Or: How I Learned to Stop Worrying and Love the Dirichlet Process Kurt Miller CS 294: Practical Machine Learning November 19, 2009 Today we will discuss Nonparametric Bayesian methods. Kurt T. Miller Dr.

1.63k views • 124 slides

Federated Machine Learning via Over-the-Air Computation Yuanming Shi ShanghaiTech University 1

Federated Machine Learning via Over-the-Air Computation Yuanming Shi ShanghaiTech University 1 Outline Motivations Big data, IoT,AI Three vignettes: Federated machine learning Federated model aggregation Over-the-air

1.02k views • 56 slides

A Fast New Bayesian Approach to High-Dimensional Nonparametric Regression Without MCMC Ray Bai

A Fast New Bayesian Approach to High-Dimensional Nonparametric Regression Without MCMC Ray Bai Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Joint work with Gemma E. Moran (co-first author), Joseph

775 views • 55 slides

Fair Resource Allocation in Federated Learning Tian Li (CMU) , Maziar Sanjabi (Facebook AI), Ahmad

Fair Resource Allocation in Federated Learning Tian Li (CMU) , Maziar Sanjabi (Facebook AI), Ahmad Beirami (Facebook AI), Virginia Smith (CMU) tianli@cmu.edu 1 Federated Learning 2 2 Federated Learning Privacy-preserving training in

1.19k views • 55 slides

On a Class of Nonparametric Bayesian Autoregressive Models Maria Anna Di Lucca 1 , Alessandra

On a Class of Nonparametric Bayesian Autoregressive Models Maria Anna Di Lucca 1 , Alessandra Guglielmi 2 , uller 3 , Fernando A. Quintana 4 Peter M Karolinska Institutet 1 Politecnico di Milano 2 University of Texas, Austin 3 olica de Chile 4

1.02k views • 89 slides

Introduction to Nonparametric Bayesian Modeling and Gaussian Process Regression Piyush Rai Dept.

Introduction to Nonparametric Bayesian Modeling and Gaussian Process Regression Piyush Rai Dept. of CSE, IIT Kanpur (Mini-course: lecture 3) Nov 07, 2015 Piyush Rai (IIT Kanpur) Nonparametric Bayesian Modeling and Gaussian Process Regression

1.03k views • 54 slides

Nonparametric Bayesian Storyline Detection from Microtexts Vinodh Krishnan and Jacob Eisenstein

Nonparametric Bayesian Storyline Detection from Microtexts Vinodh Krishnan and Jacob Eisenstein Georgia Institute of Technology Clustering microtexts into storylines Strong start for Barcelona Dog tuxedo bought with county credit card Messi

665 views • 33 slides

Unsupervised Coreference Resolution in a Nonparametric Bayesian Model Aria Haghighi and Dan

Unsupervised Coreference Resolution in a Nonparametric Bayesian Model Aria Haghighi and Dan Klein Computer Science Division University of California Berkeley Coreference Resolution The Weir Group , whose headquarters is in the U.S , is a

1.75k views • 143 slides

Outline Intro to RL and Bayesian Learning History of Bayesian RL Model-based Bayesian

Outline Intro to RL and Bayesian Learning History of Bayesian RL Model-based Bayesian RL Prior knowledge, policy optimization, discussion, Bayesian approaches for other RL variants Model-free Bayesian RL Gaussian

1.23k views • 63 slides

A Nonparametric Bayesian Basket Trial Slide 4 Design Report (decision) Peter M uller , UT

recursive splits @ Mdn( x j s ), s = 1 , 2 , 3 Slide 1 A Nonparametric Bayesian Basket Trial Slide 4 Design Report (decision) Peter M uller , UT Austin www.math.utexas.edu/users/pmueller/oncostat.pdf subset-specific mean outome p ( y i

417 views • 3 slides

Nonparametric Bayesian Word Sense Induction Xuchen Yao 1 and Benjamin Van Durme 1,2 1 Department of

Nonparametric Bayesian Word Sense Induction Xuchen Yao 1 and Benjamin Van Durme 1,2 1 Department of Computer Science 2 Human Language Technology Center of Excellence Johns Hopkins University TextGraphs-6 June 23 2011 1 / 14 Word Sense Induction

665 views • 21 slides

Unsupervised Coreference Resolution in a Nonparametric Bayesian Model Aria Haghighi and Dan Klein

Unsupervised Coreference Resolution in a Nonparametric Bayesian Model Aria Haghighi and Dan Klein Presented by Brandon Norick Overview Introduction Preliminaries Coreference Resolution Models Experiments Conclusion

656 views • 37 slides

Differentially-Private Federated Linear Bandits Introduction Federated Learning Contextual

Differentially- Private Federated Linear Bandits Dubey and Pentland June 2020 Differentially-Private Federated Linear Bandits Introduction Federated Learning Contextual Bandits Summary Background Abhimanyu Dubey and Alex Pentland

437 views • 20 slides