A Framework for Bayesian Optimization in Embedded Subspaces - PowerPoint PPT Presentation

Dec 08, 2022 •486 likes •554 views

Post ster #236 : Tue 6:30pm @ Pacific Ballroom A Framework for Bayesian Optimization in Embedded Subspaces Alexander Munteanu Amin Nayebi Matthias Poloczek TU Dortmund University of Arizona Uber AI Labs & University of Arizona ICML

Post ster #236 : Tue 6:30pm @ Pacific Ballroom A Framework for Bayesian Optimization in Embedded Subspaces Alexander Munteanu Amin Nayebi Matthias Poloczek TU Dortmund University of Arizona Uber AI Labs & University of Arizona ICML 2019
Baye yesi sian Global Global Optimiza zation of of Exp xpensi sive ve Functi Functions ons The he Goal oal Optimize an expensive-to-evaluate, black-box function f(x) x) over a feasible region of parameter vectors x specified in D dimensions. We can access for any x the value of f(x) x) possibly with some noise, i.e., f(x) x)+ ε . Typically: D < 20. Here: D large, but f(x) x) depends only on a d -dimensional active subspace. A. M A. Munteanu unteanu, A. , A. Naye yebi, M , M. . Polocze zek: A Framework k for Baye yesi sian Optimiza zation in Embedded Subsp spaces
Applications s of high-di dim. m. BO BO are re ubi biqu quitous Policy search in Reinforcement Learning • Aerospace design • Network architecture search • Calibration of simulations to observed data • Control of chemical processes • Drug design • A. M A. Munteanu unteanu, A. , A. Naye yebi, M , M. . Polocze zek: A Framework k for Baye yesi sian Optimiza zation in Embedded Subsp spaces
The The HeS HeSBO Framework k for high-dimensi sional BO Theorem Theorem : Active subspace embedding accurately preserves GP-prior (with constant probability) For a variety of popular kernels: linear, polynomial, • (squared) exponential, Matérn. The embedding can be combined with many GP-based BO algorithms, • e.g., Knowledge Gradient (KG), BLOSSOM, Expected Improvement (EI). Experiments demonstrate Efficient and easy to code using hash functions. • Robustness to ambient dimension D. D. • Outperforms st state-of of-th the-art art: REMBO, BOCK, EBO, additive BO. • A. M A. Munteanu unteanu, A. , A. Naye yebi, M , M. . Polocze zek: A Framework k for Baye yesi sian Optimiza zation in Embedded Subsp spaces
Visi sit us s at the post ster prese sentation! Great performance even if subspace assumption not not met met, e.g., for Neural Network Parameter Search 100-dim. Styblinski-Tang (Oh, Gavves, and Welling ’18) Function ster #236 : Post Tue 6:30pm Visit https://github.com/aminnayebi/HesBO @ Pacific Ballroom for He HeSBO fo for K KG, B G, BLOS OSSOM OM, a , and E EI . A. Munteanu A. M unteanu, A. , A. Naye yebi, M , M. . Polocze zek: A Framework k for Baye yesi sian Optimiza zation in Embedded Subsp spaces

Recommend

Design Space Optimization of Embedded Memory Design Space Optimization of Embedded Memory Systems

Design Space Optimization of Embedded Memory Design Space Optimization of Embedded Memory Systems via Data Remapping Systems via Data Remapping Krishna V. Palem, Rodric M. Rabbah, Vincent J. Mooney III, Pinar Korkmaz and Kiran Puttaswamy Center

775 views • 38 slides

Bayesian Optimization CSC2541 - Topics in Machine Learning Scalable and Flexible Models of

Bayesian Optimization CSC2541 - Topics in Machine Learning Scalable and Flexible Models of Uncertainty University of Toronto - Fall 2017 Overview 1. Bayesian Optimization of Machine Learning Algorithms 2. Gaussian Process Optimization in the

1.53k views • 94 slides

Outline Intro to RL and Bayesian Learning History of Bayesian RL Model-based Bayesian

Outline Intro to RL and Bayesian Learning History of Bayesian RL Model-based Bayesian RL Prior knowledge, policy optimization, discussion, Bayesian approaches for other RL variants Model-free Bayesian RL Gaussian

1.23k views • 63 slides

Safe Bayesian Optimization for Optimal Control Schillinger, M., Hartmann, B., Skalecki, P.,

Safe Bayesian Optimization for Optimal Control Schillinger, M., Hartmann, B., Skalecki, P., Meister, M., Nguyen-Tuong, D., & Nelles, O. (2017). Safe Active Learning and Safe Bayesian Optimization for Tuning a PI-Controller. IFAC-PapersOnLine ,

203 views • 6 slides

Knowing The What, But Not The Where in Bayesian Optimization Vu Nguyen & Michael A. Osborne

Knowing The What, But Not The Where in Bayesian Optimization Vu Nguyen & Michael A. Osborne University of Oxford Vu Nguyen Bayesian Optimization 1 Black-box Optimization The relationship from to is through the black-box. Output

259 views • 24 slides

My research over Bayesian Optimization and Gaussian Processes Eduardo C. GarridoMerch an

My research over Bayesian Optimization and Gaussian Processes Eduardo C. GarridoMerch an PhD student. Teacher Assistant. Universidad Aut onoma de Madrid 1/13 Index Introduction: Bayesian Optimization and Gaussian Processes. 2/13

923 views • 70 slides

Ebba: An Embedded DSL for Bayesian Inference Linkping University, 17 June 2014 Henrik Nilsson

Ebba: An Embedded DSL for Bayesian Inference Linkping University, 17 June 2014 Henrik Nilsson School of Computer Science University of Nottingham Joint work with Tom Nielsen, OpenBrain Ltd Ebba: An Embedded DSL for Bayesian Inference

1.14k views • 80 slides

The Bayesian Network Framework 89 / 384 The network formalism, informal A Bayesian network

Chapter 4: The Bayesian Network Framework 89 / 384 The network formalism, informal A Bayesian network combines two types of domain knowledge to represent a joint probability distribution: qualitative knowledge: a (minimal) directed I-map

1.63k views • 134 slides

Scaling Bayesian Optimization in High Dimensions Stefanie Jegelka, MIT BayesOpt Workshop 2017

Scaling Bayesian Optimization in High Dimensions Stefanie Jegelka, MIT BayesOpt Workshop 2017 joint work with Zi Wang , Chengtao Li, Clement Gehring (MIT) and Pushmeet Kohli (DeepMind) Bayesian Optimization with GPs f ( x )

421 views • 23 slides

Scalable Global Optimization via Local Bayesian Optimization David Eriksson Uber AI

Scalable Global Optimization via Local Bayesian Optimization David Eriksson Uber AI eriksson@uber.com Matthias Poloczek Michael Pearce Jake Gardner Ryan Turner Global Optimization Find such that

436 views • 11 slides

Regret bounds for meta Bayesian optimization with an unknown Gaussian process prior Zi Wang*

Regret bounds for meta Bayesian optimization with an unknown Gaussian process prior Zi Wang* Beomjoon Kim* Leslie Pack Kaelbling Dec 5 @ NeurIPS 18 Poster #22 Bayesian optimization x * = argmax Goal: f ( x ) x

346 views • 20 slides

Bayesian Optimization for Likelihood-Free Inference Michael Gutmann

Bayesian Optimization for Likelihood-Free Inference Michael Gutmann https://sites.google.com/site/michaelgutmann University of Edinburgh 14th September 2016 Reference For further information: M.U. Gutmann and J. Corander Bayesian

984 views • 24 slides

Embedded Optimization for Control and Signal Processing Moritz Diehl Optimization in Engineering

Embedded Optimization for Control and Signal Processing Moritz Diehl Optimization in Engineering Center (OPTEC) & Electrical Engineering Department (ESAT) K.U. Leuven Belgium Linkoping, June 16, 2011 OPTEC - Optimization in Engineering

1.13k views • 82 slides

CSC321 Lecture 21: Bayesian Hyperparameter Optimization Roger Grosse Roger Grosse CSC321

CSC321 Lecture 21: Bayesian Hyperparameter Optimization Roger Grosse Roger Grosse CSC321 Lecture 21: Bayesian Hyperparameter Optimization 1 / 25 Overview Todays lecture: a neat application of Bayesian parameter estimation to automatically

745 views • 25 slides

Learning Bayesian networks viewed as an optimization problem Milan Studen y Institute of

Learning Bayesian networks viewed as an optimization problem Milan Studen y Institute of Information Theory and Automation of the AS CR Prague COSA Workshop Combinatorial Optimization, Statistics, and Applications Munich, Germany, March

458 views • 29 slides

Bayesian Optimization of Composite Functions Ral Astudillo Cornell University Joint work

Bayesian Optimization of Composite Functions Ral Astudillo Cornell University Joint work with Peter I. Frazier ICML 2019 Ral Astudillo ra598@cornell.edu Bayesian Optimization of Composite Functions 1 0 1 log10(regret) 2 3

196 views • 16 slides

Bayesian Optimization in Reduced Eigenbases David Gaudrie 1 , Rodolphe Le Riche 2 , Victor Picheny

Parametric shape optimization Intrinsic dimensionality Meta-modeling in eigenbasis Optimization in eigenbasis Conclusions Bayesian Optimization in Reduced Eigenbases David Gaudrie 1 , Rodolphe Le Riche 2 , Victor Picheny 3 , t Enaux 1 ,

581 views • 55 slides

Bayesian Neural Networks - Presenters Group 1: A Practical Bayesian Framework for Backpropagation

Bayesian Neural Networks - Presenters Group 1: A Practical Bayesian Framework for Backpropagation Networks - Slides 2-40 Paul Vicol Shane Baccas George Alexandru Adam Group 2: Priors for Infinite Networks - Slides 41-64

1.37k views • 110 slides

Budgeted Bayesian Multiobjective Optimization David Gaudrie 1 , Rodolphe Le Riche 2 , Victor

Introduction Preferences Budgeted optimization Batch criteria Budgeted Bayesian Multiobjective Optimization David Gaudrie 1 , Rodolphe Le Riche 2 , Victor Picheny 3 , t Enaux 1 , Vincent Herbert 1 Beno 1 Groupe PSA 2 CNRS LIMOS,

963 views • 84 slides

Bayesian Optimization under Heavy-tailed Payoffs Sayak Ray Chowdhury Joint work with Aditya

Bayesian Optimization under Heavy-tailed Payoffs Sayak Ray Chowdhury Joint work with Aditya Gopalan Department of ECE, Indian Institute of Science NeurIPS, Dec. 2019 Black-box optimization 1.5 Problem: Maximize an unknown 1 utility

967 views • 17 slides

Using Meta Learning to Initialize Bayesian Optimization Albert-Ludwigs-Universitt Freiburg

Using Meta Learning to Initialize Bayesian Optimization Albert-Ludwigs-Universitt Freiburg Matthias Feurer 1 Jost Tobias Springenberg 2 Frank Hutter 1 1 Research Group on Learning, Optimization, and Automated Algorithm Design 2 Machine Learning

852 views • 48 slides

. Inverse problems in signal and image processing and Bayesian inference framework: from basic to

. Inverse problems in signal and image processing and Bayesian inference framework: from basic to advanced Bayesian computation Ali Mohammad-Djafari Laboratoire des Signaux et Syst` emes (L2S) UMR8506 CNRS-CentraleSup elec-UNIV PARIS SUD

990 views • 77 slides

A Nonlinear Trust Region Framework for PDE-Constrained Optimization Using

Motivation PDE-Constrained Optimization ROM-Constrained Optimization Numerical Experiments Conclusion References A Nonlinear Trust Region Framework for PDE-Constrained Optimization Using Progressively-Constructed Reduced-Order Models

458 views • 34 slides

Design Optimization of Time- and Cost-Constrained Fault-Tolerant Distributed Embedded Systems

Design Optimization of Time- and Cost-Constrained Fault-Tolerant Distributed Embedded Systems Viaceslav Izosimov, Paul Pop, Petru Eles, Zebo Peng Embedded Systems Lab (ESLAB) Linkping University, Sweden 1/21 1 of 14 Motivation Faults

408 views • 21 slides