pac identification of many good arms in stochastic multi
play

PAC Identification of Many Good Arms in Stochastic Multi-Armed - PowerPoint PPT Presentation

Oct 28, 2023 •102 likes •347 views

PAC Identification of Many Good Arms in Stochastic Multi-Armed Bandits Arghya Roy Chaudhuri under the guidance of Prof. Shivaram Kalyanakrishnan Indian Institute of Technology Bombay, India 1 / 8 What Is It All About? 2 / 8 What Is It All

  1. PAC Identification of Many Good Arms in Stochastic Multi-Armed Bandits Arghya Roy Chaudhuri under the guidance of Prof. Shivaram Kalyanakrishnan Indian Institute of Technology Bombay, India 1 / 8

  2. What Is It All About? 2 / 8

  3. What Is It All About? 2 / 8

  4. What Is It All About? 2 / 8

  5. What Is It All About? 3 / 8

  6. What Is a Multi-Armed Bandit? 1.0 0.9 0.5 0.5 0.2 0.0 Mean (Unknown) Bandits: Slot machines Mean: Pr[Reward = 1] 4 / 8

  7. What Is a Multi-Armed Bandit? To identify the best arm: � n 1.0 ǫ 2 log 1 � E [SC] = Ω 0.9 δ To identify the best subset of size 0.5 0.5 m : � n ǫ 2 log m � E [SC] = Ω 0.2 δ 0.0 Mean (Unknown) Bandits: Slot machines Mean: Pr[Reward = 1] 4 / 8

  8. What Is a Multi-Armed Bandit? To identify the best arm: � n 1.0 ǫ 2 log 1 � E [SC] = Ω 0.9 δ To identify the best subset of size 0.5 0.5 m : � n ǫ 2 log m � E [SC] = Ω 0.2 δ 0.0 Mean (Unknown) We need an alternative. Bandits: Slot machines Mean: Pr[Reward = 1] 4 / 8

  9. Large Bandit Instances Difficulty for n ≫ T : ǫ 2 log 1 lim n →∞ n δ = ∞ . 5 / 8

  10. Large Bandit Instances Difficulty for n ≫ T : ǫ 2 log 1 lim n →∞ n δ = ∞ . Get around: Identifying 1 from the best ρ -fraction is possible. 5 / 8

  11. Large Bandit Instances Difficulty for n ≫ T : ǫ 2 log 1 lim n →∞ n δ = ∞ . Get around: Identifying 1 from the best ρ -fraction is possible. 5 / 8

  12. Large Bandit Instances Difficulty for n ≫ T : ǫ 2 log 1 lim n →∞ n δ = ∞ . Get around: Identifying 1 from the best ρ -fraction is possible. 5 / 8

  13. Large Bandit Instances Difficulty for n ≫ T : ǫ 2 log 1 lim n →∞ n δ = ∞ . Get around: Identifying 1 from the best ρ -fraction is possible. 5 / 8

  14. Large Bandit Instances Difficulty for n ≫ T : ǫ 2 log 1 lim n →∞ n δ = ∞ . Get around: Identifying 1 from the best ρ -fraction is possible. 5 / 8

  15. Large Bandit Instances Difficulty for n ≫ T : ǫ 2 log 1 lim n →∞ n δ = ∞ . Get around: Identifying 1 from the best ρ -fraction is possible. 5 / 8

  16. Large Bandit Instances Difficulty for n ≫ T : ǫ 2 log 1 lim n →∞ n δ = ∞ . Get around: Identifying 1 from the best ρ -fraction is possible. Redefine the problem to identify 1 from the best m arms. Defining ρ = m n , generalise the problem. What if we n is relatively small? 5 / 8

  17. Finite-Armed Bandit Instances ( k , m , n ): To identify any distinct k arms from the best m arms in a set of n arms. 6 / 8

  18. Finite-Armed Bandit Instances ( k , m , n ): To identify any distinct k arms from the best m arms in a set of n arms. k = 1 : Any 1 arm out of the best subset of size m . 6 / 8

  19. Finite-Armed Bandit Instances ( k , m , n ): To identify any distinct k arms from the best m arms in a set of n arms. k = m : Best subset identification. 6 / 8

  20. Finite-Armed Bandit Instances ( k , m , n ): To identify any distinct k arms from the best m arms in a set of n arms. k = m = 1 : Best arm identification. 6 / 8

  21. Finite-Armed Bandit Instances ( k , m , n ): To identify any distinct k arms from the best m arms in a set of n arms. k = 1 : Any 1 arm out of the best subset of size m . k = m : Best subset identification. k = m = 1 : Best arm identification. Contributions: LUCB -k-m (Fully sequential + Adaptive). Worst case upper and lower bound. 6 / 8

  22. Infinite-Armed Bandit Instances ( k , ρ ): To identify any distinct k arms from the best ρ fraction of arms. 7 / 8

  23. Infinite-Armed Bandit Instances ( k , ρ ): To identify any distinct k arms from the best ρ fraction of arms. 7 / 8

  24. Thank You! Poster: #54 Email: arghya@cse.iitb.ac.in 8 / 8

Recommend

Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems, Part I S ebastien

Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems, Part I S ebastien

Regret Analysis of Stochastic and Nonstochastic Multi-armed Bandit Problems, Part I S ebastien Bubeck Theory Group i.i.d. multi-armed bandit, Robbins [1952] i.i.d. multi-armed bandit, Robbins [1952] Known parameters: number of arms n and

809 views • 67 slides
Multi-Armed Bandits: Non-adaptive and Adaptive Sampling Instructor: Sham Kakade 1 The

Multi-Armed Bandits: Non-adaptive and Adaptive Sampling Instructor: Sham Kakade 1 The

CSE 547/Stat 548: Machine Learning for Big Data Lecture Multi-Armed Bandits: Non-adaptive and Adaptive Sampling Instructor: Sham Kakade 1 The (stochastic) multi-armed bandit problem The basic paradigm is as follows: K Independent Arms: a

273 views • 5 slides
On parameter identification in linear stochastic differential equations by Gaussian statistics

On parameter identification in linear stochastic differential equations by Gaussian statistics

Introduction Parameter identification by direct observation Parameter identification by indirect observation Conclusion and future work On parameter identification in linear stochastic differential equations by Gaussian statistics Shuai Lu

659 views • 36 slides
Multi-level stochastic approximation algorithms Noufel Frikha Universit e Paris Diderot, LPMA

Multi-level stochastic approximation algorithms Noufel Frikha Universit e Paris Diderot, LPMA

Introduction Analysis of the SA scheme Multi-level stochastic approximation algorithms Numerical results Multi-level stochastic approximation algorithms Noufel Frikha Universit e Paris Diderot, LPMA 28th October, 2013. Stochastic

746 views • 35 slides
Multi-Index Monte Carlo and Multi-Index Stochastic Collocation Ra ul Tempone Alexander von

Multi-Index Monte Carlo and Multi-Index Stochastic Collocation Ra ul Tempone Alexander von

Multi-Index Monte Carlo and Multi-Index Stochastic Collocation Ra ul Tempone Alexander von Humboldt Professor, RWTH Aachen, Germany and KAUST, Saudi Arabia. Course material developed together with Fabio Nobile (CSQI-MATHICSE, Ecole

1.57k views • 154 slides
A parameter identification problem in stochastic homogenization William Minvielle CERMICS,

A parameter identification problem in stochastic homogenization William Minvielle CERMICS,

An introduction to homogenization Setting Least square formulation A parameter identification problem in stochastic homogenization William Minvielle CERMICS, cole des Ponts ParisTech, Matherials research-team, INRIA Rocquencourt

390 views • 38 slides
Upper confidence bound strategy on stochastical bandits Multiarmed bandit: K arms, at each step we

Upper confidence bound strategy on stochastical bandits Multiarmed bandit: K arms, at each step we

Upper confidence bound strategy on stochastical bandits Multiarmed bandit: K arms, at each step we can choose one arm to be pulled while the other K-1 arms stay frozen (no reward). Stochastic bandit: Each arm has fixed distribution in all

278 views • 3 slides
SHARK-FV 2014 The MOOD Ideas MOOD for multi-mat. flows The good The not-yet-good-enough

SHARK-FV 2014 The MOOD Ideas MOOD for multi-mat. flows The good The not-yet-good-enough

LA-UR 14-22765 Extension of the MOOD Method to the Saurel-Petitpas-Berry Model for Multi-Material Compressible Flows Fourth-order 2D results: The good & the not-yet-good-enough a Steven Diot , a Marianne Franois, b Edward Dendy. a Fluid

897 views • 67 slides
Large Deviations for Multi-valued Stochastic Differential Equations Large Deviations for

Large Deviations for Multi-valued Stochastic Differential Equations Large Deviations for

Large Deviations for Multi-valued Stochastic Differential Equations Large Deviations for Multi-valued Stochastic Differential Equations Joint work with Siyan Xu, Xicheng Zhang We shall talk about the Freidlin-Wentzell Large Deviation Principle

2.55k views • 169 slides
Sergeant at Arms Training Session www.toastmasters.org Sergeant at Arms TRAINING DISTRICT 37

Sergeant at Arms Training Session www.toastmasters.org Sergeant at Arms TRAINING DISTRICT 37

Sergeant at Arms Training Session www.toastmasters.org Sergeant at Arms TRAINING DISTRICT 37 TLI EAST RALEIGH, NC JANUARY 17, 2015 Sergeant at Arms (SAA) Sergeant at Arms Role Sergeant at Arms Responsibilities Sergeant at Arms

874 views • 21 slides
CS 5150 So(ware Engineering 2. Steps in the so(ware development process William Y. Arms So(ware

CS 5150 So(ware Engineering 2. Steps in the so(ware development process William Y. Arms So(ware

Cornell University Compu1ng and Informa1on Science CS 5150 So(ware Engineering 2. Steps in the so(ware development process William Y. Arms So(ware Process Fundamental Assump1on: Good processes lead to good so(ware Good processes reduce risk

260 views • 22 slides
Multi-resolution Inference of Stochastic Models from Partially Observed Data Samuel Kou

Multi-resolution Inference of Stochastic Models from Partially Observed Data Samuel Kou

Multi-resolution Inference of Stochastic Models from Partially Observed Data Samuel Kou Department of Statistics Harvard University Joint with Ben Olding Stochastic Differential Equations n Models based on stochastic differential equations

579 views • 39 slides
A Calculus for Stochastic Interventions: Causal Effect Identification and Surrogate Experiments

A Calculus for Stochastic Interventions: Causal Effect Identification and Surrogate Experiments

A Calculus for Stochastic Interventions: Causal Effect Identification and Surrogate Experiments Juan D. Correa and Elias Bareinboim {jdcorrea, eb}@cs.columbia.edu February, 2020, New York 1 Outline 2 Outline Hard/atomic interventions

1.36k views • 80 slides
Multi-level Stochastic Local Search for SAT Camilo Rostoker and Chris Dabrowski

Multi-level Stochastic Local Search for SAT Camilo Rostoker and Chris Dabrowski

Multi-level Stochastic Local Search for SAT Camilo Rostoker and Chris Dabrowski {rostokec,kdabrows}@cs.ubc.ca Department of Computer Science University of British Columbia Outline Introduction to SAT 1. Systematic vs. Stochastic Local

1.02k views • 28 slides
RIG THE PERFECT ANGLE EVERY TIME Two picking arms each with five adjustment holes make

RIG THE PERFECT ANGLE EVERY TIME Two picking arms each with five adjustment holes make

RIG THE PERFECT ANGLE EVERY TIME Two picking arms each with five adjustment holes make it easy. NEW! MULTI ROLLERS Picking arms have Unlimited configuration 8 of adjustment. opportunities Extra deep sidewalls Made of

472 views • 3 slides
Identification Multi-Echelles par Ondelettes Continues de la Signature des Etats de Surface H.

Identification Multi-Echelles par Ondelettes Continues de la Signature des Etats de Surface H.

Identification Multi-Echelles par Ondelettes Continues de la Signature des Etats de Surface H. ZAHOUANI Laboratoire de Tribologie et Dynamique des Systmes UMR CNRS 5513 ENISE - ECL Caractre Multi-Echelles des Surfaces Procds de

510 views • 29 slides
On the Complexity of Best Arm Identification in Multi-Armed Bandit Models Aurlien Garivier

On the Complexity of Best Arm Identification in Multi-Armed Bandit Models Aurlien Garivier

On the Complexity of Best Arm Identification in Multi-Armed Bandit Models Aurlien Garivier Institut de Mathmatiques de Toulouse Information Theory, Learning and Big Data Simons Institute, Berkeley, March 2015 Simple Multi-Armed Bandit

514 views • 29 slides
Multi-stage Stochastic Fluid Models for Congestion Control Magorzata OReilly * * University

Multi-stage Stochastic Fluid Models for Congestion Control Magorzata OReilly * * University

Introduction: Stochastic Fluid Model Multi-stage SFMs with congestion control Multi-stage Stochastic Fluid Models for Congestion Control Magorzata OReilly * * University of Tasmania, Australia Australia New Zealand Applied Probability

1.04k views • 57 slides
An Optimal Private Stochastic-MAB Algorithm Based on an Optimal Private Stopping Rule Touqir Sajed

An Optimal Private Stochastic-MAB Algorithm Based on an Optimal Private Stopping Rule Touqir Sajed

An Optimal Private Stochastic-MAB Algorithm Based on an Optimal Private Stopping Rule Touqir Sajed & Or Sheffet K-Armed Stochastic Bandit Problem There are K arms The learner pulls an arm at rounds : 1, , T Pulling an arm i

246 views • 6 slides
IGA-based Multi-Index Stochastic Collocation for Uncertainty Quantification J. Beck 1 , L.

IGA-based Multi-Index Stochastic Collocation for Uncertainty Quantification J. Beck 1 , L.

IGA-based Multi-Index Stochastic Collocation for Uncertainty Quantification J. Beck 1 , L. Tamellini 2 , R. Tempone 1 , 3 1 King Abdullah University of Science and Technology, Saudi Arabia 2 CNR-IMATI, Italy 3 Alexander von Humboldt Professor in

691 views • 65 slides
EVENTS AND CELEBRATIONS THE LYTTELTON ARMS WWW.THELYTTELTONARMS.CO.UK WELCOME TO THE LYTTELTON

EVENTS AND CELEBRATIONS THE LYTTELTON ARMS WWW.THELYTTELTONARMS.CO.UK WELCOME TO THE LYTTELTON

EVENTS AND CELEBRATIONS THE LYTTELTON ARMS WWW.THELYTTELTONARMS.CO.UK WELCOME TO THE LYTTELTON ARMS Situated on the edge of the beautiful Clent Hills, the Lyttelton Arms is the ideal location for your event or celebration. Our welcoming

78 views • 6 slides
Stochastic Identification of Jet particles in pPb and PbPb Martin Schmidt Physikalisches

Stochastic Identification of Jet particles in pPb and PbPb Martin Schmidt Physikalisches

BMBF Forschungsschwerpunkt BMBF Forschungsschwerpunkt 202 ALICE Experiment ALICE Experiment Stochastic Identification of Jet particles in pPb and PbPb Martin Schmidt Physikalisches Institut, University of Tbingen Jet Meeting

581 views • 34 slides
EC3062 ECONOMETRICS IDENTIFICATION OF ARMA MODELS A stationary stochastic process can be

EC3062 ECONOMETRICS IDENTIFICATION OF ARMA MODELS A stationary stochastic process can be

EC3062 ECONOMETRICS IDENTIFICATION OF ARMA MODELS A stationary stochastic process can be characterised, equivalently, by its autocovariance function or its partial autocovariance function. It can also be characterised by is spectral density

438 views • 32 slides
Surrogate models for Single and Multi-Objective Stochastic Optimization: Integrating Support

Surrogate models for Single and Multi-Objective Stochastic Optimization: Integrating Support

Covariance Matrix Adaptation-Evolution Strategy Support Vector Machines Comparison-Based Surrogate Model for CMA-ES Dominance-based Surrogate Model for Multi-Objective Optimization Surrogate models for Single and Multi-Objective Stochastic

1.1k views • 53 slides

More recommend