a closer look at adaptive regret
play

A Closer Look at Adaptive Regret Dmitry Adamskiy Joint work with - PowerPoint PPT Presentation

Oct 15, 2022 •25 likes •525 views

A Closer Look at Adaptive Regret Dmitry Adamskiy Joint work with Wouter Koolen, Volodya Vovk and Alexey Chernov Department of Computer Science Royal Holloway, University of London 15/11/2014 Adamskiy (RHUL) A Closer Look at Adaptive Regret

  1. A Closer Look at Adaptive Regret Dmitry Adamskiy Joint work with Wouter Koolen, Volodya Vovk and Alexey Chernov Department of Computer Science Royal Holloway, University of London 15/11/2014 Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 1 / 21

  2. Outline Why adaptive regret? 1 Setup 2 Results 3 Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 2 / 21

  3. Why adaptive regret? 1 Setup 2 Results 3 Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 3 / 21

  4. Weather forecasting: adaptivity Predictor Expert Expert Expert Nature Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

  5. Weather forecasting: adaptivity Predictor Expert 30% Expert 90% Expert 20% Nature Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

  6. Weather forecasting: adaptivity Predictor 55% Expert 30% Expert 90% Expert 20% Nature Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

  7. Weather forecasting: adaptivity Predictor 55% Expert 30% Expert 90% Expert 20% Nature Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

  8. Weather forecasting: adaptivity Predictor 55% Expert 30% 40% Expert 90% 70% Expert 20% 65% Nature Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

  9. Weather forecasting: adaptivity Predictor 55% 65% Expert 30% 40% Expert 90% 70% Expert 20% 65% Nature Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

  10. Weather forecasting: adaptivity Predictor 55% 65% Expert 30% 40% Expert 90% 70% Expert 20% 65% Nature Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

  11. Weather forecasting: adaptivity Predictor 55% 65% Expert 30% 40% . . . Expert 90% 70% . . . Expert 20% 65% . . . Nature Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

  12. Weather forecasting: adaptivity Predictor 55% 65% . . . Expert 30% 40% . . . Expert 90% 70% . . . Expert 20% 65% . . . Nature Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

  13. Weather forecasting: adaptivity Predictor 55% 65% . . . Expert 30% 40% . . . 90% 70% . . . Expert Expert 20% 65% . . . Nature . . . Goal: close to the best expert overall (solution: AA) Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

  14. Weather forecasting: adaptivity Predictor 55% 65% . . . Expert 30% 40% . . . is bad on foggy days! 90% 70% . . . Expert Expert 20% 65% . . . Nature . . . Goal: close to the best expert overall (solution: AA) Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

  15. Weather forecasting: adaptivity Predictor 55% 65% . . . Expert 30% 40% . . . is bad on foggy days! 90% 70% . . . Expert Expert 20% 65% . . . drunk on weekends! Nature . . . Goal: close to the best expert overall (solution: AA) Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

  16. Weather forecasting: adaptivity Predictor 55% 65% . . . Expert 30% 40% . . . is bad on foggy days! 90% 70% . . . goes on training! Expert Expert 20% 65% . . . drunk on weekends! Nature . . . Goal: close to the best expert overall (solution: AA) Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

  17. Weather forecasting: adaptivity Predictor 55% 65% . . . Expert 30% 40% . . . is bad on foggy days! Expert 90% 70% . . . goes on training! Expert 20% 65% . . . drunk on weekends! Nature . . . Goal: close to the best expert overall (solution: AA) Adaptive goal : close to the best expert on every time interval Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 4 / 21

  18. Example continued Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 5 / 21

  19. Example continued Non-adaptive predictor would lose trust in the first guy. Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 5 / 21

  20. Adaptive algorithms We studied several approaches to adaptivity: Blowing up the set of experts to compete with virtual sleeping experts [DA, Koolen, Chernov, Vovk, 2012] Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 6 / 21

  21. Adaptive algorithms We studied several approaches to adaptivity: Blowing up the set of experts to compete with virtual sleeping experts [DA, Koolen, Chernov, Vovk, 2012] Turned out to be Fixed Share [Herbster, Warmuth 1998]! Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 6 / 21

  22. Adaptive algorithms We studied several approaches to adaptivity: Blowing up the set of experts to compete with virtual sleeping experts [DA, Koolen, Chernov, Vovk, 2012] Turned out to be Fixed Share [Herbster, Warmuth 1998]! Restarting existing algorithms and combining their predictions [Hazan, Seshadhri, 2009] Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 6 / 21

  23. Adaptive algorithms We studied several approaches to adaptivity: Blowing up the set of experts to compete with virtual sleeping experts [DA, Koolen, Chernov, Vovk, 2012] Turned out to be Fixed Share [Herbster, Warmuth 1998]! Restarting existing algorithms and combining their predictions [Hazan, Seshadhri, 2009] Also turned out to be Fixed Share! Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 6 / 21

  24. Adaptive properties of Fixed Share: results Fixed Share is known for tracking . Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 7 / 21

  25. Adaptive properties of Fixed Share: results Fixed Share is known for tracking . L FS [ 1 , T ] − L S [ 1 , T ] ≤ ln N +( m − 1 ) ln ( N − 1 ) − ( m − 1 ) ln α − ( T − m ) ln ( 1 − α ) , Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 7 / 21

  26. Adaptive properties of Fixed Share: results Fixed Share is known for tracking . L FS [ 1 , T ] − L S [ 1 , T ] ≤ ln N +( m − 1 ) ln ( N − 1 ) − ( m − 1 ) ln α − ( T − m ) ln ( 1 − α ) , What about its adaptivity? Our results Figured out the Worst-Case adaptive regret of Fixed Share 1 Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 7 / 21

  27. Adaptive properties of Fixed Share: results Fixed Share is known for tracking . L FS [ 1 , T ] − L S [ 1 , T ] ≤ ln N +( m − 1 ) ln ( N − 1 ) − ( m − 1 ) ln α − ( T − m ) ln ( 1 − α ) , What about its adaptivity? Our results Figured out the Worst-Case adaptive regret of Fixed Share 1 Proved the optimality of Fixed Share — “no algorithm could have 2 better guarantees on all time intervals” Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 7 / 21

  28. Why adaptive regret? 1 Setup 2 Results 3 Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 8 / 21

  29. Protocol: Mix loss for t = 1 , 2 , . . . do Learner announces probability vector � w t ∈ △ N Reality announces loss vector � ℓ t ∈ [ −∞ , ∞ ] N t e − ℓ n n w n Learner suffers loss ℓ t := − ln � t end for Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 9 / 21

  30. Adaptive Regret Goal: On every time interval [ t 1 , t 2 ] we want to be not much worse than the best expert on that interval. We are interested in small adaptive regret Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 10 / 21

  31. Adaptive Regret Goal: On every time interval [ t 1 , t 2 ] we want to be not much worse than the best expert on that interval. We are interested in small adaptive regret Definition The adaptive regret of the algorithm on the interval [ t 1 , t 2 ] is the loss of the algorithm there minus the lost of the best expert there: L j R [ t 1 , t 2 ] := L [ t 1 , t 2 ] − min [ t 1 , t 2 ] j Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 10 / 21

  32. AA and Fixed Share Aggregating Algorithm [Vovk 1990] updates weights as: t e − ℓ n w n t w n t + 1 := t . t e − ℓ n n w n � Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 11 / 21

  33. AA and Fixed Share Aggregating Algorithm [Vovk 1990] updates weights as: t e − ℓ n w n t w n t + 1 := t . t e − ℓ n n w n � Fixed Share family is defined by the sequence of “switching rates” α t . Then the weight update is t e − ℓ n w n α t + 1 � N � t w n t + 1 := N − 1 + 1 − N − 1 α t + 1 t . t e − ℓ n n w n � Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 11 / 21

  34. AA and Fixed Share Aggregating Algorithm [Vovk 1990] updates weights as: t e − ℓ n w n t w n t + 1 := t . t e − ℓ n n w n � Fixed Share family is defined by the sequence of “switching rates” α t . Then the weight update is t e − ℓ n w n α t + 1 � N � t w n t + 1 := N − 1 + 1 − N − 1 α t + 1 t . t e − ℓ n n w n � Adaptivity hides in the first term. Adamskiy (RHUL) A Closer Look at Adaptive Regret GTP-2014 Guanajuato Mexico 11 / 21

Recommend

L M A D A Learning And Mining from DatA NANJING UNIVERSITY Adaptive Regret of Convex and

L M A D A Learning And Mining from DatA NANJING UNIVERSITY Adaptive Regret of Convex and

L M A D A Learning And Mining from DatA NANJING UNIVERSITY Adaptive Regret of Convex and Smooth Functions Lijun Zhang 1 Tie-Yan Liu 2 Zhi-Hua Zhou 1 1 National Key Laboratory for Novel Software Technology, Nanjing University 2 Microsoft

404 views • 13 slides
On adaptive regret bounds for non- stochastic bandits Gergely Neu INRIA Lille, SequeL team

On adaptive regret bounds for non- stochastic bandits Gergely Neu INRIA Lille, SequeL team

On adaptive regret bounds for non- stochastic bandits Gergely Neu INRIA Lille, SequeL team Universitat Pompeu Fabra, Barcelona Online learning and bandits Adaptive bounds in online learning Adaptive bounds for bandits Outline

1.22k views • 75 slides
Trials Designs for Adaptive Interventions Research Questions Closer to Practice in Trials

Trials Designs for Adaptive Interventions Research Questions Closer to Practice in Trials

Trials Designs for Adaptive Interventions Research Questions Closer to Practice in Trials Maya Petersen Div. Epidemiology & Biostatistics School of Public Health, University of California, Berkeley Precision Public Health

414 views • 37 slides
No-Regret Learning in Convex Games Geoff Gordon, Amy Greenwald, Casey Marks, and Martin Zinkevich

No-Regret Learning in Convex Games Geoff Gordon, Amy Greenwald, Casey Marks, and Martin Zinkevich

No-Regret Learning in Convex Games Geoff Gordon, Amy Greenwald, Casey Marks, and Martin Zinkevich No-Regret Learning in Convex Games p. 1 Introduction The connection between regret and equilibria is well understood in matrix games. Most

554 views • 42 slides
Composability of Regret Minimizers Gabriele Farina 1 Christian Kroer 2 Tuomas Sandholm 1,3,4,5 1

Composability of Regret Minimizers Gabriele Farina 1 Christian Kroer 2 Tuomas Sandholm 1,3,4,5 1

Regret Circuits: Composability of Regret Minimizers Gabriele Farina 1 Christian Kroer 2 Tuomas Sandholm 1,3,4,5 1 Computer Science Department, Carnegie Mellon University 2 IEOR Department, Columbia University 3 Strategic Machine, Inc. 4 Strategy

219 views • 19 slides
Counterfactual Regret Minimization and Domination in Extensive-Form Games Richard Gibson

Counterfactual Regret Minimization and Domination in Extensive-Form Games Richard Gibson

Counterfactual Regret Minimization and Domination in Extensive-Form Games Richard Gibson University of Alberta Edmonton, Alberta, Canada Overview Counterfactual Regret Minimization (CFR) Overview Counterfactual Regret Minimization (CFR)

479 views • 36 slides
Our Virtual Welcome Welcome We want to welcome you to this presentation. Whilst we regret we are

Our Virtual Welcome Welcome We want to welcome you to this presentation. Whilst we regret we are

Y6 Open Evening September 2020 Our Virtual Welcome Welcome We want to welcome you to this presentation. Whilst we regret we are not able to meet you in person, we hope that this presentation will give you a good understanding of our school,

750 views • 38 slides
TS 83 DORMA DORMA TS 83 Easy-action Door Closer Easy-action door closer Data and features TS

TS 83 DORMA DORMA TS 83 Easy-action Door Closer Easy-action door closer Data and features TS

Easy-action Door Closer TS 83 DORMA DORMA TS 83 Easy-action Door Closer Easy-action door closer Data and features TS 83 with thinking backcheck Variable Spring EN EN closing force strength 2 6 7 Easy to fix and even

178 views • 6 slides
Royal Economic Society The history of Regret Theory Robert Sugden Contribution to Economic

Royal Economic Society The history of Regret Theory Robert Sugden Contribution to Economic

Royal Economic Society The history of Regret Theory Robert Sugden Contribution to Economic Journal 125 th anniversary session On and beyond regret theory Royal Economic Society Conference University of Manchester 30 March 2015 1

1.23k views • 106 slides
A Closer Look at IEPs What you need to know!!!

A Closer Look at IEPs What you need to know!!!

A Closer Look at IEPs What you need to know!!! Pa# Mackey Educa-onal Consultant December 2015 Lets take a minute. The Principles

372 views • 11 slides
Closer to Home Health & Care Strategy Bringing care closer to home What is the problem? A

Closer to Home Health & Care Strategy Bringing care closer to home What is the problem? A

Closer to Home Health & Care Strategy Bringing care closer to home What is the problem? A challenged health and care system: Increasing demand Increasing complexity Increasing cost Workforce under pressure Current models

317 views • 9 slides
Regret Bounds for Lifelong Learning Pierre Alquier Groupe de Travail de Machine learning du CMLA

Regret Bounds for Lifelong Learning Pierre Alquier Groupe de Travail de Machine learning du CMLA

Transfer learning, multitask learning, lifelong learning... A strategy for lifelong learning, with regret analysis Open questions Regret Bounds for Lifelong Learning Pierre Alquier Groupe de Travail de Machine learning du CMLA ENS Paris-Saclay

1.33k views • 106 slides
Regret bounds for online variational inference Pierre Alquier ACML Nagoya, Nov. 18, 2019

Regret bounds for online variational inference Pierre Alquier ACML Nagoya, Nov. 18, 2019

Regret bounds for online variational inference Pierre Alquier ACML Nagoya, Nov. 18, 2019 Pierre Alquier, RIKEN AIP Regret bounds for online variational inference Co-authors Emtiyaz Khan Badr-Eddine Chrief-Abdellatif Approximate

779 views • 45 slides
From passivity-based adaptive control to LMI tuned adaptive control or how Alexander Fradkov

From passivity-based adaptive control to LMI tuned adaptive control or how Alexander Fradkov

From passivity-based adaptive control to LMI tuned adaptive control or how Alexander Fradkov convinced me that direct adaptive control can be robust Dimitri Peaucelle FRontiers of ADaptive and nonlinear CONtrol (FRADCON 2018) ECC Workshop,

628 views • 19 slides
5: Tyranny and Madness I Barrett on Caligula The Roman senate soon came to regret that they

5: Tyranny and Madness I Barrett on Caligula The Roman senate soon came to regret that they

5: Tyranny and Madness I Barrett on Caligula The Roman senate soon came to regret that they had so enthusiastically handed over supreme power to a personable but totally inexperienced young man. Fewer than fifteen years later they went

660 views • 11 slides
An Improved Regret Bound for Thompson Sampling in the Gaussian Linear Bandit Setting Cem

An Improved Regret Bound for Thompson Sampling in the Gaussian Linear Bandit Setting Cem

An Improved Regret Bound for Thompson Sampling in the Gaussian Linear Bandit Setting Cem Kalkanl, Ayfer Ozg ur Stanford University ISIT, June 2020 An Improved Regret Bound for Thompson Sampling in the Gaussian Linear Bandit Setting 1

379 views • 13 slides
Wireless data networks More on 802.11 Martin Heusse A closer look at 802.11 1 Channels

Wireless data networks More on 802.11 Martin Heusse A closer look at 802.11 1 Channels

Wireless data networks More on 802.11 Martin Heusse A closer look at 802.11 1 Channels 802.11b 1 6 11 2412 MHz 2437 MHz 2462 MHz f 22MHz A closer look at 802.11 2 802.11a ) &#)*+, &#)*+, ! ! ! ! ! ! ! !

419 views • 15 slides
Using the Network Layering DAG CS 118 Computer Network Fundamentals Peter Reiher Lecture 13 CS

Using the Network Layering DAG CS 118 Computer Network Fundamentals Peter Reiher Lecture 13 CS

Using the Network Layering DAG CS 118 Computer Network Fundamentals Peter Reiher Lecture 13 CS 118 Page 1 Winter 2016 Outline The DAG A closer look at tables A closer look at directed links A closer look at FSMs and their

1.03k views • 64 slides
Acceleration through Optimistic No-Regret Dynamics Jun-Kun Wang and Jacob Abernethy Georgia Tech

Acceleration through Optimistic No-Regret Dynamics Jun-Kun Wang and Jacob Abernethy Georgia Tech

Acceleration through Optimistic No-Regret Dynamics Jun-Kun Wang and Jacob Abernethy Georgia Tech Jun-Kun Wang and Jacob Abernethy Acceleration through Optimistic No-Regret Dynamics Convex Optimization min x X f ( x ) (1) Method: Gradient

394 views • 13 slides
Group Sequential and Adaptive Designs Part II: Adaptive Designs May 2, 2015 Cyrus Mehta, Ph.D.

Group Sequential and Adaptive Designs Part II: Adaptive Designs May 2, 2015 Cyrus Mehta, Ph.D.

Group Sequential and Adaptive Designs Part II: Adaptive Designs May 2, 2015 Cyrus Mehta, Ph.D. Cytel Inc. Adaptive Designs Adaptive designs are defined as: changes in design or analyses guided by examination of the accumulated

1.11k views • 64 slides
Neural Nets for Adaptive Filter and Adaptive Neural Nets as Adaptive Filters Pattern Recognition

Neural Nets for Adaptive Filter and Adaptive Neural Nets as Adaptive Filters Pattern Recognition

Neural Nets for Adaptive Filter and Adaptive Pattern Recognition Brian Young Article Context Neural Nets for Adaptive Filter and Adaptive Neural Nets as Adaptive Filters Pattern Recognition Adaptive Filters Min. Disturb. and LMS

976 views • 23 slides
R2-B2: Recursive Reasoning-Based Bayesian Optimization for No-Regret Learning in Games Zhongxiang

R2-B2: Recursive Reasoning-Based Bayesian Optimization for No-Regret Learning in Games Zhongxiang

R2-B2: Recursive Reasoning-Based Bayesian Optimization for No-Regret Learning in Games Zhongxiang Dai 1 Yizhou Chen 1 Bryan Kian Hsiang Low 1 Patrick Jaillet 2 Teck-Hua Ho 3 1 Department of Computer Science, National University of Singapore 2

534 views • 18 slides
REGRET INTRODUCTION We focus on a semantically rich dataset that captures detailed information

REGRET INTRODUCTION We focus on a semantically rich dataset that captures detailed information

ROUTING GAMES IN THE WILD: EFFICIENCY, EQUILIBRATION AND Hassan Nikaein Amin Sabbaghian REGRET INTRODUCTION We focus on a semantically rich dataset that captures detailed information about the daily behavior of thousands of Singaporean

444 views • 23 slides
Informed Consent in Research in Japan: A Case for Broad Consent Addendum: I regret I could not

Informed Consent in Research in Japan: A Case for Broad Consent Addendum: I regret I could not

Informed Consent in Research in Japan: A Case for Broad Consent Addendum: I regret I could not give adequate explanations in response to the questions posed after my presentation in Bali Convention Center on August 22, 2014. Supplemental slides

241 views • 21 slides

More recommend