active learning for regression active learning for
play

Active Learning for Regression: Active Learning for Regression: - PowerPoint PPT Presentation

Jan 31, 2024 •225 likes •870 views

LAMDA group, Nanjing University Nov. 5, 2009. Active Learning for Regression: Active Learning for Regression: Algorithms and Applications Algorithms and Applications Masashi Sugiyama Tokyo Institute of Technology sugi@cs.titech.ac.jp

  1. LAMDA group, Nanjing University Nov. 5, 2009. Active Learning for Regression: Active Learning for Regression: Algorithms and Applications Algorithms and Applications Masashi Sugiyama Tokyo Institute of Technology sugi@cs.titech.ac.jp http://sugiyama-www.cs.titech.ac.jp/~sugi/

  2. 2 Supervised Learning � Learn a target function from input-output samples . � This allows us to predict outputs of unseen inputs: “generalization” output input

  3. 3 Active Learning (AL) � Choice of input location affects the generalization performance. � Goal: choose the best input location! Learning target Learned function Good location Bad location

  4. 4 Motivation of AL � AL is effective when sampling cost is high. � Ex.) Predicting the length of a patient’s life � Input : features of patients � Output : the length of life � In order to observe the outputs, the patients need to be nursed for years � It is highly valuable to optimize the choice of input locations!

  5. 5 Organization of My Talk 1. Formulation. 2. AL for correctly specified models. 3. AL for misspecified models. 4. Choosing inputs from unlabeled samples. 5. AL with model selection.

  6. 6 Problem Formulation output input � Training samples: � Input: � Output: � Noise:

  7. 7 Problem Formulation � Use a linear model for learning: ： parameter ： basis function � Generalization error: :Test input density (assumed known) � � Goal of AL: Choose so that the generalization error is minimized.

  8. 8 Difficulty of AL � Gen err is unknown. � In AL, gen error needs to be estimated before observing output samples . � Thus standard gen err estimators such as cross-validation or Akaike’s information criterion cannot be used in AL.

  9. 9 Bias-Variance Decomposition :Expectation over noise � Gen err ： � Bias ： � Variance ： Gen err Bias Variance

  10. 10 Bias and Variance � Bias: depends on the unknown target function , so it is not possible to estimate it before observing output samples . � Variance: for linear estimator ,

  11. 11 Basic Strategy for AL � For an unbiased linear estimator, we have � Thus, gen error can be minimized before observing output samples !

  12. 12 Organization of My Talk 1. Formulation. 2. AL for correctly specified models. 3. AL for misspecified models. 4. Choosing inputs from unlabeled samples. 5. AL with model selection.

  13. 13 Correctly Specified Models � Assume that the target function is included in the model: � Learn the parameters by ordinary least-squares (OLS):

  14. 14 Properties of LS � OLS estimator is linear: Variance is � OLS estimator is unbiased: Bias is

  15. 15 AL for Correctly Specified Models � When OLS is used, � Thus Fedorov, Theory of Optimal Experiments , Academic Press, 1972.

  16. 16 Illustrative Examples � Learning target: � Model: � Test input density: � Training input density:

  17. 17 Obtained Generalization Error Mean ± Std (1000 trials) 1.45 ± 1.82 2.56 ± 2.24 113 ± 63.7 OLS-AL 3.10 ± 2.61 3.13 ± 2.61 5.75 ± 3.09 Passive � When model is correctly specified, OLS-AL works well. � Even when model is slightly misspecified, the performance degrades significantly. � When model is highly misspecified, the performance is very poor.

  18. 18 OLS-based AL: Summary � Pros: � Gen err estimation is exact. � Easy to implement. � Cons: � Correctly specified models are not available in practice. � Performance degradation for model misspecification is significant.

  19. 19 Organization of My Talk 1. Formulation. 2. AL for correctly specified models. 3. AL for misspecified models. 4. Choosing inputs from unlabeled samples. 5. AL with model selection.

  20. 20 Misspecified Models � Consider general cases where the target function is not included in the model: � However, if the model is completely misspecified, learning itself is meaningless (need model selection, discussed later) � Here we assume that the model is approximately correct.

  21. 21 Orthogonal Decomposition ( and are orthogonal) � Approximately correct model:

  22. 22 Further Decomposition of Bias � Bias: � Out-model bias: � In-model bias:

  23. 23 Difficulty of AL for Misspecified Models � Out-model bias remains, so bias cannot be zero. � Out-model bias is constant, so it can be ignored. � However, OLS does not reduce in-model bias to zero. � “Covariate shift” is the cause!

  24. 24 Covariate Shift � Training and test inputs follow different distributions: Covariate = Input � In AL, covariate shift always occurs! � Difference of input distributions causes OLS not to reduce in-model bias to zero. Shimodaira, Improving predictive inference under covariate shift by weighting the log-likelihood function, Journal of Statistical Planning and Inference , vol. 90, pp. 227-244, 2000.

  25. 25 Example of Covariate Shift Input densities Training samples Test samples

  26. 26 Bias of OLS under Covariate Shift � OLS: � Unbiased for correctly specified models. � For misspecified models, in-model bias remains even asymptotically.

  27. 27 The Law of Large Numbers � Sample average converges to the population mean: � We want to estimate the expectation over test distribution using training samples (following training distribution).

  28. 28 Importance-Weighted Average � Importance: the ratio of input densities � Importance-weighted average: (cf. importance sampling)

  29. 29 Importance-Weighted LS (WLS) � WLS: � Even for misspecified models, in-model bias vanishes asymptotically. � For approximately correct models, in-model bias is very small.

  30. 30 Importance-Weighted LS (WLS) � WLS is linear: � Thus variance is given by

  31. 31 AL for Approximately Correct Models using WLS � Use WLS for learning: Constant � Thus Sugiyama, Active learning in approximately linear regression based on conditional expectation of generalization error, Journal of Machine Learning Research , vol.7, pp.141-166, 2006.

  32. 32 Obtained Generalization Error Mean ± Std (1000 trials) T-test (95%) 2.07 ± 1.90 2.09 ± 1.90 4.28 ± 2.02 WLS-AL 1.45 ± 1.82 2.56 ± 2.24 113 ± 63.7 OLS-AL 3.10 ± 2.61 3.13 ± 2.61 5.75 ± 3.09 Passive � When model is exactly correct, OLS-AL works well. � However, when model is misspecified, it is totally unreliable. � WLS-AL works well even when model is misspecified.

  33. 33 Application to Robot Control � Golf robot: control the robot arm so that the ball is driven as far as possible. � State : joint angles, angular velocities � Action : torque to be applied to joints � We use reinforcement learning (RL). � In RL, reward (carry distance of the ball) is given to the robot. � Robot updates its control policy so that the maximum amount of rewards is obtained.

  34. 34 Policy Iteration � Value function : sum of rewards when taking action at state and then following policy . Gather samples Learn using current policy value function Update policies Sutton & Barto, Reinforcement Learning: An Introduction, MIT Press, 1998.

  35. 35 Covariate Shift in Policy Iteration Gather samples Learn using current policy value function Update policies � When policies are updated, the distribution of and changes. � Thus we need to use importance weighting for being consistent. Hachiya, Akiyama, Sugiyama & Peters. Adaptive importance sampling for value function approximation in off-policy reinforcement learning . Neural Networks , to appear

  36. 36 AL in Policy Iteration � Sampling cost is high in golf robot control (manually measuring carry distance is painful). Gather samples Learn using optimized policy value function Update policies Akiyama, Hachiya & Sugiyama. Active policy iteration, IJCAI2009 .

  37. 37 Experimental Results 70 Active 65 Performance(average) learning 60 Passive learning 55 50 45 Passive Learning 40 Active Learning 35 1 2 3 4 5 6 7 Iteration The difference of the performances at 7-th iteration is statistically significant by the t-test at the significance level 1%. � AL improves the performance!

  38. 38 Passive Learning

  39. 39 Active Learning

  40. 40 WLS-based AL: Summary � Pros: � Robust against model misspecification. � Easy to implement. � Cons: � Test input density could be unknown in practice.

  41. 41 Organization of My Talk 1. Formulation. 2. AL for correctly specified models. 3. AL for misspecified models. 4. Choosing inputs from unlabeled samples. 5. AL with model selection.

  42. 42 Pool-based AL: Setup � Test input density is unknown. � A pool of input samples following is available. � From the pool, we choose sample and gather output values .

  43. 43 Difficulty of Pool-based AL � in are unknown, so AL criterion cannot be directly computed.

  44. 44 Naïve Approach � Estimate test density from . � Plug-in the estimator : � However, density estimation is hard and thus this approach is not reliable.

Recommend

Agenda Intro to Active Learning Activity Design Resources for Active Learning Lunch with Active

Agenda Intro to Active Learning Activity Design Resources for Active Learning Lunch with Active

Agenda Intro to Active Learning Activity Design Resources for Active Learning Lunch with Active Learning Veterans Wrap Up Introduction to Active Learning What is active learning? Brainstorm your ideas with your group and generate a

855 views • 42 slides
Multi-Task Active Learning Yi Zhang Outline Active Learning Multi-Task Active Learning

Multi-Task Active Learning Yi Zhang Outline Active Learning Multi-Task Active Learning

Multi-Task Active Learning Yi Zhang Outline Active Learning Multi-Task Active Learning Linguistic Annotations (ACL 08) Image Classification (CVPR 08) Current Work and Discussions Constraint-Driven Active Learning

945 views • 47 slides
ACTIVE 1. WELCOME and JUMPING IN LEARNING 1. WHAT IS ACTIVE LEARNING? la Carte 1.

ACTIVE 1. WELCOME and JUMPING IN LEARNING 1. WHAT IS ACTIVE LEARNING? la Carte 1.

ACTIVE 1. WELCOME and JUMPING IN LEARNING 1. WHAT IS ACTIVE LEARNING? la Carte 1. CONSTRUCTING our TEACHING CHALLENGES in a low tech 1. ORCHESTRATION environment 1. SHOW-AND-TELL Selma Hamdani Psychology / DALC / Neuroscience

836 views • 50 slides
Active Learning Passive Learning Active Learning 1. Think 1. Acquisition of knowledge Ability

Active Learning Passive Learning Active Learning 1. Think 1. Acquisition of knowledge Ability

Active Learning Passive Learning Active Learning 1. Think 1. Acquisition of knowledge Ability 2. Decision making 2. Application of knowledge 3. Explain/communicate 1. Deep learning Approach to Uni-directional 2. Interactive Learning

282 views • 5 slides
Learning J.J. (Jia-Jie) Zhu Boston College GAAL 2 Active learning classify 400 instances 30

Learning J.J. (Jia-Jie) Zhu Boston College GAAL 2 Active learning classify 400 instances 30

Generative Adversarial Active Learning J.J. (Jia-Jie) Zhu Boston College GAAL 2 Active learning classify 400 instances 30 randomly selected 30 selected using AL using logistic regression 70% accuracy 90% accuracy pool-based active

801 views • 12 slides
12.1 Active Learning: A Review When learning, it may be the case that getting the true labels of

12.1 Active Learning: A Review When learning, it may be the case that getting the true labels of

CS/CNS/EE 253: Advanced Topics in Machine Learning Topic: Active Learning Review and Kernel Methods Lecturer: Andreas Krause Scribe: Jonathan Krause Date: Feb. 17, 2010 12.1 Active Learning: A Review When learning, it may be the case that getting

213 views • 5 slides
Predator and Prey: Active Learning Is Social Learning Kenneth Ronkowitz Active Learning

Predator and Prey: Active Learning Is Social Learning Kenneth Ronkowitz Active Learning

Predator and Prey: Active Learning Is Social Learning Kenneth Ronkowitz Active Learning Symposium, Rutgers University, May 15, 2018 Typically part of the curriculum in biology, phrenology, biology, anatomy, craniometry, physiognomy, crime

484 views • 14 slides
Learning Loss for Active Learning Rymarczyk D., Zieliski B., Tabor J., Sadowski M., Titov M.

Learning Loss for Active Learning Rymarczyk D., Zieliski B., Tabor J., Sadowski M., Titov M.

Learning Loss for Active Learning Rymarczyk D., Zieliski B., Tabor J., Sadowski M., Titov M. Agenda 1. Active Learning introduction 2. Base methods in AL for Deep Learning 3. Learning Loss for Active Learning 4. Our ideas for Active

802 views • 43 slides
10.1 Active Learning Review What questions should be asked in order to collect best data?

10.1 Active Learning Review What questions should be asked in order to collect best data?

CS/CNS/EE 253: Advanced Topics in Machine Learning Topic: Active Learning 2: the myopic version-space shrinking strategy Lecturer: Daniel Golovin Scribe: Pete Trautman Date: 8 February 2010 10.1 Active Learning Review What questions should be

264 views • 4 slides
Active Learning: Rethinking Our Teaching to Promote Deeper Learning Facilitated by Ken Silvestri,

Active Learning: Rethinking Our Teaching to Promote Deeper Learning Facilitated by Ken Silvestri,

Active Learning: Rethinking Our Teaching to Promote Deeper Learning Facilitated by Ken Silvestri, CFE Instructional Designer Adapted for Institute of Technology By Menaka Abraham OVERVIEW Why Active Learning What is Active Learning

542 views • 18 slides
Using active learning pedagogy to enable the Presentation by Donna Whitehead teaching of

Using active learning pedagogy to enable the Presentation by Donna Whitehead teaching of

Using active learning pedagogy to enable the Presentation by Donna Whitehead teaching of enterprise and entrepreneurial skills Active learning Active learning methodology has become a preferred way to change the traditional teacher

146 views • 11 slides
Using R for Active Learning and Self-assessment in an e-Learning Environment Arnau Mir, Margaret

Using R for Active Learning and Self-assessment in an e-Learning Environment Arnau Mir, Margaret

Using R for Active Learning and Self-assessment in an e-Learning Environment Arnau Mir, Margaret Mir-Juli and Monica J. Ruiz-Mir University of the Balearic Islands Palma de Mallorca, SPAIN Using R for Active Learning and Self-assessment

740 views • 12 slides
Active Learning with Active Learning with Model Selection Neil Rubens Sugiyama Lab / Tokyo

Active Learning with Active Learning with Model Selection Neil Rubens Sugiyama Lab / Tokyo

Active Learning with Active Learning with Model Selection Neil Rubens Sugiyama Lab / Tokyo Institute of Technology Active Learning (NLP Motivation) NLP (common scenario) Large amounts of unlabeled data Large amounts of unlabeled

706 views • 36 slides
Course outline Online decision making 1. Statistical active learning 2. Combinatorial

Course outline Online decision making 1. Statistical active learning 2. Combinatorial

Active Learning and Optimized Information Gathering Lecture 3 Reinforcement Learning CS 101.2 Andreas Krause Announcements Homework 1: out tomorrow Due Thu Jan 22 Project Proposal due Tue Jan 27 (start soon!) Office hours Come to

573 views • 23 slides
Approximating Learning Curves for Active-Learning-Driven Annotation Katrin T omanek and Udo Hahn

Approximating Learning Curves for Active-Learning-Driven Annotation Katrin T omanek and Udo Hahn

Approximating Learning Curves for Active-Learning-Driven Annotation Katrin T omanek and Udo Hahn Jena University Language and Information Engineering (JULIE) Lab Agenda Introduction to Active Learning Stopping Conditions

511 views • 19 slides
Active Learning Community Open House Undergraduate Academic Affairs Undergraduate Academic

Active Learning Community Open House Undergraduate Academic Affairs Undergraduate Academic

Active Learning Community Active Learning Community Open House Undergraduate Academic Affairs Undergraduate Academic Affairs Active Learning Community Rutgers Learning Centers Digital Classroom Services Mary Emenike Dave Wyrtzen Director of

705 views • 24 slides
Developing Active Learners Workshop 1 Paul Ellis Head of Teaching & Learning Strategy Sri

Developing Active Learners Workshop 1 Paul Ellis Head of Teaching & Learning Strategy Sri

Developing Active Learners Workshop 1 Paul Ellis Head of Teaching & Learning Strategy Sri Lanka, December 2014 Active learning What do you understand by the term active learning? What is it not? Traditional vs.

165 views • 15 slides
Learning Algorithms for Active Learning Plan Background Matching Networks

Learning Algorithms for Active Learning Plan Background Matching Networks

Learning Algorithms for Active Learning Plan Background Matching Networks Active Learning Model Applications: Omniglot and MovieLens Critique and discussion Background: Matching Networks (Vinyals et al. 2016)

241 views • 20 slides
By Sara Stolbach Advanced CLT, Spring 2007 Definition In Active Learning the user is given

By Sara Stolbach Advanced CLT, Spring 2007 Definition In Active Learning the user is given

By Sara Stolbach Advanced CLT, Spring 2007 Definition In Active Learning the user is given unlabelled examples where it is possible to get any label but it can be costly. Pool-Based active learning is when the General Learning Model

218 views • 18 slides
Introduction to Machine Learning Active Learning Barnabs Pczos 1 Credits Some of the

Introduction to Machine Learning Active Learning Barnabs Pczos 1 Credits Some of the

Introduction to Machine Learning Active Learning Barnabs Pczos 1 Credits Some of the slides are taken from Nina Balcan. 2 Classic Supervised Learning Paradigm is Insufficient Nowadays Modern applications: massive amounts of raw data.

1.17k views • 73 slides
Active learning Revising class materials based on formal and informal assessment of students

Active learning Revising class materials based on formal and informal assessment of students

Active learning Revising class materials based on formal and informal assessment of students learning Malgorzata Marciniak LaGuardia Community College CUNY IPDM, March 31, 2017 Abstract Active learning is described as a process where

247 views • 23 slides
Tools and Ideas for Engaging Students Through Active Learning Springboard to Active Schools is an

Tools and Ideas for Engaging Students Through Active Learning Springboard to Active Schools is an

Tools and Ideas for Engaging Students Through Active Learning Springboard to Active Schools is an initiative of the National Network of Public Health Institutes (NNPHI) October 15, 2020 and Health Resources in Action (HRiA) through Cooperative

450 views • 27 slides
STUDENT ENGAGEMENT AND ACTIVE LEARNING STRATEGIES Goal By the end of this session, you will

STUDENT ENGAGEMENT AND ACTIVE LEARNING STRATEGIES Goal By the end of this session, you will

STUDENT ENGAGEMENT AND ACTIVE LEARNING STRATEGIES Goal By the end of this session, you will have: Developed some ideas for active engagement in your classroom Planned the use of two collaboration tools in your classroom Active Learning

510 views • 15 slides
Batch Mode Active Learning and Its Application to Medical Image Classification ICML 2006 S. Hoi,

Batch Mode Active Learning and Its Application to Medical Image Classification ICML 2006 S. Hoi,

Introduction A Framework of Batch Mode Active learning Efficient Algorithms for Batch Mode Active Learning Experimental Result Conclusion Batch Mode Active Learning and Its Application to Medical Image Classification ICML 2006 S. Hoi, R.

574 views • 56 slides

More recommend