minimax angle learning for optimal treatment decision
play

Minimax-Angle Learning for Optimal Treatment Decision with - PowerPoint PPT Presentation

Sep 20, 2023 •233 likes •907 views

Minimax-Angle Learning for Optimal Treatment Decision with Heterogeneous Data Chengchun Shi Department of Statistics North Carolina State University Joint work with Wenbin Lu and Rui Song August 3, 2016 Chengchun Shi (NCSU) Minimax-Angle

  1. Minimax-Angle Learning for Optimal Treatment Decision with Heterogeneous Data Chengchun Shi Department of Statistics North Carolina State University Joint work with Wenbin Lu and Rui Song August 3, 2016 Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 1 / 26

  2. A few words on causal inference Data A : Treatment (0 or 1) X : Covariates Y : Observed outcome (usually the larger the better) Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 2 / 26

  3. A few words on causal inference Data A : Treatment (0 or 1) X : Covariates Y : Observed outcome (usually the larger the better) Y ∗ ( a ): Potential outcome a = 0 , 1 Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 2 / 26

  4. A few words on causal inference Data A : Treatment (0 or 1) X : Covariates Y : Observed outcome (usually the larger the better) Y ∗ ( a ): Potential outcome a = 0 , 1 Objective Identify the optimal regime d opt to reach the best clinical outcome Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 2 / 26

  5. A few words on causal inference Data A : Treatment (0 or 1) X : Covariates Y : Observed outcome (usually the larger the better) Y ∗ ( a ): Potential outcome a = 0 , 1 Objective Identify the optimal regime d opt to reach the best clinical outcome Maximize E Y ∗ ( d ) = E[ d ( X ) Y ∗ (1) + { 1 − d ( X ) } Y ∗ (0)] d : X → { 0 , 1 } . Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 2 / 26

  6. Q, Contrast and Value function Q ( x , a ) = E[ Y | X = x , A = a ], Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 3 / 26

  7. Q, Contrast and Value function Q ( x , a ) = E[ Y | X = x , A = a ], C ( x ) = Q ( x , 1) − Q ( x , 0), Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 3 / 26

  8. Q, Contrast and Value function Q ( x , a ) = E[ Y | X = x , A = a ], C ( x ) = Q ( x , 1) − Q ( x , 0), V ( d ) = E Y ∗ ( d ) = E[ d ( X ) Y ∗ (1) + { 1 − d ( X ) } Y ∗ (0)]. Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 3 / 26

  9. Q, Contrast and Value function Q ( x , a ) = E[ Y | X = x , A = a ], C ( x ) = Q ( x , 1) − Q ( x , 0), V ( d ) = E Y ∗ ( d ) = E[ d ( X ) Y ∗ (1) + { 1 − d ( X ) } Y ∗ (0)]. Optimal treatment regime SUTVA, no unmeasured confounders, positivity assumption optimal treatment regime d opt ( x ) = I ( C ( x ) > 0) . Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 3 / 26

  10. Q, Contrast and Value function Q ( x , a ) = E[ Y | X = x , A = a ], C ( x ) = Q ( x , 1) − Q ( x , 0), V ( d ) = E Y ∗ ( d ) = E[ d ( X ) Y ∗ (1) + { 1 − d ( X ) } Y ∗ (0)]. Optimal treatment regime SUTVA, no unmeasured confounders, positivity assumption optimal treatment regime d opt ( x ) = I ( C ( x ) > 0) . Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 3 / 26

  11. Heterogeneity Optimal treatment regime (OTR): captures patient’s heterogeneous response Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 4 / 26

  12. Heterogeneity Optimal treatment regime (OTR): captures patient’s heterogeneous response However, OTR may vary across patients. Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 4 / 26

  13. Heterogeneity Optimal treatment regime (OTR): captures patient’s heterogeneous response However, OTR may vary across patients. Data integration (Meta analysis) Results combined from different studies to identify similar patterns. Heterogeneity due to different populations of the data Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 4 / 26

  14. Heterogeneity Optimal treatment regime (OTR): captures patient’s heterogeneous response However, OTR may vary across patients. Data integration (Meta analysis) Results combined from different studies to identify similar patterns. Heterogeneity due to different populations of the data Examples Schizophrenia study: OTR varies across patients locations Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 4 / 26

  15. Heterogeneity Optimal treatment regime (OTR): captures patient’s heterogeneous response However, OTR may vary across patients. Data integration (Meta analysis) Results combined from different studies to identify similar patterns. Heterogeneity due to different populations of the data Examples Schizophrenia study: OTR varies across patients locations Health assessment questionnaire (HAQ) progression data: OTR varies across patients enrollment time Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 4 / 26

  16. Schizophrenia study A multi-center, randomized trial with an 18 months follow-up Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 5 / 26

  17. Schizophrenia study A multi-center, randomized trial with an 18 months follow-up Over 400 patients from three geographical locations (Manchester/Salford, Liverpool and North Nottinghamshire) Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 5 / 26

  18. Schizophrenia study A multi-center, randomized trial with an 18 months follow-up Over 400 patients from three geographical locations (Manchester/Salford, Liverpool and North Nottinghamshire) HAQ data An observational study which enrolled 847 patients enrolled from 1990 to 2000. Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 5 / 26

  19. Schizophrenia study A multi-center, randomized trial with an 18 months follow-up Over 400 patients from three geographical locations (Manchester/Salford, Liverpool and North Nottinghamshire) HAQ data An observational study which enrolled 847 patients enrolled from 1990 to 2000. Patients enrolled at different times showing heterogeneity; we considered three groups: 1990 - 1992 ( G = 1); 1993 - 1996 ( G = 2); 1997 - 2000 ( G = 3). Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 5 / 26

  20. How to recommend treatment rule for future patients? Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 6 / 26

  21. How to recommend treatment rule for future patients? Strategy 1: recommend OTR according to patients groups Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 6 / 26

  22. How to recommend treatment rule for future patients? Strategy 1: recommend OTR according to patients groups (What if the future patients don’t belong to any of current groups) Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 6 / 26

  23. How to recommend treatment rule for future patients? Strategy 1: recommend OTR according to patients groups (What if the future patients don’t belong to any of current groups) Strategy 2: combine the data together and obtain OTR based on the pooled data Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 6 / 26

  24. How to recommend treatment rule for future patients? Strategy 1: recommend OTR according to patients groups (What if the future patients don’t belong to any of current groups) Strategy 2: combine the data together and obtain OTR based on the pooled data (Doesn’t take population heterogeneity into account) Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 6 / 26

  25. How to recommend treatment rule for future patients? Strategy 1: recommend OTR according to patients groups (What if the future patients don’t belong to any of current groups) Strategy 2: combine the data together and obtain OTR based on the pooled data (Doesn’t take population heterogeneity into account) Our Strategy: focus on a single treatment regime Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 6 / 26

  26. How to recommend treatment rule for future patients? Strategy 1: recommend OTR according to patients groups (What if the future patients don’t belong to any of current groups) Strategy 2: combine the data together and obtain OTR based on the pooled data (Doesn’t take population heterogeneity into account) Our Strategy: focus on a single treatment regime that accounts for population heterogeneities. Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 6 / 26

  27. Models G different population groups: Y gj = h g ( X gj ) + A gj ψ g ( X T gj β g ) + ε gj || β g || 2 = 1 , g = 1 , . . . , G , j = 1 , . . . , m h g arbitrary baseline function ψ g arbitrary monotone function X gj mean 0, covariance matrix I . Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 7 / 26

  28. Models G different population groups: Y gj = h g ( X gj ) + A gj ψ g ( X T gj β g ) + ε gj || β g || 2 = 1 , g = 1 , . . . , G , j = 1 , . . . , m h g arbitrary baseline function ψ g arbitrary monotone function X gj mean 0, covariance matrix I . Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 7 / 26

  29. Objective Group-wise optimal regime: I ( X T 0 β g > ψ − 1 g (0)) Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 8 / 26

  30. Objective Group-wise optimal regime: I ( X T 0 β g > ψ − 1 g (0)) Overall decision: I ( X T 0 β 0 > c 0 ) subject to || β 0 || 2 = 1 Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 8 / 26

  31. Objective Group-wise optimal regime: I ( X T 0 β g > ψ − 1 g (0)) Overall decision: I ( X T 0 β 0 > c 0 ) subject to || β 0 || 2 = 1 Two steps strategy: Step 1: Fix c 0 , search for some β 0 ( c 0 ) achieves some “optimality” Chengchun Shi (NCSU) Minimax-Angle Learning August 3, 2016 8 / 26

Recommend

On the construction of minimax-distance (sub-)optimal designs Luc Pronzato Universit Cte

On the construction of minimax-distance (sub-)optimal designs Luc Pronzato Universit Cte

On the construction of minimax-distance (sub-)optimal designs Luc Pronzato Universit Cte dAzur, CNRS, I3S, France 1) Introduction 1) Introduction & motivation Objective: Approximation/interpolation of a function f : x X R d

1.14k views • 87 slides
Minimax Statistical Learning with Wasserstein distances Jaeho Lee & Maxim Raginsky NeurIPS

Minimax Statistical Learning with Wasserstein distances Jaeho Lee & Maxim Raginsky NeurIPS

Minimax Statistical Learning with Wasserstein distances Jaeho Lee & Maxim Raginsky NeurIPS 2018 Poster #86 Minimax learning Goal: find the hypothesis minimizing the worst-case risk is an ambiguity set representing

272 views • 10 slides
max ( | ) ( ) P s a U s preferences, must exist consistent utility function a s

max ( | ) ( ) P s a U s preferences, must exist consistent utility function a s

The Winding Path to RL Markov Decision Processes & Decision Theory Descriptive theory of optimal Reinforcement Learning behavior Markov Decision Processes Mathematical/Algorithmic realization of Decision Theory

86 views • 8 slides
Optimal Learning Rate What is the optimal value opt of the learning rate? Consider 1 -dim.

Optimal Learning Rate What is the optimal value opt of the learning rate? Consider 1 -dim.

Optimal Learning Rate What is the optimal value opt of the learning rate? Consider 1 -dim. case. Use first-order Taylor expansion around current weight w c E ( w ) = E ( w c ) + ( w w c ) E ( w c ) . w Differentiating both sides

420 views • 24 slides
Module 15 POMDP Bounds CS 886 Sequential Decision Making and Reinforcement Learning University

Module 15 POMDP Bounds CS 886 Sequential Decision Making and Reinforcement Learning University

Module 15 POMDP Bounds CS 886 Sequential Decision Making and Reinforcement Learning University of Waterloo Bounds POMDP algorithms typically find approximations to optimal value function or optimal policy Need some performance

402 views • 12 slides
The Nearest Neighbor Information Estimator is Adaptively Near Minimax Rate-Optimal Jiantao Jiao

The Nearest Neighbor Information Estimator is Adaptively Near Minimax Rate-Optimal Jiantao Jiao

The Nearest Neighbor Information Estimator is Adaptively Near Minimax Rate-Optimal Jiantao Jiao Berkeley EECS Weihao Gao UIUC ECE Yanjun Han Stanford EE NIPS 2018, Montr eal, Canada The Nearest Neighbor Information Estimator is

395 views • 23 slides
Right Angle: An angle whose measure is 90. Straight Angle: An angle whose measure is 180.

Right Angle: An angle whose measure is 90. Straight Angle: An angle whose measure is 180.

Connect Proofs to Section 2-4: Special Pairs of Angles Right Angle: An angle whose measure is 90. Straight Angle: An angle whose measure is 180. Complementary Angles: Two angles whose measures sum to 90. Supplementary Angles: Two angles

278 views • 8 slides
Stochastic Optimal Control part 2 discrete time, Markov Decision Processes, Reinforcement

Stochastic Optimal Control part 2 discrete time, Markov Decision Processes, Reinforcement

Stochastic Optimal Control part 2 discrete time, Markov Decision Processes, Reinforcement Learning Marc Toussaint Machine Learning & Robotics Group TU Berlin mtoussai@cs.tu-berlin.de ICML 2008, Helsinki, July 5th, 2008 Why

383 views • 25 slides
CMU 15-896 Noncooperative games 2: Learning and minimax Teacher: Ariel Procaccia Reminder: The

CMU 15-896 Noncooperative games 2: Learning and minimax Teacher: Ariel Procaccia Reminder: The

CMU 15-896 Noncooperative games 2: Learning and minimax Teacher: Ariel Procaccia Reminder: The Minimax Theorem Theorem [von Neumann, 1928]: Every 2-player zero-sum game has a unique value such that: Player 1 can guarantee value at o

566 views • 22 slides
Phase Angle What is it & why is it Important? Phase Angle Basics What is Phase Angle and

Phase Angle What is it & why is it Important? Phase Angle Basics What is Phase Angle and

RJL Systems is the Industry Innovator and First Company to introduce Bio-electrical Impedance Analysis (BIA) to the market in 1979. Phase Angle What is it & why is it Important? Phase Angle Basics What is Phase Angle and Why is it

289 views • 10 slides
AIXI: Universal Optimal Sequential Decision Making Marcus Hutter (2005) Reinforcement Learning

AIXI: Universal Optimal Sequential Decision Making Marcus Hutter (2005) Reinforcement Learning

AIXI: Universal Optimal Sequential Decision Making Marcus Hutter (2005) Reinforcement Learning State space , Action space , Policy , Reward (, ) Goal: Find policy which maximizes expected cumulative reward.

552 views • 27 slides
Development of decision support system for optimal agricultural system for optimal agricultural

Development of decision support system for optimal agricultural system for optimal agricultural

APAN Geo ICT and Sensor Network based Decision Support Geo-ICT and Sensor Network based Decision Support Systems in Agriculture and Environment Assessmen 2011.8.24 Development of decision support system for optimal agricultural system for

429 views • 13 slides
Bayesian decision theory Andrea Passerini passerini@disi.unitn.it Machine Learning Bayesian

Bayesian decision theory Andrea Passerini passerini@disi.unitn.it Machine Learning Bayesian

Bayesian decision theory Andrea Passerini passerini@disi.unitn.it Machine Learning Bayesian decision theory Introduction Overview Bayesian decision theory allows to take optimal decisions in a fully probabilistic setting It assumes all

331 views • 29 slides
Sample mple-Opt Optimal imal Pa Para rametric metric Q-Le Learning arning Usi Using ng

Sample mple-Opt Optimal imal Pa Para rametric metric Q-Le Learning arning Usi Using ng

Sample mple-Opt Optimal imal Pa Para rametric metric Q-Le Learning arning Usi Using ng Li Line nearly arly Ad Additive ditive Fea eatur tures es Lin in F. Yan ang, , Meng ngdi di Wan ang A Basic RL Model: Markov Decision

380 views • 8 slides
Infotheory for Statistics and Learning Lecture 4 Binary hypothesis testing The

Infotheory for Statistics and Learning Lecture 4 Binary hypothesis testing The

Infotheory for Statistics and Learning Lecture 4 Binary hypothesis testing The NeymanPearson lemma Minimum P e test and total variation General theory Bayes and minimax The minimax theorem Mikael Skoglund 1/15 Binary

369 views • 8 slides
Policy Certificates: Towards Accountable Reinforcement Learning Christoph Dann, Lihong Li,

Policy Certificates: Towards Accountable Reinforcement Learning Christoph Dann, Lihong Li,

Policy Certificates: Towards Accountable Reinforcement Learning Christoph Dann, Lihong Li, Wei Wei, Emma Brunskill CMU Google Research Stanford University Minimax-Optimal PAC

489 views • 12 slides
Complex learning example: curve fitting t = sin(2 x ) + noise t n t 1 y ( x n , w ) 0 1

Complex learning example: curve fitting t = sin(2 x ) + noise t n t 1 y ( x n , w ) 0 1

Artificial Intelligence: Representation and Problem Solving 15-381 April 10, 2007 Introduction to Learning & Decision Trees Learning and Decision Trees to learning aka: regression What is learning? pattern recognition

462 views • 17 slides
Towards Principled Methodologies and Efficient Algorithms for Minimax Machine Learning Tuo Zhao

Towards Principled Methodologies and Efficient Algorithms for Minimax Machine Learning Tuo Zhao

Towards Principled Methodologies and Efficient Algorithms for Minimax Machine Learning Tuo Zhao Georgia Tech, Jun. 26. 2019 Joint work with Haoming Jiang, Minshuo Chen (Georgia Tech), Bo Dai (Google Brain), Zhaoran Wang (Northwestern U) and

994 views • 66 slides
Optimal Decision Making with Limited, Imperfect Information Thomas Adams and Carolyn Atterbury

Optimal Decision Making with Limited, Imperfect Information Thomas Adams and Carolyn Atterbury

Optimal Decision Making with Limited, Imperfect Information Thomas Adams and Carolyn Atterbury Department of Computer Science University of New Mexico 26 February 2019 Decision Problems in Nature: Working with Unclear Data Animals in the

368 views • 26 slides
Decision Tree R Greiner Cmput 466 / 551 Learning Decision Trees Def'n: Decision Trees

Decision Tree R Greiner Cmput 466 / 551 Learning Decision Trees Def'n: Decision Trees

HTF: 9.2 B: 14.4 R N , C h a p t e r 1 8 1 8 . 3 Decision Tree R Greiner Cmput 466 / 551 Learning Decision Trees Def'n: Decision Trees Algorithm for Learning Decision Trees Entropy, Inductive Bias (Occam's

1.43k views • 87 slides
4. Minimax and planning problems Optimizing piecewise linear functions Minimax problems

4. Minimax and planning problems Optimizing piecewise linear functions Minimax problems

CS/ECE/ISyE 524 Introduction to Optimization Spring 201718 4. Minimax and planning problems Optimizing piecewise linear functions Minimax problems Example: Chebyshev center Multi-period planning problems Example: building a

411 views • 25 slides
Theoretical Analysis of Adversarial Learning: A Minimax Approach Zhuozhuo Tu 1 , Jingwei Zhang 2,1

Theoretical Analysis of Adversarial Learning: A Minimax Approach Zhuozhuo Tu 1 , Jingwei Zhang 2,1

Theoretical Analysis of Adversarial Learning: A Minimax Approach Zhuozhuo Tu 1 , Jingwei Zhang 2,1 , Dacheng Tao 1 1 The University of Sydney 2 The Hong Kong University of Science and Technology NeurIPS 2019

469 views • 17 slides
3D orientation Rotation matrix Fixed angle and Euler angle Axis angle

3D orientation Rotation matrix Fixed angle and Euler angle Axis angle

3D orientation Rotation matrix Fixed angle and Euler angle Axis angle Quaternion Exponential map Joints and rotations Rotational DOFs are widely used in character animation 3 translational DOFs 48 rotational DOFs Each

570 views • 42 slides
3D orientation Rotation matrix Fixed angle and Euler angle Axis angle

3D orientation Rotation matrix Fixed angle and Euler angle Axis angle

3D orientation Rotation matrix Fixed angle and Euler angle Axis angle Quaternion Exponential map Joints and rotations Rotational DOFs are widely used in character animation 3 translational DOFs 48 rotational DOFs Each

1.04k views • 45 slides

More recommend