Regularization in Directable Environments with Application to - PowerPoint PPT Presentation

Regularization in Directable Environments 
 with Application to Tetris Jan Malte Lichtenberg Özgür Ş im ş ek

Shrinkage Toward Equal Weights (STEW) Lichtenberg J.M. and Ş im ş ek Ö. Regularization in Directable Environments with Application to Tetris.

Shrinkage Toward Equal Weights (STEW) 0.3 Weight estimates 0.2 0.1 0.0 10 − 4 10 − 2 10 0 10 2 10 4 λ ( log scale ) STEW (q = 2) Lichtenberg J.M. and Ş im ş ek Ö. Regularization in Directable Environments with Application to Tetris.

Shrinkage Toward Equal Weights (STEW) Equal Weights 0.3 Weight estimates 0.2 0.1 0.0 10 − 4 10 − 2 10 0 10 2 10 4 λ ( log scale ) STEW (q = 2) Lichtenberg J.M. and Ş im ş ek Ö. Regularization in Directable Environments with Application to Tetris.

Shrinkage Toward Equal Weights (STEW) Equal Weights 0.3 0.3 Weight estimates Weight estimates 0.2 0.2 0.1 0.1 0.0 0.0 10 − 3 10 − 2 10 − 1 10 0 10 − 4 10 − 2 10 0 10 2 10 4 10 1 10 2 λ ( log scale ) λ ( log scale ) Ridge regression : STEW (q = 2) Lichtenberg J.M. and Ş im ş ek Ö. Regularization in Directable Environments with Application to Tetris.

Lichtenberg J.M. and Ş im ş ek Ö. Regularization in Directable Environments with Application to Tetris.

“1 / N rule“ (DeMiguel et al., 2009) Lichtenberg J.M. and Ş im ş ek Ö. Regularization in Directable Environments with Application to Tetris.

Lichtenberg J.M. and Ş im ş ek Ö. Regularization in Directable Environments with Application to Tetris.

Are feature directions known? Lichtenberg J.M. and Ş im ş ek Ö. Regularization in Directable Environments with Application to Tetris.

Are feature directions known? “Direct” all features Lichtenberg J.M. and Ş im ş ek Ö. Regularization in Directable Environments with Application to Tetris.

Are feature directions known? “Direct” all features Lichtenberg J.M. and Ş im ş ek Ö. Regularization in Directable Environments with Application to Tetris.

Are feature directions known? “Direct” all features Lichtenberg J.M. and Ş im ş ek Ö. Regularization in Directable Environments with Application to Tetris.

1 Density 0 − 2 − 1 0 1 2 β Lichtenberg J.M. and Ş im ş ek Ö. Regularization in Directable Environments with Application to Tetris.

1 Density 0 − 2 − 1 0 1 2 β EW 25 Ridge ● 20 Lasso ● NNLasso ● 15 STEW MSE 10 ● ● 5 ● ● ● 0 15 20 30 50 100 200 Training set size (log scale) Lichtenberg J.M. and Ş im ş ek Ö. Regularization in Directable Environments with Application to Tetris.

Environment less directable 1 1 1 Density Density Density 0 0 0 − 2 − 1 0 1 2 − 2 − 1 0 1 2 − 2 − 1 0 1 2 β β β EW 25 10.0 8 Ridge ● 20 Lasso ● ● ● NNLasso 7.5 6 ● 15 STEW ● MSE MSE MSE ● 5.0 4 10 ● ● ● ● ● ● 5 2.5 2 ● ● ● ● ● ● ● ● ● 0 15 20 30 50 100 200 15 20 30 50 100 200 15 20 30 50 100 200 Training set size (log scale) Training set size (log scale) Training set size (log scale) Lichtenberg J.M. and Ş im ş ek Ö. Regularization in Directable Environments with Application to Tetris.

1 Density 0 − 2 0 2 β Ridge Lasso ● 15 NNLasso ● STEW 10 MSE ● ● 5 ● ● ● ● 15 20 30 50 100 200 Training set size (log scale) Lichtenberg J.M. and Ş im ş ek Ö. Regularization in Directable Environments with Application to Tetris.

1 Density 0 − 2 0 2 β 15 Ridge Sq. Bias Ridge Lasso ● Sq. Bias STEW 15 NNLasso Variance Ridge Error component ● STEW Variance STEW 10 10 MSE ● 5 ● 5 ● ● ● 0 ● 15 20 30 50 100 200 15 20 30 50 100 200 Training set size (log scale) Training set size (log scale) Lichtenberg J.M. and Ş im ş ek Ö. Regularization in Directable Environments with Application to Tetris.

Tetris 5000 M − learning + STEW M − learning + NN M − learning + Ridge 4000 M − learning Equal Weights Mean score 3000 2000 1000 0 3 5 10 20 50 100 200 300 Iteration (log scale) Poster #137 @ Pacific Ballroom Lichtenberg J.M. and Ş im ş ek Ö. Regularization in Directable Environments with Application to Tetris.