Diagnostics & Kernel Methods Visualized Ondej Bojar April 3, - PowerPoint PPT Presentation

RBF Kernels … Totally “fmips” the space: 69/110 𝑙( x , y ) = 𝑓𝑦𝑞(−𝛿‖ x − y ‖ 2 ); 𝛿 > 0 1 exp(-0.5*x*x) 0.9 exp(-1*x*x) exp(-2*x*x) 0.8 exp(-3*x*x) 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0 -10 -5 0 5 10 • Each training point creates its bell. • Overall shape is the sum of the bells. • Kind of “all nearest neighbours”. • The axes are now closeness to other objects.

RBF Kernel Parameters Peaked must be close to training examples High can be far from training examples Low Afgected Points gamma High 𝐷 Complex Low High Low High Simple Smooth Low Variance Bias Model Decision Surface 70/110 • Does higher gamma lead to higher variance? • Choice critical for SVM performance. • Advised to use GridSearchCV for 𝐷 and gamma: • exponentially spaced probes • wide range

SVM RBF (C=0.05, gamma=2) 71/110

SVM RBF (C=0.1, gamma=2) 72/110

SVM RBF (C=0.2, gamma=2) 73/110

SVM RBF (C=0.5, gamma=2) 74/110

SVM RBF (C=0.6, gamma=2) 75/110

SVM RBF (C=0.7, gamma=2) 76/110

SVM RBF (C=1, gamma=2) 77/110

SVM RBF (C=2, gamma=2) 78/110

SVM RBF (C=1, gamma=2) 79/110

SVM RBF (C=0.5, gamma=2) 80/110

SVM RBF (C=0.5, gamma=5) 81/110

SVM RBF (C=0.5, gamma=10) 82/110

SVM RBF (C=0.5, gamma=5) 83/110

SVM RBF (C=0.5, gamma=2) 84/110

SVM RBF (C=0.5, gamma=1) 85/110

SVM RBF (C=0.5, gamma=0.7) 86/110

SVM RBF (C=0.5, gamma=0.5) 87/110

SVM RBF (C=0.5, gamma=0.2) 88/110

SVM RBF (C=0.5, gamma=0.1) 89/110

SVM RBF (C=0.5, gamma=0.05) 90/110

Cross-validation Heatmap http: //scikit-learn.org/stable/auto_examples/svm/plot_rbf_parameters.html 91/110

Multi-class SVM Two implementations in scikit-learn: 92/110 • SVC: one-against-one • 𝑜(𝑜 − 1)/2 classifjers constructed • supports various kernels, incl. custom ones • LinearSVC: one-vs-the-rest • 𝑜 classifjers trained

PAMAP-easy Training Data 93/110 34 1 12 13 16 33.5 17 2 24 3 33 4 5 6 7 32.5 32 31.5 31 30.5 30 60 80 100 120 140 160 180 200

Default View (every 200) 94/110

Default View (every 300) 95/110

Default View (every 400) 96/110

Regularization C=0.5 97/110