Support Vector Machines Alex Leblang and Sam Birch ML Framework - PowerPoint PPT Presentation

Support Vector Machines Alex Leblang and Sam Birch

ML Framework Data projected into feature space, each feature is a dimension. Via Wikipedia, scikit-learn examples

SVMs ● Supervised learning model: train / test ● Binary, discriminative classifier ○ Models the boundary, not the data ● Linear classification boundary defined with support vectors ● Non-linear decision boundaries can be introduced via the kernel trick

Where SVMs fit (Hays)

Linear separators (Hays)

Non-linearity (Andrew Moore via Hays)

Kernel Trick

Demo MLDemo

Challenges ● Non-linear classification takes > O(n) time ○ Have to compute entire kernel matrix ○ Most large-scale classification tasks use linear SVMs ● Validation ● Feature choice is challenging in many domains

LibSVM An efficient implementation of SVM routines Functions in LibSVM: ● C-Support vector classification ● V-Support vector classification ● Distribution Estimation ● v-Support vector regression

Optimization Moving training and classification to the GPU can provide 1-2 orders of magnitude performance advantage “Fast Support Vector Machine Training and Classification on Graphics Processors” (Catanzaro, Sundaram, Keutzer)

Conclusion ● Solid go-to classification algorithm ● Features > specifics ● Tuning and validation make a big difference