A two-step method to incorporate task features spaces for large - PowerPoint PPT Presentation

KERMIT A two-step method for large output A two-step method to incorporate task features spaces for large output spaces Michiel Stock twitter: @michielstock Motivation Introductory Michiel Stock 1 , Tapio Pahikkala 2 , Antti Airola 2 , Bernard example De Baets 1 & Willem Waegeman 1 Relational learning Other applications Pairwise 1 KERMIT learning methods Department of Mathematical Modelling, Statistics and Bioinformatics Kronecker kernel Ghent University ridge regression Two-step kernel ridge regression 2 Department of Computer Science Computational University of Turku aspects Cross-validation Exact online NIPS: extreme classification workshop learning Take home December 12, 2015 messages

What will we read next? KERMIT A two-step method for large output spaces Alice Bob Cedric Daphne Michiel Stock twitter: @michielstock 5 4 Motivation Introductory 1 4 example Relational learning Other 4 applications Pairwise learning 2 1 methods Kronecker kernel ridge regression Two-step kernel 4 3 ridge regression Computational aspects Cross-validation Exact online learning Take home messages

What will we read next? KERMIT A two-step Social method for large output graph spaces Alice Bob Cedric Daphne Genre Michiel Stock twitter: @michielstock 1 1 0 1 5 4 Motivation Introductory 0 0 1 0 1 4 example Relational learning Other 1 1 0 0 4 applications Pairwise learning 0 0 1 0 2 1 methods Kronecker kernel ridge regression 0 1 0 1 Two-step kernel 4 3 ridge regression Computational aspects Cross-validation Exact online learning Take home messages

What will we read next? KERMIT A two-step Social method for large output graph spaces Alice Bob Cedric Daphne Genre Michiel Stock twitter: @michielstock 1 1 0 1 5 2.3 4 3.1 Motivation Introductory 0 0 1 0 1 4.5 1.3 4 example Relational learning Other 1 1 0 0 3.9 4 3.8 0.8 applications Pairwise learning 0 0 1 0 2 5.2 1 4.5 methods Kronecker kernel ridge regression 0 1 0 1 Two-step kernel 4 2.5 3 3.6 ridge regression Computational aspects Cross-validation Exact online learning Take home messages

What will we read next? KERMIT A two-step Social method for large output graph spaces Alice Bob Cedric Daphne Genre Michiel Stock twitter: @michielstock 1 1 0 1 5 2.3 4 3.1 Motivation Introductory 0 0 1 0 1 4.5 1.3 4 example Relational learning Other 1 1 0 0 3.9 4 3.8 0.8 applications Pairwise learning 0 0 1 0 2 5.2 1 4.5 methods Kronecker kernel ridge regression 0 1 0 1 Two-step kernel 4 2.5 3 3.6 ridge regression Computational aspects Cross-validation 1 1 0 1 Exact online 4.8 1.1 3.7 2.3 learning Take home messages

What will we read next? KERMIT A two-step Social method for large output graph spaces Alice Bob Cedric Daphne Eric Genre Michiel Stock twitter: @michielstock 1 1 0 1 5 2.3 4 3.1 2.3 Motivation Introductory 0 0 1 0 1 4.5 1.3 4 4.0 example Relational learning Other 1 1 0 0 3.9 4 3.8 0.8 1.7 applications Pairwise learning 0 0 1 0 2 5.2 1 4.5 4.8 methods Kronecker kernel ridge regression 0 1 0 1 Two-step kernel 4 2.5 3 3.6 2.9 ridge regression Computational aspects Cross-validation 1 1 0 1 Exact online 4.8 1.1 3.7 2.3 learning Take home messages

What will we read next? KERMIT A two-step Social method for large output graph spaces Alice Bob Cedric Daphne Eric Genre Michiel Stock twitter: @michielstock 1 1 0 1 5 2.3 4 3.1 2.3 Motivation Introductory 0 0 1 0 1 4.5 1.3 4 4.0 example Relational learning Other 1 1 0 0 3.9 4 3.8 0.8 1.7 applications Pairwise learning 0 0 1 0 2 5.2 1 4.5 4.8 methods Kronecker kernel ridge regression 0 1 0 1 Two-step kernel 4 2.5 3 3.6 2.9 ridge regression Computational aspects Cross-validation 1 1 0 1 Exact online 4.8 1.1 3.7 2.3 2.4 learning Take home messages

Learning relations KERMIT A two-step method for large output spaces Michiel Stock twitter: @michielstock In-sample Motivation instances Introductory example Relational learning Other Training applications Pairwise Setting A learning methods Out-of- Kronecker kernel Setting B ridge regression sample Two-step kernel instances ridge regression Setting C Computational aspects Setting D Cross-validation Exact online learning In-sample Out-of-sample Take home tasks tasks messages

Other cool applications: drug design KERMIT A two-step method for large output spaces Michiel Stock twitter: @michielstock Motivation Introductory example Relational learning Other applications Pairwise learning methods Kronecker kernel ridge regression Two-step kernel ridge regression Computational aspects Cross-validation Exact online learning Predicting interaction between proteins and small compounds Take home messages

Other cool applications: social network analysis KERMIT A two-step method for large output spaces Michiel Stock twitter: @michielstock Motivation Introductory example Relational learning Other applications Pairwise learning methods Kronecker kernel ridge regression Two-step kernel ridge regression Computational aspects Cross-validation Exact online learning Predicting links between people Take home messages

Other cool applications: food pairing KERMIT A two-step method for large output spaces Michiel Stock twitter: @michielstock Motivation Introductory example Relational learning Other applications Pairwise learning methods Kronecker kernel ridge regression Two-step kernel ridge regression Computational aspects Cross-validation Exact online learning Finding ingredients that pair well Take home messages

Learning with pairwise feature representations KERMIT A two-step method for Features Features large output spaces books readers Michiel Stock twitter: @michielstock Motivation Introductory example Relational learning Other applications Φ Pairwise Ψ learning methods Kronecker kernel d : instance (e.g. book) t : task (e.g. reader) ridge regression Two-step kernel ridge regression φ ( d ) : instance features ψ ( t ) : task features (e.g. Computational (e.g. genre) social network) aspects Cross-validation Exact online learning Take home messages

Learning with pairwise feature representations KERMIT A two-step method for Features Features large output spaces books readers Michiel Stock twitter: @michielstock Motivation ⊗ = Introductory example Relational learning Other applications Φ Pairwise Ψ Φ ⊗ Ψ learning methods Kronecker kernel d : instance (e.g. book) t : task (e.g. reader) ridge regression Two-step kernel ridge regression φ ( d ) : instance features ψ ( t ) : task features (e.g. Computational (e.g. genre) social network) aspects Cross-validation Exact online learning Take home messages

Learning with pairwise feature representations KERMIT A two-step method for Features Features large output spaces books readers Michiel Stock twitter: @michielstock Motivation ⊗ = Introductory example Relational learning Other applications Φ Pairwise Ψ Φ ⊗ Ψ learning methods Kronecker kernel d : instance (e.g. book) t : task (e.g. reader) ridge regression Two-step kernel ridge regression φ ( d ) : instance features ψ ( t ) : task features (e.g. Computational (e.g. genre) social network) aspects Cross-validation Exact online Pairwise prediction function: f ( d , t ) = w | ( φ ( d ) ⊗ ψ ( t )) learning Take home messages

Learning relations in two steps KERMIT A two-step method for large output In-sample Out-of-sample spaces tasks tasks Michiel Stock twitter: @michielstock 1 Build a ridge Motivation regression model to Introductory example generalize to new Relational In-sample Instance learning instances KRR instances Other applications 2 Build a ridge Pairwise learning regression model to methods Kronecker kernel generalize to new ridge regression Out-of- Two-step kernel ridge regression Virtual instances sample tasks instances Computational aspects Cross-validation Exact online learning Take home Task KRR messages

The two-step ridge regression KERMIT A two-step method for large output spaces Michiel Stock twitter: Prediction function: @michielstock Motivation f ( d , t ) = φ ( d ) | W ψ ( t ) Introductory example Relational Parameters can be found by solving: learning Other applications Pairwise Φ | Y Ψ = ( Φ | Φ + λ d I ) W ( Ψ | Ψ + λ t I ) learning methods Kronecker kernel ridge regression Two-step kernel ridge regression Computational aspects Cross-validation Exact online learning Take home messages

The two-step ridge regression KERMIT A two-step method for large output spaces Michiel Stock twitter: Prediction function: @michielstock Motivation f ( d , t ) = φ ( d ) | W ψ ( t ) Introductory example Relational Parameters can be found by solving: learning Other applications Pairwise Φ | Y Ψ = ( Φ | Φ + λ d I ) W ( Ψ | Ψ + λ t I ) learning methods Kronecker kernel Two hyperparameters: λ d and λ t ! ridge regression Two-step kernel ridge regression Computational aspects Cross-validation Exact online learning Take home messages