Learning Structural SVMs with Latent Variables Presented By- - PowerPoint PPT Presentation

Learning Structural SVMs with Latent Variables Presented By- – Subhabrata Debnath(Roll- 13111063) – Anjan Banerjee(Roll-13111008)

Basics Machine Learning Blood Sugar Hyper Pressure Tension 10 15 No 5 5 No 25 25 Yes 30 36 Yes Y-Values 40 35 30 25 20 15 10 5 0 0 5 10 15 20 25 30 35

Basics Machine Learning Support Vectors Y 1 w Y 2

Objective Function of SVM s.t.

Objective Function of SVM s.t.

Score Higher the value of W T x,higher the chance of belonging to this class w

Latent Structured SVM Final Objective Function: Non-Convex Objective Function Can be solved by CCCP

Soft-Margin SVM

Soft-Margin SVM s. t.

Soft-Margin SVM

Multiclass SVM w Y3 Y 1 w Y1 Y 2 Y 3 w 2 Y Predicted Class:

Multi-Class SVM s.t. here,

Multi-class SVM • What if we don’t want the same amount of margin for all the classes? • E.g.: Given age, sex of an user and the movie genre, predict the rating(1-5) that the user will give. • Highly Incorrect Class and Lesser Incorrect Class Actual Rating Predicted Loss Rating 5 4 Less 5 1 High

Multi-Class SVM s.t.

Multi-Class SVM s.t.

Structured SVM

Structured SVM

Structured SVM

Structured SVM

Structured SVM

Structured SVM Could have been solved using any convex solver The only problem is the number of classes, hence the number of constraints are exponentially large. e.g. Number of possible parse trees for a given sentence is exponential in the number of words.

Cutting Plane Method However, this method gives a solution of the given convex optimization problem with precision ε. Our Convex Cutting Plane Method W*, ξ i Objective Function

Latent Information Hidden Information present in the training set that can improve our learning Let us denote these hidden/latent information as h i . x i y i h i (given/observed) (hidden/unobserved)

Latent Information Noun Phrase Coreference Problem: Input x: Noun Phrases with edge features Labels y: Clusters Of Noun Phrases Latent Variable h: ‘Strong’ links as trees

Latent Information Noun Phrase Coreference Problem:

Latent Structured SVM Objective function:

Latent Structured SVM Objective function:

Latent Structured SVM Objective function:

Latent Structured SVM Objective function:

Latent Structured SVM Final Objective Function: Non-Convex Objective Function Can’t be solved using Cutting plane

Property Of Concave Property Of Convex Function Function f ( x2 ) >= f ( x1 ) + ( x2 - x1 ) * f’ f ( x1 ) <= f ( x2 ) + ( x1 – x2 ) * f’ ( x2 ) ( x1 ) ) 2 x f(x2) ( ’ f f f(x2) ) 1 x ( ’ f f(x1) f(x1) x1 x1 x2 x2 x

Concave-Convex Procedure

Concave-Convex Procedure

Concave-Convex Procedure

Concave-Convex Procedure Minimize the resulting sum Cutting plane W t+1 Algorithm Iterate till desired precision

Overview of the CCCP • Initialize w 0 repeat – Find h* using the w i – Obtain w i+1 by optimizing the convex function using cutting plane. – Set w i =w i+1 till objective function improves by at least ε

THANK YOU