From ranking to intransitive preference learning: - PowerPoint PPT Presentation

From ranking to intransitive preference learning: rock-paper-scissors and beyond Tapio Pahikkala 1 Willem Waegeman 2 Evgeni Tsivtsivadze 3 Tapio Salakoski 1 Bernard De Baets 2 1 TUCS, University of Turku, Finland 2 KERMIT, Ghent University, Belgium 3 Institute for Computing and Information Sciences Radboud University Nijmegen, The Netherlands Pahikkala et al. (TUCS, UGent) Intransitive Preference Learning September, 11 2009 1 / 25

Introduction Outline 1 Introduction Stochastic transitivity and ranking representability 2 Learning intransitive reciprocal relations 3 Experiments 4 Pahikkala et al. (TUCS, UGent) Intransitive Preference Learning September, 11 2009 2 / 25

Introduction The transitivity property: a classical example Pahikkala et al. (TUCS, UGent) Intransitive Preference Learning September, 11 2009 3 / 25

Introduction Examples of intransitivity are found in many fields... Pahikkala et al. (TUCS, UGent) Intransitive Preference Learning September, 11 2009 4 / 25

Stochastic transitivity and ranking representability Outline 1 Introduction Stochastic transitivity and ranking representability 2 Learning intransitive reciprocal relations 3 Experiments 4 Pahikkala et al. (TUCS, UGent) Intransitive Preference Learning September, 11 2009 5 / 25

Stochastic transitivity and ranking representability Q ( x , x ′ ) = 5 / 9 Q ( x ′ , x ′′ ) = 5 / 9 Q ( x ′′ , x ) = 5 / 9 Q ( x ′ , x ) = 4 / 9 Q ( x ′′ , x ′ ) = 4 / 9 Q ( x , x ′′ ) = 4 / 9 Proposition A relation Q : X 2 → [ 0 , 1 ] is called a reciprocal relation if ∀ ( x , x ′ ) ∈ X 2 . Q ( x , x ′ ) + Q ( x ′ , x ) = 1 Pahikkala et al. (TUCS, UGent) Intransitive Preference Learning September, 11 2009 6 / 25

Stochastic transitivity and ranking representability Q ( x , x ′ ) = 5 / 9 Q ( x ′ , x ′′ ) = 5 / 9 Q ( x ′′ , x ) = 5 / 9 Q ( x ′ , x ) = 4 / 9 Q ( x ′′ , x ′ ) = 4 / 9 Q ( x , x ′′ ) = 4 / 9 Proposition A relation Q : X 2 → [ 0 , 1 ] is called a reciprocal relation if ∀ ( x , x ′ ) ∈ X 2 . Q ( x , x ′ ) + Q ( x ′ , x ) = 1 Pahikkala et al. (TUCS, UGent) Intransitive Preference Learning September, 11 2009 6 / 25

Stochastic transitivity and ranking representability Q ( x , x ′ ) = 5 / 9 Q ( x ′ , x ′′ ) = 5 / 9 Q ( x ′′ , x ) = 5 / 9 Q ( x ′ , x ) = 4 / 9 Q ( x ′′ , x ′ ) = 4 / 9 Q ( x , x ′′ ) = 4 / 9 Proposition A relation Q : X 2 → [ 0 , 1 ] is called a reciprocal relation if ∀ ( x , x ′ ) ∈ X 2 . Q ( x , x ′ ) + Q ( x ′ , x ) = 1 Pahikkala et al. (TUCS, UGent) Intransitive Preference Learning September, 11 2009 6 / 25

Stochastic transitivity and ranking representability Q ( x , x ′ ) = 5 / 9 Q ( x ′ , x ′′ ) = 5 / 9 Q ( x ′′ , x ) = 5 / 9 Q ( x ′ , x ) = 4 / 9 Q ( x ′′ , x ′ ) = 4 / 9 Q ( x , x ′′ ) = 4 / 9 Proposition A relation Q : X 2 → [ 0 , 1 ] is called a reciprocal relation if ∀ ( x , x ′ ) ∈ X 2 . Q ( x , x ′ ) + Q ( x ′ , x ) = 1 Pahikkala et al. (TUCS, UGent) Intransitive Preference Learning September, 11 2009 6 / 25

Stochastic transitivity and ranking representability Ranking representability Definition A reciprocal relation Q : X 2 → [ 0 , 1 ] is called weakly ranking representable if there exists a ranking function f : X → R such that for any ( x , x ′ ) ∈ X 2 it holds that Q ( x , x ′ ) ≤ 1 2 ⇔ f ( x ) ≤ f ( x ′ ) . Q ( x , x ′ ) = 5 / 9 ⇔ x ≻ x ′ Q ( x ′ , x ′′ ) = 5 / 9 ⇔ x ′ ≻ x ′′ Q ( x ′′ , x ) = 5 / 9 ⇔ x ′′ ≻ x Pahikkala et al. (TUCS, UGent) Intransitive Preference Learning September, 11 2009 7 / 25

Stochastic transitivity and ranking representability Ranking representability Definition A reciprocal relation Q : X 2 → [ 0 , 1 ] is called weakly ranking representable if there exists a ranking function f : X → R such that for any ( x , x ′ ) ∈ X 2 it holds that Q ( x , x ′ ) ≤ 1 2 ⇔ f ( x ) ≤ f ( x ′ ) . Q ( x , x ′ ) = 5 / 9 ⇔ x ≻ x ′ Q ( x ′ , x ′′ ) = 5 / 9 ⇔ x ′ ≻ x ′′ Q ( x ′′ , x ) = 5 / 9 ⇔ x ′′ ≻ x Pahikkala et al. (TUCS, UGent) Intransitive Preference Learning September, 11 2009 7 / 25

Stochastic transitivity and ranking representability Weak stochastic transitivity Proposition (Luce and Suppes, 1965) A reciprocal relation Q is weakly ranking representable if and only if it satisfies weak stochastic transitivity, i.e., for any ( x , x ′ , x ′′ ) ∈ X 3 it holds that Q ( x , x ′ ) ≥ 1 / 2 ∧ Q ( x ′ , x ′′ ) ≥ 1 / 2 ⇒ Q ( x , x ′′ ) ≥ 1 / 2 . Q ( x , x ′ ) = 6 / 9 Q ( x ′ , x ′′ ) = 5 / 9 Q ( x ′′ , x ) = 2 / 9 ⇔ x ≻ x ′ ≻ x ′′ Pahikkala et al. (TUCS, UGent) Intransitive Preference Learning September, 11 2009 8 / 25

Stochastic transitivity and ranking representability Weak stochastic transitivity Proposition (Luce and Suppes, 1965) A reciprocal relation Q is weakly ranking representable if and only if it satisfies weak stochastic transitivity, i.e., for any ( x , x ′ , x ′′ ) ∈ X 3 it holds that Q ( x , x ′ ) ≥ 1 / 2 ∧ Q ( x ′ , x ′′ ) ≥ 1 / 2 ⇒ Q ( x , x ′′ ) ≥ 1 / 2 . Q ( x , x ′ ) = 6 / 9 Q ( x ′ , x ′′ ) = 5 / 9 Q ( x ′′ , x ) = 2 / 9 ⇔ x ≻ x ′ ≻ x ′′ Pahikkala et al. (TUCS, UGent) Intransitive Preference Learning September, 11 2009 8 / 25

Learning intransitive reciprocal relations Outline 1 Introduction Stochastic transitivity and ranking representability 2 Learning intransitive reciprocal relations 3 Experiments 4 Pahikkala et al. (TUCS, UGent) Intransitive Preference Learning September, 11 2009 9 / 25

Learning intransitive reciprocal relations Definition of our framework Training data E = ( e i , y i ) N i = 1 Training data are here couples: e = ( x , x ′ ) Labels y i = 2 Q ( x i , x ′ i ) + 1 Minimizing the regularized empirical error: N 1 � L ( h ( e i ) , y i ) + λ � h � 2 A ( E ) = argmin F N h ∈F i = 1 Least-squares loss function: regularized least-squares Pahikkala et al. (TUCS, UGent) Intransitive Preference Learning September, 11 2009 10 / 25

Learning intransitive reciprocal relations Reciprocal relations are learned by defining a specific kernel construction Consider the following joint feature representation for a couple: Φ( e i ) = Φ( x i , x ′ i ) = Ψ( x i , x ′ i ) − Ψ( x ′ i , x i ) , This yields the following kernel defined on couples: K Φ ( e i , e j ) K Φ ( x i , x ′ i , x j , x ′ = j ) � Ψ( x i , x ′ i ) − Ψ( x ′ i , x i ) , Ψ( x j , x ′ j ) − Ψ( x ′ = j , x j ) � K Ψ ( x i , x ′ j ) + K Ψ ( x ′ i , x j , x ′ i , x i , x ′ = j , x j ) − K Ψ ( x ′ i , x i , x j , x ′ j ) − K Ψ ( x i , x ′ i , x ′ j , x j ) . And the model becomes: N � h ( x , x ′ ) = � w , Ψ( x , x ′ ) − Ψ( x ′ , x ) � = a i K Φ ( x i , x ′ i , x , x ′ ) . i = 1 Pahikkala et al. (TUCS, UGent) Intransitive Preference Learning September, 11 2009 11 / 25

Learning intransitive reciprocal relations Reciprocal relations are learned by defining a specific kernel construction Consider the following joint feature representation for a couple: Φ( e i ) = Φ( x i , x ′ i ) = Ψ( x i , x ′ i ) − Ψ( x ′ i , x i ) , This yields the following kernel defined on couples: K Φ ( e i , e j ) K Φ ( x i , x ′ i , x j , x ′ = j ) � Ψ( x i , x ′ i ) − Ψ( x ′ i , x i ) , Ψ( x j , x ′ j ) − Ψ( x ′ = j , x j ) � K Ψ ( x i , x ′ j ) + K Ψ ( x ′ i , x j , x ′ i , x i , x ′ = j , x j ) − K Ψ ( x ′ i , x i , x j , x ′ j ) − K Ψ ( x i , x ′ i , x ′ j , x j ) . And the model becomes: N � h ( x , x ′ ) = � w , Ψ( x , x ′ ) − Ψ( x ′ , x ) � = a i K Φ ( x i , x ′ i , x , x ′ ) . i = 1 Pahikkala et al. (TUCS, UGent) Intransitive Preference Learning September, 11 2009 11 / 25

Learning intransitive reciprocal relations Reciprocal relations are learned by defining a specific kernel construction Consider the following joint feature representation for a couple: Φ( e i ) = Φ( x i , x ′ i ) = Ψ( x i , x ′ i ) − Ψ( x ′ i , x i ) , This yields the following kernel defined on couples: K Φ ( e i , e j ) K Φ ( x i , x ′ i , x j , x ′ = j ) � Ψ( x i , x ′ i ) − Ψ( x ′ i , x i ) , Ψ( x j , x ′ j ) − Ψ( x ′ = j , x j ) � K Ψ ( x i , x ′ j ) + K Ψ ( x ′ i , x j , x ′ i , x i , x ′ = j , x j ) − K Ψ ( x ′ i , x i , x j , x ′ j ) − K Ψ ( x i , x ′ i , x ′ j , x j ) . And the model becomes: N � h ( x , x ′ ) = � w , Ψ( x , x ′ ) − Ψ( x ′ , x ) � = a i K Φ ( x i , x ′ i , x , x ′ ) . i = 1 Pahikkala et al. (TUCS, UGent) Intransitive Preference Learning September, 11 2009 11 / 25

Learning intransitive reciprocal relations Ranking can be considered as a specific case in this framework Consider the following joint feature representation Ψ for a couple: Ψ T ( x , x ′ ) = φ ( x ) . This yields the following kernel K Ψ : K Ψ T ( x i , x ′ i , x j , x ′ j ) = K φ ( x i , x j ) = � φ ( x i ) , φ ( x j ) � , And the model becomes: h ( x , x ′ ) = � w , φ ( x ) � − � w , φ ( x ′ ) � = f ( x ) − f ( x ′ ) , Pahikkala et al. (TUCS, UGent) Intransitive Preference Learning September, 11 2009 12 / 25