Binary Factorization Models for Statistical Relational Learning - PowerPoint PPT Presentation

Binary Factorization Models for Statistical Relational Learning Guillaume Bouchard Collaborators Machine Learning for Services Beyza Ermis Xerox Research Centre Europe Behrouz Behmardi Cedric Archambeau Dawei Yin Ehsan Abbasnejad Julien Perez Shengbo Guo

Motivating example Human biological data (1) Patients data Year 1993 9 variables Source: UCI repository stability of stability of stability of patient's patient's internal patient's surface oxygen blood discharge patient's patient's patient's blood perceived temperature temperature saturation pressure decision surface temp core temp pressure comfort mid low excellent mid stable stable stable 15 A mid high excellent high stable stable stable 10 S high low excellent high stable stable mod-stable 10 A mid low good high stable unstable mod-stable 15 A mid mid excellent high stable stable stable 10 A high low good mid stable stable unstable 15 S mid low excellent high stable stable mod-stable 5 S 90 patients high mid excellent mid unstable unstable stable 10 S mid high good mid stable stable stable 10 S mid low excellent mid unstable stable mod-stable 10 S mid mid good mid stable stable stable 15 A mid low good high stable stable mod-stable 10 A high high excellent high unstable stable unstable 15 A mid high good mid unstable stable mod-stable 10 A mid low good high unstable unstable stable 15 S Objective: statistical modelling (tests, predictions, visualisation)

Motivating example Human biological data (2) Genetic data Year2012 38M variables Source: 1000 genomes project 1092 individuals

Example Human biological data (3) Biological data Year 2013 Source: KEGG/DBGET/LinkDB

Example Social Network (1) Monk relationships Year 1969 Source: Sampson Monk j 18x18x10 Influences 0 0 2 0 3 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 2 0 3 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 2 0 3 0 0 0 0 0 0 0 0 1 0 0 0 0 Does not like 3 0 0 0 0 0 2 0 0 0 0 0 0 1 0 0 0 0 0 0 2 0 3 0 0 0 0 0 0 0 0 1 0 0 0 0 3 0 0 0 0 0 2 0 0 0 0 0 0 1 0 0 0 0 0 0 2 0 3 0 0 0 0 0 0 0 0 1 0 0 0 0 3 0 0 0 0 0 2 0 0 0 0 0 0 1 0 0 0 0 0 0 2 0 3 0 0 0 0 0 0 0 0 1 0 0 0 0 Like 2 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 3 0 0 0 0 0 2 0 0 0 0 0 0 1 0 0 0 0 0 0 2 2 3 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 3 0 0 0 0 0 2 0 0 0 0 0 0 1 0 0 0 0 0 0 2 0 3 0 0 0 0 0 0 0 0 1 0 0 0 0 2 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 3 0 0 0 0 0 2 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 3 1 0 0 0 2 0 0 0 0 0 0 0 0 0 0 2 0 3 0 0 0 0 0 0 0 0 1 0 0 0 0 2 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 3 0 0 0 0 0 2 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 3 1 0 0 0 2 0 0 0 0 0 0 0 0 0 0 2 2 3 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 1 0 3 0 0 0 0 0 2 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 3 1 0 0 0 2 0 0 0 0 0 0 0 0 2 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 3 0 0 0 0 0 0 2 0 1 0 0 0 0 0 3 0 0 0 0 0 2 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 3 1 0 0 0 2 0 0 0 0 0 0 0 0 2 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 3 0 0 0 0 0 0 2 0 1 0 0 0 0 0 3 0 0 0 0 0 2 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 3 1 0 0 0 2 0 0 0 0 0 0 0 0 2 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 3 0 0 0 0 0 0 2 0 1 0 0 0 0 0 0 0 0 0 3 1 0 0 0 2 0 0 0 0 0 0 0 0 1 0 0 3 0 0 0 0 2 0 0 0 0 0 0 0 0 0 2 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 3 0 0 0 0 0 0 2 0 1 0 0 0 0 0 0 0 0 0 3 1 0 0 0 2 0 0 0 0 0 0 0 0 1 0 0 3 0 0 0 0 2 0 0 0 0 0 0 0 0 0 2 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 3 0 0 0 0 0 0 2 0 1 0 0 0 0 0 0 0 1 0 0 0 0 3 3 1 0 0 0 0 0 0 0 2 2 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 2 0 1 0 0 0 0 0 0 2 0 0 0 0 0 1 0 0 0 0 0 0 0 3 0 0 0 0 0 0 3 1 0 0 0 2 0 0 0 0 0 0 0 0 1 0 0 3 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 2 0 1 0 0 0 0 0 0 2 0 0 0 0 0 1 0 0 0 0 0 0 0 3 0 0 0 0 0 0 3 1 0 0 0 2 0 0 0 0 0 0 0 0 1 0 0 3 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 2 0 1 0 0 0 0 0 0 2 0 0 0 0 0 1 0 0 0 0 0 0 0 3 0 0 1 0 0 3 0 0 0 0 2 0 0 0 0 0 0 0 0 0 3 2 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 2 0 1 0 0 0 0 0 0 2 0 0 0 0 0 1 0 0 0 0 0 0 0 3 0 0 1 0 0 3 0 0 0 0 2 0 0 0 0 0 0 0 0 0 3 2 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 2 0 1 0 0 0 0 0 0 2 0 0 0 0 0 1 0 0 0 0 0 0 0 3 0 0 1 0 0 3 0 0 0 0 2 0 0 0 0 0 0 0 0 0 3 2 0 0 Dislikes 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 0 0 0 2 0 0 1 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 1 0 0 0 1 0 0 3 0 0 0 0 2 0 0 0 0 0 0 0 0 0 3 2 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 Likes 0 2 0 0 0 0 0 1 0 0 0 0 0 0 0 3 0 0 0 0 0 0 2 0 0 3 0 0 0 0 0 0 0 1 0 0 1 0 0 3 0 0 0 0 2 0 0 0 0 0 0 0 0 0 3 2 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 2 0 0 0 0 0 1 0 0 0 0 0 0 0 3 0 0 0 0 0 0 2 0 0 3 0 0 0 0 0 0 0 1 0 0 Dislikes 3 2 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 1 0 0 0 0 0 2 0 0 0 0 0 2 0 0 0 0 0 0 0 0 2 0 0 1 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 1 0 0 0 3 2 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 Likes 3 0 0 0 1 0 0 0 0 0 2 0 0 0 0 0 2 0 0 0 0 0 1 0 0 0 0 0 0 0 3 0 0 0 0 0 0 2 0 0 3 0 0 0 0 0 0 0 1 0 0 3 2 0 0 Dislikes 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 1 0 0 0 0 0 2 0 0 0 0 0 0 0 0 2 0 0 3 0 0 0 0 0 0 0 1 0 0 0 0 0 0 3 0 0 1 0 0 0 0 0 2 0 0 0 0 3 2 0 0 Likes 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 1 0 0 0 0 0 2 0 0 0 0 0 0 0 0 0 0 0 2 0 0 3 0 0 3 0 0 1 0 0 0 0 0 0 0 0 0 0 0 2 1 0 0 0 0 0 0 3 2 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 1 0 0 0 0 0 2 0 0 0 0 0 0 0 0 2 0 0 3 0 0 0 0 0 0 0 1 0 0 0 0 0 0 3 0 0 1 0 0 0 0 0 2 0 0 0 0 Dislikes 0 0 0 3 0 0 0 1 0 0 0 0 0 2 0 0 0 0 3 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 2 0 0 3 0 0 0 0 0 0 0 1 0 0 0 0 0 0 3 0 0 1 0 0 0 0 0 2 0 0 0 0 0 0 0 Likes 3 0 0 0 1 0 0 0 0 0 2 0 0 0 0 3 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 2 0 0 3 0 0 3 0 0 1 0 0 0 0 0 0 0 0 0 0 0 2 1 0 0 0 0 0 0 0 0 0 3 0 0 0 1 0 0 0 0 0 2 0 0 0 0 3 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 3 0 0 1 0 0 0 0 0 2 0 0 0 0 0 0 0 0 2 0 1 0 0 0 0 0 0 0 0 0 0 3 0 0 0 3 0 0 0 1 0 0 0 0 0 2 0 0 0 0 3 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 3 0 0 1 0 0 0 0 0 2 0 0 0 0 0 0 0 0 2 0 1 0 0 0 0 0 0 0 0 0 0 3 0 0 0 3 0 0 0 1 0 0 0 0 0 2 0 0 0 0 3 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 3 0 0 1 0 0 0 0 0 2 0 0 0 0 0 0 0 0 2 0 1 0 0 0 0 0 0 0 0 0 0 3 3 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 Monk i 3 0 0 0 0 0 0 0 0 0 1 2 0 0 2 0 0 0 0 0 0 0 3 0 0 1 0 0 0 0 0 2 0 0 0 0 0 0 0 0 2 0 1 0 0 0 0 0 0 0 0 0 0 3 3 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 3 0 0 0 0 0 0 0 0 0 1 2 0 0 2 0 0 0 0 0 0 0 3 0 0 1 0 0 0 0 0 2 0 0 0 0 0 0 0 0 2 0 1 0 0 0 0 0 0 0 0 0 0 3 3 2 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 0 2 1 0 0 0 0 2 0 0 0 0 0 0 0 0 0 2 0 1 0 0 0 0 0 0 0 0 0 0 3 3 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 3 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 3 0 0 0 0 0 0 0 0 0 1 2 0 0 2 0 0 0 0 0 0 0 2 0 1 0 0 0 0 0 0 0 0 0 0 3 3 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 3 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 3 0 0 0 0 0 0 0 0 0 1 2 0 0 2 0 0 0 0 0 0 0 2 0 1 0 0 0 0 0 0 0 0 0 0 3 3 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 3 0 0 0 0 0 0 0 0 0 1 2 0 0 2 0 0 0 1 2 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 1 0 0 0 0 0 0 0 0 0 0 3 3 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 3 0 0 0 0 0 0 0 0 0 1 2 0 0 2 0 0 0 1 2 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 2 0 1 0 0 0 0 0 0 0 0 0 0 3 3 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 3 0 0 0 0 0 0 0 0 0 1 2 0 0 2 0 0 0 1 2 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 3 2 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 0 0 0 0 0 0 0 0 1 3 0 0 0 0 0 0 0 0 0 1 2 0 0 2 0 0 0 1 2 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 3 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 2 0 0 0 0 1 3 0 0 0 0 0 0 0 0 0 1 2 0 0 2 0 0 0 1 2 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 3 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 2 0 0 0 0 1 1 2 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 2 3 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 3 2 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 0 0 0 0 0 0 0 0 1 1 2 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 2 3 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 3 2 0 0 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 2 0 0 0 0 1 1 2 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 2 3 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 2 0 0 0 0 1 1 2 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 2 3 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 2 0 0 0 0 1 1 2 0 0 0 0 3 0 0 0 0 0 0 0 0 0 0 0 2 3 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 2 0 0 0 0 1 2 3 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 2 0 0 0 0 1 2 3 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 3 0 0 0 0 0 0 0 0 0 2 0 0 0 0 1 2 3 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 2 3 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 2 3 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0

Example Social Network (2) Bloggers community Year 2012 Source: LiveJournal online social network http://snap.stanford.edu/data 4Mx4Mx300K

Evolution of Machine Learning • Early age : One task  many models • Today : One task  one model • Wish : multiple tasks  one model We need a generic data model! Xerox Internal Use Only

Example Social Network Relational data Year 2012 Data source: FlickR C3 Tag C1 User Several prediction tasks: - Item recommendation Item feature - Friend recommendation C4 - Automatic tagging - Data cleaning C2 Item comment - Predicting the comment type

Statistical Relational Learning Also called multi-relational learning Goal: predict relationships • Probabilistic relational models • Markov logic networks • Recently, distributed representations have shown great performances – Collective Factorization [Singh&Gordon2005] – Tensor factorization [Sutskever et al., 2009, Nickel&Tresp, 2010]