June 2012 Entity Detection Structured Recommendations Play - PowerPoint PPT Presentation

Patrick Pantel Joint work with : Tom Lin (UW), Michael Gamon (MSR), Anitha Kannan (MSR), Ariel Fuxman (MSR) June 2012

Entity Detection Structured Recommendations Play trailer Structured Data

Price prediction Task completion Aggregate ratings Task completion

Structured Data Direct Answer Active Objects 4

Current Experience Active Objects 21

Better experience Actions easily Recognize Active Objects entity in query accessible 23

Better experience Actions easily Recognize Active Objects entity in query accessible 24

Better experience Actions easily Recognize Active Objects entity in query accessible 25

Politicians Actions easily Recognize Active Objects entity in query accessible 29

Films Actions easily Recognize Active Objects entity in query accessible 30

t t q q t t T T y y T f T f e a a K K s w w s s n c K K 2 K K 2 Q Q Active Objects 32

Actions vs Intents User Intents plan vacation get in shape and Goals Query hilton orlando reviews sea world location how to lose weight Query Intent Informational Navigational Transactional [Broder, 2002] Finer-grained Locate Interact … Obtain … Advice Download Intents [Rose and Levinson, 2004] add to Netflix queue(film) get address(landmark) Actions on Entities buy(camera) read reviews(hotel) Active Objects 36

Do web queries contain entities? Schema.org types for entity- event Entity Distribution in Web Search Queries bearing queries other 3% 3% person category 8% 4% product 9% other creativework category + 40% 15% organization refiner 37% 10% 43% entity (e.g., “GoldenEye”, “Horne Auto”) 14% entity category entity website 29% (e.g., “golf cart battery”, “global sim card”) 28% 15% no entity (e.g., “xxx”, “good reading quotes”) 28% website entity + refiner 14% (e.g., “yahoo mail”, “girlybox.com”) * From a query traffic-weighted sample Active Objects 38

Ontology of Actions Transactional (navigating to a web-mediated action) Navigational 1x Apply for Job (on a LocalBusiness / Organization entity) 10x Login Action (on a Website entity) 4x Search Action (on a Website entity) Buy (Shopping on a Product entity) Informational (need satisfied by reading content, or could be satisfied by written Buy Tickets (on an Event / Product / Person entity) transcript of content) 3x Content Creation (on a Website entity) 1x Find Location(s) (on an Organization entity) Discuss Online (on any entity) 1x Find Lyrics (on a CreativeWork / MusicalTrack entity) 2x Find Recipe For (on a food) 5x Download (on a CreativeWork or Software entity) 1x Find Where to Buy (on a Product entity) 1x Listen to Music (on a CreativeWork or Website entity) 2x Get Contact Information (on an Organization entity) Manage Account (on a Local Business / Website / Org entity) 1x Get Directions To (on an Organization / Location entity) Pay Bill (on a Website / Organization entity) 2x Get Domain Information (on a Website entity) 14x Play Game (on a Game entity) 1x Get Event Details (on an Event entity) 2x Get Event Results (on an Event entity) Rent (on a CreativeWork / Product entity) 4x Product Detail (on a Product entity) 2x Reservation (on a Hotel entity) 29x Learn (on any entity) Schedule Appointment (on a LocalBusiness entity) 6x Learn / Educational (on a Person / Product / Organization entity) Sell (Shopping on a Product entity) 1x Learn / Trivia (on any entity) 1x Use Service On (e.g., translate, on a Website) 1x Operating Hours (on an Organization entity) 3x Read Articles (on a News / Magazine entity) 6x Watch Video About (on any entity) 1x Read Guide (on a Product entity) 1x Web Chat 1x Read Help (on a Product entity) Other 8x Read News About (on any entity) 13x Shopping (category of actions including reviews and buying) 3x Read Reviews (Shopping on a CreativeWork / Product / Service entity) 19x Various/Unknown 1x Read Spoilers (on a CreativeWork) 8x Research (focused information gathering, on any entity) 8x Search Database of (e.g., obituaries, on an Organization / Website) Actions are tied to entity types See Menu (on a Restaurant) 3x See Pictures (on a Person / Product / Organization entity) 47 actions in current list Side Effects / Safety (on a Product entity) Stock Price (on an Organization entity) Note: No existing Actions equivalent for Schema.org Active Objects 43

How many Actions should there be? Discovery Rate of New Actions 10 9 New Actions per 10 Annotations (average) 8 7 6 5 4 Rapidly 3 decreasing 2 discovery rate 1 0 0 50 100 150 200 250 Annotations Active Objects 44

t t q q t t T T y y T f T f e a a K K s w w s s n c K K 2 K K 2 Q Q Active Objects 45

Learning Actions from Web Usage Logs • Three months of us-en 2,164,579 21 types 235,385 entities web logs (query, host) pairs • 129,088 contexts 58,123 hosts over 3 months Annotate with Freebase entities • Keep queries with an entity in set of 21 types • Filter out navigational queries • Filter out clicked hosts that weren’t clicked at least 100 times get reviews Orlando hotel reviews ← read biography Does Hope Solo have a boyfriend? Free Winzip download download software watch shows online watch family guy online Active Objects 49

Action Context ______” “______ “ebert Star Wars review" Model 1 read reviews action read reviews action Model 1.01 Model 1.01 a a Goal : Define a theory for how actionable queries are generated. The story for p(actionable query), q q or more formally The story for p ( f , q ,a,n | a , b ) b b For each action a a 1 a 2 a 1 a 2 f a ~ Dirichlet( b ) ( action→contexts die) For each query q f q ~ Dirichlet( a ) f K For each context position in q (pre or post) K action a ~ Multinomial( q ) n 1 n 2 ngram n ~ Multinomial( f a ) Q Q Active Objects 52

Action Context “ebert Star Wars review" Model 2 read reviews action Model 1.02 Model 1.02 a a The story for p ( f , q ,a,n | a , b ) For each action a q f a ~ Dirichlet( b ) q For each query q q ~ Dirichlet( a ) b b For each context position in q (pre or post) action a ~ Multinomial( q ) ngram n ~ Multinomial( f a ) a a action a ~ Multinomial( q ) ngram n 1 ~ Multinomial( f a ) f f ngram n 2 ~ Multinomial( f a ) K K n 1 n 2 Q Q Active Objects 53

Action Context Model 3 Clicked hosts matter… buy action read reviews action amazon.com rottentomatoes.com metacritic.com ebay.com walmart.com efilmcritic.com Active Objects 54

www.rottentomatoes.com Action Context Host ______” “______ review” “ebert Star Wars Model 3 read reviews action Model 1.13 Model 1.13 a a The story for P ( f , q,w ,a,n,c | a , b ,  ) w , ,  ,c For each action a q q f a ~ Dirichlet( b ) (to contexts) w a ~ Dirichlet(i) (to clicks) For each query q q ~ Dirichlet( a )  b  b a a action a ~ Multinomial( q ) ngram n 1 ~ Multinomial( f a ) ngram n 2 ~ Multinomial( f a ) click c ~ Multinomial( w a ) f w f w n c K 2 K 2 K K Q Q Active Objects - 55 -

Action Context Host Model 3 The type matters… Active Objects 57

Type Action Context Host www.rottentomatoes.com Model 4 “______ Star Wars ______” “ebert r eview” read reviews action film type The story for P ( f , q,t , w ,t,a,n,c | a , b , g, ) t , t, g, Model 1.03 Model 1.14 Model 1.03 Model 1.14 a a For each action a f a ~ Dirichlet( b ) (to contexts) a a (to clicks) w a ~ Dirichlet(i) q q g g For each type t t t ~ Dirichlet( g ) (to actions) For each query q q q q ~ Dirichlet( a ) t t t t T type t ~ Multinomial( q ) T action a ~ Multinomial( q ) action a ~ Multinomial( q ) action a ~ Multinomial( t t )   b b a a ngram n 1 ~ Multinomial( f a ) ngram n 2 ~ Multinomial( f a ) click c ~ Multinomial( w a ) f w f w n c 2 K 2 K K K Q Q Q Q Active Objects 58

Type Action Context Host Model 3 We also have entity data… Active Objects 60

Type Entity Action Context Host Model 5 The story for P ( f , q,t , y , w ,t,a,e,n,c | a , b , g,h ,  ) h , y , e, For each action a Model 1.05b Model 1.05b f a ~ Dirichlet( b ) b a g b a g w a ~ Dirichlet(i) For each type t t t ~ Dirichlet( g ) f q t f q t y t ~ Dirichlet( h ) T T K K For each query q q ~ Dirichlet( a ) h  h  type t ~ Multinomial( q ) t a t a action a ~ Multinomial( t t ) entity e ~ Multinomial( y t ) y ngram n 1 ~ Multinomial( f a ) w y w n c e T 2 T ngram n 2 ~ Multinomial( f a ) 2 K K Q Q click c ~ Multinomial( w a ) Active Objects - 61 -