Applying the User-over-Ranking Hypothesis to Query Formulation - PowerPoint PPT Presentation

Applying the User-over-Ranking Hypothesis to Query Formulation Matthias Hagen Benno Stein Bauhaus-Universit¨ at Weimar matthias.hagen@uni-weimar.de ICTIR 2011 Bertinoro, Italy September 14, 2011 Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 1

What is the User-over-Ranking hypothesis? Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 2

The User-over-Ranking Hypothesis [Stein and Hagen, ECIR 2011] Queries returning as many results as the user can consider increase retrieval performance. Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 3

The User-over-Ranking Hypothesis [Stein and Hagen, ECIR 2011] Queries returning as many results as the user can consider increase retrieval performance. Fine print: If ranking works: great! Use case is not some query like ebay . But more involved information needs, automatic systems, etc. Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 3

Assumption 1: More keywords = more specific Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 4

Assumption 1: More keywords = more specific Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 4

Assumption 1: More keywords = more specific 10 6 underspecific Result list length overspecific 0 Query 1 10 20 length Specificity of Queries Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 5

Assumption 2: User can arbitrarily specify information need 10 6 underspecific Result list length overspecific 0 Query 1 10 20 length Specificity of Queries Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 6

Assumption 2: User can arbitrarily specify information need 10 6 underspecific Result list length overspecific 0 Query 1 10 20 length Specificity of Queries Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 6

Assumption 3: User can consider about k results. 10 6 underspecific Result list length k overspecific 0 Query Result 10 3 10 4 1 10 20 0 10 100 length list length Processing k capacity Specificity of Queries Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 7

Hypothesis: Specificity matches k = Optimum retrieval optimum retrieval 10 6 underspecific ⇔ Result result set size = k list length k overspecific 0 Query Result 10 3 10 4 1 10 20 0 10 100 length list length Processing k capacity Specificity of Queries Probability for Retrieval Success Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 8

What is this hypothesis good for? Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 9

Query Formulation Scenario Given a set W of keywords Find a good query Q ⊆ W Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 10

Query Formulation Scenario Given a set W of keywords Find a good query Q ⊆ W Previous approach [Lee et al., CIKM 2009] Learnt ranking function identifies the m best keywords from W . Based on: Known relevant documents Unrestricted index access Manually tuned m for each set W Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 10

Consider for instance . . . Known-Item Finding Scenario User accessed a document But did not store it Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 11

Consider for instance . . . Known-Item Finding Scenario User accessed a document How can she find it again? But did not store it Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 11

Consider for instance . . . Known-Item Finding Scenario User accessed a document How can she find it again? But did not store it Solution Remember some keywords information retrieval query formulation web search search session user support search engine cost optimization Query a search engine Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 11

But what query to formulate with the keywords? Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 12

Single keywords? information retrieval / query / / / / / / / / / / formulation / / / / / / / / / / / / / / / / / / web / / / / / / / search / / / / / / / / / / search / / / / / / / / / / / / session / / / / / / / / / / / / user/ / / / / / / / support / / / / / / / / / / / / search / / / / / / / / / / / / engine / / / / / / / / / / cost/ / / / / / / / optimization / / / / / / / / / / / / / / / / / / / / Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 13

Single keywords? / information/ / / / / / / / / / / / / / / / / / / / retrieval / / / / / / / / / / / / / / query formulation / web / / / / / / / search / / / / / / / / / / search / / / / / / / / / / / / session / / / / / / / / / / / / user/ / / / / / / / support / / / / / / / / / / / / / search / / / / / / / / / / / engine / / / / / / / / / / cost/ / / / / / / / optimization / / / / / / / / / / / / / / / / / / / / Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 13

Single keywords? / information/ / / / / / / / / / / / / / / / / / / / retrieval / / / / / / / / / / / / / / / query / / / / / / / / / / formulation / / / / / / / / / / / / / / / / / web search / search / / / / / / / / / / / / session / / / / / / / / / / / / user/ / / / / / / / support / / / / / / / / / / / / search / / / / / / / / / / / / engine / / / / / / / / / / cost/ / / / / / / / optimization / / / / / / / / / / / / / / / / / / / / Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 13

Single keywords? Underspecific! / information/ / / / / / / / / / / / / / / / / / / / retrieval / / / / / / / / / / / / / / / query / / / / / / / / / / formulation / / / / / / / / / / / / / / / / / web search / search / / / / / / / / / / / / session / / / / / / / / / / / / user/ / / / / / / / support / / / / / / / / / / / / search / / / / / / / / / / / / engine / / / / / / / / / / cost/ / / / / / / / optimization / / / / / / / / / / / / / / / / / / / / Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 13

All keywords at once? information retrieval query formulation web search search session user support search engine cost optimization Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 13

All keywords at once? Overspecific! information retrieval query formulation web search search session user support search engine cost optimization Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 13

Remember the hypothesis . . . not too many results! Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 14

Solution: As many keywords as possible! information retrieval query formulation web search search session / user/ / / / / / / / support / / / / / / / / / / / search engine cost optimization Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 15

“As many keywords as possible”-Query Characteristics Captures most of the remembered keywords Best possible description of the known-item Not too many results user can check complete list → Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 16

“As many keywords as possible”-Query Characteristics Captures most of the remembered keywords Best possible description of the known-item Not too many results user can check complete list → Problem Relevant documents not known No web index at user site Query size not known Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 16

“As many keywords as possible”-Query Characteristics Captures most of the remembered keywords Best possible description of the known-item Not too many results user can check complete list → Problem Relevant documents not known No web index at user site Lee et al. not applicable → Query size not known Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 16

We propose an approach for this scenario . . . Matthias Hagen, Benno Stein Applying the User-over-Ranking Hypothesis to Query Formulation 17