Ranked retrieval Term weighting Vector space model Length normalization NPFL103: Information Retrieval (4) Ranked retrieval, Term weighting, Vector space model Pavel Pecina Institute of Formal and Applied Linguistics Faculty of Mathematics and Physics Charles University Original slides are courtesy of Hinrich Schütze, University of Stutugart. 1 / 62 pecina@ufal.mff.cuni.cz
Ranked retrieval Document frequency Pivot normalization Length normalization Measuring similarity Principles Vector space model tf-idf weighting Term frequency Term weighting Term weighting Qvery-document scoring Introduction Ranked retrieval Contents Length normalization Vector space model 2 / 62
Ranked retrieval Term weighting Vector space model Length normalization Ranked retrieval 3 / 62
Ranked retrieval Term weighting Vector space model Length normalization Ranked retrieval 5 / 62 ▶ So far, our queries have been boolean - document is a match or not. ▶ Good for experts: precise understanding of the needs and collection. ▶ Good for applications: can easily consume thousands of results. ▶ Not good for the majority of users. ▶ Most users are not capable or lazy to write Boolean queries. ▶ Most users don’t want to wade through 1000s of results. ▶ This is particularly true of web search.
Ranked retrieval Term weighting Vector space model Length normalization Problem with Boolean search: ”Feast” or ”famine” that produces a manageable number of hits. 6 / 62 ▶ Boolean queries ofuen result in either too few or too many results (too few ∼ 0, too many ∼ 1000s). ▶ Qvery 1 (boolean conj.): [standard user dlink 650] → 200,000 hits: ”feast” ▶ Qvery 2 (boolean conj.): [standard user dlink 650 no card found] → 0 hits: ”famine” ▶ In Boolean retrieval, it takes a lot of skill to come up with a query
Ranked retrieval Term weighting Vector space model Length normalization Feast or famine: No problem in ranked retrieval 7 / 62 ▶ With ranking, large result sets are not an issue. ▶ Just show the top 10 results. ▶ This doesn’t overwhelm the user. ▶ Premise: the ranking algorithm works. ▶ …More relevant results are ranked higher than less relevant results.
Ranked retrieval Term weighting Vector space model Length normalization Scoring as the basis of ranked retrieval documents that are less relevant. collection with respect to a query? 9 / 62 ▶ We wish to rank documents that are more relevant higher than ▶ How can we accomplish such a ranking of the documents in the ▶ Assign a score to each query-document pair, say in [0 , 1] . ▶ This score measures how well document and query “match”.
Ranked retrieval Term weighting Vector space model Length normalization Qvery-document matching scores score 10 / 62 ▶ How do we compute the score of a query-document pair? ▶ Let’s start with a one-term query. ▶ If the query term does not occur in the document: score should be 0. ▶ The more frequent the query term in the document, the higher the ▶ We will look at a number of alternatives for doing this.
Ranked retrieval Term weighting Vector space model Length normalization Take 1: Jaccard coefgicient 11 / 62 ▶ A commonly used measure of overlap of two sets ▶ Let A and B be two sets ▶ Jaccard coefgicient: jaccard ( A , B ) = | A ∩ B | | A ∪ B | , where ( A ̸ = ∅ or B ̸ = ∅ ) ▶ jaccard ( A , A ) = 1 ▶ jaccard ( A , B ) = 0 if A ∩ B = 0 ▶ A and B don’t have to be the same size. ▶ Always assigns a number between 0 and 1.
Ranked retrieval Term weighting Vector space model Length normalization Jaccard coefgicient: Example What is the query-document score the Jaccard coefgicient computes for: 12 / 62 ▶ Qvery: “ides of March” ▶ Document: “Caesar died in March” ▶ jaccard ( q , d ) = 1/6
Ranked retrieval Term weighting Vector space model Length normalization What’s wrong with Jaccard? not consider this information. document. 13 / 62 ▶ It ignores term frequency (how many occurrences a term has). ▶ Rare terms are more informative than frequent terms. Jaccard does → We need a more sophisticated way of normalizing for the length of a
Ranked retrieval Term weighting Vector space model Length normalization Term weighting 14 / 62
Ranked retrieval mercy 0 1 0 0 0 0 Cleopatra 1 0 0 0 0 0 1 Term weighting 0 1 1 1 1 worser 1 0 1 1 1 0 … Calpurnia 1 1 1 Vector space model Length normalization Binary incidence matrix Anthony Julius The Hamlet Othello Macbeth … and Caesar Tempest Cleopatra Anthony 1 1 0 0 0 1 Brutus 1 1 0 1 0 0 Caesar 1 1 0 15 / 62 ▶ Each document is represented as a binary vector ∈ { 0 , 1 } | V | .
Ranked retrieval mercy 0 10 0 0 0 0 Cleopatra 57 0 0 0 0 0 2 Term weighting 0 3 8 5 8 worser 2 0 1 1 1 5 … Calpurnia 0 1 2 Vector space model Length normalization Count matrix Anthony Julius The Hamlet Othello Macbeth … and Caesar Tempest Cleopatra Anthony 157 73 0 0 0 1 Brutus 4 157 0 2 0 0 Caesar 232 227 0 16 / 62 ▶ Each document is represented as a count vector ∈ N | V | .
Ranked retrieval Term weighting Vector space model Length normalization Bag of words model represented the same way. distinguish these two documents. course. 17 / 62 ▶ We do not consider the order of words in a document. ▶ John is quicker than Mary and Mary is quicker than John are ▶ This is called a bag of words model. ▶ In a sense, this is a step back: The positional index was able to ▶ We will look at “recovering” positional information later in this ▶ For now: bag of words model
Ranked retrieval Term weighting Vector space model Length normalization Term frequency (tf) number of times that t occurs in d . 19 / 62 ▶ The term frequency tf t , d of term t in document d is defined as the ▶ We want to use tf when computing query-document match scores. ▶ But how? ▶ Raw term frequency is not what we want because: ▶ A document with tf = 10 occurrences of the term is more relevant than a document with tf = 1 occurrence of the term. ▶ But not 10 times more relevant.
Ranked retrieval Term weighting otherwise 20 / 62 Instead of raw frequency: Log frequency weighting Length normalization Vector space model ▶ The log frequency weight of term t in d is defined as follows: { 1 + log 10 tf t , d if tf t , d > 0 w t , d = 0 ▶ tf t , d → w t , d : 0 → 0, 1 → 1, 2 → 1.3, 10 → 2, 1000 → 4, etc. ▶ Score for a document-query pair: sum over terms t in both q and d : ∑ tf-matching-score ( q , d ) = (1 + log tf t , d ) t ∈ q ∩ d ▶ The score is 0 if none of the query terms is present in the document.
Ranked retrieval Term weighting Vector space model Length normalization Frequency in document vs. frequency in collection weighting and ranking. 22 / 62 ▶ In addition, to the frequency of the term in the document … …we also want to use the frequency of the term in the collection for
Ranked retrieval Term weighting Vector space model Length normalization Desired weight for rare terms (e.g., arachnocentric). 23 / 62 ▶ Rare terms are more informative than frequent terms. ▶ Consider a term in the query that is rare in the collection ▶ A document containing this term is very likely to be relevant. → we want high weights for rare terms like arachnocentric.
Ranked retrieval Term weighting Vector space model Length normalization Desired weight for frequent terms (e.g., good, increase, line). document that doesn’t but words like good, increase and line are not sure indicators of relevance. weights but lower weights than for rare terms. 24 / 62 ▶ Frequent terms are less informative than rare terms. ▶ Consider a term in the query that is frequent in the collection ▶ A document containing this term is more likely to be relevant than a → For frequent terms like good, increase, and line, we want positive
Ranked retrieval Term weighting Vector space model Length normalization Document frequency increase, and line. matching score. collection that the term occurs in. 25 / 62 ▶ We want high weights for rare terms like arachnocentric. ▶ We want low (positive) weights forfrequent words like good, ▶ We will use document frequency to factor this into computing the ▶ The document frequency is the number of documents in the
Ranked retrieval Term weighting Vector space model Length normalization idf weight N df t document frequency. 26 / 62 ▶ df t is document frequency, the number of documents t occurs in. ▶ df t is an inverse measure of the informativeness of term t . ▶ We define the idf weight of term t in a collection of N documents as: idf t = log 10 ▶ idf t is a measure of the informativeness of the term. ▶ log N / df t instead of [ N / df t ] to “dampen” the efgect of idf ▶ Note that we use the log transformation for both term frequency and
Ranked retrieval 4 0 1,000,000 the 1 100,000 under 2 10,000 fly 3 1000 sunday 100 Term weighting animal 6 1 calpurnia idf t df t term df t Examples for idf Length normalization Vector space model 27 / 62 1 , 000 , 000 Compute idf t using the formula: idf t = log 10
Recommend
More recommend
