[PPT] - A Time Machine for Text Search Klaus Berberich, Srikanta Bedathur, PowerPoint Presentation

SLIDE 1

A Time Machine for Text Search

Klaus Berberich, Srikanta Bedathur, Thomas Neumann, Gerhard Weikum

Max-Planck Institute for Informatics, Saarbrücken, Germany

SLIDE 2

Klaus Berberich – A Time Machine for Text Search

Motivation

Historical information needs, e.g.,

Contemporary (~2001) articles about the movie “Harry Potter and the Sorcerer’s Stone”

Relevant pages have disappeared but are preserved

by Web archives (e.g., archive.org)

Search over Web archives is limited and ignores the

time-axis

SLIDE 3

Klaus Berberich – A Time Machine for Text Search

Motivation

Historical information needs, e.g.,

Contemporary (~2001) articles about the movie “Harry Potter and the Sorcerer’s Stone”

Relevant pages have disappeared but are preserved

by Web archives (e.g., archive.org)

Search over Web archives is limited and ignores the

time-axis

SLIDE 4

Klaus Berberich – A Time Machine for Text Search

Motivation

Historical information needs, e.g.,

Contemporary (~2001) articles about the movie “Harry Potter and the Sorcerer’s Stone”

Relevant pages have disappeared but are preserved

by Web archives (e.g., archive.org)

Search over Web archives is limited and ignores the

time-axis

SLIDE 5

Klaus Berberich – A Time Machine for Text Search

Motivation

Historical information needs, e.g.,

Contemporary (~2001) articles about the movie “Harry Potter and the Sorcerer’s Stone”

Relevant pages have disappeared but are preserved

by Web archives (e.g., archive.org)

Search over Web archives is limited and ignores the

time-axis

SLIDE 6

Klaus Berberich – A Time Machine for Text Search

Motivation

Historical information needs, e.g.,

Contemporary (~2001) articles about the movie “Harry Potter and the Sorcerer’s Stone”

Relevant pages have disappeared but are preserved

by Web archives (e.g., archive.org)

Search over Web archives is limited and ignores the

time-axis

SLIDE 7

Klaus Berberich – A Time Machine for Text Search

Motivation

Time-Travel Text Search extends keyword querying

by a time-point of interest t “harry potter” @ 2001/11/14

Other temporally versioned text collections

Wikis Repositories (e.g., controlled by CVS, Subversion) Your Desktop

SLIDE 8

Klaus Berberich – A Time Machine for Text Search

Outline

Motivation Collection, Query, and Relevance Model Time-Travel Inverted File Index

Reducing Index Size Tuning Index Performance

Experimental Evaluation Conclusions

SLIDE 9

Klaus Berberich – A Time Machine for Text Search

Collection Model

Document d is a sequence of time-stamped versions Version is a vector of searchable terms Document deletion results in tombstone version ⊥ Discrete time, timestamps are non-negative State of document collection as of time t

SLIDE 10

Klaus Berberich – A Time Machine for Text Search

Query Model

Time-travel query q t consists of

keyword part q (i.e., a set of query terms) time-point of interest t

Time-travel query q t is evaluated over D t so that only

versions that existed at time t are considered

SLIDE 11

Klaus Berberich – A Time Machine for Text Search

Relevance Model

We adapt Okapi BM25 as a relevance model Term-frequency score (TF) Inverse document-frequency score (IDF)

SLIDE 12

Klaus Berberich – A Time Machine for Text Search

Relevance Model

We adapt Okapi BM25 as a relevance model Term-frequency score (TF) Inverse document-frequency score (IDF)

SLIDE 13

Klaus Berberich – A Time Machine for Text Search

Relevance Model

We adapt Okapi BM25 as a relevance model Term-frequency score (TF) Inverse document-frequency score (IDF)

SLIDE 14

Klaus Berberich – A Time Machine for Text Search

Relevance Model

We adapt Okapi BM25 as a relevance model Term-frequency score (TF) Inverse document-frequency score (IDF)

SLIDE 15

Klaus Berberich – A Time Machine for Text Search

Relevance Model

We adapt Okapi BM25 as a relevance model Term-frequency score (TF) Inverse document-frequency score (IDF)

SLIDE 16

Klaus Berberich – A Time Machine for Text Search

Outline

Motivation Collection, Query, and Relevance Model Time-Travel Inverted File Index

Reducing Index Size Tuning Index Performance

Experimental Evaluation Conclusions

SLIDE 17

Klaus Berberich – A Time Machine for Text Search

Time-Travel Inverted File Index

Idea: Transparently extend “IR’s workhorse” so that

the existing wealth of extensions remains applicable

We extend postings by a validity time-interval

IDF

“harry”

“potter”

B+-Tree

TF ( d1, 11.2, [t1, t2) ) ( d1, 10.6, [t2, t5) )

“harry”

( d8, 10.9, [t7, t9) )

B+-Tree

SLIDE 18

Klaus Berberich – A Time Machine for Text Search

Time-Travel Inverted File Index

Time-travel query q t can be processed by scanning

index lists while ignoring non-relevant postings

Example: “harry”@t8

IDF

“harry”

“potter”

B+-Tree

TF ( d1, 11.2, [t1, t2) ) ( d1, 10.6, [t2, t5) )

“harry”

( d8, 10.9, [t7, t9) )

B+-Tree

SLIDE 19

Klaus Berberich – A Time Machine for Text Search

Time-Travel Inverted File Index

Time-travel query q t can be processed by scanning

index lists while ignoring non-relevant postings

Example: “harry”@t8

IDF

“harry”

“potter”

B+-Tree

TF ( d1, 11.2, [t1, t2) ) ( d1, 10.6, [t2, t5) )

“harry”

( d8, 10.9, [t7, t9) )

B+-Tree

widf (“harry”, t8) = 3.08

SLIDE 20

Klaus Berberich – A Time Machine for Text Search

Time-Travel Inverted File Index

Time-travel query q t can be processed by scanning

index lists while ignoring non-relevant postings

Example: “harry”@t8

IDF

“harry”

“potter”

B+-Tree

TF ( d1, 11.2, [t1, t2) ) ( d1, 10.6, [t2, t5) )

“harry”

( d8, 10.9, [t7, t9) )

B+-Tree

widf (“harry”, t8) = 3.08 scan

SLIDE 21

Klaus Berberich – A Time Machine for Text Search

Time-Travel Inverted File Index

Time-travel query q t can be processed by scanning

index lists while ignoring non-relevant postings

Example: “harry”@t8

IDF

“harry”

“potter”

B+-Tree

TF ( d1, 11.2, [t1, t2) ) ( d1, 10.6, [t2, t5) )

“harry”

( d8, 10.9, [t7, t9) )

B+-Tree

widf (“harry”, t8) = 3.08 scan

SLIDE 22

Klaus Berberich – A Time Machine for Text Search

Time-Travel Inverted File Index

Time-travel query q t can be processed by scanning

index lists while ignoring non-relevant postings

Example: “harry”@t8

IDF

“harry”

“potter”

B+-Tree

TF ( d1, 11.2, [t1, t2) ) ( d1, 10.6, [t2, t5) )

“harry”

( d8, 10.9, [t7, t9) )

B+-Tree

widf (“harry”, t8) = 3.08 scan

SLIDE 23

Klaus Berberich – A Time Machine for Text Search

Time-Travel Inverted File Index

Time-travel query q t can be processed by scanning

index lists while ignoring non-relevant postings

Example: “harry”@t8

IDF

“harry”

“potter”

B+-Tree

TF ( d1, 11.2, [t1, t2) ) ( d1, 10.6, [t2, t5) )

“harry”

( d8, 10.9, [t7, t9) )

B+-Tree

widf (“harry”, t8) = 3.08 scan

SLIDE 24

Klaus Berberich – A Time Machine for Text Search

Time-Travel Inverted File Index

Time-travel query q t can be processed by scanning

index lists while ignoring non-relevant postings

Example: “harry”@t8

IDF

“harry”

“potter”

B+-Tree

TF ( d1, 11.2, [t1, t2) ) ( d1, 10.6, [t2, t5) )

“harry”

( d8, 10.9, [t7, t9) )

B+-Tree

widf (“harry”, t8) = 3.08 scan

SLIDE 25

Klaus Berberich – A Time Machine for Text Search

Time-Travel Inverted File Index

Time-travel query q t can be processed by scanning

index lists while ignoring non-relevant postings

Example: “harry”@t8

IDF

“harry”

“potter”

B+-Tree

TF ( d1, 11.2, [t1, t2) ) ( d1, 10.6, [t2, t5) )

“harry”

( d8, 10.9, [t7, t9) )

B+-Tree

widf (“harry”, t8) = 3.08 scan

SLIDE 26

Klaus Berberich – A Time Machine for Text Search

Time-Travel Inverted File Index

Time-travel query q t can be processed by scanning

index lists while ignoring non-relevant postings

Example: “harry”@t8

IDF

“harry”

“potter”

B+-Tree

TF ( d1, 11.2, [t1, t2) ) ( d1, 10.6, [t2, t5) )

“harry”

( d8, 10.9, [t7, t9) )

B+-Tree

widf (“harry”, t8) = 3.08 scan

SLIDE 27

Klaus Berberich – A Time Machine for Text Search

Time-Travel Inverted File Index

Time-travel query q t can be processed by scanning

index lists while ignoring non-relevant postings

Example: “harry”@t8

IDF

“harry”

“potter”

B+-Tree

TF ( d1, 11.2, [t1, t2) ) ( d1, 10.6, [t2, t5) )

“harry”

( d8, 10.9, [t7, t9) )

B+-Tree

widf (“harry”, t8) = 3.08 scan

SLIDE 28

Klaus Berberich – A Time Machine for Text Search

Time-Travel Inverted File Index

Time-travel query q t can be processed by scanning

index lists while ignoring non-relevant postings

Example: “harry”@t8

IDF

“harry”

“potter”

B+-Tree

TF ( d1, 11.2, [t1, t2) ) ( d1, 10.6, [t2, t5) )

“harry”

( d8, 10.9, [t7, t9) )

B+-Tree

widf (“harry”, t8) = 3.08 scan

SLIDE 29

Klaus Berberich – A Time Machine for Text Search

Outline

Motivation Collection, Query, and Relevance Model Time-Travel Inverted File Index

Reducing Index Size Tuning Index Performance

Experimental Evaluation Conclusions

SLIDE 30

Klaus Berberich – A Time Machine for Text Search

Reducing Index Size

Shortcoming: Since we create one posting per

version per term, the resulting index is very large

TF ( d1, 11.2, [t1, t2) ) ( d1, 10.6, [t2, t5) )

“harry”

( d8, 10.9, [t7, t9) )

B+-Tree

HUGE!!!

(Wikipedia Revision History ~8.6B postings)

SLIDE 31

Klaus Berberich – A Time Machine for Text Search

Reducing Index Size

Observation: Changes between document versions

minor (e.g., corrected typos) have no noticeable effect on the ranked result

(e.g., 500 x “harry” vs. 510 x “harry” )

Idea: Coalesce sequences of temporally adjacent

postings having similar scores

time score

non-coalesced coalesced

SLIDE 32

Klaus Berberich – A Time Machine for Text Search

Reducing Index Size

Problem Statement: Given input sequence I find a

minimal length output sequence O with approximation errors bounded by a threshold

Approximate Temporal Coalescing (ATC)

finds an optimal output sequence using a greedy linear time algorithm

p1

score

p’ p3

Guarantee: |p’ - pi| / |pi|

p2

SLIDE 33

Klaus Berberich – A Time Machine for Text Search

Tuning Index Performance

Shortcoming: During query processing many

postings are superfluously read

time

t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 d1 d2 d3

document

1 2 3 4 5 6 7 8 9 10

SLIDE 34

Klaus Berberich – A Time Machine for Text Search

Tuning Index Performance

Shortcoming: During query processing many

postings are superfluously read

time

t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 d1 d2 d3

document

1 2 3 4 5 6 7 8 9 10

t

We read 10 postings, but only {1, 5, 8} are needed

SLIDE 35

Klaus Berberich – A Time Machine for Text Search

Tuning Index Performance

Idea: Materialize smaller sublists containing only

postings that overlap with a smaller time-interval

time

t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 d1 d2 d3

document

1 2 3 4 5 6 7 8 9 10

t

SLIDE 36

Klaus Berberich – A Time Machine for Text Search

Tuning Index Performance

Idea: Materialize smaller sublists containing only

postings that overlap with a smaller time-interval

time

t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 d1 d2 d3

document

1 2 3 4 5 6 7 8 9 10

t

By materializing a sublist for [t1, t2) we can achieve optimal performance for the query

SLIDE 37

Klaus Berberich – A Time Machine for Text Search

Tuning Index Performance

Idea: Materialize smaller sublists containing only

postings that overlap with a smaller time-interval

time

t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 d1 d2 d3

document

1 2 3 4 5 6 7 8 9 10

t

SLIDE 38

Klaus Berberich – A Time Machine for Text Search

Tuning Index Performance

Idea: Materialize smaller sublists containing only

postings that overlap with a smaller time-interval

time

t1 t2 t3 t4 t5 t6 t7 t8 t9 t10 d1 d2 d3

document

1 2 3 4 5 6 7 8 9 10

By materializing a sublist for each elementary time interval we achieve optimal performance

t

SLIDE 39

Klaus Berberich – A Time Machine for Text Search

Tuning Index Performance

So far, we have seen two extreme solutions

space-optimal: keep only a single list (SOPT) performance-optimal: keep one list per

elementary time-interval (POPT)

We propose two systematic techniques to trade-off

space and performance

performance-guarantee: consumes minimal space

while retaining a performance guarantee (PG)

space-bound: achieves best performance while

not exceeding a space limit (SB)

SLIDE 40

Klaus Berberich – A Time Machine for Text Search

Tuning Index Performance

Performance Guarantee (PG)

consumes minimal space guarantees that for any t at most ! nt postings

are read where nt is the number of postings that exist at time t

Optimal solution computable by means of induction

in O(T2) time and O(T2) space (where T is the number of distinct timestamps in the list)

SLIDE 41

Klaus Berberich – A Time Machine for Text Search

Tuning Index Performance

Space Bound (SB)

achieves minimal expected processing cost

(i.e., expected length of the list that is scanned)

consumes at most !n space where n is the

length of the original list

Optimal solution computable using dynamic

programming in O(n4) time and O(n3) space

Approximate solution computable in O(T2) time and

O(T) space using simulated annealing

SLIDE 42

Klaus Berberich – A Time Machine for Text Search

Outline

Motivation Collection, Query, and Relevance Model Time-Travel Inverted File Index

Reducing Index Size Tuning Index Performance

Experimental Evaluation Conclusions

SLIDE 43

Klaus Berberich – A Time Machine for Text Search

Experimental Evaluation – Setup

Implementation:

Java, Oracle 10g

Datasets:

WIKI: Revision history of English Wikipedia (2001-2005)

892K documents / 13,976K versions / 0.7 TBytes

UKGOV: Weekly crawls of 11 .gov.uk sites (2004-2005)

502K documents / 8,687K versions / 0.4 TBytes

Queries:

300 keyword queries from AOL query log that most frequently

produced a result click on en.wikipedia.org / .gov.uk

Each keyword query is assigned one time point per month in

the collection’s lifespan (18K / 7.2K time-travel queries in total)

SLIDE 44

Klaus Berberich – A Time Machine for Text Search

Experimental Evaluation – Setup

Implementation:

Java, Oracle 10g

Datasets:

WIKI: Revision history of English Wikipedia (2001-2005)

892K documents / 13,976K versions / 0.7 TBytes

UKGOV: Weekly crawls of 11 .gov.uk sites (2004-2005)

502K documents / 8,687K versions / 0.4 TBytes

Queries:

300 keyword queries from AOL query log that most frequently

produced a result click on en.wikipedia.org / .gov.uk

Each keyword query is assigned one time point per month in

the collection’s lifespan (18K / 7.2K time-travel queries in total)

WIKI: ten commandments, abraham lincoln, da vinci code, harlem renaissance… UKGOV: 1901 uk census, british royal family, migrant worker statistics, witness intimidation…

SLIDE 45

Klaus Berberich – A Time Machine for Text Search

Experimental Evaluation – Setup

Implementation:

Java, Oracle 10g

Datasets:

WIKI: Revision history of English Wikipedia (2001-2005)

892K documents / 13,976K versions / 0.7 TBytes

UKGOV: Weekly crawls of 11 .gov.uk sites (2004-2005)

502K documents / 8,687K versions / 0.4 TBytes

Queries:

300 keyword queries from AOL query log that most frequently

produced a result click on en.wikipedia.org / .gov.uk

Each keyword query is assigned one time point per month in

the collection’s lifespan (18K / 7.2K time-travel queries in total)

SLIDE 46

Klaus Berberich – A Time Machine for Text Search

Indexes computed for different values of threshold

Approximate Temporal Coalescing

0.01 0.1 1 0.1 0.2 0.3 0.4 0.5 Size of Original Index

!

WIKI UKGOV

SLIDE 47

Klaus Berberich – A Time Machine for Text Search

Indexes computed for different values of threshold

Approximate Temporal Coalescing

4.39% 2.38%

0.01 0.1 1 0.1 0.2 0.3 0.4 0.5 Size of Original Index

!

WIKI UKGOV

SLIDE 48

Klaus Berberich – A Time Machine for Text Search

Approximate Temporal Coalescing

Impact on top-k query results assessed using

Relative Recall @ k in [0,1] Kendall’s @ k in [-1,1]

Computed per dataset for

all time-travel queries (18K / 7.2K) k varying as 10, 25, 50, 100 varying as 0.01, 0.05, 0.10, 0.25, 0.50

We report mean, 5%-percentile, and 95%-percentile

SLIDE 49

Klaus Berberich – A Time Machine for Text Search

Approximate Temporal Coalescing

0.2 0.4 0.6 0.8 1

!

0.01 0.05 0.10 0.25 0.50 Relative Recall @ 100 (WIKI) Kendall’s " @ 100 (WIKI) Relative Recall @ 100 (UKGOV) Kendall’s " @ 100 (UKGOV)

SLIDE 50

Klaus Berberich – A Time Machine for Text Search

Sublist Materialization

Threshold for ATC fixed as = 0.10 For terms in query workloads (422/522) we apply

SOPT and POPT PG for varying between 1.10 and 3.00 SB for varying between 1.10 and 3.00

We report

Space, i.e., total number of postings in

materialized sublists

Expected Processing Cost (EPC), i.e., expected

length of scanned list for random term and time

SLIDE 51

Klaus Berberich – A Time Machine for Text Search

Performance Guarantee

WIKI UKGOV Space EPC Space EPC POPT 14,428% 100% 11,406% 100% SOPT 100% 963% 100% 147% WIKI UKGOV Space EPC Space EPC = 1.10 1,004% 106% 616% 103% = 1.50 295% 132% 233% 117% = 2.00 195% 160% 163% 125% = 3.00 145% 207% 132% 133%

Performance Guarantee

EPC = Expected Processing Cost

SLIDE 52

Klaus Berberich – A Time Machine for Text Search

Performance Guarantee

WIKI UKGOV Space EPC Space EPC POPT 14,428% 100% 11,406% 100% SOPT 100% 963% 100% 147% WIKI UKGOV Space EPC Space EPC = 1.10 1,004% 106% 616% 103% = 1.50 295% 132% 233% 117% = 2.00 195% 160% 163% 125% = 3.00 145% 207% 132% 133%

Performance Guarantee

EPC = Expected Processing Cost

SLIDE 53

Klaus Berberich – A Time Machine for Text Search

Performance Guarantee

WIKI UKGOV Space EPC Space EPC POPT 14,428% 100% 11,406% 100% SOPT 100% 963% 100% 147% WIKI UKGOV Space EPC Space EPC = 1.10 1,004% 106% 616% 103% = 1.50 295% 132% 233% 117% = 2.00 195% 160% 163% 125% = 3.00 145% 207% 132% 133%

Performance Guarantee

EPC = Expected Processing Cost

SLIDE 54

Klaus Berberich – A Time Machine for Text Search

Performance Guarantee

WIKI UKGOV Space EPC Space EPC POPT 14,428% 100% 11,406% 100% SOPT 100% 963% 100% 147% WIKI UKGOV Space EPC Space EPC = 1.10 1,004% 106% 616% 103% = 1.50 295% 132% 233% 117% = 2.00 195% 160% 163% 125% = 3.00 145% 207% 132% 133%

Performance Guarantee

EPC = Expected Processing Cost

SLIDE 55

Klaus Berberich – A Time Machine for Text Search

Performance Guarantee

WIKI UKGOV Space EPC Space EPC POPT 14,428% 100% 11,406% 100% SOPT 100% 963% 100% 147% WIKI UKGOV Space EPC Space EPC = 1.10 1,004% 106% 616% 103% = 1.50 295% 132% 233% 117% = 2.00 195% 160% 163% 125% = 3.00 145% 207% 132% 133%

Performance Guarantee

EPC = Expected Processing Cost

SLIDE 56

Klaus Berberich – A Time Machine for Text Search

Space Bound

WIKI UKGOV Space EPC Space EPC POPT 14,428% 100% 11,406% 100% SOPT 100% 963% 100% 147% WIKI UKGOV Space EPC Space EPC = 3.00 288% 139% 273% 107% = 2.00 194% 171% 180% 119% = 1.50 146% 214% 131% 131% = 1.10 109% 406% 104% 145%

Space Bound

EPC = Expected Processing Cost

SLIDE 57

Klaus Berberich – A Time Machine for Text Search

Space Bound

WIKI UKGOV Space EPC Space EPC POPT 14,428% 100% 11,406% 100% SOPT 100% 963% 100% 147% WIKI UKGOV Space EPC Space EPC = 3.00 288% 139% 273% 107% = 2.00 194% 171% 180% 119% = 1.50 146% 214% 131% 131% = 1.10 109% 406% 104% 145%

Space Bound

EPC = Expected Processing Cost

SLIDE 58

Klaus Berberich – A Time Machine for Text Search

Space Bound

WIKI UKGOV Space EPC Space EPC POPT 14,428% 100% 11,406% 100% SOPT 100% 963% 100% 147% WIKI UKGOV Space EPC Space EPC = 3.00 288% 139% 273% 107% = 2.00 194% 171% 180% 119% = 1.50 146% 214% 131% 131% = 1.10 109% 406% 104% 145%

Space Bound

EPC = Expected Processing Cost

SLIDE 59

Klaus Berberich – A Time Machine for Text Search

Space Bound

WIKI UKGOV Space EPC Space EPC POPT 14,428% 100% 11,406% 100% SOPT 100% 963% 100% 147% WIKI UKGOV Space EPC Space EPC = 3.00 288% 139% 273% 107% = 2.00 194% 171% 180% 119% = 1.50 146% 214% 131% 131% = 1.10 109% 406% 104% 145%

Space Bound

EPC = Expected Processing Cost

SLIDE 60

Klaus Berberich – A Time Machine for Text Search

Space Bound

WIKI UKGOV Space EPC Space EPC POPT 14,428% 100% 11,406% 100% SOPT 100% 963% 100% 147% WIKI UKGOV Space EPC Space EPC = 3.00 288% 139% 273% 107% = 2.00 194% 171% 180% 119% = 1.50 146% 214% 131% 131% = 1.10 109% 406% 104% 145%

Space Bound

EPC = Expected Processing Cost

SLIDE 61

Klaus Berberich – A Time Machine for Text Search

Outline

Motivation Collection, Query, and Relevance Model Time-Travel Inverted File Index

Reducing Index Size Tuning Index Performance

Experimental Evaluation Conclusions

SLIDE 62

Klaus Berberich – A Time Machine for Text Search

Conclusions

Time-Travel Text Search

an interesting & important research problem!

Our Time Machine

building on inverted file index significant reduction of index size tunable index performance

Experimental Evaluation

ver two large-scale real-world datasets

demonstrating efficiency & effectiveness

SLIDE 63

Klaus Berberich – A Time Machine for Text Search

Demo at VLDB ‘07

September 23-28, Vienna, Austria

Web-based GUI

FLUXCAPACITOR Server

Query Processor DB DB DB Time-Travel Text Index Metadata & Snippets IDF Score Time-Series

Temporally Versioned Text Collection

SLIDE 64

Klaus Berberich – A Time Machine for Text Search

Thank you! Questions?

SLIDE 65

Klaus Berberich – A Time Machine for Text Search

Experimental Evaluation – PG & SB

10000 100000 1e+09 1e+10 Space PG (UKGOV) SB (UKGOV) SOPT (UKGOV) POPT (UKGOV) 10000 100000 1e+09 1e+10 Expected Processing Cost Space PG (WIKI) SB (WIKI) SOPT (WIKI) POPT (WIKI)