Reconfigurable Inverted Index Yusuke Matsui 1 Ryota Hinami 2 Shinichi - - PowerPoint PPT Presentation

Reconfigurable Inverted Index

Yusuke Matsui1 Ryota Hinami2 Shin’ichi Satoh1

1National Institute of Informatics 2The University of Tokyo

slides: https://bit.ly/2P0KuW1

0.23 3.15 0.65 1.43 5.22 0.54 1.66 0.74 4.63 6.21 0.72 0.31 0.86 3.44 1.12 0.04

…

Query

0.20 3.25 0.72 1.68

𝒓 ∈ ℝ𝐸 𝒚1 𝒚2 𝒚𝑂 𝒚74 argmin

𝑜∈ 1,…,𝑂

𝒓 − 𝒚𝑜

2 2

Database vectors Result ANN system

Add hash-table, trees, inverted-index, etc

Approximate NN search

slides: https://bit.ly/2P0KuW1

Approximate nearest neighbor search

0.23 3.15 0.65 1.43 5.22 0.54 1.66 0.74 4.63 6.21 0.72 0.31 0.86 3.44 1.12 0.04

…

Query

0.20 3.25 0.72 1.68

𝒓 ∈ ℝ𝐸 𝒚1 𝒚2 𝒚𝑂 𝒚74 argmin

𝑜∈ 1,…,𝑂

𝒓 − 𝒚𝑜

2 2

Database vectors Result ANN system

Add hash-table, trees, inverted-index, etc

Approximate NN search

slides: https://bit.ly/2P0KuW1

Approximate nearest neighbor search

0.23 3.15 0.65 1.43 5.22 0.54 1.66 0.74 4.63 6.21 0.72 0.31 0.86 3.44 1.12 0.04

…

Query

0.20 3.25 0.72 1.68

𝒓 ∈ ℝ𝐸 𝒚1 𝒚2 𝒚𝑂 𝒚74 argmin

𝑜∈ 1,…,𝑂

𝒓 − 𝒚𝑜

2 2

Database vectors Result ANN system

Add hash-table, trees, inverted-index, etc

Approximate NN search

slides: https://bit.ly/2P0KuW1

Approximate nearest neighbor search

Related work

slides: https://bit.ly/2P0KuW1

➢ Locality-sensitive-hashing (LSH)

• FALCONN [Andoni+, 15] [Razenshteyn+, 18]

➢ Project/tree-based

• FLANN [Muja+, 14]
• Annoy [Bernhardsson, 18]

➢ Graph traversal

• NSW/HNSW on NMSLIB [Malkov+, 16][Boytsov+, 13]

➢ Product quantization (PQ)

• IVFPQ on Faiss [Jégou+, 11][Johnson+, 17] etc.
• Our Reconfigurable Inverted Index

0.23 3.15 0.65 1.43

Query

0.20 3.25 0.72 1.68

𝒓 ∈ ℝ𝐸 𝒚74 argmin

𝑜∈𝒯

𝒓 − 𝒚𝑜

2 2

Result ANN system Approximate NN Search

Subset search problem

➢ Existing ANN systems are fast for the all vectors

• Search is over 𝒯 = 1, … , 𝑂

➢ However, it is hard to run the search for a subset

• Search is over 𝒯 ⊆ 1, … , 𝑂
• e.g., searching from 𝒚1000, … , 𝒚2000
• Why? Systems are usually optimized for 𝒯 = 1, … , 𝑂

slides: https://bit.ly/2P0KuW1

There is a demand for subset search!

slides: https://bit.ly/2P0KuW1

There is a demand for subset search!

Propose: Reconfigurable inverted index (Rii) ✓Subset search ✓A comparative performance with IVFPQ (Faiss) ✓10 ms for billion-scale data

slides: https://bit.ly/2P0KuW1

Reconfigurable inverted index (Rii)

➢Preliminary

• PQ linear scan
• IVFPQ

➢Data structure ➢Search

slides: https://bit.ly/2P0KuW1

𝒯

Runtime

Fast if |𝒯| is large Fast if |𝒯| is small Cherry pick! Always fast

𝒯

Runtime

𝒯

Runtime

Reconfigurable inverted index (Rii)

➢Preliminary

• PQ linear scan
• IVFPQ

➢Data structure ➢Search

slides: https://bit.ly/2P0KuW1

𝒯

Runtime

Fast if |𝒯| is large Fast if |𝒯| is small Cherry pick! Always fast

𝒯

Runtime

𝒯

Runtime

Preliminary: Product quantization (PQ) [Jégou+, TPAMI 11]

5.22 0.54 1.66 0.74 4.63 6.21 0.72 0.31 0.86 3.44 1.12 0.04

…

𝒚1 𝒚2 𝒚𝑂

5.22 0.54 1.66 0.74

PQ: Compress a vector into a short code

ℝ4 → … 2

All database vectors are PQ-encoded beforehand PQ

1 2 N

slides: https://bit.ly/2P0KuW1

, ,

…

PQ PQ

Preliminary: Product quantization (PQ) [Jégou+, TPAMI 11]

0.23 3.15 0.65 1.43 Linearly compared

𝒓 ∈ ℝ𝐸

➢The subset search is possible with a linear cost of 𝒯 ➢The search is efficient

• nly if 𝒯 is small

e.g., 𝒯 = 2, 4, 5, 8

𝒯

Runtime

𝑂

slides: https://bit.ly/2P0KuW1

1 2 N

…

3 4 5 6

argmin

𝑜∈𝒯

𝑒 𝒓,

𝑜

✔ ✔ ✔

Reconfigurable inverted index (Rii)

➢Preliminary

• PQ linear scan
• IVFPQ

➢Data structure ➢Search ➢Evaluation

slides: https://bit.ly/2P0KuW1

𝒯

Runtime

Fast if |𝒯| is large Fast if |𝒯| is small Cherry pick! Always fast

𝒯

Runtime

𝒯

Runtime

Preliminary: Inverted Index + PQ (IVFPQ) [Jégou+, TPAMI 11]

➢Current basic data structure for a large-scale search ➢Subset-search is possible only if 𝒯 is large

126 225 13 92 188

…

𝒅13 𝒅2 𝒅1 𝒅5 Space partitioning

slides: https://bit.ly/2P0KuW1

Preliminary: Inverted Index + PQ (IVFPQ) [Jégou+, TPAMI 11]

➢Current basic data structure for a large-scale search ➢Subset-search is possible only if 𝒯 is large

126 225 13 92 188

…

𝒅13 𝒅2 𝒅1 𝒅5 Space partitioning

0.23 3.15 0.65 1.43

𝒓 ∈ ℝ𝐸

Re-rank via PQ-linear scan

1.Find the closest space: 𝑙∗ = argmin𝑙 𝒓 − 𝒅𝑙

2 2

2.Focus the 𝑙∗th space, accept items ∈ 𝒯 3.Re-rank the items via PQ-linear scan

slides: https://bit.ly/2P0KuW1

e.g., 𝒯 = 13, 92, 105, …

𝑜 ∈ 𝒯 or not

✔ ✔

92

Preliminary: Inverted Index + PQ (IVFPQ) [Jégou+, TPAMI 11]

➢Current basic data structure for a large-scale search ➢Subset-search is possible only if 𝒯 is large

126 225 13 92 188

…

𝒅13 𝒅2 𝒅1 𝒅5 Space partitioning

0.23 3.15 0.65 1.43

𝒓 ∈ ℝ𝐸

Re-rank via PQ-linear scan

1.Find the closest space: 𝑙∗ = argmin𝑙 𝒓 − 𝒅𝑙

2 2

2.Focus the 𝑙∗th space, accept items ∈ 𝒯 3.Re-rank the items via PQ-linear scan

slides: https://bit.ly/2P0KuW1

e.g., 𝒯 = 13, 92, 105, …

𝑜 ∈ 𝒯 or not

✔ ✔

92

Why is it slow for small 𝒯 ?

e.g., if 𝒯 is small and they are far away from the query, we might need to scan all items

𝒯

Runtime

𝑂

Reconfigurable inverted index (Rii)

➢Preliminary

• PQ linear scan
• IVFPQ

➢Data structure ➢Search

slides: https://bit.ly/2P0KuW1

𝒯

Runtime

Fast if |𝒯| is large Fast if |𝒯| is small Cherry pick! Always fast

𝒯

Runtime

𝒯

Runtime

Data structure

➢Store (1) PQ-codes linearly, and (2) IDs as an inverted index ➢Can run either PQ-linear-scan or IVFPQ with a single data structure

126 225 13 92 188

…

𝒅2 𝒅1 𝒅5

1 2 N

…

13

…

𝒅2 𝒅1 𝒅5

• cf. IVFPQ

slides: https://bit.ly/2P0KuW1

Key: store codes linearly

126 225 13 92 188

➢ PQ-codes are also chunked. Natural ➢ Slight, but critical change

Reconfigurable inverted index (Rii)

➢Preliminary

• PQ linear scan
• IVFPQ

➢Data structure ➢Search

slides: https://bit.ly/2P0KuW1

𝒯

Runtime

Fast if |𝒯| is large Fast if |𝒯| is small Cherry pick! Always fast

𝒯

Runtime

𝒯

Runtime

Search

➢If 𝒯 is small, run PQ-linear scan ➢If 𝒯 is large, run IVFPQ

126 225 13 92 188

…

𝒅2 𝒅1 𝒅5

1 2 N

…

13

…

0.23 3.15 0.65 1.43

𝒓 ∈ ℝ𝐸

slides: https://bit.ly/2P0KuW1

… …

𝒯

𝑂

𝒯

Runtime

𝑂

Runtime

Search

➢If 𝒯 is small, run PQ-linear scan ➢If 𝒯 is large, run IVFPQ

126 225 13 92 188

…

𝒅2 𝒅1 𝒅5

1 2 N

…

13

…

0.23 3.15 0.65 1.43

𝒓 ∈ ℝ𝐸

slides: https://bit.ly/2P0KuW1

… …

𝒯

𝑂

𝒯

Runtime

𝑂

Runtime

Search

➢If 𝒯 is small, run PQ-linear scan ➢If 𝒯 is large, run IVFPQ

126 225 13 92 188

…

𝒅2 𝒅1 𝒅5

1 2 N

…

13

…

0.23 3.15 0.65 1.43

𝒓 ∈ ℝ𝐸

fetch

slides: https://bit.ly/2P0KuW1

… …

𝒯

𝑂

𝒯

Runtime

𝑂

Runtime

Search

➢If 𝒯 is small, run PQ-linear scan ➢If 𝒯 is large, run IVFPQ

126 225 13 92 188

…

𝒅2 𝒅1 𝒅5

1 2 N

…

13

…

0.23 3.15 0.65 1.43

𝒓 ∈ ℝ𝐸

fetch

slides: https://bit.ly/2P0KuW1

… …

𝒯

𝑂

𝒯

Runtime

𝑂

Runtime

Search

➢If 𝒯 is small, run PQ-linear scan ➢If 𝒯 is large, run IVFPQ

126 225 13 92 188

…

𝒅2 𝒅1 𝒅5

1 2 N

…

13

…

𝒯

Runtime

𝑂

0.23 3.15 0.65 1.43

𝒓 ∈ ℝ𝐸

fetch

𝜄

➢Set a threshold 𝜄 ➢Key: Switch two methods based on 𝒯 ≶ 𝜄

slides: https://bit.ly/2P0KuW1

Use PQ-linear-scan Use IVFPQ

… …

Search

➢If 𝒯 is small, run PQ-linear scan ➢If 𝒯 is large, run IVFPQ

126 225 13 92 188

…

𝒅2 𝒅1 𝒅5

1 2 N

…

13

…

𝒯

Runtime

𝑂

0.23 3.15 0.65 1.43

𝒓 ∈ ℝ𝐸

fetch

𝜄

➢Set a threshold 𝜄 ➢Key: Switch two methods based on 𝒯 ≶ 𝜄

slides: https://bit.ly/2P0KuW1

Use PQ-linear-scan Use IVFPQ

… …

slides: https://bit.ly/2P0KuW1

Evaluation

➢SIFT1M (𝑂 = 106, 𝐸 = 128). Results for top-R search

slides: https://bit.ly/2P0KuW1

Evaluation

➢SIFT1M (𝑂 = 106, 𝐸 = 128). Results for top-R search

➢ Existing system: Annoy ➢ Force to search a subset The existing system is slow, especially when 𝒯 is small Proposed Rii is always fast regardless of 𝒯 and 𝑆

$ pip install rii

import rii import nanopq # Prepare a PQ/OPQ codec with M=32 sub spaces codec = nanopq.PQ(M=32).fit(vecs=Xt) # Trained using Xt # Instantiate a Rii class with the codec e = rii.Rii(fine_quantizer=codec) # Add vectors e.add_configure(vecs=X) # Search ids, dists = e.query(q=q, topk=3, target_ids=S) print(ids, dists) # e.g., [7484 8173 1556] [15.062 15.385 16.169]

slides: https://bit.ly/2P0KuW1

https://github.com/matsui528/rii

Summary

slides: https://bit.ly/2P0KuW1 0.23 3.15 0.65 1.43

Query

argmin

𝑜∈𝒯

𝒓 − 𝒚𝑜

2 2

ANN system

Approximate NN Search

Reconfigurable inverted index: ➢Store PQ-codes linearly ➢Switch method based on 𝒯

𝒯

Runtime

𝑂

𝜄

Use PQ-linear-scan Use IVFPQ

Result ➢PyPI: ➢One thing I couldn’t mention: ➢See our paper, or come to our poster: ✓ Poster session 5 (15:30 – 16:30) Reconfiguration: the system remains fast even after many new items are added $ pip install rii

0.23 3.15 0.65 1.43

Query

0.20 3.25 0.72 1.68

𝒓 ∈ ℝ𝐸 𝒚74 argmin

𝑜∈ 1,…,𝑂

𝒓 − 𝒚𝑜

2 2

Result ANN system Approximate NN search

Extract a VGG feature

74th image Query image

Example: Image search

𝑂 database images

slides: https://bit.ly/2P0KuW1

0.23 3.15 0.65 1.43

Query

0.20 3.25 0.72 1.68

𝒓 ∈ ℝ𝐸 𝒚92 argmin

𝑜∈𝒯

𝒓 − 𝒚𝑜

2 2

Result ANN system Approximate NN search

Extract a VGG feature

Query image 𝑂 database images

slides: https://bit.ly/2P0KuW1

Filtering by shooting-date:

• Taken in 2018
• 𝒯 = 34, 56, 92, … , 663

92nd image

Example of subset search for image

Evaluation

➢Extensive comparison against existing methods ➢For a fixed accuracy (Recall@1), check runtime and its disk space

slides: https://bit.ly/2P0KuW1

Evaluation

➢Extensive comparison against existing methods ➢For a fixed accuracy (Recall@1), check runtime and its disk space

NMSLIB is extremely fast, but consume relatively large disk space (∼memory)

slides: https://bit.ly/2P0KuW1

Evaluation

➢Extensive comparison against existing methods ➢For a fixed accuracy (Recall@1), check runtime and its disk space

NMSLIB is extremely fast, but consume relatively large disk space (∼memory) Proposed Rii achieved a comparative performance with Faiss

slides: https://bit.ly/2P0KuW1