mirror of
https://github.com/qdrant/landing_page.git
synced 2026-09-27 15:08:30 +02:00
Article: Product Quantization (#140)
* Add PQ article draft * Grammar fixes * Adapt full process image * Add an image about distance calculation * Add missing descriptions * docs auto-sync * Adapt social preview image * Change the preview image * Adapt the publishing date * Add PQ article draft * Grammar fixes * Adapt full process image * Add an image about distance calculation * Add missing descriptions * Adapt social preview image * Change the preview image * Adapt the publishing date * Change preview image * Update qdrant-landing/content/articles/product-quantization.md Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com> * Adapt publication date * Modify weights --------- Co-authored-by: qdrant <qdrant@users.noreply.github.com> Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
This commit is contained in:
Kacper Łukawski
co-authored by
Andrey Vasnetsov
qdrant
Andrey Vasnetsov
qdrant
parent
c973db23e5
commit
6cfa2b525d
@@ -5,7 +5,7 @@ description: "ChatGPT factuality might be improved with semantic search. Here is
|
||||
social_preview_image: /articles_data/chatgpt-plugin/social_preview.jpg
|
||||
small_preview_image: /articles_data/chatgpt-plugin/chatgpt-plugin-icon.svg
|
||||
preview_dir: /articles_data/chatgpt-plugin/preview
|
||||
weight: 3
|
||||
weight: 4
|
||||
author: Kacper Łukawski
|
||||
author_link: https://medium.com/@lukawskikacper
|
||||
date: 2023-03-23T18:01:00+01:00
|
||||
|
||||
@@ -0,0 +1,221 @@
|
||||
---
|
||||
title: "Qdrant under the hood: Product Quantization"
|
||||
short_description: "Vector search with low memory? Try out our brand-new Product Quantization!"
|
||||
description: "Vector search with low memory? Try out our brand-new Product Quantization!"
|
||||
social_preview_image: /articles_data/product-quantization/preview/social_preview.jpg
|
||||
small_preview_image: /articles_data/product-quantization/product-quantization-icon.svg
|
||||
preview_dir: /articles_data/product-quantization/preview
|
||||
weight: 1
|
||||
author: Kacper Łukawski
|
||||
author_link: https://medium.com/@lukawskikacper
|
||||
date: 2023-05-30T09:45:00+02:00
|
||||
draft: false
|
||||
keywords:
|
||||
- vector search
|
||||
- product quantization
|
||||
- memory optimization
|
||||
---
|
||||
|
||||
Qdrant 1.1.0 brought the support of [Scalar Quantization](/articles/scalar-quantization/),
|
||||
a technique of reducing the memory footprint by even four times, by using `int8` to represent
|
||||
the values that would be normally represented by `float32`.
|
||||
|
||||
The memory usage in vector search might be reduced even further! Please welcome **Product
|
||||
Quantization**, a brand-new feature of Qdrant 1.2.0!
|
||||
|
||||
## Product Quantization
|
||||
|
||||
Product Quantization converts floating-point numbers into integers like every other quantization
|
||||
method. However, the process is slightly more complicated than Scalar Quantization and is more
|
||||
customizable, so you can find the sweet spot between memory usage and search precision. This article
|
||||
covers all the steps required to perform Product Quantization and the way it's implemented in Qdrant.
|
||||
|
||||
Let’s assume we have a few vectors being added to the collection and that our optimizer decided
|
||||
to start creating a new segment.
|
||||
|
||||
|
||||
|
||||
### Cutting the vector into pieces
|
||||
|
||||
First of all, our vectors are going to be divided into **chunks** aka **subvectors**. The number
|
||||
of chunks is configurable, but as a rule of thumb - the lower it is, the higher the compression rate.
|
||||
That also comes with reduced search precision, but in some cases, you may prefer to keep the memory
|
||||
usage as low as possible.
|
||||
|
||||
|
||||
|
||||
Qdrant API allows choosing the compression ratio from 4x up to 64x. In our example, we selected 16x,
|
||||
so each subvector will consist of 4 floats (16 bytes), and it will eventually be represented by
|
||||
a single byte.
|
||||
|
||||
### Clustering
|
||||
|
||||
The chunks of our vectors are then used as input for clustering. Qdrant uses the K-means algorithm,
|
||||
with $ K = 256 $. It was selected a priori, as this is the maximum number of values a single byte
|
||||
represents. As a result, we receive a list of 256 centroids for each chunk and assign each of them
|
||||
a unique id. **The clustering is done separately for each group of chunks.**
|
||||
|
||||
|
||||
|
||||
Each chunk of a vector might now be mapped to the closest centroid. That’s where we lose the precision,
|
||||
as a single point will only represent a whole subspace. Instead of using a subvector, we can store
|
||||
the id of the closest centroid. If we repeat that for each chunk, we can approximate the original
|
||||
embedding as a vector of subsequent ids of the centroids. The dimensionality of the created vector
|
||||
is equal to the number of chunks, in our case 2.
|
||||
|
||||
|
||||
|
||||
### Full process
|
||||
|
||||
All those steps build the following pipeline of Product Quantization:
|
||||
|
||||
|
||||
|
||||
## Measuring the distance
|
||||
|
||||
Vector search relies on the distances between the points. Enabling Product Quantization slightly changes
|
||||
the way it has to be calculated. The query vector is divided into chunks, and then we figure the overall
|
||||
distance as a sum of distances between the subvectors and the centroids assigned to the specific id of
|
||||
the vector we compare to. We know the coordinates of the centroids, so that's easy.
|
||||
|
||||
|
||||
|
||||
#### Qdrant implementation
|
||||
|
||||
Search operation requires calculating the distance to multiple points. Since we calculate the
|
||||
distance to a finite set of centroids, those might be precomputed and reused. Qdrant creates
|
||||
a lookup table for each query, so it can then simply sum up several terms to measure the
|
||||
distance between a query and all the centroids.
|
||||
|
||||
| | Centroid 0 | Centroid 1 | ... |
|
||||
|-------------|------------|------------|-----|
|
||||
| **Chunk 0** | 0.14213 | 0.51242 | |
|
||||
| **Chunk 1** | 0.08421 | 0.00142 | |
|
||||
| **...** | ... | ... | ... |
|
||||
|
||||
## Benchmarks
|
||||
|
||||
Product Quantization comes with a cost - there are some additional operations to perform so
|
||||
that the performance might be reduced. However, memory usage might be reduced drastically as
|
||||
well. As usual, we did some benchmarks to give you a brief understanding of what you may expect.
|
||||
|
||||
Again, we reused the same pipeline as in [the other benchmarks we published](/benchmarks). We
|
||||
selected [Arxiv-titles-384-angular-no-filters](https://github.com/qdrant/ann-filtering-benchmark-datasets)
|
||||
and [Glove-100](https://github.com/erikbern/ann-benchmarks/) datasets to measure the impact
|
||||
of Product Quantization on precision and time. Both experiments were launched with $ EF = 128 $.
|
||||
The results are summarized in the tables:
|
||||
|
||||
#### Glove-100
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th></th>
|
||||
<th>Original</th>
|
||||
<th>1D clusters</th>
|
||||
<th>2D clusters</th>
|
||||
<th>3D clusters</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<th>Mean precision</th>
|
||||
<td>0.7158</td>
|
||||
<td>0.7143</td>
|
||||
<td>0.6731</td>
|
||||
<td>0.5854</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Mean search time</th>
|
||||
<td>2336 µs</td>
|
||||
<td>2750 µs</td>
|
||||
<td>2597 µs</td>
|
||||
<td>2534 µs</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Compression</th>
|
||||
<td>x1</td>
|
||||
<td>x4</td>
|
||||
<td>x8</td>
|
||||
<td>x12</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Upload & indexing time</th>
|
||||
<td>147 s</td>
|
||||
<td>339 s</td>
|
||||
<td>217 s</td>
|
||||
<td>178 s</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
Product Quantization increases both indexing and searching time. The higher the compression ratio,
|
||||
the lower the search precision. The main benefit is undoubtedly the reduced usage of memory.
|
||||
|
||||
#### Arxiv-titles-384-angular-no-filters
|
||||
|
||||
<table>
|
||||
<thead>
|
||||
<tr>
|
||||
<th></th>
|
||||
<th>Original</th>
|
||||
<th>1D clusters</th>
|
||||
<th>2D clusters</th>
|
||||
<th>4D clusters</th>
|
||||
<th>8D clusters</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody>
|
||||
<tr>
|
||||
<th>Mean precision</th>
|
||||
<td>0.9837</td>
|
||||
<td>0.9677</td>
|
||||
<td>0.9143</td>
|
||||
<td>0.8068</td>
|
||||
<td>0.6618</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Mean search time</th>
|
||||
<td>2719 µs</td>
|
||||
<td>4134 µs</td>
|
||||
<td>2947 µs</td>
|
||||
<td>2175 µs</td>
|
||||
<td>2053 µs</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Compression</th>
|
||||
<td>x1</td>
|
||||
<td>x4</td>
|
||||
<td>x8</td>
|
||||
<td>x16</td>
|
||||
<td>x32</td>
|
||||
</tr>
|
||||
<tr>
|
||||
<th>Upload & indexing time</th>
|
||||
<td>332 s</td>
|
||||
<td>921 s</td>
|
||||
<td>597 s</td>
|
||||
<td>481 s</td>
|
||||
<td>474 s</td>
|
||||
</tr>
|
||||
</tbody>
|
||||
</table>
|
||||
|
||||
It turns out that in some cases, Product Quantization may not only reduce the memory usage,
|
||||
but also the search time.
|
||||
|
||||
## Good practices
|
||||
|
||||
Compared to Scalar Quantization, Product Quantization offers a higher compression rate. However, this comes with considerable trade-offs in accuracy, and at times, in-RAM search speed.
|
||||
|
||||
Product Quantization tends to be favored in certain specific scenarios:
|
||||
|
||||
- Deployment in a low-RAM environment where the limiting factor is the number of disk reads rather than the vector comparison itself
|
||||
- Situations where the dimensionality of the original vectors is sufficiently high
|
||||
- Cases where indexing speed is not a critical factor
|
||||
|
||||
In circumstances that do not align with the above, Scalar Quantization should be the preferred choice.
|
||||
|
||||
Qdrant documentation on [Product Quantization](/documentation/quantization/#setting-up-product-quantization)
|
||||
will help you to set and configure the new quantization for your data and achieve even
|
||||
up to 64x memory reduction.
|
||||
@@ -5,7 +5,7 @@ description: "Check out what Qdrant 1.2 brings to vector search"
|
||||
social_preview_image: /articles_data/qdrant-1.2.x/social_preview.png
|
||||
small_preview_image: /articles_data/qdrant-1.2.x/icon.svg
|
||||
preview_dir: /articles_data/qdrant-1.2.x/preview
|
||||
weight: 1
|
||||
weight: 2
|
||||
author: Kacper Łukawski
|
||||
author_link: https://medium.com/@lukawskikacper
|
||||
date: 2023-05-24T10:45:00+02:00
|
||||
|
||||
@@ -5,7 +5,7 @@ description: "Scalar Quantization is a newly introduced mechanism of reducing th
|
||||
social_preview_image: /articles_data/scalar-quantization/social_preview.png
|
||||
small_preview_image: /articles_data/scalar-quantization/scalar-quantization-icon.svg
|
||||
preview_dir: /articles_data/scalar-quantization/preview
|
||||
weight: 2
|
||||
weight: 3
|
||||
author: Kacper Łukawski
|
||||
author_link: https://medium.com/@lukawskikacper
|
||||
date: 2023-03-27T10:45:00+01:00
|
||||
|
||||
@@ -5,7 +5,7 @@ description: We announce Qdrant seed round investment and share our thoughts on
|
||||
preview_dir: /articles_data/seed-round/preview
|
||||
social_preview_image: /articles_data/seed-round/seed-social.png
|
||||
small_preview_image: /articles_data/quantum-quantization/icon.svg
|
||||
weight: 3
|
||||
weight: 4
|
||||
author: Andre Zayarni
|
||||
draft: false
|
||||
author_link: https://www.linkedin.com/in/zayarni
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 152 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 341 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 387 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 636 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 39 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 32 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 249 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 104 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 85 KiB |
@@ -0,0 +1,7 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<!-- Uploaded to: SVG Repo, www.svgrepo.com, Generator: SVG Repo Mixer Tools -->
|
||||
<svg width="800px" height="800px" viewBox="0 0 1024 1024" class="icon" version="1.1"
|
||||
xmlns="http://www.w3.org/2000/svg">
|
||||
<path d="M697.051429 674.157714c-29.622857 0-59.245714 11.922286-83.017143 23.917715L412.013714 542.866286c5.924571-17.92 11.849143-35.84 11.849143-59.757715 0-23.917714-5.851429-41.837714-11.849143-59.757714l194.413715-149.430857c17.773714 17.92 47.469714 29.842286 77.165714 29.842286a147.456 147.456 0 0 0 148.406857-149.430857A147.456 147.456 0 0 0 683.52 4.900571 147.456 147.456 0 0 0 535.259429 154.331429c0 29.915429 11.849143 59.830857 23.698285 83.675428L382.317714 369.517714c-29.696-35.84-65.316571-59.757714-106.788571-65.755428v-65.755429c53.394286-11.922286 89.014857-59.757714 89.014857-113.517714 0-65.828571-53.394286-119.588571-118.710857-119.588572S127.122286 58.733714 127.122286 124.489143c0 53.76 35.547429 101.595429 89.014857 113.517714v65.828572C127.122286 321.682286 61.805714 393.362286 61.805714 483.035429s65.316571 161.426286 148.406857 173.348571v65.755429c-53.394286 11.922286-89.088 59.757714-89.088 113.590857 0 65.755429 53.394286 119.515429 118.710858 119.515428s118.784-53.76 118.784-119.515428c0-53.833143-35.620571-101.668571-89.088-113.590857v-65.755429c41.545143-5.997714 83.090286-29.915429 106.861714-65.755429l196.022857 149.211429c-11.849143 23.917714-23.698286 53.76-23.698286 83.675429 0 83.675429 65.316571 149.430857 148.406857 149.430857a147.456 147.456 0 0 0 148.406858-149.430857 147.456 147.456 0 0 0-148.48-149.430858zM683.666286 64.658286c47.469714 0 89.014857 41.910857 89.014857 89.673143 0 47.835429-41.545143 89.673143-89.014857 89.673142-47.542857 0-89.088-41.837714-89.088-89.673142 0-47.762286 41.545143-89.673143 89.088-89.673143z m-441.782857 300.032c62.244571 0 116.736 54.857143 116.736 117.467428 0 62.683429-54.491429 117.467429-116.662858 117.467429s-116.662857-54.857143-116.662857-117.467429c0-62.610286 54.418286-117.467429 116.662857-117.467428z m-61.293715-240.201143c0-35.84 23.771429-59.830857 59.318857-59.830857 35.620571 0 59.392 23.917714 59.392 59.830857 0 35.84-23.771429 59.757714-59.392 59.757714-35.547429 0-59.318857-23.917714-59.318857-59.757714z m118.710857 717.238857c0 35.84-23.771429 59.830857-59.392 59.830857-35.547429 0-59.318857-23.917714-59.318857-59.830857 0-35.84 23.771429-59.757714 59.318857-59.757714 35.620571 0 59.392 23.917714 59.392 59.757714z m397.897143 71.533714c-47.542857 0-89.014857-41.837714-89.014857-89.673143s41.545143-89.673143 89.014857-89.673142c47.542857 0 89.014857 41.837714 89.014857 89.673142s-41.545143 89.673143-89.014857 89.673143z"
|
||||
fill="#ffffff"/>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 2.7 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 93 KiB |
Binary file not shown.
|
After Width: | Height: | Size: 7.4 KiB |
Reference in New Issue
Block a user