krzysztof/landing_page
1
0
Fork 0
mirror of https://github.com/qdrant/landing_page.git synced 2026-10-06 11:28:31 +02:00
Code Issues Packages Projects Releases Wiki Activity

Initial draft of tutorials restructure

Browse Source
This commit is contained in:
kanungle
2025-12-22 15:03:47 -08:00
parent 00c6a700e2
commit 5993298548
26 changed files with 186 additions and 257 deletions
Download Patch File Download Diff File Expand all files Collapse all files
qdrant-landing/content/documentation/tutorials-search-engineering/_index.md
+24
View File
@@ -0,0 +1,24 @@
---
title: Search Engineering
weight: 18
is_empty: false
aliases:
- how-to
- tutorials
partition: qdrant
---
# Search Engineering Tutorials
*Master vector search modalities, reranking, and retrieval quality.*
| Tutorial | Objective | Stack | Time | Level |
| :--- | :--- | :--- | :--- | :--- |
| [Neural Search Service](https://qdrant.tech/documentation/tutorials-search-engineering/neural-search/) | Deploy a search service for company descriptions. | FastAPI | 30m | Beginner |
| [Hybrid Search with FastEmbed](https://qdrant.tech/documentation/tutorials-search-engineering/hybrid-search-fastembed/) | Combine dense and sparse search for startups. | FastAPI | 20m | Beginner |
| [Movie Recommendations](https://qdrant.tech/documentation/tutorials-search-engineering/collaborative-filtering/) | Collaborative filtering using sparse embeddings. | Python | 45m | Intermediate |
| [Advanced PDF Retrieval](https://qdrant.tech/documentation/tutorials-search-engineering/pdf-retrieval-at-scale/) | PDF RAG using ColPali and embedding pooling. | Python | 30m | Intermediate |
| [Retrieval Quality Benchmarking](https://qdrant.tech/documentation/tutorials-search-engineering/retrieval-quality/) | Measure quality and tune HNSW parameters. | Python | 30m | Intermediate |
| [Multivector Reranking](https://qdrant.tech/documentation/search-precision/reranking-semantic-search/) | Use multivector representations for better ranking. | Python | 30m | Intermediate |
| [Hybrid Search Reranking](https://qdrant.tech/documentation/tutorials-search-engineering/reranking-hybrid-search/) | Implement late interaction and sparse reranking. | Python | 40m | Intermediate |
| [Semantic Code Search](https://qdrant.tech/documentation/tutorials-search-engineering/code-search/) | Navigate codebases using vector similarity. | Python | 45m | Intermediate |
| [Static Embeddings Analysis](https://qdrant.tech/documentation/tutorials-search-engineering/static-embeddings/) | Evaluate the renaissance of static embeddings. | Python | 20m | Intermediate |
qdrant-landing/content/documentation/tutorials-search-engineering/code-search.md
+438
View File
@@ -0,0 +1,438 @@
---
title: Search Through Your Codebase
aliases:
- /documentation/tutorials/code-search/
weight: 2
---
# Navigate Your Codebase with Semantic Search and Qdrant
| Time: 45 min | Level: Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/qdrant/examples/blob/master/code-search/code-search.ipynb) | |
|--------------|---------------------|--|----|
You too can enrich your applications with Qdrant semantic search. In this
tutorial, we describe how you can use Qdrant to navigate a codebase, to help
you find relevant code snippets. As an example, we will use the [Qdrant](https://github.com/qdrant/qdrant)
source code itself, which is mostly written in Rust.
<aside role="status">This tutorial might not work on code bases that are not disciplined or structured. For good code search, you may need to refactor the project first.</aside>
## The approach
We want to search codebases using natural semantic queries, and searching for code based on similar logic. You can set up these tasks with embeddings:
1. General usage neural encoder for Natural Language Processing (NLP), in our case
`sentence-transformers/all-MiniLM-L6-v2`.
2. Specialized embeddings for code-to-code similarity search. We use the
`jina-embeddings-v2-base-code` model.
To prepare our code for `all-MiniLM-L6-v2`, we preprocess the code to text that
more closely resembles natural language. The Jina embeddings model supports a
variety of standard programming languages, so there is no need to preprocess the
snippets. We can use the code as is.
NLP-based search is based on function signatures, but code search may return
smaller pieces, such as loops. So, if we receive a particular function signature
from the NLP model and part of its implementation from the code model, we merge
the results and highlight the overlap.
## Data preparation
Chunking the application sources into smaller parts is a non-trivial task. In
general, functions, class methods, structs, enums, and all the other language-specific
constructs are good candidates for chunks. They are big enough to
contain some meaningful information, but small enough to be processed by
embedding models with a limited context window. You can also use docstrings,
comments, and other metadata can be used to enrich the chunks with additional
information.
![Code chunking strategy](/documentation/tutorials/code-search/data-chunking.png)
### Parsing the codebase
While our example uses Rust, you can use our approach with any other language.
You can parse code with a [Language Server Protocol](https://microsoft.github.io/language-server-protocol/) (**LSP**)
compatible tool. You can use an LSP to build a graph of the codebase, and then extract chunks.
We did our work with the [rust-analyzer](https://rust-analyzer.github.io/).
We exported the parsed codebase into the [LSIF](https://microsoft.github.io/language-server-protocol/specifications/lsif/0.4.0/specification/)
format, a standard for code intelligence data. Next, we used the LSIF data to
navigate the codebase and extract the chunks. For details, see our [code search
demo](https://github.com/qdrant/demo-code-search).
<aside role="status">
For other languages, you can use the same approach. There are
<a href="https://microsoft.github.io/language-server-protocol/implementors/servers/">plenty of implementations available
</a>.
</aside>
We then exported the chunks into JSON documents with not only the code itself,
but also context with the location of the code in the project. For example, see
the description of the `await_ready_for_timeout` function from the `IsReady`
struct in the `common` module:
```json
{
"name":"await_ready_for_timeout",
"signature":"fn await_ready_for_timeout (& self , timeout : Duration) -> bool",
"code_type":"Function",
"docstring":"= \" Return `true` if ready, `false` if timed out.\"",
"line":44,
"line_from":43,
"line_to":51,
"context":{
"module":"common",
"file_path":"lib/collection/src/common/is_ready.rs",
"file_name":"is_ready.rs",
"struct_name":"IsReady",
"snippet":" /// Return `true` if ready, `false` if timed out.\n pub fn await_ready_for_timeout(&self, timeout: Duration) -> bool {\n let mut is_ready = self.value.lock();\n if !*is_ready {\n !self.condvar.wait_for(&mut is_ready, timeout).timed_out()\n } else {\n true\n }\n }\n"
}
}
```
You can examine the Qdrant structures, parsed in JSON, in the [`structures.jsonl`
file](https://storage.googleapis.com/tutorial-attachments/code-search/structures.jsonl)
in our Google Cloud Storage bucket. Download it and use it as a source of data for our code search.
```shell
wget https://storage.googleapis.com/tutorial-attachments/code-search/structures.jsonl
```
Next, load the file and parse the lines into a list of dictionaries:
```python
import json
structures = []
with open("structures.jsonl", "r") as fp:
for i, row in enumerate(fp):
entry = json.loads(row)
structures.append(entry)
```
### Code to *natural language* conversion
Each programming language has its own syntax which is not a part of the natural
language. Thus, a general-purpose model probably does not understand the code
as is. We can, however, normalize the data by removing code specifics and
including additional context, such as module, class, function, and file name.
We took the following steps:
1. Extract the signature of the function, method, or other code construct.
2. Divide camel case and snake case names into separate words.
3. Take the docstring, comments, and other important metadata.
4. Build a sentence from the extracted data using a predefined template.
5. Remove the special characters and replace them with spaces.
As input, expect dictionaries with the same structure. Define a `textify`
function to do the conversion. We'll use an `inflection` library to convert
with different naming conventions.
```shell
pip install inflection
```
Once all dependencies are installed, we define the `textify` function:
```python
import inflection
import re
from typing import Dict, Any
def textify(chunk: Dict[str, Any]) -> str:
# Get rid of all the camel case / snake case
# - inflection.underscore changes the camel case to snake case
# - inflection.humanize converts the snake case to human readable form
name = inflection.humanize(inflection.underscore(chunk["name"]))
signature = inflection.humanize(inflection.underscore(chunk["signature"]))
# Check if docstring is provided
docstring = ""
if chunk["docstring"]:
docstring = f"that does {chunk['docstring']} "
# Extract the location of that snippet of code
context = (
f"module {chunk['context']['module']} "
f"file {chunk['context']['file_name']}"
)
if chunk["context"]["struct_name"]:
struct_name = inflection.humanize(
inflection.underscore(chunk["context"]["struct_name"])
)
context = f"defined in struct {struct_name} {context}"
# Combine all the bits and pieces together
text_representation = (
f"{chunk['code_type']} {name} "
f"{docstring}"
f"defined as {signature} "
f"{context}"
)
# Remove any special characters and concatenate the tokens
tokens = re.split(r"\W", text_representation)
tokens = filter(lambda x: x, tokens)
return " ".join(tokens)
```
Now we can use `textify` to convert all chunks into text representations:
```python
text_representations = list(map(textify, structures))
```
This is how the `await_ready_for_timeout` function description appears:
```text
Function Await ready for timeout that does Return true if ready false if timed out defined as Fn await ready for timeout self timeout duration bool defined in struct Is ready module common file is_ready rs
```
## Ingestion pipeline
Next, we'll build a pipeline for vectorizing the data and set up a semantic search mechanism for both embedding models.
### Building Qdrant collection
We use the `qdrant-client` library with the `fastembed` extra to interact with the Qdrant server and generate vector embeddings locally. Let's install it:
```shell
pip install "qdrant-client[fastembed]"
```
Of course, we need a running Qdrant server for vector search. If you need one,
you can [use a local Docker container](/documentation/quick-start/)
or deploy it using the [Qdrant Cloud](https://cloud.qdrant.io/).
You can use either to follow this tutorial. Configure the connection parameters:
```python
QDRANT_URL = "https://my-cluster.cloud.qdrant.io:6333" # http://localhost:6333 for local instance
QDRANT_API_KEY = "THIS_IS_YOUR_API_KEY" # None for local instance
```
Then use the library to create a collection:
```python
from qdrant_client import QdrantClient, models
client = QdrantClient(QDRANT_URL, api_key=QDRANT_API_KEY)
client.create_collection(
"qdrant-sources",
vectors_config={
"text": models.VectorParams(
size=client.get_embedding_size(
model_name="sentence-transformers/all-MiniLM-L6-v2"
),
distance=models.Distance.COSINE,
),
"code": models.VectorParams(
size=client.get_embedding_size(
model_name="jinaai/jina-embeddings-v2-base-code"
),
distance=models.Distance.COSINE,
),
},
)
```
Our newly created collection is ready to accept the data. Let's upload the embeddings:
```python
import uuid
# Extract the code snippets from the structures to a separate list
code_snippets = [
structure["context"]["snippet"] for structure in structures
]
points = [
models.PointStruct(
id=uuid.uuid4().hex,
vector={
"text": models.Document(
text=text, model="sentence-transformers/all-MiniLM-L6-v2"
),
"code": models.Document(
text=code, model="jinaai/jina-embeddings-v2-base-code"
),
},
payload=structure,
)
for text, code, structure in zip(text_representations, code_snippets, structures)
]
# Note: This might take a while since inference happens implicitly.
# Parallel processing can help.
# But too many processes may trigger swap memory and hurt performance.
client.upload_points("qdrant-sources", points=points, batch_size=64)
```
Internally, `qdrant-client` uses [FastEmbed](https://github.com/qdrant/fastembed) to implicitly convert our documents into their vector representations.
The uploaded points are immediately available for search. Next, query the
collection to find relevant code snippets.
## Querying the codebase
We use one of the models to search the collection. Start with text embeddings.
Run the following query "*How do I count points in a collection?*". Review the
results.
<aside role="status">In these tables, we link to longer code excerpts from a
`file_name` in the `Qdrant` repository. The results are subject to change.
Fortunately, this model should continue to provide the results you need.</aside>
```python
query = "How do I count points in a collection?"
hits = client.query_points(
"qdrant-sources",
query=models.Document(text=query, model="sentence-transformers/all-MiniLM-L6-v2"),
using="text",
limit=5,
).points
```
Now, review the results. The following table lists the module, the file name
and score. Each line includes a link to the signature, as a code block from
the file.
| module | file_name | score | signature |
|--------------------|---------------------|------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| toc | point_ops.rs | 0.59448624 | [<img src="/documentation/tutorials/code-search/github-mark.png" width="16" style="display: inline"> `pub async fn count`](https://github.com/qdrant/qdrant/blob/7aa164bd2dda1c0fc9bf3a0da42e656c95c2e52a/lib/storage/src/content_manager/toc/point_ops.rs#L120) |
| operations | types.rs | 0.5493385 | [<img src="/documentation/tutorials/code-search/github-mark.png" width="16" style="display: inline"> `pub struct CountRequestInternal`](https://github.com/qdrant/qdrant/blob/7aa164bd2dda1c0fc9bf3a0da42e656c95c2e52a/lib/collection/src/operations/types.rs#L831) |
| collection_manager | segments_updater.rs | 0.5121002 | [<img src="/documentation/tutorials/code-search/github-mark.png" width="16" style="display: inline"> `pub(crate) fn upsert_points<'a, T>`](https://github.com/qdrant/qdrant/blob/7aa164bd2dda1c0fc9bf3a0da42e656c95c2e52a/lib/collection/src/collection_manager/segments_updater.rs#L339) |
| collection | point_ops.rs | 0.5063539 | [<img src="/documentation/tutorials/code-search/github-mark.png" width="16" style="display: inline"> `pub async fn count`](https://github.com/qdrant/qdrant/blob/7aa164bd2dda1c0fc9bf3a0da42e656c95c2e52a/lib/collection/src/collection/point_ops.rs#L213) |
| map_index | mod.rs | 0.49973983 | [<img src="/documentation/tutorials/code-search/github-mark.png" width="16" style="display: inline"> `fn get_points_with_value_count<Q>`](https://github.com/qdrant/qdrant/blob/7aa164bd2dda1c0fc9bf3a0da42e656c95c2e52a/lib/segment/src/index/field_index/map_index/mod.rs#L88) |
It seems we were able to find some relevant code structures. Let's try the same with the code embeddings:
```python
hits = client.query_points(
"qdrant-sources",
query=models.Document(text=query, model="jinaai/jina-embeddings-v2-base-code"),
using="code",
limit=5,
).points
```
Output:
| module | file_name | score | signature |
|---------------|----------------------------|------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| field_index | geo_index.rs | 0.73278356 | [<img src="/documentation/tutorials/code-search/github-mark.png" width="16" style="display: inline"> `fn count_indexed_points`](https://github.com/qdrant/qdrant/blob/7aa164bd2dda1c0fc9bf3a0da42e656c95c2e52a/lib/segment/src/index/field_index/geo_index.rs#L612) |
| numeric_index | mod.rs | 0.7254976 | [<img src="/documentation/tutorials/code-search/github-mark.png" width="16" style="display: inline"> `fn count_indexed_points`](https://github.com/qdrant/qdrant/blob/3fbe1cae6cb7f51a0c5bb4b45cfe6749ac76ed59/lib/segment/src/index/field_index/numeric_index/mod.rs#L322) |
| map_index | mod.rs | 0.7124739 | [<img src="/documentation/tutorials/code-search/github-mark.png" width="16" style="display: inline"> `fn count_indexed_points`](https://github.com/qdrant/qdrant/blob/3fbe1cae6cb7f51a0c5bb4b45cfe6749ac76ed59/lib/segment/src/index/field_index/map_index/mod.rs#L315) |
| map_index | mod.rs | 0.7124739 | [<img src="/documentation/tutorials/code-search/github-mark.png" width="16" style="display: inline"> `fn count_indexed_points`](https://github.com/qdrant/qdrant/blob/3fbe1cae6cb7f51a0c5bb4b45cfe6749ac76ed59/lib/segment/src/index/field_index/map_index/mod.rs#L429) |
| fixtures | payload_context_fixture.rs | 0.706204 | [<img src="/documentation/tutorials/code-search/github-mark.png" width="16" style="display: inline"> `fn total_point_count`](https://github.com/qdrant/qdrant/blob/3fbe1cae6cb7f51a0c5bb4b45cfe6749ac76ed59/lib/segment/src/fixtures/payload_context_fixture.rs#L122) |
While the scores retrieved by different models are not comparable, but we can
see that the results are different. Code and text embeddings can capture
different aspects of the codebase. We can use both models to query the collection
and then combine the results to get the most relevant code snippets, from a single batch request.
```python
responses = client.query_batch_points(
collection_name="qdrant-sources",
requests=[
models.QueryRequest(
query=models.Document(
text=query, model="sentence-transformers/all-MiniLM-L6-v2"
),
using="text",
with_payload=True,
limit=5,
),
models.QueryRequest(
query=models.Document(
text=query, model="jinaai/jina-embeddings-v2-base-code"
),
using="code",
with_payload=True,
limit=5,
),
],
)
results = [response.points for response in responses]
```
Output:
| module | file_name | score | signature |
|--------------------|----------------------------|------------|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| toc | point_ops.rs | 0.59448624 | [<img src="/documentation/tutorials/code-search/github-mark.png" width="16" style="display: inline"> `pub async fn count`](https://github.com/qdrant/qdrant/blob/7aa164bd2dda1c0fc9bf3a0da42e656c95c2e52a/lib/storage/src/content_manager/toc/point_ops.rs#L120) |
| operations | types.rs | 0.5493385 | [<img src="/documentation/tutorials/code-search/github-mark.png" width="16" style="display: inline"> `pub struct CountRequestInternal`](https://github.com/qdrant/qdrant/blob/7aa164bd2dda1c0fc9bf3a0da42e656c95c2e52a/lib/collection/src/operations/types.rs#L831) |
| collection_manager | segments_updater.rs | 0.5121002 | [<img src="/documentation/tutorials/code-search/github-mark.png" width="16" style="display: inline"> `pub(crate) fn upsert_points<'a, T>`](https://github.com/qdrant/qdrant/blob/7aa164bd2dda1c0fc9bf3a0da42e656c95c2e52a/lib/collection/src/collection_manager/segments_updater.rs#L339) |
| collection | point_ops.rs | 0.5063539 | [<img src="/documentation/tutorials/code-search/github-mark.png" width="16" style="display: inline"> `pub async fn count`](https://github.com/qdrant/qdrant/blob/7aa164bd2dda1c0fc9bf3a0da42e656c95c2e52a/lib/collection/src/collection/point_ops.rs#L213) |
| map_index | mod.rs | 0.49973983 | [<img src="/documentation/tutorials/code-search/github-mark.png" width="16" style="display: inline"> `fn get_points_with_value_count<Q>`](https://github.com/qdrant/qdrant/blob/7aa164bd2dda1c0fc9bf3a0da42e656c95c2e52a/lib/segment/src/index/field_index/map_index/mod.rs#L88) |
| field_index | geo_index.rs | 0.73278356 | [<img src="/documentation/tutorials/code-search/github-mark.png" width="16" style="display: inline"> `fn count_indexed_points`](https://github.com/qdrant/qdrant/blob/7aa164bd2dda1c0fc9bf3a0da42e656c95c2e52a/lib/segment/src/index/field_index/geo_index.rs#L612) |
| numeric_index | mod.rs | 0.7254976 | [<img src="/documentation/tutorials/code-search/github-mark.png" width="16" style="display: inline"> `fn count_indexed_points`](https://github.com/qdrant/qdrant/blob/3fbe1cae6cb7f51a0c5bb4b45cfe6749ac76ed59/lib/segment/src/index/field_index/numeric_index/mod.rs#L322) |
| map_index | mod.rs | 0.7124739 | [<img src="/documentation/tutorials/code-search/github-mark.png" width="16" style="display: inline"> `fn count_indexed_points`](https://github.com/qdrant/qdrant/blob/3fbe1cae6cb7f51a0c5bb4b45cfe6749ac76ed59/lib/segment/src/index/field_index/map_index/mod.rs#L315) |
| map_index | mod.rs | 0.7124739 | [<img src="/documentation/tutorials/code-search/github-mark.png" width="16" style="display: inline"> `fn count_indexed_points`](https://github.com/qdrant/qdrant/blob/3fbe1cae6cb7f51a0c5bb4b45cfe6749ac76ed59/lib/segment/src/index/field_index/map_index/mod.rs#L429) |
| fixtures | payload_context_fixture.rs | 0.706204 | [<img src="/documentation/tutorials/code-search/github-mark.png" width="16" style="display: inline"> `fn total_point_count`](https://github.com/qdrant/qdrant/blob/3fbe1cae6cb7f51a0c5bb4b45cfe6749ac76ed59/lib/segment/src/fixtures/payload_context_fixture.rs#L122) |
This is one example of how you can use different models and combine the results.
In a real-world scenario, you might run some reranking and deduplication, as
well as additional processing of the results.
### Code search demo
Our [Code search demo](https://code-search.qdrant.tech/) uses the following process:
1. The user sends a query.
1. Both models vectorize that query simultaneously. We get two different
vectors.
1. Both vectors are used in parallel to find relevant snippets. We expect
5 examples from the NLP search and 20 examples from the code search.
1. Once we retrieve results for both vectors, we merge them in one of the
following scenarios:
1. If both methods return different results, we prefer the results from
the general usage model (NLP).
1. If there is an overlap between the search results, we merge overlapping
snippets.
In the screenshot, we search for `flush of wal`. The result
shows relevant code, merged from both models. Note the highlighted
code in lines 621-629. It's where both models agree.
![Results from both models, with overlap](/documentation/tutorials/code-search/code-search-demo-example.png)
Now you see semantic code intelligence, in action.
### Grouping the results
You can improve the search results, by grouping them by payload properties.
In our case, we can group the results by the module. If we use code embeddings,
we can see multiple results from the `map_index` module. Let's group the
results and assume a single result per module:
```python
results = client.query_points_groups(
collection_name="qdrant-sources",
using="code",
query=models.Document(text=query, model="jinaai/jina-embeddings-v2-base-code"),
group_by="context.module",
limit=5,
group_size=1,
)
```
Output:
| module | file_name | score | signature |
|---------------|----------------------------|------------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| field_index | geo_index.rs | 0.73278356 | [<img src="/documentation/tutorials/code-search/github-mark.png" width="16" style="display: inline"> `fn count_indexed_points`](https://github.com/qdrant/qdrant/blob/7aa164bd2dda1c0fc9bf3a0da42e656c95c2e52a/lib/segment/src/index/field_index/geo_index.rs#L612) |
| numeric_index | mod.rs | 0.7254976 | [<img src="/documentation/tutorials/code-search/github-mark.png" width="16" style="display: inline"> `fn count_indexed_points`](https://github.com/qdrant/qdrant/blob/3fbe1cae6cb7f51a0c5bb4b45cfe6749ac76ed59/lib/segment/src/index/field_index/numeric_index/mod.rs#L322) |
| map_index | mod.rs | 0.7124739 | [<img src="/documentation/tutorials/code-search/github-mark.png" width="16" style="display: inline"> `fn count_indexed_points`](https://github.com/qdrant/qdrant/blob/3fbe1cae6cb7f51a0c5bb4b45cfe6749ac76ed59/lib/segment/src/index/field_index/map_index/mod.rs#L315) |
| fixtures | payload_context_fixture.rs | 0.706204 | [<img src="/documentation/tutorials/code-search/github-mark.png" width="16" style="display: inline"> `fn total_point_count`](https://github.com/qdrant/qdrant/blob/3fbe1cae6cb7f51a0c5bb4b45cfe6749ac76ed59/lib/segment/src/fixtures/payload_context_fixture.rs#L122) |
| hnsw_index | graph_links.rs | 0.6998417 | [<img src="/documentation/tutorials/code-search/github-mark.png" width="16" style="display: inline"> `fn num_points `](https://github.com/qdrant/qdrant/blob/3fbe1cae6cb7f51a0c5bb4b45cfe6749ac76ed59/lib/segment/src/index/hnsw_index/graph_links.rs#L477) |
With the grouping feature, we get more diverse results.
## Summary
This tutorial demonstrates how to use Qdrant to navigate a codebase. For an
end-to-end implementation, review the [code search
notebook](https://colab.research.google.com/github/qdrant/examples/blob/master/code-search/code-search.ipynb) and the
[code-search-demo](https://github.com/qdrant/demo-code-search). You can also check out [a running version of the code
search demo](https://code-search.qdrant.tech/) which exposes Qdrant codebase for search with a web interface.
qdrant-landing/content/documentation/tutorials-search-engineering/collaborative-filtering.md
+277
View File
@@ -0,0 +1,277 @@
---
title: Build a Recommendation System with Collaborative Filtering
aliases:
- /documentation/tutorials/collaborative-filtering/
short_description: "Build an effective movie recommendation system using collaborative filtering and Qdrant's similarity search."
description: "Build an effective movie recommendation system using collaborative filtering and Qdrant's similarity search."
preview_image: /blog/collaborative-filtering/social_preview.png
social_preview_image: /blog/collaborative-filtering/social_preview.png
weight: 3
---
# Use Collaborative Filtering to Build a Movie Recommendation System with Qdrant
| Time: 45 min | Level: Intermediate | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/qdrant/examples/blob/master/collaborative-filtering/collaborative-filtering.ipynb) | |
|--------------|---------------------|--|----|
Every time Spotify recommends the next song from a band you've never heard of, it uses a recommendation algorithm based on other users' interactions with that song. This type of algorithm is known as **collaborative filtering**.
Unlike content-based recommendations, collaborative filtering excels when the objects' semantics are loosely or unrelated to users' preferences. This adaptability is what makes it so fascinating. Movie, music, or book recommendations are good examples of such use cases. After all, we rarely choose which book to read purely based on the plot twists.
The traditional way to build a collaborative filtering engine involves training a model that converts the sparse matrix of user-to-item relations into a compressed, dense representation of user and item vectors. Some of the most commonly referenced algorithms for this purpose include [SVD (Singular Value Decomposition)](https://en.wikipedia.org/wiki/Singular_value_decomposition) and [Factorization Machines](https://en.wikipedia.org/wiki/Matrix_factorization_(recommender_systems)). However, the model training approach requires significant resource investments. Model training necessitates data, regular re-training, and a mature infrastructure.
## Methodology
Fortunately, there is a way to build collaborative filtering systems without any model training. You can obtain interpretable recommendations and have a scalable system using a technique based on similarity search. Let’s explore how this works with an example of building a movie recommendation system.
<p align="center"><iframe width="560" height="315" src="https://www.youtube.com/embed/9B7RrmQCQeQ?si=nHp-fM_szHynLcH8" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" referrerpolicy="strict-origin-when-cross-origin" allowfullscreen></iframe></p>
## Implementation
To implement this, you will use a simple yet powerful resource: [Qdrant with Sparse Vectors](https://qdrant.tech/articles/sparse-vectors/).
Notebook: [You can try this code here](https://githubtocolab.com/qdrant/examples/blob/master/collaborative-filtering/collaborative-filtering.ipynb)
### Setup
You have to first import the necessary libraries and define the environment.
```python
import os
import pandas as pd
import requests
from qdrant_client import QdrantClient, models
from qdrant_client.models import PointStruct, SparseVector, NamedSparseVector
from collections import defaultdict
# OMDB API Key - for movie posters
omdb_api_key = os.getenv("OMDB_API_KEY")
# Collection name
collection_name = "movies"
# Set Qdrant Client
qdrant_client = QdrantClient(
os.getenv("QDRANT_HOST"),
api_key=os.getenv("QDRANT_API_KEY")
)
```
### Define output
Here, you will configure the recommendation engine to retrieve movie posters as output.
```python
# Function to get movie poster using OMDB API
def get_movie_poster(imdb_id, api_key):
url = f"https://www.omdbapi.com/?i={imdb_id}&apikey={api_key}"
data = requests.get(url).json()
return data.get('Poster'), data
```
### Prepare the data
Load the movie datasets. These include three main CSV files: user ratings, movie titles, and OMDB IDs.
```python
# Load CSV files
ratings_df = pd.read_csv('data/ratings.csv', low_memory=False)
movies_df = pd.read_csv('data/movies.csv', low_memory=False)
# Convert movieId in ratings_df and movies_df to string
ratings_df['movieId'] = ratings_df['movieId'].astype(str)
movies_df['movieId'] = movies_df['movieId'].astype(str)
rating = ratings_df['rating']
# Normalize ratings
ratings_df['rating'] = (rating - rating.mean()) / rating.std()
# Merge ratings with movie metadata to get movie titles
merged_df = ratings_df.merge(
movies_df[['movieId', 'title']],
left_on='movieId', right_on='movieId', how='inner'
)
# Aggregate ratings to handle duplicate (userId, title) pairs
ratings_agg_df = merged_df.groupby(['userId', 'movieId']).rating.mean().reset_index()
ratings_agg_df.head()
```
| |userId |movieId |rating |
|---|-----------|---------|---------|
|0 |1 |1 |0.429960 |
|1 |1 |1036 |1.369846 |
|2 |1 |1049 |-0.509926|
|3 |1 |1066 |0.429960 |
|4 |1 |110 |0.429960 |
### Convert to sparse
If you want to search across numerous reviews from different users, you can represent these reviews in a sparse matrix.
```python
# Convert ratings to sparse vectors
user_sparse_vectors = defaultdict(lambda: {"values": [], "indices": []})
for row in ratings_agg_df.itertuples():
user_sparse_vectors[row.userId]["values"].append(row.rating)
user_sparse_vectors[row.userId]["indices"].append(int(row.movieId))
```
![collaborative-filtering](/blog/collaborative-filtering/collaborative-filtering.png)
### Upload the data
Here, you will initialize the Qdrant client and create a new collection to store the data.
Convert the user ratings to sparse vectors and include the `movieId` in the payload.
```python
# Define a data generator
def data_generator():
for user_id, sparse_vector in user_sparse_vectors.items():
yield PointStruct(
id=user_id,
vector={"ratings": SparseVector(
indices=sparse_vector["indices"],
values=sparse_vector["values"]
)},
payload={"user_id": user_id, "movie_id": sparse_vector["indices"]}
)
# Upload points using the data generator
qdrant_client.upload_points(
collection_name=collection_name,
points=data_generator()
)
```
### Define query
In order to get recommendations, we need to find users with similar tastes to ours.
Let's describe our preferences by providing ratings for some of our favorite movies.
`1` indicates that we like the movie, `-1` indicates that we dislike it.
```python
my_ratings = {
603: 1, # Matrix
13475: 1, # Star Trek
11: 1, # Star Wars
1091: -1, # The Thing
862: 1, # Toy Story
597: -1, # Titanic
680: -1, # Pulp Fiction
13: 1, # Forrest Gump
120: 1, # Lord of the Rings
87: -1, # Indiana Jones
562: -1 # Die Hard
}
```
<details>
<summary>Click to see the code for <code>to_vector</code> </summary>
```python
# Create sparse vector from my_ratings
def to_vector(ratings):
vector = SparseVector(
values=[],
indices=[]
)
for movie_id, rating in ratings.items():
vector.values.append(rating)
vector.indices.append(movie_id)
return vector
```
</details>
### Run the query
From the uploaded list of movies with ratings, we can perform a search in Qdrant to get the top most similar users to us.
```python
# Perform the search
results = qdrant_client.query_points(
collection_name=collection_name,
query=to_vector(my_ratings),
using="ratings",
limit=20
).points
```
Now we can find the movies liked by the other similar users, but we haven't seen yet.
Let's combine the results from found users, filter out seen movies, and sort by the score.
```python
# Convert results to scores and sort by score
def results_to_scores(results):
movie_scores = defaultdict(lambda: 0)
for result in results:
for movie_id in result.payload["movie_id"]:
movie_scores[movie_id] += result.score
return movie_scores
# Convert results to scores and sort by score
movie_scores = results_to_scores(results)
top_movies = sorted(movie_scores.items(), key=lambda x: x[1], reverse=True)
```
<details>
<summary> Visualize results in Jupyter Notebook </summary>
Finally, we display the top 5 recommended movies along with their posters and titles.
```python
# Create HTML to display top 5 results
html_content = "<div class='movies-container'>"
for movie_id, score in top_movies[:5]:
imdb_id_row = links.loc[links['movieId'] == int(movie_id), 'imdbId']
if not imdb_id_row.empty:
imdb_id = imdb_id_row.values[0]
poster_url, movie_info = get_movie_poster(imdb_id, omdb_api_key)
movie_title = movie_info.get('Title', 'Unknown Title')
html_content += f"""
<div class='movie-card'>
<img src="{poster_url}" alt="Poster" class="movie-poster">
<div class="movie-title">{movie_title}</div>
<div class="movie-score">Score: {score}</div>
</div>
"""
else:
continue # Skip if imdb_id is not found
html_content += "</div>"
display(HTML(html_content))
```
</details>
## Recommendations
For a complete display of movie posters, check the [notebook output](https://github.com/qdrant/examples/blob/master/collaborative-filtering/collaborative-filtering.ipynb). Here are the results without html content.
```text
Toy Story, Score: 131.2033799
Monty Python and the Holy Grail, Score: 131.2033799
Star Wars: Episode V - The Empire Strikes Back, Score: 131.2033799
Star Wars: Episode VI - Return of the Jedi, Score: 131.2033799
Men in Black, Score: 131.2033799
```
On top of collaborative filtering, we can further enhance the recommendation system by incorporating other features like user demographics, movie genres, or movie tags.
Or, for example, only consider recent ratings via a time-based filter. This way, we can recommend movies that are currently popular among users.
## Conclusion
As demonstrated, it is possible to build an interesting movie recommendation system without intensive model training using Qdrant and Sparse Vectors. This approach not only simplifies the recommendation process but also makes it scalable and interpretable. In future tutorials, we can experiment more with this combination to further enhance our recommendation systems.
qdrant-landing/content/documentation/tutorials-search-engineering/hybrid-search-fastembed.md
+394
View File
@@ -0,0 +1,394 @@
---
title: Setup Hybrid Search with FastEmbed
aliases:
- /documentation/tutorials/hybrid-search-fastembed/
weight: 3
---
# Build a Hybrid Search Service with FastEmbed and Qdrant
| Time: 20 min | Level: Beginner | Output: [GitHub](https://github.com/qdrant/qdrant_demo/) |
| --- | ----------- | ----------- |----------- |
This tutorial shows you how to build and deploy your own hybrid search service to look through descriptions of companies from [startups-list.com](https://www.startups-list.com/) and pick the most similar ones to your query.
The website contains the company names, descriptions, locations, and a picture for each entry.
As we have already written on our [blog](/articles/hybrid-search/), there is no single definition of hybrid search.
In this tutorial we are covering the case with a combination of dense and [sparse embeddings](/articles/sparse-vectors/).
The former ones refer to the embeddings generated by such well-known neural networks as BERT, while the latter ones are more related to a traditional full-text search approach.
Our hybrid search service will use [Fastembed](https://github.com/qdrant/fastembed) package to generate embeddings of text descriptions and [FastAPI](https://fastapi.tiangolo.com/) to serve the search API.
Fastembed natively integrates with Qdrant client, so you can easily upload the data into Qdrant and perform search queries.
![Hybrid Search Schema](/documentation/tutorials/hybrid-search-with-fastembed/hybrid-search-schema.png)
## Workflow
To create a hybrid search service, you will need to transform your raw data and then create a search function to manipulate it.
First, you will 1) download and prepare a sample dataset using a modified version of the BERT ML model. Then, you will 2) load the data into Qdrant, 3) create a hybrid search API and 4) serve it using FastAPI.
![Hybrid Search Workflow](/docs/workflow-neural-search.png)
## Prerequisites
To complete this tutorial, you will need:
- Docker - The easiest way to use Qdrant is to run a pre-built Docker image.
- [Raw parsed data](https://storage.googleapis.com/generall-shared-data/startups_demo.json) from startups-list.com.
- Python version >=3.9
## Prepare sample dataset
To conduct a hybrid search on startup descriptions, you must first encode the description data into vectors.
Fastembed integration into qdrant client combines encoding and uploading into a single step.
It also takes care of batching and parallelization, so you don't have to worry about it.
Let's start by downloading the data and installing the necessary packages.
1. First you need to download the dataset.
```bash
wget https://storage.googleapis.com/generall-shared-data/startups_demo.json
```
## Run Qdrant in Docker
Next, you need to manage all of your data using a vector engine. Qdrant lets you store, update or delete created vectors. Most importantly, it lets you search for the nearest vectors via a convenient API.
> **Note:** Before you begin, create a project directory and a virtual python environment in it.
1. Download the Qdrant image from DockerHub.
```bash
docker pull qdrant/qdrant
```
2. Start Qdrant inside of Docker.
```bash
docker run -p 6333:6333 \
-v $(pwd)/qdrant_storage:/qdrant/storage \
qdrant/qdrant
```
You should see output like this
```text
...
[2021-02-05T00:08:51Z INFO actix_server::builder] Starting 12 workers
[2021-02-05T00:08:51Z INFO actix_server::builder] Starting "actix-web-service-0.0.0.0:6333" service on 0.0.0.0:6333
```
Test the service by going to [http://localhost:6333/](http://localhost:6333/). You should see the Qdrant version info in your browser.
All data uploaded to Qdrant is saved inside the `./qdrant_storage` directory and will be persisted even if you recreate the container.
## Upload data to Qdrant
1. Install the official Python client to best interact with Qdrant.
```bash
pip install "qdrant-client[fastembed]>=1.14.2"
```
> **Note:** This tutorial requires fastembed of version >=0.6.1.
At this point, you should have startup records in the `startups_demo.json` file and Qdrant running on a local machine.
Now you need to write a script to upload all startup data and vectors into the search engine.
2. Create a client object for Qdrant.
```python
# Import client library
from qdrant_client import QdrantClient, models
client = QdrantClient(url="http://localhost:6333")
```
3. Choose models to encode your data and prepare collections.
In this tutorial, we will be using two pre-trained models to compute dense and sparse vectors correspondingly
The models are: `sentence-transformers/all-MiniLM-L6-v2` and `prithivida/Splade_PP_en_v1`.
As soon as the choice is made, we need to configure a collection in Qdrant.
```python
dense_vector_name = "dense"
sparse_vector_name = "sparse"
dense_model_name = "sentence-transformers/all-MiniLM-L6-v2"
sparse_model_name = "prithivida/Splade_PP_en_v1"
if not client.collection_exists("startups"):
client.create_collection(
collection_name="startups",
vectors_config={
dense_vector_name: models.VectorParams(
size=client.get_embedding_size(dense_model_name),
distance=models.Distance.COSINE
)
}, # size and distance are model dependent
sparse_vectors_config={sparse_vector_name: models.SparseVectorParams()},
)
```
Qdrant requires vectors to have their own names and configurations.
Parameters `size` and `distance` are mandatory, however, you can additionaly specify extended configuration for your vectors, like `quantization_config` or `hnsw_config`.
4. Read data from the file.
```python
import json
payload_path = "startups_demo.json"
documents = []
metadata = []
with open(payload_path) as fd:
for line in fd:
obj = json.loads(line)
description = obj["description"]
dense_document = models.Document(text=description, model=dense_model_name)
sparse_document = models.Document(text=description, model=sparse_model_name)
documents.append(
{
dense_vector_name: dense_document,
sparse_vector_name: sparse_document,
}
)
metadata.append(obj)
```
In this block of code, we read data from `startups_demo.json` file and split it into two list: `documents` and `metadata`.
Documents are models with descriptions of startups and model names to embed data. Metadata is payload associated with each startup, such as the name, location, and picture.
We will use `documents` to encode the data into vectors.
6. Encode and upload data.
```python
client.upload_collection(
collection_name="startups",
vectors=tqdm.tqdm(documents),
payload=metadata,
parallel=4, # Use 4 CPU cores to encode data.
# This will spawn a model per process, which might be memory expensive
# Make sure that your system does not use swap, and reduce the amount
# # of processes if it does.
# Otherwise, it might significantly slow down the process.
# Requires wrapping code into if __name__ == '__main__' block
)
```
<aside role="status">
Vector generation process might be time-consuming. In order to save time, you can skip this step by uploading already processed data (available under the spoiler).
</aside>
<details>
<summary>Upload processed data</summary>
Download and unpack the processed data from [here](https://storage.googleapis.com/dataset-startup-search/startup-list-com/startups_hybrid_search_processed_40k.tar.gz) or use the following script:
```bash
wget https://storage.googleapis.com/dataset-startup-search/startup-list-com/startups_hybrid_search_processed_40k.tar.gz
tar -xvf startups_hybrid_search_processed_40k.tar.gz
```
Then you can upload the data to Qdrant.
```python
import json
import numpy as np
def named_vectors(
vectors: list[float],
sparse_vectors: list[models.SparseVector]
) -> dict:
for vector, sparse_vector in zip(vectors, sparse_vectors):
yield {
dense_vector_name: vector,
sparse_vector_name: models.SparseVector(**sparse_vector),
}
with open("dense_vectors.npy", "rb") as f:
vectors = np.load(f)
with open("sparse_vectors.json", "r") as f:
sparse_vectors = json.load(f)
with open("payload.json", "r") as f:
payload = json.load(f)
client.upload_collection(
"startups",
vectors=named_vectors(vectors, sparse_vectors),
payload=payload
)
```
</details>
The `upload_collection` method will encode all documents and upload them to Qdrant.
The `parallel` parameter enables data-parallelism instead of built-in ONNX parallelism.
Additionally, you can specify ids for each document, if you want to use them later to update or delete documents.
If you don't specify ids, they will be generated automatically.
You can monitor the progress of the encoding by passing tqdm progress bar to the `upload_collection` method.
```python
from tqdm import tqdm
client.upload_collection(
collection_name="startups",
vectors=documents,
payload=metadata,
ids=tqdm(range(len(documents))),
)
```
## Build the search API
Now that all the preparations are complete, let's start building a neural search class.
In order to process incoming requests, the hybrid search class will need 3 things: 1) models to convert the query into a vector, 2) the Qdrant client to perform search queries, 3) fusion function to re-rank dense and sparse search results.
Qdrant supports 2 fusion functions for combining the results: [reciprocal rank fusion](https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf) and [distribution based score fusion](https://qdrant.tech/documentation/concepts/hybrid-queries/?q=distribution+based+sc#:~:text=Distribution%2DBased%20Score%20Fusion)
1. Create a file named `hybrid_searcher.py` and specify the following.
```python
from qdrant_client import QdrantClient, models
class HybridSearcher:
DENSE_MODEL = "sentence-transformers/all-MiniLM-L6-v2"
SPARSE_MODEL = "prithivida/Splade_PP_en_v1"
def __init__(self, collection_name):
self.collection_name = collection_name
self.qdrant_client = QdrantClient()
```
2. Write the search function.
```python
def search(self, text: str):
search_result = self.qdrant_client.query_points(
collection_name=self.collection_name,
query=models.FusionQuery(
fusion=models.Fusion.RRF # we are using reciprocal rank fusion here
),
prefetch=[
models.Prefetch(
query=models.Document(text=text, model=self.DENSE_MODEL),
using=dense_vector_name,
),
models.Prefetch(
query=models.Document(text=text, model=self.SPARSE_MODEL),
using=sparse_vector_name,
),
],
query_filter=None, # If you don't want any filters for now
limit=5, # 5 the closest results
).points
# `search_result` contains models.QueryResponse structure
# We can access list of scored points with the corresponding similarity scores,
# vectors (if `with_vectors` was set to `True`), and payload via `points` attribute.
# Select and return metadata
metadata = [point.payload for point in search_result]
return metadata
```
3. Add search filters.
With Qdrant it is also feasible to add some conditions to the search.
For example, if you wanted to search for startups in a certain city, the search query could look like this:
```python
...
city_of_interest = "Berlin"
# Define a filter for cities
city_filter = models.Filter(
must=[
models.FieldCondition(
key="city",
match=models.MatchValue(value=city_of_interest)
)
]
)
# NOTE: it is not a hybrid search! It's just a dense query for simplicity
search_result = self.qdrant_client.query_points(
collection_name=self.collection_name,
query=models.Document(text=text, model=self.DENSE_MODEL),
query_filter=city_filter,
limit=5
).points
...
```
You have now created a class for neural search queries. Now wrap it up into a service.
## Deploy the search with FastAPI
To build the service you will use the FastAPI framework.
1. Install FastAPI.
To install it, use the command
```bash
pip install fastapi uvicorn
```
2. Implement the service.
Create a file named `service.py` and specify the following.
The service will have only one API endpoint and will look like this:
```python
from fastapi import FastAPI
# The file where HybridSearcher is stored
from hybrid_searcher import HybridSearcher
app = FastAPI()
# Create a neural searcher instance
hybrid_searcher = HybridSearcher(collection_name="startups")
@app.get("/api/search")
def search_startup(q: str):
return {"result": hybrid_searcher.search(text=q)}
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=8000)
```
3. Run the service.
```bash
python service.py
```
4. Open your browser at [http://localhost:8000/docs](http://localhost:8000/docs).
You should be able to see a debug interface for your service.
![FastAPI Swagger interface](/docs/fastapi_neural_search.png)
Feel free to play around with it, make queries regarding the companies in our corpus, and check out the results.
Join our [Discord community](https://qdrant.to/discord), where we talk about vector search and similarity learning, publish other examples of neural networks and neural search applications.
qdrant-landing/content/documentation/tutorials-search-engineering/neural-search.md
+340
View File
@@ -0,0 +1,340 @@
---
title: Build a Neural Search Service
aliases:
- /documentation/tutorials/neural-search/
weight: 2
---
# Build a Neural Search Service with Sentence Transformers and Qdrant
| Time: 30 min | Level: Beginner | Output: [GitHub](https://github.com/qdrant/qdrant_demo/tree/sentense-transformers) | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/drive/1kPktoudAP8Tu8n8l-iVMOQhVmHkWV_L9?usp=sharing) |
| --- | ----------- | ----------- |----------- |
This tutorial shows you how to build and deploy your own neural search service to look through descriptions of companies from [startups-list.com](https://www.startups-list.com/) and pick the most similar ones to your query. The website contains the company names, descriptions, locations, and a picture for each entry.
A neural search service uses artificial neural networks to improve the accuracy and relevance of search results. Besides offering simple keyword results, this system can retrieve results by meaning. It can understand and interpret complex search queries and provide more contextually relevant output, effectively enhancing the user's search experience.
<aside role="status">
There is a version of this tutorial that uses <a href="https://github.com/qdrant/fastembed">Fastembed</a> model inference engine instead of Sentence Transformers.
Check it out <a href="/documentation/beginner-tutorials/hybrid-search-fastembed/">here</a>.
</aside>
## Workflow
To create a neural search service, you will need to transform your raw data and then create a search function to manipulate it. First, you will 1) download and prepare a sample dataset using a modified version of the BERT ML model. Then, you will 2) load the data into Qdrant, 3) create a neural search API and 4) serve it using FastAPI.
![Neural Search Workflow](/docs/workflow-neural-search.png)
> **Note**: The code for this tutorial can be found here: | [Step 1: Data Preparation Process](https://colab.research.google.com/drive/1kPktoudAP8Tu8n8l-iVMOQhVmHkWV_L9?usp=sharing) | [Step 2: Full Code for Neural Search](https://github.com/qdrant/qdrant_demo/tree/sentense-transformers). |
## Prerequisites
To complete this tutorial, you will need:
- Docker - The easiest way to use Qdrant is to run a pre-built Docker image.
- [Raw parsed data](https://storage.googleapis.com/generall-shared-data/startups_demo.json) from startups-list.com.
- Python version >=3.8
## Prepare sample dataset
To conduct a neural search on startup descriptions, you must first encode the description data into vectors. To process text, you can use a pre-trained models like [BERT](https://en.wikipedia.org/wiki/BERT_(language_model)) or sentence transformers. The [sentence-transformers](https://github.com/UKPLab/sentence-transformers) library lets you conveniently download and use many pre-trained models, such as DistilBERT, MPNet, etc.
1. First you need to download the dataset.
```bash
wget https://storage.googleapis.com/generall-shared-data/startups_demo.json
```
2. Install the SentenceTransformer library as well as other relevant packages.
```bash
pip install sentence-transformers numpy pandas tqdm
```
3. Import the required modules.
```python
from sentence_transformers import SentenceTransformer
import numpy as np
import json
import pandas as pd
from tqdm.notebook import tqdm
```
You will be using a pre-trained model called `all-MiniLM-L6-v2`.
This is a performance-optimized sentence embedding model and you can read more about it and other available models [here](https://www.sbert.net/docs/pretrained_models.html).
4. Download and create a pre-trained sentence encoder.
```python
model = SentenceTransformer(
"all-MiniLM-L6-v2", device="cuda"
) # or device="cpu" if you don't have a GPU
```
5. Read the raw data file.
```python
df = pd.read_json("./startups_demo.json", lines=True)
```
6. Encode all startup descriptions to create an embedding vector for each. Internally, the `encode` function will split the input into batches, which will significantly speed up the process.
```python
vectors = model.encode(
[row.alt + ". " + row.description for row in df.itertuples()],
show_progress_bar=True,
)
```
All of the descriptions are now converted into vectors. There are 40474 vectors of 384 dimensions. The output layer of the model has this dimension
```python
vectors.shape
# > (40474, 384)
```
7. Download the saved vectors into a new file named `startup_vectors.npy`
```python
np.save("startup_vectors.npy", vectors, allow_pickle=False)
```
## Run Qdrant in Docker
Next, you need to manage all of your data using a vector engine. Qdrant lets you store, update or delete created vectors. Most importantly, it lets you search for the nearest vectors via a convenient API.
> **Note:** Before you begin, create a project directory and a virtual python environment in it.
1. Download the Qdrant image from DockerHub.
```bash
docker pull qdrant/qdrant
```
2. Start Qdrant inside of Docker.
```bash
docker run -p 6333:6333 \
-v $(pwd)/qdrant_storage:/qdrant/storage \
qdrant/qdrant
```
You should see output like this
```text
...
[2021-02-05T00:08:51Z INFO actix_server::builder] Starting 12 workers
[2021-02-05T00:08:51Z INFO actix_server::builder] Starting "actix-web-service-0.0.0.0:6333" service on 0.0.0.0:6333
```
Test the service by going to [http://localhost:6333/](http://localhost:6333/). You should see the Qdrant version info in your browser.
All data uploaded to Qdrant is saved inside the `./qdrant_storage` directory and will be persisted even if you recreate the container.
## Upload data to Qdrant
1. Install the official Python client to best interact with Qdrant.
```bash
pip install qdrant-client
```
At this point, you should have startup records in the `startups_demo.json` file, encoded vectors in `startup_vectors.npy` and Qdrant running on a local machine.
Now you need to write a script to upload all startup data and vectors into the search engine.
2. Create a client object for Qdrant.
```python
# Import client library
from qdrant_client import QdrantClient
from qdrant_client.models import VectorParams, Distance
client = QdrantClient("http://localhost:6333")
```
3. Related vectors need to be added to a collection. Create a new collection for your startup vectors.
```python
if not client.collection_exists("startups"):
client.create_collection(
collection_name="startups",
vectors_config=VectorParams(size=384, distance=Distance.COSINE),
)
```
<aside role="status">
- The `vector_size` parameter defines the size of the vectors for a specific collection. If their size is different, it is impossible to calculate the distance between them. `384` is the encoder output dimensionality. You can also use `model.get_sentence_embedding_dimension()` to get the dimensionality of the model you are using.
- The `distance` parameter lets you specify the function used to measure the distance between two points.
</aside>
4. Create an iterator over the startup data and vectors.
The Qdrant client library defines a special function that allows you to load datasets into the service.
However, since there may be too much data to fit a single computer memory, the function takes an iterator over the data as input.
```python
fd = open("./startups_demo.json")
# payload is now an iterator over startup data
payload = map(json.loads, fd)
# Load all vectors into memory, numpy array works as iterable for itself.
# Other option would be to use Mmap, if you don't want to load all data into RAM
vectors = np.load("./startup_vectors.npy")
```
5. Upload the data
```python
client.upload_collection(
collection_name="startups",
vectors=vectors,
payload=payload,
ids=None, # Vector ids will be assigned automatically
batch_size=256, # How many vectors will be uploaded in a single request?
)
```
Vectors are now uploaded to Qdrant.
## Build the search API
Now that all the preparations are complete, let's start building a neural search class.
In order to process incoming requests, neural search will need 2 things: 1) a model to convert the query into a vector and 2) the Qdrant client to perform search queries.
1. Create a file named `neural_searcher.py` and specify the following.
```python
from qdrant_client import QdrantClient
from sentence_transformers import SentenceTransformer
class NeuralSearcher:
def __init__(self, collection_name):
self.collection_name = collection_name
# Initialize encoder model
self.model = SentenceTransformer("all-MiniLM-L6-v2", device="cpu")
# initialize Qdrant client
self.qdrant_client = QdrantClient("http://localhost:6333")
```
2. Write the search function.
```python
def search(self, text: str):
# Convert text query into vector
vector = self.model.encode(text).tolist()
# Use `vector` for search for closest vectors in the collection
search_result = self.qdrant_client.query_points(
collection_name=self.collection_name,
query=vector,
query_filter=None, # If you don't want any filters for now
limit=5, # 5 the most closest results is enough
).points
# `search_result` contains found vector ids with similarity scores along with the stored payload
# In this function you are interested in payload only
payloads = [hit.payload for hit in search_result]
return payloads
```
3. Add search filters.
With Qdrant it is also feasible to add some conditions to the search.
For example, if you wanted to search for startups in a certain city, the search query could look like this:
```python
from qdrant_client.models import Filter
...
city_of_interest = "Berlin"
# Define a filter for cities
city_filter = Filter(**{
"must": [{
"key": "city", # Store city information in a field of the same name
"match": { # This condition checks if payload field has the requested value
"value": city_of_interest
}
}]
})
search_result = self.qdrant_client.query_points(
collection_name=self.collection_name,
query=vector,
query_filter=city_filter,
limit=5
).points
...
```
You have now created a class for neural search queries. Now wrap it up into a service.
## Deploy the search with FastAPI
To build the service you will use the FastAPI framework.
1. Install FastAPI.
To install it, use the command
```bash
pip install fastapi uvicorn
```
2. Implement the service.
Create a file named `service.py` and specify the following.
The service will have only one API endpoint and will look like this:
```python
from fastapi import FastAPI
# The file where NeuralSearcher is stored
from neural_searcher import NeuralSearcher
app = FastAPI()
# Create a neural searcher instance
neural_searcher = NeuralSearcher(collection_name="startups")
@app.get("/api/search")
def search_startup(q: str):
return {"result": neural_searcher.search(text=q)}
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=8000)
```
3. Run the service.
```bash
python service.py
```
4. Open your browser at [http://localhost:8000/docs](http://localhost:8000/docs).
You should be able to see a debug interface for your service.
![FastAPI Swagger interface](/docs/fastapi_neural_search.png)
Feel free to play around with it, make queries regarding the companies in our corpus, and check out the results.
## Next steps
The code from this tutorial has been used to develop a [live online demo](https://qdrant.to/semantic-search-demo).
You can try it to get an intuition for cases when the neural search is useful.
The demo contains a switch that selects between neural and full-text searches.
You can turn the neural search on and off to compare your result with a regular full-text search.
> **Note**: The code for this tutorial can be found here: | [Step 1: Data Preparation Process](https://colab.research.google.com/drive/1kPktoudAP8Tu8n8l-iVMOQhVmHkWV_L9?usp=sharing) | [Step 2: Full Code for Neural Search](https://github.com/qdrant/qdrant_demo/tree/sentense-transformers). |
Join our [Discord community](https://qdrant.to/discord), where we talk about vector search and similarity learning, publish other examples of neural networks and neural search applications.
qdrant-landing/content/documentation/tutorials-search-engineering/pdf-retrieval-at-scale.md
+347
View File
@@ -0,0 +1,347 @@
---
title: Scaling PDF Retrieval with Qdrant
aliases:
- /documentation/tutorials/pdf-retrieval-at-scale/
short_description: "Optimizing PDF retrieval at scale with Qdrant and Vision Large Language Models (VLLMs) such as ColPali and ColQwen."
description: "Optimizing PDF retrieval at scale with Qdrant and Vision Large Language Models (VLLMs) such as ColPali and ColQwen. Two-stage retrieval with multivector representations mean pooling."
weight: 4
preview_image: /documentation/tutorials/pdf-retrieval-at-scale/social_preview.png
social_preview_image: /documentation/tutorials/pdf-retrieval-at-scale/social_preview.png
---
# Scaling PDF Retrieval with Qdrant
![scaling-pdf-retrieval-qdrant](/documentation/tutorials/pdf-retrieval-at-scale/image1.png)
| Time: 30 min | Level: Intermediate |Output: [GitHub](https://github.com/qdrant/examples/blob/master/pdf-retrieval-at-scale/ColPali_ColQwen2_Tutorial.ipynb)|[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/qdrant/examples/blob/master/pdf-retrieval-at-scale/ColPali_ColQwen2_Tutorial.ipynb) |
| --- | ----------- | ----------- | ----------- |
Efficient PDF documents retrieval is a common requirement in tasks like **(agentic) retrieval-augmented generation (RAG)** and many other search-based applications. At the same time, setting up PDF documents retrieval is rarely possible without additional challenges.
Many traditional PDF retrieval solutions rely on **optical character recognition (OCR)** together with use case-specific heuristics to handle visually complex elements like tables, images and charts. These algorithms are often non-transferable -- even within the same domain -- with their task-customized parsing and chunking strategies, labor-intensive, prone to errors, and difficult to scale.
Recent advancements in **Vision Large Language Models (VLLMs)**, such as [**ColPali**](https://huggingface.co/blog/manu/colpali) and its successor [**ColQwen**](https://huggingface.co/vidore/colqwen2-v0.1), started the transformation of the PDF retrieval. These multimodal models work directly with PDF pages as inputs, no pre-processing required. Anything that can be converted into an **image** (think of PDFs as screenshots of document pages) can be effectively processed by these models. Being far simpler in use, VLLMs achieve state-of-the-art performance in PDF retrieval benchmarks like the [Visual Document Retrieval (ViDoRe) Benchmark](https://huggingface.co/spaces/vidore/vidore-leaderboard).
## How VLLMs Work for PDF Retrieval
VLLMs like **ColPali** and **ColQwen** generate **multivector representations** for each PDF page; the representations are stored and indexed in a vector database. During the retrieval process, models dynamically create multivector representations for (textual) user queries, and precise retrieval -- matching between PDF pages and queries -- is achieved through [late-interaction mechanism](/blog/qdrant-colpali/#how-colpali-works-under-the-hood).
<aside role="status"> Qdrant supports <a href="/documentation/concepts/vectors/#multivectors">multivector representations</a>, making it well-suited for using embedding models such as ColPali, ColQwen, or <a href="/documentation/fastembed/fastembed-colbert/">ColBERT</a></aside>
## Challenges of Scaling VLLMs
The heavy multivector representations produced by VLLMs make PDF retrieval at scale computationally intensive. These models are inefficient for large-scale PDF retrieval tasks if used without optimization.
### Math Behind the Scaling
**ColPali** generates over **1,000 vectors per PDF page**, while its successor, **ColQwen**, generates slightly fewer — up to **768 vectors**, dynamically adjusted based on the image size. Typically, ColQwen produces **~700 vectors per page**.
To understand the impact, consider the construction of an [**HNSW index**](/articles/what-is-a-vector-database/#1-indexing-hnsw-index-and-sending-data-to-qdrant), a common indexing algorithm for vector databases. Let's roughly estimate the number of comparisons needed to insert a new PDF page into the index.
- **Vectors per page:** ~700 (ColQwen) or ~1,000 (ColPali)
- **[ef_construct](/documentation/concepts/indexing/#vector-index):** 100 (default)
The lower bound estimation for the number of vector comparisons would be:
$$
700 \times 700 \times 100 = 49 \ \text{millions}
$$
Now imagine how much it will take to build an index on **20,000 pages**!
For ColPali, this number doubles. The result is **extremely slow index construction time**.
### Our Solution
We recommend reducing the number of vectors in a PDF page representation for the **first-stage retrieval**. After the first stage retrieval with a reduced amount of vectors, we propose to **rerank** retrieved subset with the original uncompressed representation.
<aside role="status"> You might consider using <b>quantization</b> (e.g., binary quantization) to reduce computational resources. However, as you can see above, quantization does not impact the parameters that determine the number of comparisons, so it will only affect memory consumption.</aside>
The reduction of vectors can be achieved by applying a **mean pooling operation** to the multivector VLLM-generated outputs. Mean pooling averages the values across all vectors within a selected subgroup, condensing multiple vectors into a single representative vector. If done right, it allows the preservation of important information from the original page while significantly reducing the number of vectors.
VLLMs generate vectors corresponding to patches that represent different portions of a PDF page. These patches can be grouped in columns and rows of a PDF page.
For example:
- ColPali divides PDF page into **1,024 patches**.
- Applying mean pooling by rows (or columns) of this patch matrix reduces the page representation to just **32 vectors**.
![ColPali patching of a PDF page](/documentation/tutorials/pdf-retrieval-at-scale/pooling-by-rows.png)
We tested this approach with the ColPali model, mean pooling its multivectors by PDF page rows. The results showed:
- **Indexing time faster by an order of magnitude**
- **Retrieval quality comparable to the original model**
For details of this experiment refer to our [gitHub repository](https://github.com/qdrant/demo-colpali-optimized), [ColPali optimization blog post](/blog/colpali-qdrant-optimization/) or [webinar "PDF Retrieval at Scale"](https://www.youtube.com/watch?v=_h6SN1WwnLs)
## Goal of This Tutorial
In this tutorial, we will demonstrate a scalable approach to PDF retrieval using **Qdrant** and **ColPali** & **ColQwen2** VLLMs.
The presented approach is **highly recommended** to avoid the common pitfalls of long indexing times and slow retrieval speeds.
In the following sections, we will demonstrate an optimized retrieval algorithm born out of our successful experimentation:
**First-Stage Retrieval with Mean-Pooled Vectors:**
- Construct an HNSW index using **only mean-pooled vectors**.
- Use them for the first-stage retrieval.
**Reranking with Original Model Multivectors:**
- Use the original multivectors from ColPali or ColQwen2 **to rerank** the results retrieved in the first stage.
## Setup
Install & import required libraries
```python
# pip install colpali_engine>=0.3.1
from colpali_engine.models import ColPali, ColPaliProcessor
# pip install qdrant-client>=1.12.0
from qdrant_client import QdrantClient, models
```
To run these experiments, we’re using a **Qdrant cluster**. If you’re just getting started, you can set up a **free-tier cluster** for testing and exploration. Follow the instructions in the documentation ["How to Create a Free-Tier Qdrant Cluster"](/documentation/cloud/create-cluster/#free-clusters)
```python
client = QdrantClient(
url=<YOUR CLUSTER URL>,
api_key=<YOUR API KEY>
)
```
Download **ColPali** model along with its input processors. Make sure to select the backend that suits your setup.
```python
colpali_model = ColPali.from_pretrained(
"vidore/colpali-v1.3",
torch_dtype=torch.bfloat16,
device_map="mps", # Use "cuda:0" for GPU, "cpu" for CPU, or "mps" for Apple Silicon
).eval()
colpali_processor = ColPaliProcessor.from_pretrained("vidore/colpali-v1.3")
```
<details>
<summary> For <b>ColQwen</b> model </summary>
```python
from colpali_engine.models import ColQwen2, ColQwen2Processor
colqwen_model = ColQwen2.from_pretrained(
"vidore/colqwen2-v0.1",
torch_dtype=torch.bfloat16,
device_map="mps", # Use "cuda:0" for GPU, "cpu" for CPU, or "mps" for Apple Silicon
).eval()
colqwen_processor = ColQwen2Processor.from_pretrained("vidore/colqwen2-v0.1")
```
</details>
## Create Qdrant Collections
We can now create a collection in Qdrant to store the multivector representations of PDF pages generated by **ColPali** or **ColQwen**.
Collection will include **mean pooled** by rows and columns representations of a PDF page, as well as the **original** multivector representation.
<aside role="status"> For the original multivectors generated by the models, we will disable HNSW index construction </aside>
```python
client.create_collection(
collection_name=collection_name,
vectors_config={
"original":
models.VectorParams( #switch off HNSW
size=128,
distance=models.Distance.COSINE,
multivector_config=models.MultiVectorConfig(
comparator=models.MultiVectorComparator.MAX_SIM
),
hnsw_config=models.HnswConfigDiff(
m=0 #switching off HNSW
)
),
"mean_pooling_columns": models.VectorParams(
size=128,
distance=models.Distance.COSINE,
multivector_config=models.MultiVectorConfig(
comparator=models.MultiVectorComparator.MAX_SIM
)
),
"mean_pooling_rows": models.VectorParams(
size=128,
distance=models.Distance.COSINE,
multivector_config=models.MultiVectorConfig(
comparator=models.MultiVectorComparator.MAX_SIM
)
)
}
)
```
## Choose a dataset
We’ll use the **UFO Dataset** by Daniel van Strien for this tutorial. It’s available on Hugging Face; you can download it directly from there.
```python
from datasets import load_dataset
ufo_dataset = "davanstrien/ufo-ColPali"
dataset = load_dataset(ufo_dataset, split="train")
```
## Embedding and Mean Pooling
We'll use a function that generates multivector representations and their mean pooled versions of each PDF page (aka image) in batches.
For complete understanding, it's important to consider the following specifics of **ColPali** and **ColQwen**:
**ColPali:**
In theory, ColPali is designed to generate 1,024 vectors per PDF page, but in practice, it produces 1,030 vectors. This discrepancy is due to ColPali's pre-processor, which appends the text `<bos>Describe the image.` to each input. This additional text generates an extra 6 multivectors.
**ColQwen:**
ColQwen dynamically determines the number of patches in "rows and columns" of a PDF page based on its size. Consequently, the number of multivectors can vary between inputs. ColQwen pre-processor prepends `<|im_start|>user<|vision_start|>` and appends `<|vision_end|>Describe the image.<|im_end|><|endoftext|>`.
For example, that's how ColQwen multivector output is formed.
![that's how ColQwen multivector output is formed](/documentation/tutorials/pdf-retrieval-at-scale/ColQwen-preprocessing.png)
The `get_patches` function is to get the number of `x_patches` (rows) and `y_patches` (columns) ColPali/ColQwen2 models will divide a PDF page into.
For ColPali, the numbers will always be 32 by 32; ColQwen will define them dynamically based on the PDF page size.
```python
x_patches, y_patches = model_processor.get_n_patches(
image_size,
patch_size=model.patch_size
)
```
<details>
<summary> For <b>ColQwen</b> model </summary>
```python
model_processor.get_n_patches(
image_size,
patch_size=model.patch_size,
spatial_merge_size=model.spatial_merge_size
)
```
</details>
We choose to **preserve prefix and postfix multivectors**. Our **pooling** operation compresses the multivectors representing **the image tokens** based on the number of rows and columns determined by the model (static 32x32 for ColPali, dynamic XxY for ColQwen). Function retains and integrates the additional multivectors produced by the model back to pooled representations.
Simplified version of pooling for **ColPali** model:
(see the full version -- also applicable for **ColQwen** -- in the [tutorial notebook](https://githubtocolab.com/qdrant/examples/blob/master/pdf-retrieval-at-scale/ColPali_ColQwen2_Tutorial.ipynb))
```python
processed_images = model_processor.process_images(image_batch)
# Image embeddings of shape (batch_size, 1030, 128)
image_embeddings = model(**processed_images)
# (1030, 128)
image_embedding = image_embeddings[0] # take the first element of the batch
# Now we need to identify vectors that correspond to the image tokens
# It can be done by selecting tokens corresponding to special `image_token_id`
# (1030, ) - boolean mask (for the first element in the batch), True for image tokens
mask = processed_images.input_ids[0] == model_processor.image_token_id
# For convenience, we now select only image tokens
# and reshape them to (x_patches, y_patches, dim)
# (x_patches, y_patches, 128)
image_patch_embeddings = image_embedding[mask].view(x_patches, y_patches, model.dim)
# Now we can apply mean pooling by rows and columns
# (x_patches, 128)
pooled_by_rows = image_patch_embeddings.mean(dim=0)
# (y_patches, 128)
pooled_by_columns = image_patch_embeddings.mean(dim=1)
# [Optionally] we can also concatenate special tokens to the pooled representations,
# For ColPali, it's only postfix
# (x_patches + 6, 128)
pooled_by_rows = torch.cat([pooled_by_rows, image_embedding[~mask]])
# (y_patches + 6, 128)
pooled_by_columns = torch.cat([pooled_by_columns, image_embedding[~mask]])
```
</details>
## Upload to Qdrant
The upload process is trivial; the only thing to pay attention to is the compute cost for ColPali and ColQwen2 models.
In low-resource environments, it's recommended to use a smaller batch size for embedding and mean pooling.
Full version of the upload code is available in the [tutorial notebook](https://githubtocolab.com/qdrant/examples/blob/master/pdf-retrieval-at-scale/ColPali_ColQwen2_Tutorial.ipynb)
## Querying PDFs
After indexing PDF documents, we can move on to querying them using our two-stage retrieval approach.
```python
query = "Lee Harvey Oswald's involvement in the JFK assassination"
processed_queries = model_processor.process_queries([query]).to(model.device)
# Resulting query embedding is a tensor of shape (22, 128)
query_embedding = model(**processed_queries)[0]
```
Now let's design a function for the two-stage retrieval with multivectors produced by VLLMs:
- **Step 1:** Prefetch results using a compressed multivector representation & HNSW index.
- **Step 2:** Re-rank the prefetched results using the original multivector representation.
Let's query our collections using combined mean pooled representations for the first stage of retrieval.
```python
# Final amount of results to return
search_limit = 10
# Amount of results to prefetch for reranking
prefetch_limit = 100
response = client.query_points(
collection_name=collection_name,
query=query_embedding,
prefetch=[
models.Prefetch(
query=query_embedding,
limit=prefetch_limit,
using="mean_pooling_columns"
),
models.Prefetch(
query=query_embedding,
limit=prefetch_limit,
using="mean_pooling_rows"
),
],
limit=search_limit,
with_payload=True,
with_vector=False,
using="original"
)
```
And check the top retrieved result to our query *"Lee Harvey Oswald's involvement in the JFK assassination"*.
```python
dataset[response.points[0].payload['index']]['image']
```
![Results, ColPali](/documentation/tutorials/pdf-retrieval-at-scale/result-VLLMs.png)
## Conclusion
In this tutorial, we demonstrated an optimized approach using **Qdrant for PDF retrieval at scale** with VLLMs producing **heavy multivector representations** like **ColPali** and **ColQwen2**.
Without such optimization, the performance of retrieval systems can degrade severely, both in terms of indexing time and query latency, especially as the dataset size grows.
We **strongly recommend** implementing this approach in your workflows to ensure efficient and scalable PDF retrieval. Neglecting to optimize the retrieval process could result in unacceptably slow performance, hindering the usability of your system.
Start scaling your PDF retrieval today!
qdrant-landing/content/documentation/tutorials-search-engineering/reranking-hybrid-search.md
+317
View File
@@ -0,0 +1,317 @@
---
title: Reranking in Hybrid Search
weight: 2
aliases:
- /documentation/search-precision/reranking-hybrid-search/
---
# Reranking Hybrid Search Results with Qdrant Vector Database
Hybrid search combines dense and sparse retrieval to deliver precise and comprehensive results. By adding reranking with ColBERT, you can further refine search outputs for maximum relevance.
In this guide, we’ll show you how to implement hybrid search with reranking in Qdrant, leveraging dense, sparse, and late interaction embeddings to create an efficient, high-accuracy search system. Let’s get started!
## Overview
Let’s start by breaking down the architecture:
![image3.png](/documentation/examples/reranking-hybrid-search/image3.png)
Processing Dense, Sparse, and Late Interaction Embeddings in Vector Databases (VDB)
### Ingestion Stage
Here’s how we’re going to set up the advanced hybrid search. The process is similar to what we did earlier but with a few powerful additions:
1. **Documents**: Just like before, we start with the raw input—our set of documents that need to be indexed for search.
2. **Dense Embeddings**: We’ll generate dense embeddings for each document, just like in the basic search. These embeddings capture the deeper, semantic meanings behind the text.
3. **Sparse Embeddings**: This is where it gets interesting. Alongside dense embeddings, we’ll create sparse embeddings using more traditional, keyword-based methods. Specifically, we’ll use BM25, a probabilistic retrieval model. BM25 ranks documents based on how relevant their terms are to a given query, taking into account how often terms appear, document length, and how common the term is across all documents. It’s perfect for keyword-heavy searches.
4. **Late Interaction Embeddings**: Now, we add the magic of ColBERT. ColBERT uses a two-stage approach. First, it generates contextualized embeddings for both queries and documents using BERT, and then it performs late interaction—matching those embeddings efficiently using a dot product to fine-tune relevance. This step allows for deeper, contextual understanding, making sure you get the most precise results.
5. **Vector Database**: All of these embeddings—dense, sparse, and late interaction—are stored in a vector database like Qdrant. This allows you to efficiently search, retrieve, and rerank your documents based on multiple layers of relevance.
![image2.png](/documentation/examples/reranking-hybrid-search/image2.png)
Query Retrieval and Reranking Process in Search Systems
### Retrieval Stage
Now, let's talk about how we’re going to pull the best results once the user submits a query:
1. **User’s Query**: The user enters a query, and that query is transformed into multiple types of embeddings. We’re talking about representations that capture both the deeper meaning (dense) and specific keywords (sparse).
2. **Embeddings**: The query gets converted into various embeddings—some for understanding the semantics (dense embeddings) and others for focusing on keyword matches (sparse embeddings).
3. **Hybrid Search**: Our hybrid search uses both dense and sparse embeddings to find the most relevant documents. The dense embeddings ensure we capture the overall meaning of the query, while sparse embeddings make sure we don’t miss out on those key, important terms.
4. **Rerank**: Once we’ve got a set of documents, the final step is reranking. This is where late interaction embeddings come into play, giving you results that are not only relevant but tuned to your query by prioritizing the documents that truly meet the user's intent.
## Implementation
Let’s see it in action in this section.
### Additional Setup
This time around, we’re using FastEmbed—a lightweight Python library designed for generating embeddings, and it supports popular text models right out of the box. First things first, you’ll need to install it:
```python
pip install fastembed
```
---
Here are the models we’ll be pulling from FastEmbed:
```python
from fastembed import TextEmbedding, LateInteractionTextEmbedding, SparseTextEmbedding
```
---
### Ingestion
As before, we’ll convert our documents into embeddings, but thanks to FastEmbed, the process is even more straightforward because all the models you need are conveniently available in one location.
### Embeddings
First, let’s load the models we need:
```python
dense_embedding_model = TextEmbedding("sentence-transformers/all-MiniLM-L6-v2")
bm25_embedding_model = SparseTextEmbedding("Qdrant/bm25")
late_interaction_embedding_model = LateInteractionTextEmbedding("colbert-ir/colbertv2.0")
```
---
Now, let’s convert our documents into embeddings:
```python
dense_embeddings = list(dense_embedding_model.embed(doc for doc in documents))
bm25_embeddings = list(bm25_embedding_model.embed(doc for doc in documents))
late_interaction_embeddings = list(late_interaction_embedding_model.embed(doc for doc in documents))
```
---
Since we’re dealing with multiple types of embeddings (dense, sparse, and late interaction), we’ll need to store them in a collection that supports a multi-vector setup. The previous collection we created won’t work here, so we’ll create a new one designed specifically for handling these different types of embeddings.
### Create Collection
Now, we’re setting up a new collection in Qdrant for our hybrid search with the right configurations to handle all the different vector types we’re working with.
Here’s how you do it:
```python
from qdrant_client.models import Distance, VectorParams, models
client.create_collection(
"hybrid-search",
vectors_config={
"all-MiniLM-L6-v2": models.VectorParams(
size=len(dense_embeddings[0]),
distance=models.Distance.COSINE,
),
"colbertv2.0": models.VectorParams(
size=len(late_interaction_embeddings[0][0]),
distance=models.Distance.COSINE,
multivector_config=models.MultiVectorConfig(
comparator=models.MultiVectorComparator.MAX_SIM,
),
hnsw_config=models.HnswConfigDiff(m=0) # Disable HNSW for reranking
),
},
sparse_vectors_config={
"bm25": models.SparseVectorParams(modifier=models.Modifier.IDF
)
}
)
```
---
What’s happening here? We’re creating a collection called "hybrid-search", and we’re configuring it to handle:
- **Dense embeddings** from the model all-MiniLM-L6-v2 using cosine distance for comparisons.
- **Late interaction embeddings** from colbertv2.0, also using cosine distance, but with a multivector configuration to use the maximum similarity comparator. Note that we set `m=0` in the `colbertv2.0` vector to prevent indexing since it's not needed for reranking.
- **Sparse embeddings** from BM25 for keyword-based searches. They use `dot_product` for similarity calculation.
This setup ensures that all the different types of vectors are stored and compared correctly for your hybrid search.
### Upsert Data
Next, we need to insert the documents along with their multiple embeddings into the **hybrid-search** collection:
```python
from qdrant_client.models import PointStruct
points = []
for idx, (dense_embedding, bm25_embedding, late_interaction_embedding, doc) in enumerate(zip(dense_embeddings, bm25_embeddings, late_interaction_embeddings, documents)):
point = PointStruct(
id=idx,
vector={
"all-MiniLM-L6-v2": dense_embedding,
"bm25": bm25_embedding.as_object(),
"colbertv2.0": late_interaction_embedding,
},
payload={"document": doc}
)
points.append(point)
operation_info = client.upsert(
collection_name="hybrid-search",
points=points
)
```
<aside role="status">
Check how points can be uploaded with builtin Fastembed integration.
</aside>
<details>
<summary>Upload with implicit embeddings computation</summary>
```python
from qdrant_client.models import PointStruct
points = []
for idx, doc in enumerate(documents):
point = PointStruct(
id=idx,
vector={
"all-MiniLM-L6-v2": models.Document(text=doc, model="sentence-transformers/all-MiniLM-L6-v2"),
"bm25": models.Document(text=doc, model="Qdrant/bm25"),
"colbertv2.0": models.Document(text=doc, model="colbert-ir/colbertv2.0"),
},
payload={"document": doc}
)
points.append(point)
operation_info = client.upsert(
collection_name="hybrid-search",
points=points
)
```
</details>
---
This code pulls everything together by creating a list of **PointStruct** objects, each containing the embeddings and corresponding documents.
For each document, it adds:
- **Dense embeddings** for the deep, semantic meaning.
- **BM25 embeddings** for powerful keyword-based search.
- **ColBERT embeddings** for precise contextual interactions.
Once that’s done, the points are uploaded into our **"hybrid-search"** collection using the upsert method, ensuring everything’s in place.
### Retrieval
For retrieval, it’s time to convert the user’s query into the required embeddings. Here’s how you can do it:
```python
dense_vectors = next(dense_embedding_model.query_embed(query))
sparse_vectors = next(bm25_embedding_model.query_embed(query))
late_vectors = next(late_interaction_embedding_model.query_embed(query))
```
---
The real magic of hybrid search lies in the **prefetch** parameter. This lets you run multiple sub-queries in one go, combining the power of dense and sparse embeddings. Here’s how to set it up, after which we execute the hybrid search:
```python
prefetch = [
models.Prefetch(
query=dense_vectors,
using="all-MiniLM-L6-v2",
limit=20,
),
models.Prefetch(
query=models.SparseVector(**sparse_vectors.as_object()),
using="bm25",
limit=20,
),
]
```
---
This code kicks off a hybrid search by running two sub-queries:
- One using dense embeddings from "all-MiniLM-L6-v2" to capture the semantic meaning of the query.
- The other using sparse embeddings from BM25 for strong keyword matching.
Each sub-query is limited to 20 results. These sub-queries are bundled together using the prefetch parameter, allowing them to run in parallel.
### Rerank
Now that we've got our initial hybrid search results, it’s time to rerank them using late interaction embeddings for maximum precision. Here’s how you can do it:
```python
results = client.query_points(
"hybrid-search",
prefetch=prefetch,
query=late_vectors,
using="colbertv2.0",
with_payload=True,
limit=10,
)
```
<aside role="status">
Check how queries can be made with builtin Fastembed integration.
</aside>
<details>
<summary>Query points with implicit embeddings computation</summary>
```python
prefetch = [
models.Prefetch(
query=models.Document(text=query, model="sentence-transformers/all-MiniLM-L6-v2"),
using="all-MiniLM-L6-v2",
limit=20,
),
models.Prefetch(
query=models.Document(text=query, model="Qdrant/bm25"),
using="bm25",
limit=20,
),
]
results = client.query_points(
"hybrid-search",
prefetch=prefetch,
query=models.Document(text=query, model="colbert-ir/colbertv2.0"),
using="colbertv2.0",
with_payload=True,
limit=10,
)
```
</details>
---
Let’s look at how the positions change after applying reranking. Notice how some documents shift in rank based on their relevance according to the late interaction embeddings.
| | **Document** | **First Query Rank** | **Second Query Rank** | **Rank Change** |
| --- | --- | --- | --- | --- |
| | In machine learning, feature scaling is the process of normalizing the range of independent variables or features. The goal is to ensure that all features contribute equally to the model, especially in algorithms like SVM or k-nearest neighbors where distance calculations matter. | 1 | 1 | No Change |
| | Feature scaling is commonly used in data preprocessing to ensure that features are on the same scale. This is particularly important for gradient descent-based algorithms where features with larger scales could disproportionately impact the cost function. | 2 | 6 | Moved Down |
| | Unsupervised learning algorithms, such as clustering methods, may benefit from feature scaling, which ensures that features with larger numerical ranges don't dominate the learning process. | 3 | 4 | Moved Down |
| | Data preprocessing steps, including feature scaling, can significantly impact the performance of machine learning models, making it a crucial part of the modeling pipeline. | 5 | 2 | Moved Up |
Great! We've now explored how reranking works and successfully implemented it.
## Best Practices in Reranking
Reranking can dramatically improve the relevance of search results, especially when combined with hybrid search. Here are some best practices to keep in mind:
- **Implement Hybrid Reranking**: Blend keyword-based (sparse) and vector-based (dense) search results for a more comprehensive ranking system.
- **Continuous Testing and Monitoring**: Regularly evaluate your reranking models to avoid overfitting and make timely adjustments to maintain performance.
- **Balance Relevance and Latency**: Reranking can be computationally expensive, so aim for a balance between relevance and speed. Therefore, the first step is to retrieve the relevant documents and then use reranking on it.
## Conclusion
Reranking is a powerful tool that boosts the relevance of search results, especially when combined with hybrid search methods. While it can add some latency due to its complexity, applying it to a smaller, pre-filtered subset of results ensures both speed and relevance.
Qdrant offers an easy-to-use API to get started with your own search engine, so if you’re ready to dive in, sign up for free at [Qdrant Cloud](https://qdrant.tech/) and start building
qdrant-landing/content/documentation/tutorials-search-engineering/retrieval-quality.md
+229
View File
@@ -0,0 +1,229 @@
---
title: Measure Search Quality
aliases:
- /documentation/tutorials/retrieval-quality/
weight: 4
---
# Measure and Improve Retrieval Quality in Semantic Search
| Time: 30 min | Level: Intermediate | | |
|--------------|---------------------|--|----|
Semantic search pipelines are as good as the embeddings they use. If your model cannot properly represent input data, similar objects might
be far away from each other in the vector space. No surprise, that the search results will be poor in this case. There is, however, another
component of the process which can also degrade the quality of the search results. It is the ANN algorithm itself.
In this tutorial, we will show how to measure the quality of the semantic retrieval and how to tune the parameters of the HNSW, the ANN
algorithm used in Qdrant, to obtain the best results.
## Embeddings quality
The quality of the embeddings is a topic for a separate tutorial. In a nutshell, it is usually measured and compared by benchmarks, such as
[Massive Text Embedding Benchmark (MTEB)](https://huggingface.co/spaces/mteb/leaderboard). The evaluation process itself is pretty
straightforward and is based on a ground truth dataset built by humans. We have a set of queries and a set of the documents we would expect
to receive for each of them. In the [evaluation process](https://qdrant.tech/rag/rag-evaluation-guide/), we take a query, find the most similar documents in the vector space and compare
them with the ground truth. In that setup, **finding the most similar documents is implemented as full kNN search, without any approximation**.
As a result, we can measure the quality of the embeddings themselves, without the influence of the ANN algorithm.
## Retrieval quality
Embeddings quality is indeed the most important factor in the semantic search quality. However, vector search engines, such as Qdrant, do not
perform pure kNN search. Instead, they use **Approximate Nearest Neighbors** (ANN) algorithms, which are much faster than the exact search,
but can return suboptimal results. We can also **measure the retrieval quality of that approximation** which also contributes to the overall
search quality.
### Quality metrics
There are various ways of how quantify the quality of semantic search. Some of them, such as [Precision@k](https://en.wikipedia.org/wiki/Evaluation_measures_(information_retrieval)#Precision_at_k),
are based on the number of relevant documents in the top-k search results. Others, such as [Mean Reciprocal Rank (MRR)](https://en.wikipedia.org/wiki/Mean_reciprocal_rank),
take into account the position of the first relevant document in the search results. [DCG and NDCG](https://en.wikipedia.org/wiki/Discounted_cumulative_gain)
metrics are, in turn, based on the relevance score of the documents.
If we treat the search pipeline as a whole, we could use them all. The same is true for the embeddings quality evaluation. However, for the
ANN algorithm itself, anything based on the relevance score or ranking is not applicable. Ranking in vector search relies on the distance
between the query and the document in the vector space, however distance is not going to change due to approximation, as the function is
still the same.
Therefore, it only makes sense to measure the quality of the ANN algorithm by the number of relevant documents in the top-k search results,
such as `precision@k`. It is calculated as the number of relevant documents in the top-k search results divided by `k`. In case of testing
just the ANN algorithm, we can use the exact kNN search as a ground truth, with `k` being fixed. It will be a measure on **how well the ANN
algorithm approximates the exact search**.
## Measure the quality of the search results
Let's build a quality [evaluation](https://qdrant.tech/rag/rag-evaluation-guide/) of the ANN algorithm in Qdrant. We will, first, call the search endpoint in a standard way to obtain
the approximate search results. Then, we will call the exact search endpoint to obtain the exact matches, and finally compare both results
in terms of precision.
Before we start, let's create a collection, fill it with some data and then start our evaluation. We will use the same dataset as in the
[Loading a dataset from Hugging Face hub](/documentation/tutorials/huggingface-datasets/) tutorial, `Qdrant/arxiv-titles-instructorxl-embeddings`
from the [Hugging Face hub](https://huggingface.co/datasets/Qdrant/arxiv-titles-instructorxl-embeddings). Let's download it in a streaming
mode, as we are only going to use part of it.
```python
from datasets import load_dataset
dataset = load_dataset(
"Qdrant/arxiv-titles-instructorxl-embeddings", split="train", streaming=True
)
```
We need some data to be indexed and another set for the testing purposes. Let's get the first 50000 items for the training and the next 1000
for the testing.
```python
dataset_iterator = iter(dataset)
train_dataset = [next(dataset_iterator) for _ in range(60000)]
test_dataset = [next(dataset_iterator) for _ in range(1000)]
```
Now, let's create a collection and index the training data. This collection will be created with the default configuration. Please be aware that
it might be different from your collection settings, and it's always important to test exactly the same configuration you are going to use later
in production.
<aside role="status">
Distance function is another parameter that may impact the retrieval quality. If the embedding model was not trained to minimize cosine
distance, you can get suboptimal search results by using it. Please test different distance functions to find the best one for your embeddings,
if you don't know the specifics of the model training.
</aside>
```python
from qdrant_client import QdrantClient, models
client = QdrantClient("http://localhost:6333")
client.create_collection(
collection_name="arxiv-titles-instructorxl-embeddings",
vectors_config=models.VectorParams(
size=768, # Size of the embeddings generated by InstructorXL model
distance=models.Distance.COSINE,
),
)
```
We are now ready to index the training data. Uploading the records is going to trigger the indexing process, which will build the HNSW graph.
The indexing process may take some time, depending on the size of the dataset, but your data is going to be available for search immediately
after receiving the response from the `upsert` endpoint. **As long as the indexing is not finished, and HNSW not built, Qdrant will perform
the exact search**. We have to wait until the indexing is finished to be sure that the approximate search is performed.
```python
client.upload_points( # upload_points is available as of qdrant-client v1.7.1
collection_name="arxiv-titles-instructorxl-embeddings",
points=[
models.PointStruct(
id=item["id"],
vector=item["vector"],
payload=item,
)
for item in train_dataset
]
)
while True:
collection_info = client.get_collection(collection_name="arxiv-titles-instructorxl-embeddings")
if collection_info.status == models.CollectionStatus.GREEN:
# Collection status is green, which means the indexing is finished
break
```
## Standard mode vs exact search
Qdrant has a built-in exact search mode, which can be used to measure the quality of the search results. In this mode, Qdrant performs a
full kNN search for each query, without any approximation. It is not suitable for production use with high load, but it is perfect for the
evaluation of the ANN algorithm and its parameters. It might be triggered by setting the `exact` parameter to `True` in the search request.
We are simply going to use all the examples from the test dataset as queries and compare the results of the approximate search with the
results of the exact search. Let's create a helper function with `k` being a parameter, so we can calculate the `precision@k` for different
values of `k`.
```python
def avg_precision_at_k(k: int):
precisions = []
for item in test_dataset:
ann_result = client.query_points(
collection_name="arxiv-titles-instructorxl-embeddings",
query=item["vector"],
limit=k,
).points
knn_result = client.query_points(
collection_name="arxiv-titles-instructorxl-embeddings",
query=item["vector"],
limit=k,
search_params=models.SearchParams(
exact=True, # Turns on the exact search mode
),
).points
# We can calculate the precision@k by comparing the ids of the search results
ann_ids = set(item.id for item in ann_result)
knn_ids = set(item.id for item in knn_result)
precision = len(ann_ids.intersection(knn_ids)) / k
precisions.append(precision)
return sum(precisions) / len(precisions)
```
Calculating the `precision@5` is as simple as calling the function with the corresponding parameter:
```python
print(f"avg(precision@5) = {avg_precision_at_k(k=5)}")
```
Response:
```text
avg(precision@5) = 0.9935999999999995
```
As we can see, the precision of the approximate search vs exact search is pretty high. There are, however, some scenarios when we
need higher precision and can accept higher latency. HNSW is pretty tunable, and we can increase the precision by changing its parameters.
## Tweaking the HNSW parameters
HNSW is a hierarchical graph, where each node has a set of links to other nodes. The number of edges per node is called the `m` parameter.
The larger the value of it, the higher the precision of the search, but more space required. The `ef_construct` parameter is the number of
neighbours to consider during the index building. Again, the larger the value, the higher the precision, but the longer the indexing time.
The default values of these parameters are `m=16` and `ef_construct=100`. Let's try to increase them to `m=32` and `ef_construct=200` and
see how it affects the precision. Of course, we need to wait until the indexing is finished before we can perform the search.
```python
client.update_collection(
collection_name="arxiv-titles-instructorxl-embeddings",
hnsw_config=models.HnswConfigDiff(
m=32, # Increase the number of edges per node from the default 16 to 32
ef_construct=200, # Increase the number of neighbours from the default 100 to 200
)
)
while True:
collection_info = client.get_collection(collection_name="arxiv-titles-instructorxl-embeddings")
if collection_info.status == models.CollectionStatus.GREEN:
# Collection status is green, which means the indexing is finished
break
```
The same function can be used to calculate the average `precision@5`:
```python
print(f"avg(precision@5) = {avg_precision_at_k(k=5)}")
```
Response:
```text
avg(precision@5) = 0.9969999999999998
```
The precision has obviously increased, and we know how to control it. However, there is a trade-off between the precision and the search
latency and memory requirements. In some specific cases, we may want to increase the precision as much as possible, so now we know how
to do it.
## Wrapping up
Assessing the quality of retrieval is a critical aspect of [evaluating](https://qdrant.tech/rag/rag-evaluation-guide/) semantic search performance. It is imperative to measure retrieval quality when aiming for optimal quality of.
your search results. Qdrant provides a built-in exact search mode, which can be used to measure the quality of the ANN algorithm itself,
even in an automated way, as part of your CI/CD pipeline.
Again, **the quality of the embeddings is the most important factor**. HNSW does a pretty good job in terms of precision, and it is
parameterizable and tunable, when required. There are some other ANN algorithms available out there, such as [IVF*](https://github.com/facebookresearch/faiss/wiki/Faiss-indexes#cell-probe-methods-indexivf-indexes),
but they usually [perform worse than HNSW in terms of quality and performance](https://nirantk.com/writing/pgvector-vs-qdrant/#correctness).
qdrant-landing/content/documentation/tutorials-search-engineering/static-embeddings.md
+164
View File
@@ -0,0 +1,164 @@
---
title: Static Embeddings. Should you pay attention?
weight: 181
aliases:
- /blog/static-embeddings/
---
# Static Embeddings: should you pay attention?
In the world of resource-constrained computing, a quiet revolution is taking place. While transformers dominate
leaderboards with their impressive capabilities, static embeddings are making an unexpected comeback, offering
remarkable speed improvements with surprisingly small quality trade-offs. **We evaluated how Qdrant users can benefit
from this renaissance, and the results are promising**.
## What makes static embeddings different?
Transformers are often seen as the only way to go when it comes to embeddings. The use of attention mechanisms helps to
capture the relationships between the input tokens, so each token gets a vector representation that is context-aware
and defined not only by the token itself but also by the surrounding tokens. Transformer-based models easily beat the
quality of the older methods, such as word2vec or GloVe, which could only create a single vector embedding per each
word. As a result, the word "bank" would have identical representation in the context of "river bank" and "financial
institution".
![Static embeddings](/documentation/tutorials/static-embeddings/financial-river-bank.png)
Transformer-based models would represent the word "bank" differently in each of the contexts. However, transformers come
with a cost. They are computationally expensive and usually require a lot of memory, although the embeddings models
usually have fewer parameters than the Large Language Models. Still, GPUs are preferred to be used, even for inference.
Static embeddings are still a thing, though! [MinishLab](https://minishlab.github.io/) introduced their [model2vec
technique](https://huggingface.co/blog/Pringled/model2vec) in October 2024, achieving a remarkable 15x reduction in
model size and up to 500x speed increase while maintaining impressive performance levels. Their idea was to distill the
knowledge from the transformer-based sentence transformer and create a static embedding model that would be much faster
and less memory-consuming. This introduction seems to be a catalyst for the static embeddings renaissance, as we can see
static embeddings to be integrated even into popular [Sentence Transformers](https://www.sbert.net/) library. The
[recent blog post on the Hugging Face blog](https://huggingface.co/blog/static-embeddings) by [Tom
Aarsen](https://www.tomaarsen.com) reveals how to train a static embedding model using Sentence Transformers and still
get up to 85% of transformer-level quality at a fraction of computational cost. The blog post also introduces an
embedding model for English text retrieval, which is called `static-retrieval-mrl-en-v1`.
## Static embeddings in Qdrant
From the vector database perspective, static embeddings are not different from any other embedding models. They are
dense vectors after all, and you can simply store them in a Qdrant collection. Here is how you do it with the
`sentence-transformers/static-retrieval-mrl-en-v1` model:
```python
import uuid
from sentence_transformers import SentenceTransformer
from qdrant_client import QdrantClient, models
# The model produces vectors of size 1024
model = SentenceTransformer(
"sentence-transformers/static-retrieval-mrl-en-v1"
)
# Let's assume we have a collection "my_collection"
# with a single vector called "static"
client = QdrantClient("http://localhost:6333")
# Calling the sentence transformer model to encode
# the text is not different compared to any other model
client.upsert(
"my_collection",
points=[
models.PointStruct(
id=uuid.uuid4().hex,
vector=model.encode("Hello, world!"),
payload={"static": "Hello, world!"},
)
]
)
```
The retrieval is not going to be any faster just because you use static embeddings. However, **you will experience a
huge speedup in creating the vectors from your data**, what is usually a bottleneck. The Hugging Face blog post mentions
that the model might be even up to 400x faster on a CPU than the state-of-the-art embedding model.
We didn't perform any proper benchmarking of the encoding speed, but one of the experiments done on `TREC-COVID` dataset
from [BeIR](https://github.com/beir-cellar/beir) shows that we can **encode and fully index 171K documents in Qdrant in
around 7.5 minutes**. All of it done on a consumer-grade laptop, without GPU acceleration.
## Quantization of the static embeddings
What can actually make the retrieval faster is the use of Matryoshka Embeddings, as the `static-retrieval-mrl-en-v1`
model was trained with that technique in mind. However, that's not the only way to speed up search. Quantization
methods are really popular among our users, and we were curious to check if they might be applied to the static
embeddings with the same success.
We took the `static-retrieval-mrl-en-v1` model and tested it on various subsets of
[BeIR](https://github.com/beir-cellar/beir) with and without Binary Quantization, to see how much if affects the
retrieval quality. The results are really promising, as shown in our NDCG@10 measurements (a metric that evaluates the
ranking quality of search results, with higher scores indicating better performance):
<table>
<thead>
<tr>
<th></th>
<th colspan="2" style="text-align: center">NDCG@10</th>
</tr>
<tr>
<th>Dataset</th>
<th>Original vectors</th>
<th>Binary Quantization, no rescoring</th>
</tr>
</thead>
<tbody>
<tr>
<th>SciFact</th>
<td><u>0.59348</u></td>
<td>0.54195</td>
</tr>
<tr>
<th>TREC-COVID</th>
<td><u>0.4428</u></td>
<td>0.44185</td>
</tr>
<tr>
<th>ArguAna</th>
<td><u>0.44393</u></td>
<td>0.42164</td>
</tr>
<tr>
<th>NFCorpus</th>
<td><u>0.30045</u></td>
<td>0.28027</td>
</tr>
</tbody>
</table>
Binary Quantization definitely speeds up the retrieval, and make it cheaper, but also seems not to affect the quality of
the retrieval much in some cases. **However, that's something you should carefully verify on your own data**. If you are
a Qdrant user, then you can just enable quantization on an existing collection and [measure the impact on the retrieval
quality](/documentation/beginner-tutorials/retrieval-quality/).
All the tests we did were performed using [`beir-qdrant`](https://github.com/kacperlukawski/beir-qdrant), and might be
reproduced by running [the script available on the project
repo](https://github.com/kacperlukawski/beir-qdrant/blob/main/examples/retrieval/search/evaluate_static_embeddings.py).
## Who should use static embeddings?
Static embeddings seem to be a budget-friendly option for those who would like to use semantic search in their
applications, but can't afford hosting standard representation models, or cannot do it, i.e. due to hardware
constraints. Some of the use cases might be:
- **Mobile applications** - although many smartphones have powerful CPUs or even GPUs, the battery life is still a
concern, and the static embeddings might be a good compromise between the quality and the power consumption. Moreover,
the static embeddings can be used in the applications that require offline mode.
- **Web browser extensions** - running a transformer-based model in a web browser is usually not quite an option, but
static embeddings might be a good choice, as they have fewer parameters and are faster to encode.
- **Embedded systems** - the static embeddings might be a good choice for the devices with limited computational power,
such as IoT devices or microcontrollers.
If you are one of the above, then you should definitely give static embeddings a try. **However, if the search quality
is not the top of your priorities, then you might consider using static embeddings even in the high-performance
environments**. The speedup in the encoding process might be a game-changer for you.
### Customization of the static embeddings
Last, but not least. The training pipeline published by [Tom Aarsen](https://www.tomaarsen.com) can help you to train
your own static embeddings models, so **you can adjust it the specifics of your data easily**. This training process
will also be way faster than for a transformer-based model, so you can even retrain it more often. Recomputing the
embeddings is a bottleneck of the semantic search systems, and the static embeddings might be a good solution to this
problem. Whether a custom static embedding model can beat a general pre-trained model remains an open question, but it's
definitely worth trying.