Standardize on 'filterable' instead of 'filtrable'

This commit is contained in:
Abdon Pijpelink
2026-01-15 16:11:53 +01:00
parent ae335d93d8
commit 2d4ee38f37
44 changed files with 42 additions and 42 deletions
+2 -2
View File
@@ -190,10 +190,10 @@ bash -x automation/process-article-img.sh <path-to-image> <alias-for-the-article
For example:
```bash
bash -x automation/process-article-img.sh ~/Pictures/my_preview.jpg filtrable-hnsw
bash -x automation/process-article-img.sh ~/Pictures/my_preview.jpg filterable-hnsw
```
This command will create a directory `preview` in `static/article_data/filtrable-hnsw` and generate preview images in it. If the directory `static/article_data/filtrable-hnsw` doesn't exist, it will be created. If it exists, only files in the children `preview` directory will be affected. In this case, preview images will be overwritten. Your original image will not be affected.
This command will create a directory `preview` in `static/article_data/filterable-hnsw` and generate preview images in it. If the directory `static/article_data/filterable-hnsw` doesn't exist, it will be created. If it exists, only files in the children `preview` directory will be affected. In this case, preview images will be overwritten. Your original image will not be affected.
For **macOS** you'll have to make 2 adjustements to `process-img.sh` script which is run by `process-article-img.sh` script:
@@ -37,7 +37,7 @@ features:
description: Qdrant enhances search speeds and control and context understanding through filtering on any nested entry in our payload. Unique architecture allows Qdrant to avoid expensive pre-filtering and post-filtering stages, making search faster and accurate.
link:
text: Learn More
url: /articles/filtrable-hnsw/
url: /articles/filterable-hnsw/
sitemapExclude: true
---
@@ -39,7 +39,7 @@ features:
description: Qdrant’s real-time, advanced vector search enables AI agents to act instantly on live data, which is crucial for time-sensitive, autonomous decision-making.
link:
text: HNSW
url: /articles/filtrable-hnsw/
url: /articles/filterable-hnsw/
- id: 3
icon:
src: /icons/outline/server-rack-blue.svg
@@ -145,7 +145,7 @@ In many vector search solutions, filtering is approached in two ways: **pre-filt
| ❌ | **Pre-filtering** | Has the linear complexity of computing the vector mask and becomes a bottleneck for large datasets. |
| ❌ | **Post-filtering** | The problem with **post-filtering** is tied to vector search "*everything fits and doesn't at the same time*" nature: imagine a low-cardinality filter that leaves only a few matching elements in the database. If none of them are similar enough to the query to appear in the top-X retrieved results, they'll all be filtered out. |
Qdrant [**took filtering in vector search further**](/articles/vector-search-filtering/), recognizing the limitations of pre-filtering & post-filtering strategies. We developed an adaptation of HNSW — [**filterable HNSW**](/articles/filtrable-hnsw/) — that also enables **in-place filtering** during graph traversal. To make this possible, we condition HNSW index construction on possible filtering conditions reflected by [**payload indexes**](/documentation/concepts/indexing/#payload-index) (inverted indexes built on vectors' [**metadata**](/documentation/concepts/payload/)).
Qdrant [**took filtering in vector search further**](/articles/vector-search-filtering/), recognizing the limitations of pre-filtering & post-filtering strategies. We developed an adaptation of HNSW — [**filterable HNSW**](/articles/filterable-hnsw/) — that also enables **in-place filtering** during graph traversal. To make this possible, we condition HNSW index construction on possible filtering conditions reflected by [**payload indexes**](/documentation/concepts/indexing/#payload-index) (inverted indexes built on vectors' [**metadata**](/documentation/concepts/payload/)).
**Qdrant was designed with a vector index being a central component of the system.** That made it possible to organize optimizers, payload indexes and other components around the vector index, unlocking the possibility of building a filterable HNSW.
@@ -1,17 +1,17 @@
---
title: Filtrable HNSW
title: Filterable HNSW
short_description: How to make ANN search with custom filtering?
description: How to make ANN search with custom filtering? Search in selected subsets without loosing the results.
# external_link: https://blog.vasnetsov.com/posts/categorical-hnsw/
social_preview_image: /articles_data/filtrable-hnsw/social_preview.jpg
preview_dir: /articles_data/filtrable-hnsw/preview
small_preview_image: /articles_data/filtrable-hnsw/global-network.svg
social_preview_image: /articles_data/filterable-hnsw/social_preview.jpg
preview_dir: /articles_data/filterable-hnsw/preview
small_preview_image: /articles_data/filterable-hnsw/global-network.svg
weight: 60
date: 2019-11-24T22:44:08+03:00
author: Andrei Vasnetsov
author_link: https://blog.vasnetsov.com/
category: qdrant-internals
# aliases: [ /articles/filtrable-hnsw/ ]
aliases: [ /articles/filtrable-hnsw/ ]
---
If you need to find some similar objects in vector space, provided e.g. by embeddings or matching NN, you can choose among a variety of libraries: Annoy, FAISS or NMSLib.
@@ -37,7 +37,7 @@ We need to build a navigation graph among all indexed points so that the greedy
This graph is constructed by sequentially adding points that are connected by a fixed number of edges to previously added points.
In the resulting graph, the number of edges at each point does not exceed a given threshold $m$ and always contains the nearest considered points.
![NSW](/articles_data/filtrable-hnsw/NSW.png)
![NSW](/articles_data/filterable-hnsw/NSW.png)
### How can we modify it?
@@ -55,9 +55,9 @@ Therefore, the theoretical conclusions obtained in the [Percolation theory](http
This statement also confirmed by experiments:
{{< figure src=/articles_data/filtrable-hnsw/exp_connectivity_glove_m0.png caption="Dependency of connectivity to the number of edges" >}}
{{< figure src=/articles_data/filterable-hnsw/exp_connectivity_glove_m0.png caption="Dependency of connectivity to the number of edges" >}}
{{< figure src=/articles_data/filtrable-hnsw/exp_connectivity_glove_num_elements.png caption="Dependency of connectivity to the number of point (no dependency)." >}}
{{< figure src=/articles_data/filterable-hnsw/exp_connectivity_glove_num_elements.png caption="Dependency of connectivity to the number of point (no dependency)." >}}
There is a clear threshold when the search begins to fail.
@@ -79,13 +79,13 @@ In this case, the total number of edges will increase by no more than 2 times, r
Second case is a little harder. A connection may be lost between two categories if they lie in different clusters.
![category clusters](/articles_data/filtrable-hnsw/hnsw_graph_category.png)
![category clusters](/articles_data/filterable-hnsw/hnsw_graph_category.png)
The idea here is to build same navigation graph but not between nodes, but between categories.
Distance between two categories might be defined as distance between category entry points (or, for precision, as the average distance between a random sample). Now we can estimate expected graph connectivity by number of excluded categories, not nodes.
It still does not guarantee that two random categories will be connected, but allows us to switch to multiple searches in each category if connectivity threshold passed. In some cases, multiple searches can be even faster if you take advantage of parallel processing.
{{< figure src=/articles_data/filtrable-hnsw/exp_random_groups.png caption="Dependency of connectivity to the random categories included in search" >}}
{{< figure src=/articles_data/filterable-hnsw/exp_random_groups.png caption="Dependency of connectivity to the random categories included in search" >}}
Third case might be resolved in a same way it is resolved in classical databases.
Depending on labeled subsets size ration we can go for one of the following scenarios:
@@ -100,7 +100,7 @@ Next we also connect neighboring buckets to achieve graph connectivity. We still
Geographical case is a lot like a numerical one.
Usual geographical search involves [geohash](https://en.wikipedia.org/wiki/Geohash), which matches any geo-point to a fixes length identifier.
![Geohash example](/articles_data/filtrable-hnsw/geohash.png)
![Geohash example](/articles_data/filterable-hnsw/geohash.png)
We can use this identifiers as categories and additionally make connections between neighboring geohashes.
It will ensure that any selected geographical region will also contain connected HNSW graph.
@@ -102,7 +102,7 @@ Let's take a look at a non-exhaustive list of data structures and potential impr
| Tenant Isolation | Vector Storage | Defragmented Vector Storage | Faster access to on-disk data |
For more info on payload-aware connections in HNSW, read our [previous article](/articles/filtrable-hnsw/).
For more info on payload-aware connections in HNSW, read our [previous article](/articles/filterable-hnsw/).
This time around, we will focus on the latest additions to Qdrant:
- **the immutable hash map with perfect hashing**
@@ -214,7 +214,7 @@ But let's first see how much RAM we need to serve 1 million vectors and then we
### Vectors and HNSW graph stored using MMAP
In the third experiment, we tested how well our system performs when vectors and [HNSW](https://qdrant.tech/articles/filtrable-hnsw/) graph are stored using the memory-mapped files.
In the third experiment, we tested how well our system performs when vectors and [HNSW](https://qdrant.tech/articles/filterable-hnsw/) graph are stored using the memory-mapped files.
Create collection with:
```http
@@ -61,7 +61,7 @@ negative examples.
## HNSW ANN example and strategy
Let’s start with an example to help you understand the [HNSW graph](/articles/filtrable-hnsw/). Assume you want
Let’s start with an example to help you understand the [HNSW graph](/articles/filterable-hnsw/). Assume you want
to travel to a small city on another continent:
1. You start from your hometown and take a bus to the local airport.
@@ -145,7 +145,7 @@ significantly lower. However, if the best negative score is higher than the best
further away from the negatives. That procedure effectively **pulls the traversal procedure away from the negative examples**.
If you want to know more about the internals of HNSW, you can check out the article about the
[Filtrable HNSW](/articles/filtrable-hnsw/) that covers the topic thoroughly.
[Filterable HNSW](/articles/filterable-hnsw/) that covers the topic thoroughly.
## Food Discovery demo
@@ -50,7 +50,7 @@ Our plan for the current [open-source roadmap](https://github.com/qdrant/qdrant/
Qdrant started more than two years ago with the mission of building a vector database powered by a well-thought-out tech stack. Using Rust as the system programming language and technical architecture decision during the development of the engine made Qdrant the leading and one of the most popular vector database solutions.
Our unique custom modification of the [HNSW algorithm](/articles/filtrable-hnsw/) for Approximate Nearest Neighbor Search (ANN) allows querying the result with a state-of-the-art speed and applying filters without compromising on results. Cloud-native support for distributed deployment and replications makes the engine suitable for high-throughput applications with real-time latency requirements. Rust brings stability, efficiency, and the possibility to make optimization on a very low level. In general, we always aim for the best possible results in [performance](/benchmarks/), code quality, and feature set.
Our unique custom modification of the [HNSW algorithm](/articles/filterable-hnsw/) for Approximate Nearest Neighbor Search (ANN) allows querying the result with a state-of-the-art speed and applying filters without compromising on results. Cloud-native support for distributed deployment and replications makes the engine suitable for high-throughput applications with real-time latency requirements. Rust brings stability, efficiency, and the possibility to make optimization on a very low level. In general, we always aim for the best possible results in [performance](/benchmarks/), code quality, and feature set.
Most importantly, we want to say a big thank you to our [open-source community](https://qdrant.to/discord), our adopters, our contributors, and our customers. Your active participation in the development of our products has helped make Qdrant the best vector database on the market. I cannot imagine how we could do what we’re doing without the community or without being open-source and having the TRUST of the engineers. Thanks to all of you!
@@ -68,7 +68,7 @@ Many users configure complex filters but may not be aware of the need to create
> As a result, every query scans thousands of vectors and their bare payloads before discarding the majority that failed the filter condition. This leads to soaring CPU usage and long response times, especially under higher traffic loads.
Filtering after retrieving thousands of vectors can get expensive. If you don't filter with your queries, then Qdrant will evaluate more vectors than you need. This will make the entire system slower and more resource intensive. Because of this, we have developed out own version of HNSW - [**The Filterable Vector Index**](https://qdrant.tech/articles/filtrable-hnsw/).
Filtering after retrieving thousands of vectors can get expensive. If you don't filter with your queries, then Qdrant will evaluate more vectors than you need. This will make the entire system slower and more resource intensive. Because of this, we have developed out own version of HNSW - [**The Filterable Vector Index**](https://qdrant.tech/articles/filterable-hnsw/).
Unlike some other engines, Qdrant lets you make the optimal choice of which fields to index for your use case rather than creating indexes for every field by default.
@@ -82,7 +82,7 @@ To ensure our catalog is accurate, we can use a dissimilarity search to highligh
To do this, we only need to search for the most dissimilar items using the
embedding of the category title itself as a query.
This can be too broad, so, by combining it with filters —a [Qdrant superpower](/articles/filtrable-hnsw/)—, we can narrow down the search to a specific category.
This can be too broad, so, by combining it with filters —a [Qdrant superpower](/articles/filterable-hnsw/)—, we can narrow down the search to a specific category.
{{< figure src=/articles_data/vector-similarity-beyond-search/mislabelling.png caption="Mislabeling Detection" >}}
@@ -184,7 +184,7 @@ In Qdrant, indexing is modular. You can configure indexes for **both vectors and
<img src="/articles_data/what-is-a-vector-database/hnsw-search.png" alt="Searching Data with the HNSW algorithm" width="300">
You need to build the payload index for **each field** you'd like to search. The magic here is in the combination: HNSW finds similar vectors, and the payload index makes sure only the ones that fit your criteria come through. Learn more about Qdrant's [Filtrable HNSW](https://qdrant.tech/articles/filtrable-hnsw/) and why it was built like this.
You need to build the payload index for **each field** you'd like to search. The magic here is in the combination: HNSW finds similar vectors, and the payload index makes sure only the ones that fit your criteria come through. Learn more about Qdrant's [Filterable HNSW](https://qdrant.tech/articles/filterable-hnsw/) and why it was built like this.
> Combining [full-text search](https://qdrant.tech/documentation/concepts/indexing/#full-text-index) with vector-based search gives you even more versatility. You can simultaneously search for conceptually similar documents while ensuring specific keywords are present, all within the same query.
@@ -31,4 +31,4 @@ On top of it, there is also a problem with search accuracy.
It appears if too many vectors are filtered out, so the HNSW graph becomes disconnected.
Qdrant uses a different approach, not requiring pre- or post-filtering while addressing the accuracy problem.
Read more about the Qdrant approach in our [Filtrable HNSW](/articles/filtrable-hnsw/) article.
Read more about the Qdrant approach in our [Filterable HNSW](/articles/filterable-hnsw/) article.
@@ -46,7 +46,7 @@ With the emergence of pure-play, native vector search engines, Nyris conducted e
As part of their selection process, Nyris evaluated several critical factors to ensure they chose the best vector search engine solution:
- **Accuracy and Speed**: These were primary considerations. Nyris needed to understand the performance differences between the [HNSW](https://qdrant.tech/articles/filtrable-hnsw/) graph-based approach and brute-force search. In particular, they examined edge cases that required numerous filters, sometimes necessitating a switch to brute-force search. Even in these scenarios, Qdrant demonstrated impressive speed and reliability, meeting Nyris's stringent performance requirements.
- **Accuracy and Speed**: These were primary considerations. Nyris needed to understand the performance differences between the [HNSW](https://qdrant.tech/articles/filterable-hnsw/) graph-based approach and brute-force search. In particular, they examined edge cases that required numerous filters, sometimes necessitating a switch to brute-force search. Even in these scenarios, Qdrant demonstrated impressive speed and reliability, meeting Nyris's stringent performance requirements.
- **Insert Speed**: Nyris assessed how quickly data could be inserted into the database, including the performance during simultaneous data ingests and query requests. Qdrant excelled in this area, providing the necessary efficiency for their operations.
- **Total Cost of Ownership**: Nyris analyzed the infrastructure costs and licensing fees associated with each solution. Qdrant offered a competitive total cost of ownership, making it an economically viable option.
- **Data Sovereignty**: The ability to deploy Qdrant in their own clusters was a key aspect for Nyris, ensuring they maintained control over their data and complied with relevant data sovereignty requirements.
+1 -1
View File
@@ -281,7 +281,7 @@ This modification, in combinations with [incremental HNSW indexing](/blog/qdrant
### HNSW Graph connectivity estimation
Qdrant builds [addtitional HNSW links](/articles/filtrable-hnsw/) to ensure that filtered searches are performed fast and accurate.
Qdrant builds [addtitional HNSW links](/articles/filterable-hnsw/) to ensure that filtered searches are performed fast and accurate.
It does, however, introduce an overhead for indexing complexity, especially when the number of payload indexes is large.
With v1.15, Qdrant introduces an optimization, which quickly estimates graph connectivity before creating additional links.
+2 -2
View File
@@ -50,9 +50,9 @@ To use Tiered Multitenancy, after [setting up a collection with a shared fallbac
![Section 2](/blog/qdrant-1.16.x/section-2.png)
To enhance the scalability and speed of vector search, Qdrant employs a graph-based index structure known as [HNSW (Hierarchical Navigable Small World)](/documentation/concepts/indexing/#vector-index). While traditional HNSW is primarily designed for unfiltered searches, Qdrant has addressed this limitation by implementing a [filterable HSNW index](/articles/filtrable-hnsw/). This innovative approach extends the HNSW graph with additional edges that correspond to indexed payload values. This enables Qdrant to maintain search quality even with high filtering selectivity, without introducing any runtime overhead during the search process.
To enhance the scalability and speed of vector search, Qdrant employs a graph-based index structure known as [HNSW (Hierarchical Navigable Small World)](/documentation/concepts/indexing/#vector-index). While traditional HNSW is primarily designed for unfiltered searches, Qdrant has addressed this limitation by implementing a [filterable HSNW index](/articles/filterable-hnsw/). This innovative approach extends the HNSW graph with additional edges that correspond to indexed payload values. This enables Qdrant to maintain search quality even with high filtering selectivity, without introducing any runtime overhead during the search process.
Even with filterable HNSW graphs, there are instances where the quality of search results can deteriorate significantly. This can happen when you use a combination of high cardinality filters, leading to the HNSW graph becoming [disconnected](/documentation/concepts/indexing/#filtrable-index). It is impractical to build additional links for every possible combination of filters in advance due to the potentially vast number of combinations. Another case where filterable HNSW may break down is in situations where filtering criteria are not known in advance.
Even with filterable HNSW graphs, there are instances where the quality of search results can deteriorate significantly. This can happen when you use a combination of high cardinality filters, leading to the HNSW graph becoming [disconnected](/documentation/concepts/indexing/#filterable-index). It is impractical to build additional links for every possible combination of filters in advance due to the potentially vast number of combinations. Another case where filterable HNSW may break down is in situations where filtering criteria are not known in advance.
To address these limitations, in version 1.16 we are introducing support for [ACORN](/documentation/concepts/search/#acorn-search-algorithm), based on the ACORN-1 algorithm described in the paper [ACORN: Performant and Predicate-Agnostic Search Over Vector Embeddings and Structured Data](https://arxiv.org/abs/2403.04871). With ACORN enabled, Qdrant not only traverses direct neighbors (the first hop) in the HNSW graph but also examines neighbors of neighbors (the second hop) if the direct neighbors have been filtered out. This enhancement improves search accuracy at the expense of performance, especially when multiple low-selectivity filters are applied.
@@ -27,7 +27,7 @@ The rise of generative AI in the last few years has shone a spotlight on vector
## What sets Qdrant apart?
To meet the needs of the next generation of AI applications, Qdrant has always been built with four keys in mind: efficiency, scalability, performance, and flexibility. Our goal is to give our users unmatched speed and reliability, even when they are building massive-scale AI applications requiring the handling of billions of vectors. We did so by building Qdrant on Rust for performance, memory safety, and scale. Additionally, [our custom HNSW search algorithm](/articles/filtrable-hnsw/) and unique [filtering](/documentation/concepts/filtering/) capabilities consistently lead to [highest RPS](/benchmarks/), minimal latency, and high control with accuracy when running large-scale, high-dimensional operations.
To meet the needs of the next generation of AI applications, Qdrant has always been built with four keys in mind: efficiency, scalability, performance, and flexibility. Our goal is to give our users unmatched speed and reliability, even when they are building massive-scale AI applications requiring the handling of billions of vectors. We did so by building Qdrant on Rust for performance, memory safety, and scale. Additionally, [our custom HNSW search algorithm](/articles/filterable-hnsw/) and unique [filtering](/documentation/concepts/filtering/) capabilities consistently lead to [highest RPS](/benchmarks/), minimal latency, and high control with accuracy when running large-scale, high-dimensional operations.
Beyond performance, we provide our users with the most flexibility in cost savings and deployment options. A combination of cutting-edge efficiency features, like [built-in compression options](/documentation/guides/quantization/), [multitenancy](/documentation/guides/multiple-partitions/) and the ability to [offload data to disk](/documentation/concepts/storage/), dramatically reduce memory consumption. Committed to privacy and security, crucial for modern AI applications, Qdrant now also offers on-premise and hybrid SaaS solutions, meeting diverse enterprise needs in a data-sensitive world. This approach, coupled with our open-source foundation, builds trust and reliability with engineers and developers, making Qdrant a game-changer in the vector database domain.
@@ -135,7 +135,7 @@ collection_info = client.get_collection(collection_name)
print("Collection info:", collection_info)
```
Expected output: Detailed collection information showing `points_count=2`, vector configuration, and [HNSW](https://qdrant.tech/articles/filtrable-hnsw/) settings.
Expected output: Detailed collection information showing `points_count=2`, vector configuration, and [HNSW](https://qdrant.tech/articles/filterable-hnsw/) settings.
## Step 8: Run Your First Similarity Search
@@ -349,4 +349,4 @@ encoder_large = SentenceTransformer("all-mpnet-base-v2") # Larger, potentially
encoder_fast = SentenceTransformer("all-MiniLM-L12-v2") # Different size/speed tradeoff
```
**Ready for Day 2?** Tomorrow you'll learn how Qdrant makes vector search lightning-fast through [HNSW](https://qdrant.tech/articles/filtrable-hnsw/) indexing and how to optimize for production workloads.
**Ready for Day 2?** Tomorrow you'll learn how Qdrant makes vector search lightning-fast through [HNSW](https://qdrant.tech/articles/filterable-hnsw/) indexing and how to optimize for production workloads.
@@ -9,7 +9,7 @@ weight: 30
# Indexing and Performance
Master [HNSW](https://qdrant.tech/articles/filtrable-hnsw/) indexing and practical tuning for fast retrieval.
Master [HNSW](https://qdrant.tech/articles/filterable-hnsw/) indexing and practical tuning for fast retrieval.
---
@@ -8,7 +8,7 @@ weight: 4
# Demo: HNSW Performance Tuning
Learn how to improve vector search speed with [HNSW](https://qdrant.tech/articles/filtrable-hnsw/) tuning and payload indexing on a real 100K dataset.
Learn how to improve vector search speed with [HNSW](https://qdrant.tech/articles/filterable-hnsw/) tuning and payload indexing on a real 100K dataset.
**Follow along in Colab:** <a href="https://colab.research.google.com/github/qdrant/examples/blob/master/course/day_2/hnsw_performance_tuning.ipynb">
<img src="https://colab.research.google.com/assets/colab-badge.svg" style="display:inline; margin:0;" alt="Open In Colab"/>
@@ -8,7 +8,7 @@ weight: 3
# Combining Vector Search and Filtering
We've talked about how Qdrant uses the [HNSW](/documentation/concepts/indexing/#filtrable-index) graph to efficiently search dense vectors. But in real-world applications, you'll often want to constrain your search using filters. This creates unique challenges for graph traversal that Qdrant solves elegantly.
We've talked about how Qdrant uses the [HNSW](/documentation/concepts/indexing/#filterable-index) graph to efficiently search dense vectors. But in real-world applications, you'll often want to constrain your search using filters. This creates unique challenges for graph traversal that Qdrant solves elegantly.
<div class="video">
<iframe
@@ -199,4 +199,4 @@ See more in [the docs](/documentation/concepts/filtering/).
In the next section, we'll define a collection with structured payloads, configure payload indexing, and evaluate how different HNSW parameters impact filtered search performance.
Learn more: [Filterable HNSW Article](https://qdrant.tech/articles/filtrable-hnsw/)
Learn more: [Filterable HNSW Article](https://qdrant.tech/articles/filterable-hnsw/)
@@ -8,7 +8,7 @@ weight: 5
# Project: HNSW Performance Benchmarking
Now that you've seen how [HNSW](https://qdrant.tech/articles/filtrable-hnsw/) parameters and payload indexes affect performance with the DBpedia dataset, it's time to optimize for your own domain and use case.
Now that you've seen how [HNSW](https://qdrant.tech/articles/filterable-hnsw/) parameters and payload indexes affect performance with the DBpedia dataset, it's time to optimize for your own domain and use case.
## Your Mission
@@ -25,7 +25,7 @@ At this point, you've learned how vector search retrieves the nearest vectors to
You might wonder if Qdrant calculates the distance to every single vector in your collection for each query. This method, known as brute force search, technically works but with millions or billions of vectors this is too slow per query.
Fortunately, Qdrant speeds things up with **[HNSW — Hierarchical Navigable Small World](https://qdrant.tech/articles/filtrable-hnsw/)**.
Fortunately, Qdrant speeds things up with **[HNSW — Hierarchical Navigable Small World](https://qdrant.tech/articles/filterable-hnsw/)**.
### The Library Analogy
@@ -88,7 +88,7 @@ Qdrant is an AI-native vector database and a semantic search engine. You can use
## Qdrant's most popular features:
||||
|:-|:-|:-|
|[Filtrable HNSW](/documentation/filtering/) </br> Single-stage payload filtering | [Recommendations & Context Search](/documentation/concepts/explore/#explore-the-data) </br> Exploratory advanced search| [Pure-Vector Hybrid Search](/documentation/hybrid-queries/)</br>Full text and semantic search in one|
|[Filterable HNSW](/documentation/filtering/) </br> Single-stage payload filtering | [Recommendations & Context Search](/documentation/concepts/explore/#explore-the-data) </br> Exploratory advanced search| [Pure-Vector Hybrid Search](/documentation/hybrid-queries/)</br>Full text and semantic search in one|
|[Multitenancy](/documentation/guides/multiple-partitions/) </br> Payload-based partitioning|[Custom Sharding](/documentation/guides/distributed_deployment/#sharding) </br> For data isolation and distribution|[Role Based Access Control](/documentation/guides/security/?q=jwt#granular-access-control-with-jwt)</br>Secure JWT-based access |
|[Quantization](/documentation/guides/quantization/) </br> Compress data for drastic speedups|[Multivector Support](/documentation/concepts/vectors/?q=multivect#multivectors) </br> For ColBERT late interaction |[Built-in IDF](/documentation/concepts/indexing/?q=inverse+docu#idf-modifier) </br> Advanced similarity calculation|
@@ -339,7 +339,7 @@ Where:
- `N` is the total number of documents in the collection.
- `n` is the number of documents containing non-zero values for the given vector element.
## Filtrable Index
## Filterable Index
Separately, a payload index and a vector index cannot solve the problem of search using the filter completely.
@@ -356,7 +356,7 @@ On the other hand, the HNSW graph starts to fall apart when using too strict fil
Qdrant solves this problem by extending the HNSW graph with additional edges based on the stored payload values.
Extra edges allow you to efficiently search for nearby vectors using the HNSW index and apply filters as you search in the graph.
You can find more information on this approach in our [article](/articles/filtrable-hnsw/).
You can find more information on this approach in our [article](/articles/filterable-hnsw/).
However, in some cases, these additional edges might not be enough.
These extra edges are added per each payload index separately, but not per each possible combination of them.
@@ -175,7 +175,7 @@ Accessing array elements by index is currently not supported.
*Available as of v1.16.0*
For filtered vector search, you are recommended to create a [payload index](/documentation/concepts/indexing/#payload-index) for the fields you want to filter by.
During the search, Qdrant will use a combined [filterable index](/documentation/concepts/indexing/#filtrable-index).
During the search, Qdrant will use a combined [filterable index](/documentation/concepts/indexing/#filterable-index).
However, when combining multiple strict payload filters, this mechanism might not provide sufficient accuracy.
In such cases, you can use the ACORN search algorithm.

Before

Width:  |  Height:  |  Size: 55 KiB

After

Width:  |  Height:  |  Size: 55 KiB

Before

Width:  |  Height:  |  Size: 70 KiB

After

Width:  |  Height:  |  Size: 70 KiB

Before

Width:  |  Height:  |  Size: 79 KiB

After

Width:  |  Height:  |  Size: 79 KiB

Before

Width:  |  Height:  |  Size: 454 KiB

After

Width:  |  Height:  |  Size: 454 KiB

Before

Width:  |  Height:  |  Size: 5.8 KiB

After

Width:  |  Height:  |  Size: 5.8 KiB

Before

Width:  |  Height:  |  Size: 55 KiB

After

Width:  |  Height:  |  Size: 55 KiB

Before

Width:  |  Height:  |  Size: 29 KiB

After

Width:  |  Height:  |  Size: 29 KiB

Before

Width:  |  Height:  |  Size: 25 KiB

After

Width:  |  Height:  |  Size: 25 KiB

Before

Width:  |  Height:  |  Size: 40 KiB

After

Width:  |  Height:  |  Size: 40 KiB

Before

Width:  |  Height:  |  Size: 67 KiB

After

Width:  |  Height:  |  Size: 67 KiB

Before

Width:  |  Height:  |  Size: 110 KiB

After

Width:  |  Height:  |  Size: 110 KiB

Before

Width:  |  Height:  |  Size: 23 KiB

After

Width:  |  Height:  |  Size: 23 KiB

Before

Width:  |  Height:  |  Size: 4.2 KiB

After

Width:  |  Height:  |  Size: 4.2 KiB

Before

Width:  |  Height:  |  Size: 153 KiB

After

Width:  |  Height:  |  Size: 153 KiB