fixes & images

This commit is contained in:
Евгения Суходольская
2025-02-17 17:51:21 +01:00
parent baf6a3e532
commit 82f5a21165
6 changed files with 59 additions and 61 deletions
@@ -1,8 +1,8 @@
---
title: "Dedicated Vector Search"
title: "Built for Vector Search"
short_description: "Why add-on vector search looks good — until you actually use it."
description: "Why add-on vector search looks good — until you actually use it."
#social_preview_image: TBD
social_preview_image: /articles_data/dedicated-vector-search/preview/social_preview.jpg
preview_dir: /articles_data/dedicated-vector-search/preview
weight: -170
author: Evgeniya Sukhodolskaya & Andrey Vasnetsov
@@ -15,55 +15,54 @@ keywords:
category: qdrant-internals
---
Every complex enough problem requires a specialized solution.
You can use a Swiss Army to open a bottle or poke a hole in a cardboard box, but you are going to need an axe to chop wood.
The same goes for software.
Any problem with even a bit of complexity requires a specialized solution.
You can use a Swiss Army knife to open a bottle or poke a hole in a cardboard box, but you will need an axe to chop wood -- the same goes for software.
In this article we are going to describe what unique challenges and optimization opportunities vector search poses, and why a dedicated solution is the best way to tackle them.
In this article, we will describe what unique challenges vector search poses and why a dedicated solution is the best way to tackle them.
## Vectors
Let's take a look at the central concept of vector databases - vectors.
Let's look at the central concept of vector databases -- vectors.
Vectors (aka embeddings) are high-dimensional representations of various data points - texts, images, e.t.c..
Many state-of-the-art text models generate embeddings of over 1,500 dimensions. When it comes to state-of-the-art PDF retrieval, the representations can reach [over 100,000 dimensions per page](https://qdrant.tech/documentation/advanced-tutorials/pdf-retrieval-at-scale/).
Vectors (aka embeddings) are high-dimensional representations of various data points -- texts, images, videos, etc.
Many state-of-the-art (SOTA) embedding models generate representations of over 1,500 dimensions. When it comes to state-of-the-art PDF retrieval, the representations can reach [over 100,000 dimensions per page](https://qdrant.tech/documentation/advanced-tutorials/pdf-retrieval-at-scale/).
This brings us to the first challenge of vector search - vectors are heavy.
This brings us to the first challenge of vector search -- **vectors are heavy**.
### Vectors are heavy
To put this in perspective, consider one million records stored in relational database.
A relatively small amount of data for modern databases, which can be handled evem by a free tier of many cloud providers.
To put this in perspective, consider one million records stored in a relational database.
It's a relatively small amount of data for modern databases, which a free tier of many cloud providers could easily handle.
But apply a 1536-dimensional embedding from OpenAI's `text-embedding-ada-002` to each record, and you are looking at around 6GB of storage.
As a result, vector search workloads, especially if not optimized, will quickly dominate main use cases of a database.
Now, generate a 1536-dimensional embedding with OpenAI's `text-embedding-ada-002` model from each record, and you are looking at around **6GB of storage**.
As a result, vector search workloads, especially if not optimized, will quickly dominate the main use cases of a non-vector database.
Having vectors as a part of a main database is a potential issue for one another reason - vectors are always a transformation of other data.
Having vectors as a part of a main database is a potential issue for one another reason -- vectors are always a transformation of other data.
### Vectors are a transformation
Vectors are always obtained from some other source-of-truth data.
If lost, it can be restored with the same embedding model used previously.
Vectors are obtained from some other source-of-truth data.
It can be restored if lost with the same embedding model previously used.
At the same time, even small changes in that model can shift the geometry of the vector space, so if you update or change the embedding model, you need to update and reindex all the data to maintain accurate vector comparisons.
This update process, if coupled with the main database, can lead to a significant complications and even unavailability of the whole system.
If coupled with the main database, this update process can lead to significant complications and even unavailability of the whole system.
<aside role="status">
Decouple vector workloads even if you are planning to use a general-purpose database for vectors.
Decouple vector workloads even if you plan to use a general-purpose database for vectors.
</aside>
But there are a positive properties of vectors as well. One of the most important is that vectors are fixed-size.
However, vectors have positive properties as well. One of the most important is that vectors are fixed-size.
### Vectors are fixed-size
Embedding models are designed to produce vectors of a fixed size. We have to use it to our advantage.
For fast search, vectors need to be instantly accessible. Stored either in RAM or disk, vectors should be stored in a format that allows for quick access and comparison.
For fast search, vectors need to be instantly accessible. Whether in RAM or disk, vectors should be stored in a format that allows quick access and comparison.
This is essential, as vector comparison is a very hot operation in vector search workloads.
In make cases it is performed thousands of times per search query, so even a small overhead can lead to a significant slowdown.
It is often performed thousands of times per search query, so even a small overhead can lead to a significant slowdown.
For a dedicated storage, their fixed size comes as a blessing.
Knowing exactly how much space one data point needs, we don’t have to deal with the usual overhead of locating data, the location of elements in storage is straightforward to calculate.
For dedicated storage, vectors fixed size comes as a blessing.
Knowing how much space one data point needs, we don’t have to deal with the usual overhead of locating data -- the location of elements in storage is straightforward to calculate.
Everything becomes far less intuitive if vectors are stored together with other data types, for example, texts or JSONs. The size of a single data point is not fixed anymore, so accessing it becomes non-trivial, especially if data is added, updated and deleted over time.
@@ -78,15 +77,14 @@ Just as Google or Bing can handle billions of queries at once, vector databases
{{<figure src=/articles_data/dedicated-vector-search/compass.png caption="Databse Compass" width=80% >}}
### Pick Any Two
Distributed systems are perfect for scalability – horizontal scaling in these systems allows you to add more machines as needed. In the world of distributed systems, one well-known principle – the **CAP theorem** — illustrates that you cannot have it all. The theorem states that a distributed system can guarantee only two out of three properties: **Consistency**, **Availability**, and **Partition Tolerance**.
Distributed systems are perfect for scalability -- horizontal scaling in these systems allows you to add more machines as needed. In the world of distributed systems, one well-known principle -- the **CAP theorem** -- illustrates that you cannot have it all. The theorem states that a distributed system can guarantee only two out of three properties: **Consistency**, **Availability**, and **Partition Tolerance**.
Because network partitions are inevitable in any real-world distributed system, all modern distributed databases are designed with partition tolerance in mind, forcing a trade-off between **consistency** (providing the most up-to-date data) and **availability** (remaining responsive).
As network partitions are inevitable in any real-world distributed system, all modern distributed databases are designed with partition tolerance in mind, forcing a trade-off between **consistency** (providing the most up-to-date data) and **availability** (remaining responsive).
<aside role="status">
<strong>CP systems</strong> are still available to clients under normal operation — they prioritize data correctness over availability during failures. <br/>
<strong>CP systems</strong> are still available to clients under normal operation -- they prioritize data correctness over availability during failures. <br/>
<strong>AP systems</strong> deliver quick responses by relaxing immediate consistency guarantees but eventually converge to a correct state.
</aside>
@@ -96,35 +94,35 @@ There are two main design philosophies for databases in this context:
The ACID model ensures that every transaction (a group of operations treated as a single unit, such as transferring money between accounts) is executed fully or not at all (reverted), leaving the database in a valid state.
When a system is distributed, achieving ACID properties requires complex coordination between nodes.
Each node must communicate and agree on the state of a transaction, which can **limit system availability** — if a node is uncertain about the state of another, it may refuse to process a transaction until consistency is assured.
Each node must communicate and agree on the state of a transaction, which can **limit system availability** -- if a node is uncertain about the state of another, it may refuse to process a transaction until consistency is assured.
This coordination also makes **scaling more challenging**.
Financial institutions use ACID-compliant databases when dealing with money transfers, where even a momentary discrepancy in an account balance is unacceptable.
### BASE: Prioritizing Availability
On the other hand, the BASE model favours high availability and partition tolerance. BASE systems distribute data and workload across multiple nodes, enabling them to respond to read and write requests immediately. They operate under the principle of **eventual consistency** — although data may be temporarily out-of-date, the system will converge on a consistent state given time.
On the other hand, the BASE model favours high availability and partition tolerance. BASE systems distribute data and workload across multiple nodes, enabling them to respond to read and write requests immediately. They operate under the principle of **eventual consistency** -- although data may be temporarily out-of-date, the system will converge on a consistent state given time.
Social media platforms, streaming services, and search engines all benefit from the BASE approach. For these applications, having immediate responsiveness is more critical than strict consistency.
### BASEd Vector Search
Considering the specifics of vector search – its nature demanding availability & scalability – it should be served on BASE-oriented architecture. This choice is made due to the need for horizontal scaling, high availability, low latency and high throughput. For example, having BASE-focused architecture allows us to [easily manage resharding](https://qdrant.tech/documentation/cloud/cluster-scaling/#resharding).
A strictly consistent transactional approach also loses its attractiveness when we remember that vectors are heavy transformations of data at our disposal – what’s the point in limiting data protection mechanisms if we can always restore vectorized data through a transformation?
Considering the specifics of vector search -- its nature demanding availability & scalability -- it should be served on BASE-oriented architecture. This choice is made due to the need for horizontal scaling, high availability, low latency and high throughput. For example, having BASE-focused architecture allows us to [easily manage resharding](https://qdrant.tech/documentation/cloud/cluster-scaling/#resharding).
A strictly consistent transactional approach also loses its attractiveness when we remember that vectors are heavy transformations of data at our disposal -- what’s the point in limiting data protection mechanisms if we can always restore vectorized data through a transformation?
## Vector Index
Vector search relies on high-dimensional vector mathematics, making it computationally heavy at scale.
A brute-force similarity search would require comparing a query against every vector in the database.
In a database with 100 million 1536-dimensional vectors, performing 100 million comparisons per one query is unfeasible for production scenarios.
Instead of using brute-force approach, vector databases have specialized approximate nearest neighbor (ANN) indexes that balance search precision with speed.
Instead of a brute-force approach, vector databases have specialized approximate nearest neighbour (ANN) indexes that balance search precision and speed.
These indexes require carefully designed architectures to make their maintenance in production feasible.
{{< figure src=/articles_data/dedicated-vector-search/hnsw.png caption="HNSW Index" width=80% >}}
One of the most popular vector indexes is **HNSW (Hierarchical Navigable Small World)**, which we picked for its capability to provide simultaneously high search speed and accuracy.
High performance came with a cost – implementing it in production is untrivial due to several challenges, so to make it shine all the system’s architecture has to be structured around it, serving the capricious index.
High performance came with a cost -- implementing it in production is untrivial due to several challenges, so to make it shine all the system’s architecture has to be structured around it, serving the capricious index.
### Index Complexity
@@ -133,8 +131,8 @@ With a new data point inserted, the algorithm must compare it to existing nodes
As the number of vectors grows, these comparisons will noticeably slow down the construction process, making updates increasingly time-consuming.
The indexing operation can quickly become the bottleneck in the system, slowing down search requests.
Building HNSW monolith means limiting the scalability of your solution - its size has to be capped, as its construction time scales **non-linearly** with the number of elements.
To keep the construction process feasible and ensure it doesn't affect the search time, we came up with layered architecture that breaks down all data management into small units called **segments**.
Building an HNSW monolith means limiting the scalability of your solution -- its size has to be capped, as its construction time scales **non-linearly** with the number of elements.
To keep the construction process feasible and ensure it doesn't affect the search time, we came up with a layered architecture that breaks down all data management into small units called **segments**.
{{<figure src=/articles_data/dedicated-vector-search/segments.png caption="Storage structure" width=80% >}}
@@ -154,43 +152,43 @@ These optimizations come from an idea that working with immutable structures int
| **Mutable Segments** | These are used for quickly ingesting new data and handling changes (updates) to existing data. |
| **Immutable Segments** | Once a mutable segment reaches a certain size, an optimization process converts it into an immutable segment, constructing an HNSW index – you could read about these optimizers [here](https://qdrant.tech/documentation/concepts/optimizer/#optimizer) in detail. This immutability trick allowed us, for example, to ensure effective [tenant isolation](https://qdrant.tech/documentation/concepts/indexing/#tenant-index). |
Immutable segments are an implementation detail transparent for users – they can delete vectors at any time, while additions and updates are applied to a mutable segment instead. This combination of mutability and immutability allows search and indexing to smoothly run simultaneously, even under heavy loads.
Immutable segments are an implementation detail transparent for users -- they can delete vectors at any time, while additions and updates are applied to a mutable segment instead. This combination of mutability and immutability allows search and indexing to smoothly run simultaneously, even under heavy loads.
This approach minimizes the performance impact of indexing time and allows on-the-fly configuration changes on a collection level (such as enabling or disabling data quantization) without downtimes.
### Filterable Index
Vector search wasn't hisorically designed for filtering — imposing strict constraints on results. It’s inherently fuzzy; every document is, to some extent, both similar and dissimilar to any query — there’s no binary “*fits/doesn’t fit*” segregation.
As a result, vector search algorithms weren’t originally built with filtering in mind.
Vector search wasn't historically designed for filtering -- imposing strict constraints on results. It's inherently fuzzy; every document is, to some extent, both similar and dissimilar to any query -- there's no binary "*fits/doesn't fit*" segregation.
As a result, vector search algorithms weren't originally built with filtering in mind.
At the same time, filtering is unavoidable in many vector search applications, such as e-commerce search/recommendations.
Searching for a Christmas present, you might want to filter out everything over 100 euros while still benefiting from vector search semantic nature.
Searching for a Christmas present, you might want to filter out everything over 100 euros while still benefiting from the vector search's semantic nature.
In many vector search solutions filtering is approached in two ways: **pre-filtering** (computes a binary mask for all vectors before running HNSW search) or **post-filtering** (running HNSW as usual and then filtering the results).
In many vector search solutions, filtering is approached in two ways: **pre-filtering** (computes a binary mask for all vectors fitting the condition before running HNSW search) or **post-filtering** (running HNSW as usual and then filtering the results).
| | | |
|----|------------------|---------|
| ❌ | **Pre-filtering** | Has linear complexity of computing vector mask, becomes a bottleneck for large datasets. |
| ❌ | **Pre-filtering** | Has the linear complexity of computing the vector mask and becomes a bottleneck for large datasets. |
| ❌ | **Post-filtering** | The problem with **post-filtering** is tied to vector search "*everything fits and doesn't at the same time*" nature: imagine a low-cardinality filter that leaves only a few matching elements in the database. If none of them are similar enough to the query to appear in the top-X retrieved results, they’ll all be filtered out. |
Qdrant [took filtering in vector search further](https://qdrant.tech/articles/vector-search-filtering/), recognizing the limitations of pre-filtering & post-filtering strategies. We developed an adaptation of HNSW — [filterable HNSW](https://qdrant.tech/articles/filtrable-hnsw/) — that also enables **in-place filtering** during graph traversal. To make this possible, we condition HNSW index construction on possible filtering conditions reflected by [payload indexes](https://qdrant.tech/documentation/concepts/indexing/#payload-index) (inverted indexes built on vectors’ [metadata](https://qdrant.tech/documentation/concepts/payload/)).
Qdrant [took filtering in vector search further](https://qdrant.tech/articles/vector-search-filtering/), recognizing the limitations of pre-filtering & post-filtering strategies. We developed an adaptation of HNSW -- [filterable HNSW](https://qdrant.tech/articles/filtrable-hnsw/) -- that also enables **in-place filtering** during graph traversal. To make this possible, we condition HNSW index construction on possible filtering conditions reflected by [payload indexes](https://qdrant.tech/documentation/concepts/indexing/#payload-index) (inverted indexes built on vectors’ [metadata](https://qdrant.tech/documentation/concepts/payload/)).
Qdrant was designed with vector index being a central component of the system.
That made it possible to organize optimizers, payload indexes and other components around the vector index, unlocking a possibility to build a filterable index.
**Qdrant was designed with a vector index being a central component of the system.**
That made it possible to organize optimizers, payload indexes and other components around the vector index, unlocking the possibility of building a filterable HNSW.
{{<figure src=/articles_data/dedicated-vector-search/filterable-vector-index.png caption="Filterable Vector Index" width=80% >}}
In general, optimizing vector search requires a custom, finely tuned approach to data & index management that secures high performance even as data grows and changes dynamically.
This specialized architecture is the key reason why dedicated vector databases will always outperform general-purpose databases in production settings.
In general, optimizing vector search requires a custom, finely tuned approach to data and index management that secures high performance even as data grows and changes dynamically.
This specialized architecture is the key reason why **dedicated vector databases will always outperform general-purpose databases in production settings**.
## Vector Search Beyond RAG
{{<figure src=/articles_data/dedicated-vector-search/venn-diagram.png caption="Vector Search is not Text Search Extension" width=80% >}}
Many discussions about the purpose of vector databases focus on RAG (Retrieval-Augmented Generation) — or its more advanced variant, agentic RAG – where vector databases are used as a knowledge source to retrieve context for large language models (LLMs). This is a legitimate use case, however, the hype wave of RAG solutions has overshadowed the broader potential of vector search, which goes [beyond augmenting generative AI](https://qdrant.tech/articles/vector-similarity-beyond-search/).
Many discussions about the purpose of vector databases focus on Retrieval-Augmented Generation (RAG) -- or its more advanced variant, agentic RAG -- where vector databases are used as a knowledge source to retrieve context for large language models (LLMs). This is a legitimate use case, however, the hype wave of RAG solutions has overshadowed the broader potential of vector search, which goes [beyond augmenting generative AI](https://qdrant.tech/articles/vector-similarity-beyond-search/).
### Discovery
The strength of vector search lies in its ability to facilitate [discovery](https://qdrant.tech/articles/discovery-search/). Vector search allows you to refine your choices as you search rather than starting with a fixed query. Say, [you're ordering food not knowing exactly what you want](https://qdrant.tech/articles/food-discovery-demo/) — just that it should contain meat but not a burger, or that it should be meat with cheese but not tacos. Instead of searching for a specific dish, vector search helps you navigate options based on similarity and dissimilarity, guiding you toward something that matches your taste without requiring you to define it upfront.
The strength of vector search lies in its ability to facilitate [discovery](https://qdrant.tech/articles/discovery-search/). Vector search allows you to refine your choices as you search rather than starting with a fixed query. Say, [you're ordering food not knowing exactly what you want](https://qdrant.tech/articles/food-discovery-demo/) -- just that it should contain meat & not a burger, or that it should be meat with cheese & not tacos. Instead of searching for a specific dish, vector search helps you navigate options based on similarity and dissimilarity, guiding you toward something that matches your taste without requiring you to define it upfront.
### Recommendations
@@ -202,15 +200,15 @@ Vector search nature makes it also ideal for [big unstructured data analysis](ht
### Fundamentally Different
**Vector search beyond RAG isn’t just another feature—it’s a fundamental shift in how we interact with data**. Dedicated solutions integrate these capabilities natively, designed from the ground up to handle high-dimensional math and (dis-)similarity-based retrieval. In contrast, databases with vector extensions are built around a different data paradigm, making it impossible to efficiently support advanced vector search capabilities.
**Vector search beyond RAG isn’t just another feature -- it’s a fundamental shift in how we interact with data**. Dedicated solutions integrate these capabilities natively and are designed from the ground up to handle high-dimensional math and (dis-)similarity-based retrieval. In contrast, databases with vector extensions are built around a different data paradigm, making it impossible to efficiently support advanced vector search capabilities.
Even if you wanted to retrofit these capabilities, it’s not just a matter of adding a new feature – it’s a structural problem. Supporting advanced vector search requires **dedicated interfaces** that enable a flexible usage of vector search from multi-stage filtering to dynamic exploration of high-dimensional spaces.
Even if you want to retrofit these capabilities, it’s not just a matter of adding a new feature -- it’s a structural problem. Supporting advanced vector search requires **dedicated interfaces** that enable flexible usage of vector search from multi-stage filtering to dynamic exploration of high-dimensional spaces.
When the underlying architecture wasn’t initially designed for this kind of interaction, integrating interfaces is a **software engineering team nightmare**. You end up breaking existing assumptions, forcing inefficient workarounds, and often introducing backward-compatibility problems. It’s why attempts to patch vector search onto traditional databases won’t match the efficiency of purpose-built systems.
When the underlying architecture wasn’t initially designed for this kind of interaction, integrating interfaces is a **software engineering team nightmare**. You end up breaking existing assumptions, forcing inefficient workarounds, and often introducing backwards-compatibility problems. It’s why attempts to patch vector search onto traditional databases won’t match the efficiency of purpose-built systems.
## Making Vector Search State-of-the-Art
Now, let’s shift focus to another key advantage of dedicated solutions — their ability to keep up with state-of-the-art solutions in the field.
Now, let’s shift focus to another key advantage of dedicated solutions -- their ability to keep up with state-of-the-art solutions in the field.
Vector databases are purpose-built for vector retrieval, and as a result, they offer cutting-edge features that are often critical for AI businesses relying on vector search. Vector database engineers invest significant time and effort into researching and implementing the most optimal ways to perform vector search. Many of these innovations come naturally to vector-native architectures, while general-purpose databases with added vector capabilities may struggle to adapt and replicate these benefits efficiently.
@@ -236,24 +234,24 @@ In addition to that, we continuously look for improvements in:
| **Vector Data Analysis & Visualization** | Tools like the [distance matrix API](https://qdrant.tech/blog/qdrant-1.12.x/#distance-matrix-api-for-data-insights) provide insights into vectorized data, and a [Web UI](https://qdrant.tech/blog/qdrant-1.11.x/#web-ui-search-quality-tool) allows for intuitive exploration of data. |
| **Search Speed & Scalability** | Includes optimizations for [multi-tenant environments](https://qdrant.tech/articles/multitenancy/) to ensure efficient and scalable search. |
**These advancements are not just incremental improvements — they define the difference between a system optimized for vector search and one that accommodates it.**
**These advancements are not just incremental improvements -- they define the difference between a system optimized for vector search and one that accommodates it.**
Staying at the cutting edge of vector search is not just about performance — it’s also about keeping pace with an evolving AI landscape.
Staying at the cutting edge of vector search is not just about performance -- it’s also about keeping pace with an evolving AI landscape.
## Summing up
When it comes to vector search, there’s a clear distinction between using a dedicated vector search solution and extending a database to support vector operations.
**For small-scale applications or prototypes handling up to a million data points, a non-optimized architecture might suffice.** However, as the volume of vectors grows, an unoptimized solution will quickly become a bottleneck — slowing down search operations and limiting scalability. Dedicated vector search solutions are engineered from the ground up to handle massive amounts of high-dimensional data efficiently.
**For small-scale applications or prototypes handling up to a million data points, a non-optimized architecture might suffice.** However, as the volume of vectors grows, an unoptimized solution will quickly become a bottleneck -- slowing down search operations and limiting scalability. Dedicated vector search solutions are engineered from the ground up to handle massive amounts of high-dimensional data efficiently.
State-of-the-art (SOTA) vector search evolves rapidly. If you plan to build on the latest advances, using a vector extension will eventually hold you back. Dedicated vector search solutions integrate these features natively, ensuring that you benefit from continuous innovations without compromising performance.
The power of vector search extends into areas such as big data analysis, recommendation systems, and discovery-based applications, and to support these vector search capabilities a dedicated solution is needed.
The power of vector search extends into areas such as big data analysis, recommendation systems, and discovery-based applications, and to support these vector search capabilities, a dedicated solution is needed.
### When to Choose a Dedicated Database over Extension:
- **High-Volume, Real-Time Search**: Ideal for applications with many simultaneous users who require fast, continuous access to search results—think search engines, e-commerce recommendations, social media, or media streaming services.
- **Dynamic, Unstructured Data**: Perfect for scenarios where data is continuously evolving, and where the goal is to discover insights from data patterns.
- **Innovative Applications**: If you’re looking to implement advanced use cases such as recommendation engines, hybrid search solutions, or exploratory data analysis where traditional exact or token based searches hold short.
- **Dynamic, Unstructured Data**: Perfect for scenarios where data is continuously evolving and where the goal is to discover insights from data patterns.
- **Innovative Applications**: If you’re looking to implement advanced use cases such as recommendation engines, hybrid search solutions, or exploratory data analysis where traditional exact or token-based searches hold short.
In summary, if your application relies on vector search at scale, keeping up with trends, or requires more than just simple small-scale similarity search, investing in a dedicated vector search engine will deliver the performance and flexibility necessary for success.
Investing in a dedicated vector search engine will deliver the performance and flexibility necessary for success if your application relies on vector search at scale, keeps up with trends, or requires more than just a simple small-scale similarity search.