From 95d39a52f3edf3ca02f5b68a3df3550497b000f2 Mon Sep 17 00:00:00 2001 From: Andrey Vasnetsov Date: Tue, 4 Mar 2025 22:06:01 +0100 Subject: [PATCH] Update sparse-vectors.md --- qdrant-landing/content/articles/sparse-vectors.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/qdrant-landing/content/articles/sparse-vectors.md b/qdrant-landing/content/articles/sparse-vectors.md index 16c6c8b49..a2b467d6b 100644 --- a/qdrant-landing/content/articles/sparse-vectors.md +++ b/qdrant-landing/content/articles/sparse-vectors.md @@ -65,7 +65,7 @@ For example, in the medical domain, many rare terms are not present in the gener Where do sparse vectors fail though? They're not great at capturing nuanced relationships between words. For example, they can't capture the relationship between "king" and "queen" as well as dense vectors. -# SPLADE +## SPLADE Let's check out [SPLADE](https://europe.naverlabs.com/research/computer-science/splade-a-sparse-bi-encoder-bert-based-model-achieves-effective-and-efficient-full-text-document-ranking/?utm_source=qdrant&utm_medium=website&utm_campaign=sparse-vectors&utm_content=article&utm_term=sparse-vectors), an excellent way to make sparse vectors. Let's look at some numbers first. Higher is better: @@ -219,7 +219,7 @@ For example, assume a 1M document corpus. Say, we use 100 sparse token ids + wei | OpenAI Embedding | 12.288 | | Sparse Vector | 1.12 | -## How SPLADE works: leveraging BERT +### How SPLADE works: leveraging BERT SPLADE leverages a transformer architecture to generate sparse representations of documents and queries, enabling efficient retrieval. Let's dive into the process. @@ -234,7 +234,7 @@ The token-level distributions that you'd expect in a standard transformer model The resulting sparse vectors are not only memory-efficient but also tailored for precise matching in the high-dimensional space of a search engine like Qdrant. -## Interpreting SPLADE +### Interpreting SPLADE A downside of dense vectors is that they are not interpretable, making it difficult to understand why a document is relevant to a query. @@ -248,7 +248,7 @@ The switch to max pooling in SPLADE improved its performance on the MS MARCO and ### Document and query Eecoder The SPLADE model variant that uses a document encoder with max pooling but no query encoder reaches the same performance level as the prior SPLADE model. This suggests a limitation in the necessity of a query encoder, potentially affecting the efficiency of the model​​. -## Other sparse vector methods +### Other sparse vector methods SPLADE is not the only method to create sparse vectors.