Update sparse-vectors.md

This commit is contained in:
Andrey Vasnetsov
2025-03-04 22:06:01 +01:00
committed by GitHub
parent 60bbc9ee6c
commit 95d39a52f3
@@ -65,7 +65,7 @@ For example, in the medical domain, many rare terms are not present in the gener
Where do sparse vectors fail though? They're not great at capturing nuanced relationships between words. For example, they can't capture the relationship between "king" and "queen" as well as dense vectors.
# SPLADE
## SPLADE
Let's check out [SPLADE](https://europe.naverlabs.com/research/computer-science/splade-a-sparse-bi-encoder-bert-based-model-achieves-effective-and-efficient-full-text-document-ranking/?utm_source=qdrant&utm_medium=website&utm_campaign=sparse-vectors&utm_content=article&utm_term=sparse-vectors), an excellent way to make sparse vectors. Let's look at some numbers first. Higher is better:
@@ -219,7 +219,7 @@ For example, assume a 1M document corpus. Say, we use 100 sparse token ids + wei
| OpenAI Embedding | 12.288 |
| Sparse Vector | 1.12 |
## How SPLADE works: leveraging BERT
### How SPLADE works: leveraging BERT
SPLADE leverages a transformer architecture to generate sparse representations of documents and queries, enabling efficient retrieval. Let's dive into the process.
@@ -234,7 +234,7 @@ The token-level distributions that you'd expect in a standard transformer model
The resulting sparse vectors are not only memory-efficient but also tailored for precise matching in the high-dimensional space of a search engine like Qdrant.
## Interpreting SPLADE
### Interpreting SPLADE
A downside of dense vectors is that they are not interpretable, making it difficult to understand why a document is relevant to a query.
@@ -248,7 +248,7 @@ The switch to max pooling in SPLADE improved its performance on the MS MARCO and
### Document and query Eecoder
The SPLADE model variant that uses a document encoder with max pooling but no query encoder reaches the same performance level as the prior SPLADE model. This suggests a limitation in the necessity of a query encoder, potentially affecting the efficiency of the model​​.
## Other sparse vector methods
### Other sparse vector methods
SPLADE is not the only method to create sparse vectors.