mirror of
https://github.com/qdrant/landing_page.git
synced 2026-09-25 22:18:30 +02:00
Apply Neil's review nits on the fusion articles
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Dylan Couzon
co-authored by
Claude Opus 5
Claude Opus 5
parent
32e7308426
commit
21d8378bd8
@@ -26,7 +26,9 @@ Some settings are there to verify correctness, not to tune performance. If a vec
|
||||
|
||||
Every query first retrieves candidates, then ranks them. In dense-only search, one vector search does both. Hybrid search adds a sparse prefetch for exact terms, then fusion combines the dense and sparse candidate lists. A reranker, if present, scores the top candidates again.
|
||||
|
||||
{{< figure src="/articles_data/before-tuning-a-qdrant-collection/retrieval-pipeline.svg" alt="Pipeline diagram: a dense prefetch with limit and hnsw_ef settings and a sparse prefetch with limit and Modifier.IDF settings both feed a fusion stage with RRF k, weights, and DBSF settings, followed by an optional reranker with candidate count and model settings." caption="The hybrid pipeline and the settings each stage owns. Dense-only search uses the dense prefetch path on its own, so `limit` and `hnsw_ef` are its only settings here." width="100%" >}}
|
||||
|
||||
|
||||
_The hybrid pipeline and the settings each stage owns. Dense-only search uses the dense prefetch path on its own, so `limit` and `hnsw_ef` are its only settings here._
|
||||
|
||||
If you run dense-only search and exact keywords are missing from results, hybrid search is the first change to test. [Tuning hybrid search](/articles/how-to-tune-hybrid-search/) covers the request shape, what the second prefetch costs, and how to check that fusion beats either prefetch on your labels.
|
||||
|
||||
|
||||
@@ -50,7 +50,9 @@ Each value is the change in `nDCG@10` from `limit=10` to 500.
|
||||
| CodeSearchNet | +0.149 | +0.010 |
|
||||
| DBPedia-entity | +0.282 | +0.003 |
|
||||
|
||||
{{< figure src="/articles_data/candidate-depth/depth-ceiling-vs-current.png" alt="Five line charts, one per dataset, showing nDCG at 10 as prefetch limit rises from 10 to 500. In each chart the best possible score climbs at every step while the current score stays almost flat, so the shaded gap between the two lines widens." caption="The full sweep behind the table. The best possible score climbs at every depth step on every dataset, while the score fusion returns stays almost flat." width="100%" >}}
|
||||
|
||||
|
||||
_The full sweep behind the table. The best possible score climbs at every depth step on every dataset, while the score fusion returns stays almost flat._
|
||||
|
||||
The best possible score change rises with corpus size across these five, from 5,183 documents on SciFact to 100,000 on DBPedia-entity, while the current score change stays flat. Size and domain move together here, so re-measure the gap as your own collection grows.
|
||||
|
||||
@@ -68,7 +70,9 @@ Depth is per shard. Each shard receives its own <code>limit</code> and searches
|
||||
|
||||
For dense vectors, `limit` decides how many candidates the dense stage returns, and `hnsw_ef` decides how wide the HNSW graph traversal searches for them, trading approximate-search recall for latency. Measure `limit` against your labels when a downstream stage can use more candidates, and measure `hnsw_ef` against exact search to see whether the traversal still misses neighbors.
|
||||
|
||||
{{< figure src="/articles_data/candidate-depth/hnsw-ef-saturation.png" alt="A wide HNSW graph with a query, an entry point, and an orange dashed path walking in. A small tinted outline marks the nodes visited at hnsw_ef 16 and a large outline marks the nodes visited at hnsw_ef 512. Four results are ringed. The node just right of the query sits outside the small outline because its only edges run to the far side of the graph, and a hollow gray ring marks the result it displaces once the wider search reaches it." caption="`hnsw_ef` widens the set of nodes the search visits, not the number of results. Here the wider walk reaches a neighbor the narrow one missed, and it displaces the weakest result." width="100%" >}}
|
||||
|
||||
|
||||
_`hnsw_ef` widens the set of nodes the search visits, not the number of results. Here the wider walk reaches a neighbor the narrow one missed, and it displaces the weakest result._
|
||||
|
||||
Run the same check on your own data, with `limit` set to the value your dense-only stage or dense `prefetch` uses.
|
||||
|
||||
|
||||
@@ -50,7 +50,9 @@ The second prefetch also needs a second index and a second vector per point. Kee
|
||||
|
||||
[Reciprocal Rank Fusion](/documentation/search/hybrid-queries/#reciprocal-rank-fusion-rrf) (RRF) uses only a candidate's position in each prefetch. A document at rank 1 scores the same whether it beat rank 2 by a wide margin or a narrow one. [Distribution-based score fusion](/documentation/search/hybrid-queries/#distribution-based-score-fusion-dbsf) (DBSF) puts both lists on one scale for each query, using each list's average score and how spread out its scores are. Adding the two rescaled scores carries the size of a lead into the fused ranking, and a document only one prefetch retrieved keeps that single rescaled score.
|
||||
|
||||
{{< figure src="/articles_data/how-to-tune-hybrid-search/fusion-signals.png" alt="Two panels of dot plots, RRF on the left and DBSF on the right. Each panel has a dense line, a sparse line, and a fused line holding documents A, B, C, and D. The RRF lines space every document evenly and label the slots 4, 3, 2, 1. The DBSF lines keep the raw score spacing on one shared axis, dense running 0.55 to 0.91 with A far out to the right and B, C, and D clustered, sparse running 12.9 to 14.8. The fused lines put B first under RRF and A first under DBSF." caption="RRF reads each document's slot, so A's dense lead flattens to one step and B, ranked near the top by both prefetches, wins. DBSF keeps the spacing on a shared axis, so A's lead survives the sum and A wins." width="100%" >}}
|
||||
|
||||
|
||||
_RRF reads each document's slot, so A's dense lead flattens to one step and B, ranked near the top by both prefetches, wins. DBSF keeps the spacing on a shared axis, so A's lead survives the sum and A wins._
|
||||
|
||||
RRF ignores score scale, so a cosine similarity and a BM25 score combine without either dominating. DBSF assumes the size of a score gap means something, so one outlying score can move the result. Which one wins depends on your data, so run both against your labels.
|
||||
|
||||
@@ -58,19 +60,16 @@ RRF ignores score scale, so a cosine similarity and a BM25 score combine without
|
||||
|
||||
Use your [labeled query set](/articles/before-tuning-a-qdrant-collection/#make-sure-your-labels-can-detect-a-gain) to compare RRF and DBSF over the same prefetches. Run RRF at `k=2` and equal weights, then run DBSF.
|
||||
|
||||
Both queries read the same two candidate lists, so connect once and build the prefetches once. The prefetches must use the models the collection was indexed with.
|
||||
|
||||
```python
|
||||
from qdrant_client import QdrantClient, models
|
||||
from your_embedding_setup import dense_query, sparse_query
|
||||
|
||||
client = QdrantClient(
|
||||
url="https://YOUR-CLUSTER.cloud.qdrant.io",
|
||||
api_key="<your-api-key>",
|
||||
)
|
||||
```
|
||||
|
||||
Both queries read the same two candidate lists, so build the prefetches once. They must use the models the collection was indexed with.
|
||||
|
||||
```python
|
||||
from your_embedding_setup import dense_query, sparse_query
|
||||
|
||||
dense_prefetch = models.Prefetch(query=dense_query, using="dense", limit=200)
|
||||
sparse_prefetch = models.Prefetch(query=sparse_query, using="bm25", limit=200)
|
||||
@@ -119,7 +118,9 @@ DBSF takes no parameters: `k` and the weight pair are RRF settings, and the publ
|
||||
|
||||
Qdrant scores a document at position `pos` in one prefetch as `1 / ((pos + 1) / weight + k - 1)`, then sums across prefetches. With equal weights that reduces to `1 / (pos + k)`, and `k` alone decides how steeply the head of a list outranks its tail.
|
||||
|
||||
{{< figure src="/articles_data/how-to-tune-hybrid-search/rrf-k-rank-weight.png" alt="Grouped bar chart comparing the share of a retrieval prefetch's top-10 score mass at each rank, for k equal to 2 and k equal to 61. At k=2 rank 1 takes 24.8 percent and rank 10 takes 4.5 percent. At k=61 the shares are nearly flat, 10.7 percent at rank 1 and 9.3 percent at rank 10." caption="At Qdrant's default of k=2, rank 1 carries 5.50 times the score weight of rank 10. At k=61, it carries 1.15 times the weight, so a candidate's presence in a prefetch matters almost as much as its position." width="100%" >}}
|
||||
|
||||
|
||||
_At Qdrant's default of k=2, rank 1 carries 5.50 times the score weight of rank 10. At k=61, it carries 1.15 times the weight, so a candidate's presence in a prefetch matters almost as much as its position._
|
||||
|
||||
Rank 1 outweighs rank 10 by 2.80 times at `k=5` and 1.45 times at `k=20`, so most of the movement sits below `k=20`. A sweep in even steps of five would spend most of its runs past the point where the curve stops moving.
|
||||
|
||||
@@ -135,7 +136,7 @@ The table gives `nDCG@10` at equal weights across five values of `k`, with `k=2`
|
||||
| DBPedia-entity | 400 | 38.2 | 0.4625 | 0.4638 | 0.4641 | 0.4682* | 0.4606 |
|
||||
| WANDS | 480 | 358.9 | 0.7232 | 0.7254 | 0.7336 | 0.7571 | 0.7614* |
|
||||
|
||||
On WANDS, `k=2` and `k=61` chose a different top result for 202 of 480 queries, while `nDCG@10` rose by 0.0360. A small aggregate gain can still change what a user sees first.
|
||||
On WANDS, `k=2` and `k=61` chose a different top result for 42% of queries, while `nDCG@10` rose by 0.0360. A small aggregate gain can still change what a user sees first.
|
||||
|
||||
These five datasets suggest a direction: with about one relevant document per query, the best `k` was 2 or 5; with tens or hundreds, it was 20 or 61. Count relevant documents per query in your labeled query set, then try that part of the range first.
|
||||
|
||||
|
||||
@@ -129,7 +129,9 @@ If you find a mismatch, swap in a model whose window and training data fit your
|
||||
|
||||
Start with 10 candidates, and confirm on your labeled queries that the reranker beats tuned fusion before you change the count. Every configuration that trailed tuned fusion at 10 candidates still trailed it at 200, so a deeper list does not rescue a reranker that loses at 10. `nDCG@10` grades the same top 10 results at every count, so the count changes only what the reranker gets to choose from.
|
||||
|
||||
{{< figure src="/articles_data/when-a-reranker-is-worth-it/reranker-gain-by-candidate-count.png" alt="Five small line charts, one per dataset, showing the best nDCG@10 change over tuned fusion at candidate counts 10, 25, 50, 100, and 200. SciFact, CodeSearchNet, and DBPedia-entity stay above the zero line, WANDS stays below it at every count, and ArguAna peaks at 25 then falls to zero by 200." caption="The best nDCG@10 change over tuned fusion among the four models, by candidate count. A line above zero is a reranker win; WANDS never crosses it." width="100%" >}}
|
||||
|
||||
|
||||
_The best nDCG@10 change over tuned fusion among the four models, by candidate count. A line above zero is a reranker win; WANDS never crosses it._
|
||||
|
||||
Step 1 confirmed that your relevant documents reach the candidate list. Run that same check at each count you are considering, before you run the reranker at any of them. The share of queries whose relevant documents are already in the candidate list limits how much increasing the count can help. Beyond that point, extra candidates only add documents that can push the relevant ones out of the top 10.
|
||||
|
||||
|
||||
@@ -124,7 +124,9 @@ What rescoring recovers depends on how much precision the bit depth discarded. A
|
||||
|
||||
At a deep bit depth, rescoring is what makes the quantization usable. One pass raised `bits1` from 0.605 to 0.951 `Recall@10`. Qdrant [enables `rescore` by default](/documentation/manage-data/quantization/#searching-with-quantization) for `bits1`, `bits1_5`, `bits2`, and binary quantization for this reason.
|
||||
|
||||
{{< figure src="/articles_data/when-your-collection-outgrows-ram/bits1-rescore-recovery.png" alt="Line chart of the share of the exact top 10 that bits1 returns, across rescore off and rescore on at oversampling 1, 2, and 4. The share jumps from 0.605 with rescore off to 0.951 at oversampling 1, crossing the dashed float32 reference at 0.957, then flattens at 0.977 and 0.988." caption="One rescoring pass does most of the recovery at bits1. Raising oversampling past 1 buys little, which is why the disk reads it adds are the cost to watch." width="100%" >}}
|
||||
|
||||
|
||||
_One rescoring pass does most of the recovery at bits1. Raising oversampling past 1 buys little, which is why the disk reads it adds are the cost to watch._
|
||||
|
||||
After `oversampling` 1, extra candidates add disk reads for little recall. `bits1` reached 0.977 `Recall@10` at `oversampling` 2 and 0.988 at `oversampling` 4.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user