mirror of
https://github.com/qdrant/landing_page.git
synced 2026-09-30 00:18:32 +02:00
Rename ANN precision tutorial to ANN recall
Recall@k is the standard metric for ANN benchmarks. Rename the tutorial title, headings, helper function, and prose; update the recent inline edits and merge sentence; and update cross-links from the relevance and pipeline-output tutorials and the tutorials index. Generic Precision@k mentions in the ranx metric list are unrelated and left alone. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Opus 4.7
parent
b7b2865e14
commit
9f4b4c05ea
+1
-1
@@ -5,7 +5,7 @@
|
||||
| [Relevance Feedback](/documentation/tutorials-search-engineering/using-relevance-feedback/) | Relevance Feedback Retrieval in Qdrant | <span class="pill">Python</span> | 30m | <span class="text-yellow">Intermediate</span> |
|
||||
| [Collaborative Filtering](/documentation/tutorials-search-engineering/collaborative-filtering/) | Collaborative filtering using sparse embeddings. | <span class="pill">Python</span> | 45m | <span class="text-yellow">Intermediate</span> |
|
||||
| [Multivector Document Retrieval](/documentation/tutorials-search-engineering/pdf-retrieval-at-scale/) | PDF RAG using ColPali and embedding pooling. | <span class="pill">Python</span> | 30m | <span class="text-yellow">Intermediate</span> |
|
||||
| [Measuring ANN Precision](/documentation/tutorials-search-engineering/retrieval-quality/) | Measure ANN precision with the Web UI and tune HNSW parameters. | <span class="pill">Web UI</span> | 15m | <span class="text-green">Beginner</span> |
|
||||
| [Measuring ANN Recall](/documentation/tutorials-search-engineering/retrieval-quality/) | Measure ANN recall with the Web UI and tune HNSW parameters. | <span class="pill">Web UI</span> | 15m | <span class="text-green">Beginner</span> |
|
||||
| [Measuring Retrieval Relevance](/documentation/tutorials-search-engineering/retrieval-quality-golden-set/) | Build a labeled golden set and score retrieval relevance with ranx. | <span class="pill">Python</span> | 40m | <span class="text-yellow">Intermediate</span> |
|
||||
| [Evaluating Pipeline Output Quality](/documentation/tutorials-search-engineering/retrieval-quality-pipeline-output/) | Score a RAG pipeline with Ragas and isolate retrieval vs generation failures. | <span class="pill">Python</span> | 45m | <span class="text-yellow">Intermediate</span> |
|
||||
| [Hybrid Search with Reranking](/documentation/tutorials-search-engineering/reranking-hybrid-search/) | Implement late interaction and sparse reranking. | <span class="pill">Python</span> | 40m | <span class="text-yellow">Intermediate</span> |
|
||||
|
||||
@@ -68,7 +68,7 @@ partition: develop
|
||||
| [Semantic Search Basics](/documentation/tutorials-search-engineering/neural-search/) | Deploy a search service for company descriptions. | <span class="pill">FastAPI</span> | 30m | <span class="text-green">Beginner</span> |
|
||||
| [Collaborative Filtering](/documentation/tutorials-search-engineering/collaborative-filtering/) | Collaborative filtering using sparse embeddings. | <span class="pill">Python</span> | 45m | <span class="text-yellow">Intermediate</span> |
|
||||
| [Multivector Document Retrieval](/documentation/tutorials-search-engineering/pdf-retrieval-at-scale/) | PDF RAG using ColPali and embedding pooling. | <span class="pill">Python</span> | 30m | <span class="text-yellow">Intermediate</span> |
|
||||
| [Measuring ANN Precision](/documentation/tutorials-search-engineering/retrieval-quality/) | Measure ANN precision with the Web UI and tune HNSW parameters. | <span class="pill">Web UI</span> | 15m | <span class="text-green">Beginner</span> |
|
||||
| [Measuring ANN Recall](/documentation/tutorials-search-engineering/retrieval-quality/) | Measure ANN recall with the Web UI and tune HNSW parameters. | <span class="pill">Web UI</span> | 15m | <span class="text-green">Beginner</span> |
|
||||
| [Measuring Retrieval Relevance](/documentation/tutorials-search-engineering/retrieval-quality-golden-set/) | Build a labeled golden set and score retrieval relevance with ranx. | <span class="pill">Python</span> | 40m | <span class="text-yellow">Intermediate</span> |
|
||||
| [Evaluating Pipeline Output Quality](/documentation/tutorials-search-engineering/retrieval-quality-pipeline-output/) | Score a RAG pipeline with Ragas and isolate retrieval vs generation failures. | <span class="pill">Python</span> | 45m | <span class="text-yellow">Intermediate</span> |
|
||||
| [Reranking for Better Search](/documentation/search-precision/reranking-semantic-search/) | Use multivector representations for better ranking. | <span class="pill">Python</span> | 30m | <span class="text-yellow">Intermediate</span> |
|
||||
|
||||
+1
-1
@@ -13,7 +13,7 @@ aliases:
|
||||
This tutorial focuses on **retrieval relevance**: how well retrieved results match real user intent.
|
||||
To measure retrieval relevance, you need a labeled dataset of queries paired with their expected relevant documents (commonly called a *golden query set* or *ground truth*). This tutorial covers both building that dataset and running it through Qdrant to compute relevance metrics.
|
||||
|
||||
This tutorial is part of a four-layer retrieval evaluation framework; see [Measuring ANN Precision](/documentation/tutorials-search-engineering/retrieval-quality/#the-four-layers-of-retrieval-evaluation) for the full overview.
|
||||
This tutorial is part of a four-layer retrieval evaluation framework; see [Measuring ANN Recall](/documentation/tutorials-search-engineering/retrieval-quality/#the-four-layers-of-retrieval-evaluation) for the full overview.
|
||||
|
||||
**Prerequisites.** A Qdrant collection populated with your documents as points (vectors + optional payload), an embedding model available to encode queries at evaluation time, and Python with `ranx` installed.
|
||||
|
||||
|
||||
+2
-2
@@ -13,7 +13,7 @@ aliases:
|
||||
This tutorial focuses on **pipeline output quality**: whether the full retrieval pipeline produces the right output once retrieved results reach a consumer, most often an LLM generator in a RAG system.
|
||||
To measure pipeline output quality, you run your golden set through the full pipeline, capture each `(question, retrieved_context, answer)` triple, and score the triples against judgment metrics like faithfulness, answer relevancy, and context precision.
|
||||
|
||||
This tutorial is part of a four-layer retrieval evaluation framework; see [Measuring ANN Precision](/documentation/tutorials-search-engineering/retrieval-quality/#the-four-layers-of-retrieval-evaluation) for the full overview.
|
||||
This tutorial is part of a four-layer retrieval evaluation framework; see [Measuring ANN Recall](/documentation/tutorials-search-engineering/retrieval-quality/#the-four-layers-of-retrieval-evaluation) for the full overview.
|
||||
|
||||
**Prerequisites.** A Qdrant collection populated with your documents as points (vectors + a `text` payload field for the chunk content), a labeled golden set (see [Measuring Retrieval Relevance](/documentation/tutorials-search-engineering/retrieval-quality-golden-set/)), LLM access for generation and judging, and Python with `ragas` installed.
|
||||
|
||||
@@ -188,4 +188,4 @@ Ragas's metrics assume the consumer is an LLM generator. If retrieval feeds some
|
||||
|
||||
## Wrapping Up
|
||||
|
||||
That completes the three retrieval-evaluation layers covered in this series: ANN precision, retrieval relevance, and pipeline output quality. Each catches a different class of regression.
|
||||
That completes the three retrieval-evaluation layers covered in this series: ANN recall, retrieval relevance, and pipeline output quality. Each catches a different class of regression.
|
||||
|
||||
+18
-18
@@ -1,17 +1,17 @@
|
||||
---
|
||||
title: Measuring ANN Precision
|
||||
title: Measuring ANN Recall
|
||||
aliases:
|
||||
- /documentation/tutorials/retrieval-quality/
|
||||
- /documentation/beginner-tutorials/retrieval-quality/
|
||||
weight: 5
|
||||
---
|
||||
|
||||
# Measuring ANN Precision
|
||||
# Measuring ANN Recall
|
||||
|
||||
| Time: 15 min | Level: Beginner | | |
|
||||
|--------------|---------------------|--|----|
|
||||
|
||||
This tutorial focuses on **ANN precision**: how closely approximate nearest-neighbor (ANN) search matches exact kNN search, measured with `precision@k`.
|
||||
This tutorial focuses on **ANN recall**: how closely approximate nearest-neighbor (ANN) search matches exact kNN search, measured with `recall@k`.
|
||||
|
||||
**Prerequisites.** A Qdrant collection populated with your documents as points (vectors + optional payload).
|
||||
|
||||
@@ -19,50 +19,50 @@ This tutorial focuses on **ANN precision**: how closely approximate nearest-neig
|
||||
|
||||
This tutorial is part of a four-layer retrieval evaluation framework.
|
||||
|
||||
- **Layer 1: ANN precision** (this tutorial). How closely approximate nearest-neighbor search matches exact kNN.
|
||||
- **Layer 1: ANN recall** (this tutorial). How closely approximate nearest-neighbor search matches exact kNN.
|
||||
- **Layer 2: Retrieval relevance** ([Measuring Retrieval Relevance](/documentation/tutorials-search-engineering/retrieval-quality-golden-set/)). How well the results match query intent against a labeled dataset.
|
||||
- **Layer 3: Pipeline output quality** ([Evaluating Pipeline Output Quality](/documentation/tutorials-search-engineering/retrieval-quality-pipeline-output/)). Whether the full pipeline (retrieval plus an LLM generator, a ranker, or a UI) produces the right output.
|
||||
- **Layer 4: Business impact**. Whether better retrieval moves the KPIs the business cares about. This layer is application-specific and out of scope for these tutorials.
|
||||
|
||||
Retrieval quality sits on top of embedding quality, measured separately by benchmarks like [MTEB](https://huggingface.co/spaces/mteb/leaderboard), which sets the ceiling on every downstream metric.
|
||||
|
||||
## Measure ANN Precision with the Web UI
|
||||
## Measure ANN Recall with the Web UI
|
||||
|
||||
Qdrant's Web UI includes a Search Quality tab that measures the gap between approximate and exact search without writing evaluation code. Open the dashboard at `http://localhost:6333/dashboard` (or your cluster's dashboard on Qdrant Cloud), navigate to your collection, open the Search Quality tab, and click **Check Index Quality** to run the comparison.
|
||||
|
||||

|
||||
|
||||
The tab reports average **precision@k** (1.0 = perfect overlap; 0.95+ is typical for well-tuned HNSW).
|
||||
The tab reports average **recall@k** (1.0 = perfect overlap; 0.95+ is typical for well-tuned HNSW).
|
||||
|
||||
## Tuning Search Precision
|
||||
## Tuning Search Recall
|
||||
|
||||
Toggle **advanced mode** in the Search Quality tab to tune search-time parameters inline. The main one is `hnsw_ef`: the number of candidates evaluated during a search. Raising it explores more of the graph, improving precision at the cost of higher query latency. To see the effect, raise `hnsw_ef` (for example, to 256) and run the evaluation again.
|
||||
Toggle **advanced mode** in the Search Quality tab to tune search-time parameters inline. The main one is `hnsw_ef`: the number of candidates evaluated during a search. Raising it explores more of the graph, improving recall at the cost of higher query latency. To see the effect, raise `hnsw_ef` (for example, to 256) and run the evaluation again.
|
||||
|
||||

|
||||
|
||||
Precision should increase at the cost of higher query latency.
|
||||
Recall should increase at the cost of higher query latency.
|
||||
|
||||

|
||||
|
||||
If `hnsw_ef` alone does not get you to your precision target, the build-time parameters `m` and `ef_construct` set the ceiling on how precise approximate search can be. Changing them requires rebuilding the HNSW index. For the trade-offs and how to choose values, see [HNSW Indexing Fundamentals](/course/essentials/day-2/what-is-hnsw/) in the Qdrant Essentials course.
|
||||
If `hnsw_ef` alone does not get you to your recall target, the build-time parameters `m` and `ef_construct` set the ceiling on the recall approximate search can achieve. Changing them requires rebuilding the HNSW index. For the trade-offs and how to choose values, see [HNSW Indexing Fundamentals](/course/essentials/day-2/what-is-hnsw/) in the Qdrant Essentials course.
|
||||
|
||||
## Automate in CI with Python
|
||||
|
||||
The Web UI is the fastest way to check precision interactively. For continuous integration or scripted regression tests, the Qdrant client exposes the same exact-search mode via `search_params=models.SearchParams(exact=True)`. Compare the ANN and exact top-k sets yourself and compute precision@k.
|
||||
The Web UI is the fastest way to check recall interactively. For continuous integration or scripted regression tests, the Qdrant client exposes the same exact-search mode via `search_params=models.SearchParams(exact=True)`. Compare the ANN and exact top-k sets yourself and compute recall@k.
|
||||
|
||||
This helper takes a list of query vectors and returns the average precision@k. Use a representative sample of query vectors from your workload (typically 20–50, embedded with the same model your collection uses) as your test set.
|
||||
This helper takes a list of query vectors and returns the average recall@k. Use a representative sample of query vectors from your workload (typically 20–50, embedded with the same model your collection uses) as your test set.
|
||||
|
||||
```python
|
||||
from qdrant_client import QdrantClient, models
|
||||
|
||||
|
||||
def avg_precision_at_k(
|
||||
def avg_recall_at_k(
|
||||
client: QdrantClient,
|
||||
collection_name: str,
|
||||
test_vectors: list,
|
||||
k: int,
|
||||
) -> float:
|
||||
precisions = []
|
||||
recalls = []
|
||||
for vector in test_vectors:
|
||||
ann_ids = {
|
||||
p.id for p in client.query_points(
|
||||
@@ -79,13 +79,13 @@ def avg_precision_at_k(
|
||||
search_params=models.SearchParams(exact=True),
|
||||
).points
|
||||
}
|
||||
precisions.append(len(ann_ids & knn_ids) / k)
|
||||
recalls.append(len(ann_ids & knn_ids) / k)
|
||||
|
||||
return sum(precisions) / len(precisions)
|
||||
return sum(recalls) / len(recalls)
|
||||
```
|
||||
|
||||
Wire it into CI and fail the job when precision falls below your target threshold. This catches regressions from embedding model swaps or index config changes before they reach production.
|
||||
Wire it into CI and fail the job when recall falls below your target threshold. This catches regressions from embedding model swaps or index config changes before they reach production.
|
||||
|
||||
## Next Steps
|
||||
|
||||
Once ANN precision is on target, continue with [Measuring Retrieval Relevance](/documentation/tutorials-search-engineering/retrieval-quality-golden-set/) to check how well those results match user intent.
|
||||
Once ANN recall is on target, continue with [Measuring Retrieval Relevance](/documentation/tutorials-search-engineering/retrieval-quality-golden-set/) to check how well those results match user intent.
|
||||
Reference in New Issue
Block a user