Measuring ANN Precision

Time: 15 min	Level: Beginner

This tutorial focuses on ANN precision: how closely approximate nearest-neighbor (ANN) search matches exact kNN search. To measure ANN precision, you compare Qdrant's approximate top-k against the exact kNN top-k using precision@k, then tune HNSW parameters to trade memory and build time for higher precision.

Prerequisites. A Qdrant collection populated with your documents as points (vectors + optional payload).

The Four Layers of Retrieval Evaluation

This tutorial is part of a four-layer retrieval evaluation framework.

Layer 1: ANN precision (this tutorial). How closely approximate nearest-neighbor search matches exact kNN. Run on every index or embedding change.
Layer 2: Retrieval relevance (Measuring Retrieval Relevance). How well the results match query intent against a labeled dataset. Run weekly, or on retrieval-stack changes.
Layer 3: Pipeline output quality (Evaluating Pipeline Output Quality). Whether the full pipeline (retrieval plus an LLM generator, a ranker, or a UI) produces the right output. Run weekly, or on retrieval or generator changes.
Layer 4: Business impact. Whether better retrieval moves the KPIs the business cares about. Measured per release once the offline layers pass.

Retrieval quality sits on top of embedding quality, measured separately by benchmarks like MTEB, which sets the ceiling on every downstream metric.

Measure ANN Precision with the Web UI

Qdrant's Web UI includes a Search Quality tab that measures the gap between approximate and exact search without writing evaluation code. Open the dashboard at http://localhost:6333/dashboard (or your cluster's dashboard on Qdrant Cloud), navigate to your collection, open the Search Quality tab, and click Check Index Quality to run the comparison.

The tab reports average precision@k (1.0 = perfect overlap; 0.95+ is typical for well-tuned HNSW). HNSW has tunable parameters that trade memory and index build time for higher precision.

Tuning the HNSW Parameters

HNSW is a hierarchical graph where each node has a set of links to other nodes. The m parameter controls the number of edges per node: higher m means higher precision at the cost of more memory. The ef_construct parameter controls how many neighbors are considered during index building: higher ef_construct means higher precision at the cost of longer indexing time. The defaults are m=16 and ef_construct=100.

For the full list of HNSW parameters, including on-disk storage and precision/memory trade-offs, see Optimize Performance.

Toggle advanced mode in the Search Quality tab to tune these parameters inline. To see the effect, try raising m and ef_construct (for example, to 32 and 200), then run the evaluation again.

Precision should increase at the cost of higher build time and memory.

Tune until the balance between precision and cost matches your targets.

Automate in CI with Python

The Web UI is the fastest way to check precision interactively. For continuous integration or scripted regression tests, the Qdrant client exposes the same exact-search mode via search_params=models.SearchParams(exact=True). Compare the ANN and exact top-k sets yourself and compute precision@k.

This helper takes a list of query vectors and returns the average precision@k. Use a representative sample of query vectors from your workload (typically 20–50, embedded with the same model your collection uses) as your test set.

from qdrant_client import QdrantClient, models


def avg_precision_at_k(
    client: QdrantClient,
    collection_name: str,
    test_vectors: list,
    k: int,
) -> float:
    precisions = []
    for vector in test_vectors:
        ann_ids = {
            p.id for p in client.query_points(
                collection_name=collection_name,
                query=vector,
                limit=k,
            ).points
        }
        knn_ids = {
            p.id for p in client.query_points(
                collection_name=collection_name,
                query=vector,
                limit=k,
                search_params=models.SearchParams(exact=True),
            ).points
        }
        precisions.append(len(ann_ids & knn_ids) / k)

    return sum(precisions) / len(precisions)

Wire it into CI and fail the job when precision falls below your target threshold. This catches regressions from embedding model swaps or index config changes before they reach production.

Next Steps

Once ANN precision is on target, continue with Measuring Retrieval Relevance to check how well those results match user intent.

5.5 KiB Raw Blame History Unescape Escape