change embedding at query time

qdrant-landing/content/documentation/improve-search/pipeline-output-quality.md

@@ -22,7 +22,7 @@ Two related tutorials cover the other retrieval-evaluation concerns: [Measuring
 
 <a href="https://docs.ragas.io/" target="_blank">Ragas</a> is a Python library that uses an LLM as a judge to score RAG outputs (rating each answer against criteria like faithfulness and relevancy). It expects samples shaped as `(question, retrieved_context, answer)` triples, so you build a fresh evaluation set from your labeled data. Three steps: prepare the evaluation data, define a grounding prompt, and run the retrieve-generate-record loop.
 
-**1. Prepare the evaluation data.** Each entry needs a `query_id`, a `query_text` (for prompting the generator), a `query_vector` (for retrieval), and `labels`. For `context_precision` only, also include a `ground_truth` reference answer.
+**1. Prepare the evaluation data.** Each entry needs a `query_id`, a `query_text` (used for both prompting the generator and embedding for retrieval), and `labels`. For `context_precision` only, also include a `ground_truth` reference answer.
 
 If your queries came from synthetic generation, they don't carry ground-truth answers natively. A simple workaround: make one more LLM pass per query, constrained to the source document, asking for a one-to-two-sentence reference answer. Skip this step if you're only scoring `faithfulness` and `answer_relevancy` (both are reference-free).
 
@@ -31,7 +31,6 @@ If your queries came from synthetic generation, they don't carry ground-truth an
 {
     "query_id": "q1",
     "query_text": "how does X work",
-    "query_vector": [0.12, -0.48, 0.33, ...],
     "labels": {"doc_42": 1},
     "ground_truth": "...",  # optional; required for context_precision only
 }
@@ -68,6 +67,8 @@ import anthropic
 from qdrant_client import QdrantClient
 from ragas import SingleTurnSample
 
+from your_embedding_model import embed  # must match the model your Qdrant collection uses
+
 client = QdrantClient("http://localhost:6333")  # or QdrantClient(url="https://<id>.cloud.qdrant.io", api_key="...") for Qdrant Cloud
 
 # The example uses Anthropic, but any LLM provider works.
@@ -95,7 +96,7 @@ def build_eval_set(golden_set: list, collection: str, k: int = 10) -> list:
         # Retrieve top-k chunks from Qdrant.
         results = client.query_points(
             collection_name=collection,
-            query=entry["query_vector"],
+            query=embed(entry["query_text"]),
             limit=k,
         ).points
         contexts = [p.payload["text"] for p in results]  # adjust the payload key to match your schema

qdrant-landing/content/documentation/improve-search/retrieval-relevance.md

@@ -67,13 +67,12 @@ Document:
 
 The evaluation runs in three steps: load the labeled queries into the shape ranx expects, run each through Qdrant, then compute metrics.
 
-**1. Load and assemble.** For each labeled query, build an entry with `query_id`, `query_text`, `query_vector` (embedded with the same model your Qdrant collection uses), and `labels`:
+**1. Load and assemble.** For each labeled query, build an entry with `query_id`, `query_text`, and `labels`:
 
 ```python
 {
     "query_id": "q1",
     "query_text": "how does X work",
-    "query_vector": [0.12, -0.48, 0.33, ...],  # embedding of query_text
     "labels": {"doc_42": 1},  # source doc for synthetic queries, relevant docs otherwise
 }
 ```
@@ -81,8 +80,6 @@ The evaluation runs in three steps: load the labeled queries into the shape ranx
 Build the full `golden_set` by normalizing whatever your generation pipeline produced, then looping through it:
 
 ```python
-from your_embedding_model import embed
-
 # Normalize whatever your generation pipeline produced into this shape:
 #   - Synthetic: one item per generated query, labels = {source_doc_id: 1}
 #   - Logs: one item per query-click pair, labels = {clicked_doc_id: 1}
@@ -98,7 +95,6 @@ for i, item in enumerate(labeled_data):
     golden_set.append({
         "query_id": f"q{i}",
         "query_text": item["query_text"],
-        "query_vector": embed(item["query_text"]),
         "labels": item["labels"],
     })
 ```
@@ -112,6 +108,8 @@ for i, item in enumerate(labeled_data):
 from qdrant_client import QdrantClient
 from ranx import Qrels, Run, evaluate
 
+from your_embedding_model import embed  # must match the model your Qdrant collection uses
+
 client = QdrantClient("http://localhost:6333")  # or QdrantClient(url="https://<id>.cloud.qdrant.io", api_key="...") for Qdrant Cloud
 
 def retrieval_run(golden_set: list, collection: str, k: int = 10) -> Run:
@@ -119,7 +117,7 @@ def retrieval_run(golden_set: list, collection: str, k: int = 10) -> Run:
     for entry in golden_set:
         results = client.query_points(
             collection_name=collection,
-            query=entry["query_vector"],
+            query=embed(entry["query_text"]),
             limit=k,
         ).points
         # p.id type must match the doc_id type in labels (ranx matches by equality).