add honeyhive framework documentation (#1623)

2026-10-04 10:28:29 +02:00 · 2025-05-09 12:37:10 +05:30
parent a5fa55c2d8
commit 80a601337c
2 changed files with 320 additions and 0 deletions
@@ -24,6 +24,7 @@ aliases: ["/documentation/frameworks/memgpt/"]
 | [Fifty-One](/documentation/frameworks/fifty-one/)             | Toolkit for building high-quality datasets and computer vision models.                                               |
 | [Genkit](/documentation/frameworks/genkit/)                   | Framework to build, deploy, and monitor production-ready AI-powered apps.                                            |
 | [Haystack](/documentation/frameworks/haystack/)               | LLM orchestration framework to build customizable, production-ready LLM applications.                                |
+| [HoneyHive](/documentation/frameworks/honeyhive/)               | AI observability and evaluation platform that provides tracing and monitoring tools for GenAI pipelines.                                |
 | [Lakechain](/documentation/frameworks/lakechain/)             | Python framework for deploying document processing pipelines on AWS using infrastructure-as-code.                    |
 | [Langchain](/documentation/frameworks/langchain/)             | Python framework for building context-aware, reasoning applications using LLMs.                                      |
 | [Langchain-Go](/documentation/frameworks/langchain-go/)       | Go framework for building context-aware, reasoning applications using LLMs.                                          |
@@ -0,0 +1,319 @@
+---
+title: HoneyHive
+---
+
+# HoneyHive
+
+[HoneyHive](https://www.honeyhive.ai/) is an AI evaluation and observability platform for Generative AI applications. HoneyHive’s platform gives developers enterprise-grade tools to debug complex retrieval pipelines, evaluate performance over large test suites, monitor usage in real-time, and manage prompts within a shared workspace. Teams use HoneyHive to iterate faster, detect failures at scale, and deliver exceptional AI products.
+
+By integrating Qdrant with HoneyHive, you can:
+
+- Trace vector database operations
+- Monitor latency, embedding quality, and context relevance
+- Evaluate retrieval performance in your RAG pipelines
+- Optimize paramaters such as `chunk_size` or `chunk_overlap`
+
+## Prerequisites
+
+- A HoneyHive account and API key
+- Python 3.8+
+
+## Installation
+
+Install the required packages:
+
+```bash
+pip install qdrant-client openai honeyhive
+```
+
+## Basic Integration Example
+
+The following example demonstrates a complete RAG pipeline with HoneyHive tracing for Qdrant operations. We'll break down each component step by step.
+
+### Initialize Clients and Setup
+
+First, set up the necessary clients and configuration for HoneyHive, OpenAI, and Qdrant:
+
+```python
+from qdrant_client import QdrantClient
+from qdrant_client.http.models import PointStruct, VectorParams, Distance
+import openai
+import os
+from honeyhive.tracer import HoneyHiveTracer
+from honeyhive.tracer.custom import trace
+from openai import OpenAI
+
+# Set API Keys
+openai.api_key = os.getenv("OPENAI_API_KEY")
+honeyhive_api_key = os.getenv("HONEYHIVE_API_KEY")
+
+# Initialize HoneyHive Tracer
+HoneyHiveTracer.init(
+    api_key=honeyhive_api_key,
+    project="qdrant-rag-example",
+    session_name="qdrant-integration-demo"
+)
+
+# Initialize OpenAI client
+openai_client = OpenAI(api_key=openai.api_key)
+```
+
+### Connect to Qdrant
+
+You can connect to Qdrant in two ways: self-hosted (local) or cloud-hosted (Qdrant Cloud):
+
+#### Option 1: Self-Hosted Qdrant (Local)
+
+To run Qdrant locally, you need to have Docker installed and run the following command:
+
+```bash
+docker pull qdrant/qdrant
+docker run -p 6333:6333 -p 6334:6334 -v "$(pwd)/qdrant_storage:/qdrant/storage" qdrant/qdrant
+```
+
+Then connect to the local Qdrant instance:
+
+```python
+# Connect to local Qdrant
+client = QdrantClient(url="http://localhost:6333")
+print("Connected to local Qdrant instance")
+```
+
+#### Option 2: Qdrant Cloud
+
+For Qdrant Cloud, you'll need your cluster host and API key:
+
+```python
+# Qdrant Cloud configuration
+QDRANT_HOST = os.getenv("QDRANT_HOST")  # e.g., "your-cluster-id.eu-central.aws.cloud.qdrant.io"
+QDRANT_API_KEY = os.getenv("QDRANT_API_KEY")
+
+# Connect to Qdrant Cloud
+client = QdrantClient(url=QDRANT_HOST, api_key=QDRANT_API_KEY)
+print("Connected to Qdrant Cloud")
+```
+
+### Create a Collection
+
+Create a collection to store document embeddings:
+
+```python
+collection_name = "documents"
+vector_size = 1536  # For text-embedding-3-small
+vector_distance = Distance.COSINE
+
+# Create collection if it doesn't exist
+if not client.collection_exists(collection_name):
+    client.create_collection(
+        collection_name=collection_name,
+        vectors_config=VectorParams(size=vector_size, distance=vector_distance)
+    )
+```
+
+### Define Embedding Function with Tracing
+
+Create a function to generate embeddings with HoneyHive tracing:
+
+```python
+@trace()
+def embed_text(text: str) -> list:
+    """Generate embeddings for a text using OpenAI's API."""
+    response = openai_client.embeddings.create(
+        model="text-embedding-3-small",
+        input=text
+    )
+    return response.data[0].embedding
+```
+
+### Insert Documents with Tracing
+
+Create a function to insert documents into Qdrant with tracing:
+
+```python
+@trace()
+def insert_documents(docs):
+    """Insert documents into Qdrant collection."""
+    points = []
+    for idx, doc in enumerate(docs):
+        vector = embed_text(doc)
+        points.append(PointStruct(
+            id=idx + 1,
+            vector=vector,
+            payload={"text": doc}
+        ))
+    
+    client.upsert(
+        collection_name=collection_name,
+        points=points
+    )
+    return len(points)
+
+# Sample documents
+documents = [
+    "Qdrant is a vector database optimized for storing and searching high-dimensional vectors.",
+    "HoneyHive provides observability for AI applications, including RAG pipelines.",
+    "Retrieval-Augmented Generation (RAG) combines retrieval systems with generative models.",
+    "Vector databases like Qdrant are essential for efficient similarity search in RAG systems.",
+    "OpenAI's embedding models convert text into high-dimensional vectors for semantic search."
+]
+
+# Insert documents
+num_inserted = insert_documents(documents)
+```
+
+### Retrieve Documents with Tracing
+
+Create a function to retrieve relevant documents from Qdrant with tracing:
+
+```python
+@trace()
+def get_relevant_docs(query: str, top_k: int = 3) -> list:
+    """Retrieve relevant documents for a query."""
+    # Embed the query
+    q_vector = embed_text(query)
+    
+    # Search in Qdrant
+    search_response = client.query_points(
+        collection_name=collection_name,
+        query=q_vector,
+        limit=top_k,
+        with_payload=True
+    )
+    
+    # Extract results
+    docs = []
+    for point in search_response.points:
+        docs.append({
+            "id": point.id,
+            "text": point.payload.get("text"),
+            "score": point.score
+        })
+    
+    return docs
+```
+
+### Generate Response with Tracing
+
+Create a function to generate a response using OpenAI with tracing:
+
+```python
+@trace()
+def answer_query(query: str, relevant_docs: list) -> str:
+    """Generate an answer for a query using retrieved documents."""
+    if not relevant_docs:
+        return "Could not retrieve relevant documents to answer the query."
+
+    # Format context from retrieved documents
+    context_parts = []
+    for i, doc in enumerate(relevant_docs):
+        context_parts.append(f"Document {i+1} (ID: {doc['id']}, Score: {doc['score']:.4f}):\n{doc['text']}")
+    context = "\n\n".join(context_parts)
+
+    # Create prompt
+    prompt = f"""Answer the question based ONLY on the following context:
+
+Context:
+{context}
+
+Question: {query}
+
+Answer:"""
+
+    # Generate answer
+    completion = openai_client.chat.completions.create(
+        model="gpt-4o-mini",
+        messages=[
+            {"role": "system", "content": "You are a helpful assistant that answers questions based strictly on the provided context. If the answer is not in the context, say so clearly."},
+            {"role": "user", "content": prompt}
+        ],
+        temperature=0.2
+    )
+
+    return completion.choices[0].message.content.strip()
+```
+
+### Complete RAG Pipeline
+
+Create a function to run the complete RAG pipeline with tracing:
+
+```python
+@trace()
+def rag_pipeline(query: str) -> dict:
+    """End-to-end RAG pipeline."""
+    # Get relevant documents
+    relevant_docs = get_relevant_docs(query)
+    
+    # Generate answer
+    answer = answer_query(query, relevant_docs)
+    
+    return {
+        "query": query,
+        "answer": answer,
+        "retrieved_documents": relevant_docs
+    }
+```
+
+### Batch Processing
+
+For larger document sets, you can use batch processing to improve performance:
+
+```python
+@trace()
+def batch_insert_documents(documents_to_insert, batch_size=10, start_id_offset=0):
+    """Insert documents in batches."""
+    total_inserted = 0
+    
+    for i in range(0, len(documents_to_insert), batch_size):
+        batch_docs = documents_to_insert[i:i+batch_size]
+        points = []
+        
+        for local_idx, doc in enumerate(batch_docs):
+            relative_idx = i + local_idx
+            vector = embed_text(doc)
+            point_id = relative_idx + start_id_offset + 1
+            points.append(PointStruct(
+                id=point_id,
+                vector=vector,
+                payload={"text": doc}
+            ))
+        
+        if points:
+            client.upsert(
+                collection_name=collection_name,
+                points=points
+            )
+            total_inserted += len(points)
+    
+    return total_inserted
+```
+
+### Test the RAG Pipeline
+
+Here's how to test the complete RAG pipeline:
+
+```python
+# Test query
+test_query = "What is Qdrant used for?"
+result = rag_pipeline(test_query)
+
+print(f"Query: {result['query']}")
+print(f"Answer: {result['answer']}")
+print("\nRetrieved Documents:")
+for i, doc in enumerate(result['retrieved_documents']):
+    print(f"Document {i+1} (ID: {doc['id']}, Score: {doc['score']:.4f}): {doc['text']}")
+```
+
+## Viewing Traces in HoneyHive
+
+After running your RAG pipeline with Qdrant, you can view the traces in the HoneyHive UI:
+
+1. Navigate to your project in the HoneyHive dashboard
+2. Click on the "Traces" tab to see all the traces from your RAG pipeline
+3. Click on a specific trace to see detailed information about each step in the pipeline
+4. Analyze the performance of your vector operations, embeddings, and retrieval processes
+
+With HoneyHive, you can easily monitor and optimize your Qdrant-powered RAG pipeline, ensuring that it delivers the best possible results for your users.
+
+## Further Reading
+
+- [HoneyHive Documentation](https://docs.honeyhive.ai/introduction/what-is-hhai)