Merge branch 'master' into neo4j-int

This commit is contained in:
David Myriel
2024-11-14 15:08:21 -08:00
committed by GitHub
127 changed files with 2299 additions and 387 deletions
@@ -2,7 +2,74 @@
title: Home
weight: 2
hideTOC: true
breadcrumb: false
content:
- partial: "documentation/banners/banner-a"
title: Qdrant Documentation
description: Qdrant is an AI-native vector database and a semantic search engine. You can use it to extract meaningful information from unstructured data.
linkDescription: <a href="https://github.com/qdrant/qdrant_demo/" target="_blank">Clone this repo now</a> and build a search engine in five minutes.
cloudButton:
text: Cloud Quickstart
url: /documentation/quickstart-cloud/
localButton:
text: Local Quickstart
url: /documentation/quickstart/
contained: true
- partial: documentation/banners/banner-d
developingTitle: Ready to start developing?
developingDescription: Qdrant is open-source and can be self-hosted. However, the quickest way to get started is with our <a href="https://qdrant.to/cloud" target="_blank">free tier</a> on Qdrant Cloud. It scales easily and provides a UI where you can interact with data.
developingBlock:
title: Create your first Qdrant Cloud cluster today
button:
text: Get Started
url: https://qdrant.to/cloud
image:
src: /img/rocket.svg
alt: Rocket
- partial: documentation/sections/cards-section
title: Optimize Qdrant's performance
description: Boost search speed, reduce latency, and improve the accuracy and memory usage of your Qdrant deployment.
button:
text: Learn More
url: /documentation/guides/optimize/
cardsPartial: documentation/cards/docs-cards
cards:
- id: 1
tag: Documents
icon:
src: /icons/outline/documentation-blue.svg
alt: Documents
title: Distributed Deployment
description: Scale Qdrant beyond a single node and optimize for high availability, fault tolerance, and billion-scale performance.
link:
url: /documentation/guides/distributed_deployment/
text: Read More
- id: 2
tag: Documents
icon:
src: /icons/outline/documentation-blue.svg
alt: Documents
title: Multitenancy
description: Build vector search apps that serve millions of users. Learn about data isolation, security, and performance tuning.
link:
url: /documentation/guides/multiple-partitions/
text: Read More
- id: 3
tag: Blog
tagColor: violet
icon:
src: /icons/outline/blog-purple.svg
alt: Blog
title: Vector Quantization
description: Learn about cutting-edge techniques for vector quantization and how they can be used to improve search performance.
link:
url: /articles/what-is-vector-quantization/
text: Read More
partition: qdrant
---
THIS CONTENT IS GOING TO BE IGNORED FOR NOW
# Documentation
Qdrant is an AI-native vector database and a semantic search engine. You can use it to extract meaningful information from unstructured data. Want to see how it works? [Clone this repo now](https://github.com/qdrant/qdrant_demo/) and build a search engine in five minutes.
@@ -0,0 +1,49 @@
---
title: Build World-Class Applications
slug: build
breadcrumb: false
content:
- partial: documentation/banners/banner-c
title: Build World-Class Applications
description: Dev-portal Build
image:
src: /img/dev-portal-build/spanish-ai-app-hero.png
alt: Spanish AI app.png
startedButton:
text: Get Started
url: https://qdrant.to/cloud
- partial: documentation/sections/cards-section
title: Start Building
description: Deploy and manage high-performance vector search clusters across cloud environments. Easily scale with fully managed cloud solutions, integrate seamlessly across hybrid setups, or maintain complete control with private cloud deployments in Kubernetes.
cardsPartial: documentation/cards/docs-cards
cards:
- id: 1
image:
src: /img/dev-portal-build/search.png
alt: Search
title: Search
description: Build a simple neural search service with Qdrant and FastEmbed. Learn how to upload data, create indexes, and run search queries.
link:
url: /documentation/tutorials/hybrid-search-fastembed/
text: Read More
- id: 2
image:
src: /img/dev-portal-build/rag.png
alt: RAG
title: RAG
description: Build end-to-end prototype chatbots. Learn how Qdrant integrates with popular RAG frameworks like LangChain and Llamaindex.
link:
url: /documentation/frameworks/langchain/
text: Read More
- id: 3
image:
src: /img/dev-portal-build/pipelines.png
alt: Pipelines
title: Pipelines
description: Integrate Qdrant into your data infrastructure by connecting with popular data engineering tools.
link:
url: /documentation/send-data/
text: Read More
partition: build
hideInSidebar: true
---
@@ -0,0 +1,73 @@
---
title: Welcome to Qdrant Cloud
slug: cloud-intro
breadcrumb: false
content:
- partial: documentation/banners/banner-b
title: Welcome to Qdrant Cloud
description: Dev-portal Cloud
image:
src: /img/dev-portal-cloud/dev-portal-cloud-hero.png
alt: Qdrant cloud dashboard
startedButton:
text: Get Started
url: https://qdrant.to/cloud
- partial: documentation/sections/cards-section
title: Managed Services
description: Deploy and manage high-performance vector search clusters across cloud environments. Easily scale with fully managed cloud solutions, integrate seamlessly across hybrid setups, or maintain complete control with private cloud deployments in Kubernetes.
cardsPartial: documentation/cards/docs-cards
cards:
- id: 1
image:
src: /img/dev-portal-cloud/managed-cloud.png
alt: Qdrant Cloud
title: Qdrant Cloud
description: Qdrant Managed Cloud is our SaaS solution, providing managed Qdrant database clusters on the cloud.
link:
url: /documentation/cloud/
text: Read More
- id: 2
image:
src: /img/dev-portal-cloud/hybrid-cloud.png
alt: Hybrid Cloud
title: Hybrid Cloud
description: Deploy and manage your vector database across diverse environments, ensuring performance, security, and cost efficiency.
link:
url: /documentation/hybrid-cloud/
text: Read More
- id: 3
image:
src: /img/dev-portal-cloud/private-cloud.png
alt: Private Cloud
title: Private Cloud
description: Qdrant Private Cloud allows you to manage Qdrant database clusters in any Kubernetes cluster on any infrastructure.
link:
url: /documentation/private-cloud/
text: Read More
- partial: documentation/sections/cards-section
title: Customer Support
description: Stream, index, and migrate data to Qdrant with these essential tools and strategies.
cardsPartial: documentation/cards/docs-cards
cardsPerRow: 2
cards:
- id: 1
icon:
src: /icons/outline/discord-purple.svg
alt: Discord icon
title: Community Support
description: Join 6,000+ active members to learn, collaborate, and participate in Qdrant’s latest activities.
link:
text: Join our Discord
url: https://qdrant.to/discord
- id: 2
icon:
src: /icons/outline/support-blue.svg
alt: Support icon
title: Qdrant Cloud Support
description: Paying customers have access to our Support team. Links to the support portal are available in the Qdrant Cloud Console.
link:
text: Join Qdrant
url: https://qdrant.to/cloud
partition: cloud
hideInSidebar: true
---
@@ -3,6 +3,7 @@ title: Managed Cloud
weight: 12
aliases:
- /documentation/overview/qdrant-alternatives/documentation/cloud/
partition: cloud
---
# About Qdrant Managed Cloud
@@ -2,6 +2,7 @@
title: Concepts
weight: 8
# If the index.md file is empty, the link to the section will be hidden from the sidebar
partition: qdrant
---
# Concepts
@@ -859,7 +859,7 @@ curl -X PATCH http://localhost:6333/collections/{collection_name} \
```python
client.update_collection(
collection_name="{collection_name}",
optimizer_config=models.OptimizersConfigDiff(indexing_threshold=10000),
optimizers_config=models.OptimizersConfigDiff(indexing_threshold=10000),
)
```
@@ -1,6 +1,7 @@
---
title: Data Management
weight: 18
partition: build
---
## Data Management Integrations
@@ -1,6 +1,7 @@
---
title: Practice Datasets
weight: 29
partition: build
---
# Common Datasets in Snapshot Format
@@ -0,0 +1,11 @@
---
#Delimiter files are used to separate the list of documentation pages into sections.
title: "Interfaces"
type: delimiter
weight: 25 # Change this weight to change order of sections
sitemapExclude: True
_build:
publishResources: false
render: never
partition: cloud
---
@@ -2,9 +2,10 @@
#Delimiter files are used to separate the list of documentation pages into sections.
title: "Support"
type: delimiter
weight: 27 # Change this weight to change order of sections
weight: 30 # Change this weight to change order of sections
sitemapExclude: True
_build:
publishResources: false
render: never
partition: cloud
---
@@ -3,6 +3,7 @@
title: "Examples"
type: delimiter
weight: 24 # Change this weight to change order of sections
partition: build
sitemapExclude: True
_build:
publishResources: false
@@ -7,4 +7,5 @@ sitemapExclude: True
_build:
publishResources: false
render: never
---
partition: qdrant
---
@@ -7,4 +7,5 @@ sitemapExclude: True
_build:
publishResources: false
render: never
partition: build
---
@@ -7,4 +7,5 @@ sitemapExclude: True
_build:
publishResources: false
render: never
partition: cloud
---
@@ -0,0 +1,11 @@
---
#Delimiter files are used to separate the list of documentation pages into sections.
title: "Support"
type: delimiter
weight: 30 # Change this weight to change order of sections
sitemapExclude: True
_build:
publishResources: false
render: never
partition: qdrant
---
@@ -7,4 +7,5 @@ sitemapExclude: True
_build:
publishResources: false
render: never
partition: qdrant
---
@@ -2,6 +2,7 @@
---
title: Embeddings
weight: 19
partition: build
---
# Supported Embedding Providers & Models
@@ -1,6 +1,7 @@
---
title: Build Prototypes
weight: 26
partition: build
---
# Examples
@@ -1,5 +1,6 @@
---
title: FAQ
weight: 28
weight: 31
partition: qdrant
is_empty: true
---
@@ -1,6 +1,7 @@
---
title: "FastEmbed"
weight: 6
partition: qdrant
---
# What is FastEmbed?
@@ -1,31 +1,33 @@
---
title: Frameworks
weight: 20
partition: build
---
## Framework Integrations
| Framework | Description |
| ------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------- |
| [AutoGen](/documentation/frameworks/autogen/) | Framework from Microsoft building LLM applications using multiple conversational agents. |
| [Canopy](/documentation/frameworks/canopy/) | Framework from Pinecone for building RAG applications using LLMs and knowledge bases. |
| [Cheshire Cat](/documentation/frameworks/cheshire-cat/) | Framework to create personalized AI assistants using custom data. |
| [DocArray](/documentation/frameworks/docarray/) | Python library for managing data in multi-modal AI applications. |
| [DSPy](/documentation/frameworks/dspy/) | Framework for algorithmically optimizing LM prompts and weights. |
| [Fifty-One](/documentation/frameworks/fifty-one/) | Toolkit for building high-quality datasets and computer vision models. |
| [Genkit](/documentation/frameworks/genkit/) | Framework to build, deploy, and monitor production-ready AI-powered apps. |
| [Haystack](/documentation/frameworks/haystack/) | LLM orchestration framework to build customizable, production-ready LLM applications. |
| [Lakechain](/documentation/frameworks/lakechain/) | Python framework for deploying document processing pipelines on AWS using infrastructure-as-code. |
| [Langchain](/documentation/frameworks/langchain/) | Python framework for building context-aware, reasoning applications using LLMs. |
| [Langchain-Go](/documentation/frameworks/langchain-go/) | Go framework for building context-aware, reasoning applications using LLMs. |
| [Langchain4j](/documentation/frameworks/langchain4j/) | Java framework for building context-aware, reasoning applications using LLMs. |
| [LlamaIndex](/documentation/frameworks/llama-index/) | A data framework for building LLM applications with modular integrations. |
| [Mem0](/documentation/frameworks/mem0/) | Self-improving memory layer for LLM applications, enabling personalized AI experiences. |
| [MemGPT](/documentation/frameworks/memgpt/) | System to build LLM agents with long term memory & custom tools |
| [Neo4j GraphRAG](/documentation/frameworks/neo4j-graphrag/) | Package to build graph retrieval augmented generation (GraphRAG) applications using Neo4j and Python. |
| [Pandas-AI](/documentation/frameworks/pandas-ai/) | Python library to query/visualize your data (CSV, XLSX, PostgreSQL, etc.) in natural language |
| [Semantic Router](/documentation/frameworks/semantic-router/) | Python library to build a decision-making layer for AI applications using vector search. |
| [Spring AI](/documentation/frameworks/spring-ai/) | Java AI framework for building with Spring design principles such as portability and modular design. |
| [Sycamore](/documentation/frameworks/sycamore/) | Document processing engine for ETL, RAG, LLM-based applications, and analytics on unstructured data. |
| [txtai](/documentation/frameworks/txtai/) | Python library for semantic search, LLM orchestration and language model workflows. |
| [Vanna AI](/documentation/frameworks/vanna-ai/) | Python RAG framework for SQL generation and querying. |
| Framework | Description |
| ------------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------- |
| [AutoGen](/documentation/frameworks/autogen/) | Framework from Microsoft building LLM applications using multiple conversational agents. |
| [Canopy](/documentation/frameworks/canopy/) | Framework from Pinecone for building RAG applications using LLMs and knowledge bases. |
| [Cheshire Cat](/documentation/frameworks/cheshire-cat/) | Framework to create personalized AI assistants using custom data. |
| [DocArray](/documentation/frameworks/docarray/) | Python library for managing data in multi-modal AI applications. |
| [DSPy](/documentation/frameworks/dspy/) | Framework for algorithmically optimizing LM prompts and weights. |
| [Fifty-One](/documentation/frameworks/fifty-one/) | Toolkit for building high-quality datasets and computer vision models. |
| [Genkit](/documentation/frameworks/genkit/) | Framework to build, deploy, and monitor production-ready AI-powered apps. |
| [Haystack](/documentation/frameworks/haystack/) | LLM orchestration framework to build customizable, production-ready LLM applications. |
| [Lakechain](/documentation/frameworks/lakechain/) | Python framework for deploying document processing pipelines on AWS using infrastructure-as-code. |
| [Langchain](/documentation/frameworks/langchain/) | Python framework for building context-aware, reasoning applications using LLMs. |
| [Langchain-Go](/documentation/frameworks/langchain-go/) | Go framework for building context-aware, reasoning applications using LLMs. |
| [Langchain4j](/documentation/frameworks/langchain4j/) | Java framework for building context-aware, reasoning applications using LLMs. |
| [LlamaIndex](/documentation/frameworks/llama-index/) | A data framework for building LLM applications with modular integrations. |
| [Mem0](/documentation/frameworks/mem0/) | Self-improving memory layer for LLM applications, enabling personalized AI experiences. |
| [MemGPT](/documentation/frameworks/memgpt/) | System to build LLM agents with long term memory & custom tools |
| [Neo4j GraphRAG](/documentation/frameworks/neo4j-graphrag/) | Package to build graph retrieval augmented generation (GraphRAG) applications using Neo4j and Python. |
| [Pandas-AI](/documentation/frameworks/pandas-ai/) | Python library to query/visualize your data (CSV, XLSX, PostgreSQL, etc.) in natural language |
| [Ragbits](/documentation/frameworks/ragbits/) | Python package that offers essential "bits" for building powerful Retrieval-Augmented Generation (RAG) applications. |
| [Semantic Router](/documentation/frameworks/semantic-router/) | Python library to build a decision-making layer for AI applications using vector search. |
| [Spring AI](/documentation/frameworks/spring-ai/) | Java AI framework for building with Spring design principles such as portability and modular design. |
| [Sycamore](/documentation/frameworks/sycamore/) | Document processing engine for ETL, RAG, LLM-based applications, and analytics on unstructured data. |
| [txtai](/documentation/frameworks/txtai/) | Python library for semantic search, LLM orchestration and language model workflows. |
| [Vanna AI](/documentation/frameworks/vanna-ai/) | Python RAG framework for SQL generation and querying. |
@@ -0,0 +1,83 @@
---
title: Ragbits
---
# Ragbits
[Ragbit](https://ragbits.deepsense.ai) is a Python package that offers essential "bits" for building powerful Retrieval-Augmented Generation (RAG) applications. It prioritizes developer experience by providing a simple and intuitive API. It also includes a comprehensive set of tools for seamlessly building, testing, and deploying your RAG applications efficiently.
Qdrant is available as a vectorstore in Ragbits to ingest and search search documents from a collection.
## Installation
Install the Python package that comes bundled with the Qdrant integration.
```bash
pip install ragbits
```
## Usage
An example usage of Ragbits and Qdrant would look something like this:
The following example uses [OpenAI embeddings](https://platform.openai.com/docs/guides/embeddings) via [LiteLLM](https://www.litellm.ai).
```python
import asyncio
from qdrant_client import AsyncQdrantClient
from ragbits.core.embeddings.litellm import LiteLLMEmbeddings
from ragbits.core.vector_stores.qdrant import QdrantVectorStore
from ragbits.document_search import DocumentSearch, SearchConfig
from ragbits.document_search.documents.document import DocumentMeta
documents = [
DocumentMeta.create_text_document_from_literal(
"RIP boiled water. You will be mist."
),
DocumentMeta.create_text_document_from_literal(
"Why programmers don't like to swim? Because they're scared of the floating points."
),
DocumentMeta.create_text_document_from_literal("This one is completely unrelated."),
]
async def main() -> None:
embedder = LiteLLMEmbeddings(
model="text-embedding-3-small",
)
vector_store = QdrantVectorStore(
client=AsyncQdrantClient(url="http://localhost:6333"),
collection_name="{collection_name}",
)
document_search = DocumentSearch(
embedder=embedder,
vector_store=vector_store,
)
await document_search.ingest(documents)
all_documents = await vector_store.list()
print([doc.metadata["content"] for doc in all_documents])
query = "I write computer software. Tell me something."
vector_store_kwargs = {
"k": 1,
"max_distance": None,
}
results = await document_search.search(
query,
config=SearchConfig(vector_store_kwargs=vector_store_kwargs),
)
print(f"Documents similar to: {query}")
print([element.get_key() for element in results])
```
</details>
## 📚 Further Reading
- Ragbits [Documentation](http://ragbits.deepsense.ai)
- [Source Code](https://github.com/deepsense-ai/ragbits)
@@ -3,4 +3,5 @@ title: Guides
weight: 9
# If the index.md file is empty, the link to the section will be hidden from the sidebar
is_empty: true
partition: qdrant
---
@@ -1,6 +1,7 @@
---
title: Hybrid Cloud
weight: 13
partition: cloud
---
# Qdrant Hybrid Cloud
@@ -1,6 +1,7 @@
---
title: Infrastructure
weight: 21
partition: build
---
## Infrastructure Integrations
@@ -1,6 +1,7 @@
---
title: API & SDKs
weight: 6
partition: qdrant
aliases:
- /documentation/interfaces/
---
@@ -1,6 +1,7 @@
---
title: Observability
weight: 22
partition: build
---
## Observability Integrations
@@ -3,6 +3,7 @@ title: What is Qdrant?
weight: 3
aliases:
- overview
partition: qdrant
---
# Introduction
@@ -1,16 +1,18 @@
---
title: Platforms
weight: 23
partition: build
---
## Platform Integrations
| Platform | Description |
| --------------------------- | ---------------------------------------------------------------------------------------- |
| Platform | Description |
| -------------------------------------------------- | ---------------------------------------------------------------------------------------- |
| [Apify](/documentation/platforms/apify/) | Platform to build web scrapers and automate web browser tasks. |
| [Bubble](/documentation/platforms/bubble/) | Development platform for application development with a no-code interface |
| [BuildShip](/documentation/platforms/buildship/) | Low-code visual builder to create APIs, scheduled jobs, and backend workflows. |
| [Bubble](/documentation/platforms/bubble/) | Development platform for application development with a no-code interface |
| [BuildShip](/documentation/platforms/buildship/) | Low-code visual builder to create APIs, scheduled jobs, and backend workflows. |
| [DocsGPT](/documentation/platforms/docsgpt/) | Tool for ingesting documentation sources and enabling conversations and queries. |
| [Kotaemon](/documentation/platforms/kotaemon/) | Open-source & customizable RAG UI for chatting with your documents. |
| [Make](/documentation/platforms/make/) | Cloud platform to build low-code workflows by integrating various software applications. |
| [N8N](/documentation/platforms/n8n/) | Platform for node-based, low-code workflow automation. |
| [Pipedream](/documentation/platforms/pipedream/) | Platform for connecting apps and developing event-driven automation. |
@@ -0,0 +1,33 @@
---
title: Kotaemon
---
# Kotaemon
[Kotaemon](https://github.com/Cinnamon/kotaemon) is open-source clean & customizable RAG UI for chatting with your documents. Built with both end users and developers in mind.
Qdrant is supported as a vectorstore in Kotaemon for ingesting and retrieving documents.
## Configuration
- Refer to [Getting started](https://cinnamon.github.io/kotaemon/) guide to set up Kotaemon.
- To configure Kotaemon to use Qdrant as the vector store, update the `flowsettings.py` as follows.
```python
KH_VECTORSTORE = {
"__type__": "kotaemon.storages.QdrantVectorStore",
"url": "https://xyz-example.eu-central.aws.cloud.qdrant.io:6333",
"api_key": "<provide-your-own-key>'",
"client_kwargs": {} # Additional options to pass to qdrant_client.QdrantClient
}
```
- Restart Kotaemon for the changes to take effect.
The reference for all the Qdrant client options can be found [here](https://python-client.qdrant.tech/qdrant_client.qdrant_client)
## Further reading
- [Kotaemon Documentation](https://cinnamon.github.io/kotaemon/)
- [Source](https://github.com/Cinnamon/kotaemon)
@@ -1,6 +1,7 @@
---
title: Private Cloud
weight: 14
partition: cloud
---
# Qdrant Private Cloud
@@ -1,6 +1,7 @@
---
title: Qdrant Cloud API
weight: 15
weight: 26
partition: cloud
---
# Qdrant Cloud API
@@ -1,8 +1,8 @@
---
title: Cloud Quickstart
weight: 4
partition: qdrant
aliases:
- quickstart-cloud
- ../cloud-quick-start
- cloud-quick-start
- cloud-quickstart
@@ -124,3 +124,4 @@ When ready, use the Console and our complete REST API to try other operations.
Now that you have a Qdrant Cloud cluster up and running, you should [test remote access](/documentation/cloud/authentication/#test-cluster-access) with a Qdrant Client.
For more about Qdrant Cloud, check our [dedicated documentation](/documentation/cloud-intro/).
@@ -1,10 +1,10 @@
---
title: Local Quickstart
weight: 5
partition: qdrant
aliases:
- quick_start
- quick-start
- quickstart
---
# How to Get Started with Qdrant Locally
@@ -1,6 +1,7 @@
---
title: Release Notes
weight: 30
weight: 32
partition: qdrant
type: external-link
external_url: https://github.com/qdrant/qdrant/releases
sitemapExclude: True
@@ -1,6 +1,7 @@
---
title: Send Data to Qdrant
weight: 25
partition: build
---
## How to Send Your Data to a Qdrant Cluster
@@ -1,6 +1,7 @@
---
title: Support
weight: 16
weight: 35
partition: cloud
aliases:
- /documentation/cloud/support/
---
@@ -6,6 +6,7 @@ is_empty: false
aliases:
- how-to
- tutorials
partition: qdrant
---
# Tutorials
@@ -159,7 +159,7 @@ Image.open(client.search(
![Coffee Image](/docs/coffee.jpg)
<h3 style="font-size: 1.25em;">Image-to-Text</h3>
### Image-to-Text
Now, let's do a reverse search with an image: