mirror of
https://github.com/qdrant/landing_page.git
synced 2026-09-26 22:48:30 +02:00
Improved Nested Docs and Link Grouping Support (#147)
* added a table of content * wide layout for docs, styles for the table of content * fixes for docs layout * wide footer at the docs section * added support for nested docs, added toggling groups of links, delimiters, external links * external link icon * added active state for nested links, styles for the external link icon * styles fix * remove doc sync * update directory structure and doc titles * fix outstanding links * fix more links for merge * Revert "fix more links for merge" This reverts commit 46c9ccaf1b7765f2cda8dc85d625fa6b4e3f5436. * Revert "fix outstanding links" This reverts commit 28e6380b74f1ab74690c8184551f186656d6d4e9. * fix remaining broken links * move how-to tutorials in the different page * split tutorials * fix link * upd github edit link * skip empty index pages --------- Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com> Co-authored-by: David Sertic <62056091+davidmyriel@users.noreply.github.com>
This commit is contained in:
trean
co-authored by
Andrey Vasnetsov
David Sertic
Andrey Vasnetsov
David Sertic
parent
b9874e5c56
commit
3215985d1d
@@ -1,9 +1,5 @@
|
||||
name: doc-sync
|
||||
on:
|
||||
pull_request:
|
||||
branches: [ master ]
|
||||
schedule:
|
||||
- cron: "45 23 * * *"
|
||||
# Allows you to run this workflow manually from the Actions tab
|
||||
workflow_dispatch:
|
||||
|
||||
|
||||
@@ -83,3 +83,29 @@ Preview images set consists of the following images:
|
||||
`title.jpg` - 898x300px (used on the article's page as the main image before the article title **for browsers, not supporting webp**)
|
||||
`title.webp` - 898x300px (used on the article's page as the main image before the article title **for browsers, supporting webp**)
|
||||
`social_preview.jpg` - 1200x630px (used in social media previews)
|
||||
|
||||
## Documentation sidebar
|
||||
|
||||
### Delimiter
|
||||
|
||||
To create a delimiter in the sidebar, use the following command:
|
||||
|
||||
``` bash
|
||||
cd qdrant-landing
|
||||
hugo new --kind delimiter documentation/<delimiter-title>.md
|
||||
```
|
||||
|
||||
It will create a file `content/documentation/<delimiter-title>.md`.
|
||||
|
||||
To put a delimiter to desired place in the sidebar, set the `weight` parameter to the desired value. The lower the value, the higher the delimiter will be in the sidebar.
|
||||
|
||||
### External link
|
||||
|
||||
To create an external link in the sidebar, use the following command:
|
||||
|
||||
``` bash
|
||||
cd qdrant-landing
|
||||
hugo new --kind external-link documentation/<link-title>.md
|
||||
```
|
||||
|
||||
It will create a file `content/documentation/<link-title>.md`. Open it and set the `external_link` parameter to the desired value.
|
||||
@@ -51,8 +51,7 @@ keywords = "search engine, vector database, neural network, matching, filter, Sa
|
||||
|
||||
gdpr = "We use cookies to learn more about you. At any time you can delete or block cookies through your browser settings."
|
||||
|
||||
githubDocPrefix = "https://github.com/qdrant/docs/tree/master/qdrant/v1.2.x/"
|
||||
githubCloudDocPrefix = "https://github.com/qdrant/docs/tree/master/cloud/v0.1.x/"
|
||||
githubDocPrefix = "https://github.com/qdrant/landing_page/tree/master/qdrant-landing/content/documentation/"
|
||||
|
||||
docVersion = "v1.2.x"
|
||||
cloudDocVersion = "v0.1.x"
|
||||
|
||||
@@ -1,17 +1,15 @@
|
||||
---
|
||||
title: Documentation
|
||||
section_title: Everything you need to start working with Qdrant
|
||||
subtitle:
|
||||
title: Qdrant Documentation
|
||||
weight: 10
|
||||
---
|
||||
|
||||
|
||||
# Qdrant Documentation
|
||||
# Overview
|
||||
|
||||
Qdrant (read: quadrant ) is a vector similarity search engine. It provides a production-ready service with a convenient API to store, search, and manage points - vectors with an additional payload. Qdrant is tailored to extended filtering support. It makes it useful for all sorts of neural network or semantic-based matching, faceted search, and other applications.
|
||||
|
||||
Qdrant is released under the open-source Apache License 2.0. Its source code is available on [GitHub](https://github.com/qdrant/qdrant).
|
||||
|
||||
## Overview
|
||||
## Common uses
|
||||
|
||||
Qdrant is ideal for deploying applications based on the matching of embeddings produced by neural network encoders.
|
||||
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
---
|
||||
title: Qdrant Cloud
|
||||
section_title: How Qdrant Cloud operates
|
||||
subtitle:
|
||||
weight: 20
|
||||
---
|
||||
|
||||
# Getting started with Qdrant Cloud
|
||||
|
||||
Qdrant Cloud is an official cloud-based managed solution by the creators of the [Qdrant](https://github.com/qdrant/qdrant) Vector Search Engine.
|
||||
It provides the same fast and reliable similarity search engine, but without a need to maintain your own infrastructure.
|
||||
@@ -14,4 +14,4 @@ The transition is even easier if you use the official client libraries.
|
||||
|
||||
For example, the Python Qdrant client has the support of the API key already built-in, so you only need to provide it once, when the QdrantClient instance is created.
|
||||
|
||||
Please see [**Quick Start**](https://qdrant.tech/documentation/cloud/cloud-quick-start/) section for details.
|
||||
Please see [**Quick Start**](../cloud/cloud-quick-start/) section for details.
|
||||
|
||||
@@ -3,6 +3,8 @@ title: Backups
|
||||
weight: 20
|
||||
---
|
||||
|
||||
# Backups
|
||||
|
||||
There are situations where you need to restore your cluster because of application or system failure.
|
||||
In most cases you will have a source of truth for your data in a regular database and would be able to reindex the data into your Qdrant vector search cluster.
|
||||
However, encoding and uploading a big amount of data might require a long time.
|
||||
@@ -12,17 +14,17 @@ For less critical use-cases you can make use of one of the available options.
|
||||
## Self-service backups
|
||||
|
||||
Qdrant engine offers a snapshot API that allows to create a snapshot of a particular collection or even the whole storage.
|
||||
Please refer to the [snapshot documentation](../../snapshots/) for details.
|
||||
Please refer to the [snapshot documentation](../../concepts/snapshots/) for details.
|
||||
|
||||
A quick recipe for successfully snapshotting and recovering a collection:
|
||||
|
||||
1. Take a snapshot
|
||||
- In case of a single node cluster, simply call the snapshot endpoint on the exposed url.
|
||||
- In case of a multi node cluster you’d need to take a snapshot on each node that the collection resides upon. To achieve this, you simply prepend `node-{num}-` to your cluster url and call the [snapshot endpoint](../..//snapshots/#create-snapshot) on the individual hosts, starting with node 0 up to the number of nodes minus one.
|
||||
- In case of a multi node cluster you’d need to take a snapshot on each node that the collection resides upon. To achieve this, you simply prepend `node-{num}-` to your cluster url and call the [snapshot endpoint](../../concepts/snapshots/#create-snapshot) on the individual hosts, starting with node 0 up to the number of nodes minus one.
|
||||
- In the response you'll get the name of the snapshot taken.
|
||||
2. Delete and recreate the collection.
|
||||
3. Recover the snapshot
|
||||
- Call the [recover endpoint](../../snapshots/#recover-in-cluster-deployment) with location pointing to the snapshot file (`file:///qdrant/snapshots/{collection_name}/{snapshot_file_name}`) you got for each host.
|
||||
- Call the [recover endpoint](../../concepts/snapshots/#recover-in-cluster-deployment) with location pointing to the snapshot file (`file:///qdrant/snapshots/{collection_name}/{snapshot_file_name}`) you got for each host.
|
||||
|
||||
|
||||
## Automatic backups
|
||||
|
||||
@@ -3,6 +3,7 @@ title: Capacity and sizing
|
||||
weight: 20
|
||||
---
|
||||
|
||||
# Capacity and sizing
|
||||
|
||||
We have been asked a lot about the optimal cluster configuration to serve a number of vectors.
|
||||
The only right answer is “It depends”.
|
||||
@@ -19,13 +20,13 @@ memory_size = number_of_vectors * vector_dimension * 4 bytes * 1.5
|
||||
|
||||
Extra 50% is needed for metadata (indexes, point versions, etc.) as well as for temporary segments constructed during the optimization process.
|
||||
|
||||
If you need to have payloads along with the vectors, it is recommended to store it on the disc, and only keep [indexed fields](../../indexing/#payload-index) in RAM.
|
||||
Read more about the payload storage in the [Storage](../../storage/#payload-storage) section.
|
||||
If you need to have payloads along with the vectors, it is recommended to store it on the disc, and only keep [indexed fields](../../concepts/indexing/#payload-index) in RAM.
|
||||
Read more about the payload storage in the [Storage](../../concepts/storage/#payload-storage) section.
|
||||
|
||||
|
||||
## Storage focused configuration
|
||||
|
||||
If your priority is to serve large amount of vectors with an average search latency, it is recommended to configure [mmap storage](../../storage/#configuring-memmap-storage).
|
||||
If your priority is to serve large amount of vectors with an average search latency, it is recommended to configure [mmap storage](../../concepts/storage/#configuring-memmap-storage).
|
||||
In this case vectors will be stored on the disc in memory-mapped files, and only the most frequently used vectors will be kept in RAM.
|
||||
|
||||
The amount of available RAM will significantly affect the performance of the search.
|
||||
|
||||
@@ -3,6 +3,8 @@ title: Quick Start in Cloud
|
||||
weight: 10
|
||||
---
|
||||
|
||||
# Quickstart
|
||||
|
||||
## Create cluster
|
||||
|
||||
To start working with the Qdrant cloud platform, you have to create at least one cluster. You can do so on the clusters overview section by clicking the “Create” button.
|
||||
|
||||
@@ -3,6 +3,7 @@ title: Cluster scaling
|
||||
weight: 30
|
||||
---
|
||||
|
||||
# Cluster scaling
|
||||
|
||||
The amount of data is always growing and at some point you might need to upgrade the capacity of your cluster.
|
||||
There are different options for how it can be done.
|
||||
@@ -26,11 +27,11 @@ In such cases, horizontal scaling may be a more effective solution.
|
||||
Horizontal scaling, also known as horizontal expansion, is the process of increasing the capacity of a cluster by adding more nodes and distributing the load and data among them.
|
||||
The horizontal scaling at Qdrant starts on the collection level.
|
||||
You have to choose the number of shards you want to distribute your collection around while creating the collection.
|
||||
Please refer to the [sharding documentation](../../distributed_deployment/#sharding) section for details.
|
||||
Please refer to the [sharding documentation](../../guides/distributed_deployment/#sharding) section for details.
|
||||
|
||||
|
||||
Important: The number of shards means the maximum amount of nodes you can add to your cluster. In the beginning, all the shards can reside on one node.
|
||||
With the growing amount of data you can add nodes to your cluster and move shards to the dedicated nodes using the [cluster setup API](../../distributed_deployment/#cluster-scaling).
|
||||
With the growing amount of data you can add nodes to your cluster and move shards to the dedicated nodes using the [cluster setup API](../../guides/distributed_deployment/#cluster-scaling).
|
||||
|
||||
We, the Qdrant team, will be glad to consult you on an optimal strategy for scaling.
|
||||
[Let us know](mailto:cloud@qdrant.io) your needs and decide together on a proper solution. We plan to introduce an auto-scaling functionality. Since it is one of most desired features, it has a high priority on our Cloud roadmap.
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
---
|
||||
title: Concepts
|
||||
weight: 30
|
||||
# If the index.md file is empty, the link to the section will be hidden from the sidebar
|
||||
is_empty: true
|
||||
---
|
||||
+4
-5
@@ -3,7 +3,7 @@ title: Collections
|
||||
weight: 30
|
||||
---
|
||||
|
||||
## Collections
|
||||
# Collections
|
||||
|
||||
A collection is a named set of points (vectors with a payload) among which you can search.
|
||||
Vectors within the same collection must have the same dimensionality and be compared by a single metric.
|
||||
@@ -53,9 +53,9 @@ In addition to the required options, you can also specify custom values for the
|
||||
* `hnsw_config` - see [indexing](../indexing/#vector-index) for details.
|
||||
* `wal_config` - Write-Ahead-Log related configuration. See more details about [WAL](../storage/#versioning)
|
||||
* `optimizers_config` - see [optimizer](../optimizer) for details.
|
||||
* `shard_number` - which defines how many shards the collection should have. See [distributed deployment](../distributed_deployment#sharding) section for details.
|
||||
* `shard_number` - which defines how many shards the collection should have. See [distributed deployment](../../guides/distributed_deployment#sharding) section for details.
|
||||
* `on_disk_payload` - defines where to store payload data. If `true` - payload will be stored on disk only. Might be useful for limiting the RAM usage in case of large payload.
|
||||
* `quantization_config` - see [quantization](../quantization/#setting-up-quantization-in-qdrant) for details.
|
||||
* `quantization_config` - see [quantization](../../guides/quantization/#setting-up-quantization-in-qdrant) for details.
|
||||
|
||||
Default parameters for the optional collection parameters are defined in [configuration file](https://github.com/qdrant/qdrant/blob/master/config/config.yaml).
|
||||
|
||||
@@ -148,7 +148,7 @@ For rare use cases, it is possible to create a collection without any vector sto
|
||||
|
||||
For each named vector you can optionally specify
|
||||
[`hnsw_config`](../indexing/#vector-index) or
|
||||
[`quantization_config`](../quantization/#setting-up-quantization-in-qdrant) to
|
||||
[`quantization_config`](../../guides/quantization/#setting-up-quantization-in-qdrant) to
|
||||
deviate from the collection configuration. This can be useful to fine-tune
|
||||
search performance on a vector level.
|
||||
|
||||
@@ -273,7 +273,6 @@ In a production environment, it is sometimes necessary to switch different versi
|
||||
For example, when upgrading to a new version of the neural network.
|
||||
|
||||
There is no way to stop the service and rebuild the collection with new vectors in these situations.
|
||||
To avoid this, you can use aliases.
|
||||
Aliases are additional names for existing collections.
|
||||
All queries to the collection can also be done identically, using an alias instead of the collection name.
|
||||
|
||||
+2
@@ -3,6 +3,8 @@ title: Filtering
|
||||
weight: 60
|
||||
---
|
||||
|
||||
# Filtering
|
||||
|
||||
With Qdrant, you can set conditions when searching or retrieving points.
|
||||
For example, you can impose conditions on both the [payload](../payload) and the `id` of the point.
|
||||
|
||||
+2
@@ -3,6 +3,8 @@ title: Indexing
|
||||
weight: 90
|
||||
---
|
||||
|
||||
# Indexing
|
||||
|
||||
A key feature of Qdrant is the effective combination of vector and traditional indexes. It is essential to have this because for vector search to work effectively with filters, having vector index only is not enough. In simpler terms, a vector index speeds up vector search, and payload indexes speed up filtering.
|
||||
|
||||
The indexes in the segments exist independently, but the parameters of the indexes themselves are configured for the whole collection.
|
||||
+2
@@ -3,6 +3,8 @@ title: Optimizer
|
||||
weight: 70
|
||||
---
|
||||
|
||||
# Optimizer
|
||||
|
||||
It is much more efficient to apply changes in batches than perform each change individually, as many other databases do. Qdrant here is no exception. Since Qdrant operates with data structures that are not always easy to change, it is sometimes necessary to rebuild those structures completely.
|
||||
|
||||
Storage optimization in Qdrant occurs at the segment level (see [storage](../storage)).
|
||||
+2
@@ -3,6 +3,8 @@ title: Payload
|
||||
weight: 40
|
||||
---
|
||||
|
||||
# Payload
|
||||
|
||||
One of the significant features of Qdrant is the ability to store additional information along with vectors.
|
||||
This information is called `payload` in Qdrant terminology.
|
||||
|
||||
+2
@@ -3,6 +3,8 @@ title: Points
|
||||
weight: 40
|
||||
---
|
||||
|
||||
# Points
|
||||
|
||||
The points are the central entity that Qdrant operates with.
|
||||
A point is a record consisting of a vector and an optional [payload](../payload).
|
||||
|
||||
+1
-2
@@ -3,8 +3,7 @@ title: Search
|
||||
weight: 50
|
||||
---
|
||||
|
||||
|
||||
## Similarity search
|
||||
# Similarity search
|
||||
|
||||
Searching for the nearest vectors is at the core of many representational learning applications.
|
||||
Modern neural networks are trained to transform objects into vectors so that objects close in the real world appear close in vector space.
|
||||
+4
-2
@@ -3,13 +3,15 @@ title: Snapshots
|
||||
weight: 110
|
||||
---
|
||||
|
||||
*Available as of v0.8.4*
|
||||
# Snapshots
|
||||
|
||||
*Available since v0.8.4*
|
||||
|
||||
Snapshots are performed on a per collection basis and consist in a `tar` archive file containing the necessary data to restore the collection at the time of the snapshot.
|
||||
|
||||
This feature can be used to archive data or easily replicate an existing deployment.
|
||||
|
||||
The target directory used to store generated snapshots is controlled through the [configuration](../configuration) or using the ENV variable: `QDRANT__STORAGE__SNAPSHOT_PATH=./snapshots`.
|
||||
The target directory used to store generated snapshots is controlled through the [configuration](../../guides/configuration) or using the ENV variable: `QDRANT__STORAGE__SNAPSHOT_PATH=./snapshots`.
|
||||
|
||||
```yaml
|
||||
storage:
|
||||
+4
-2
@@ -3,6 +3,8 @@ title: Storage
|
||||
weight: 80
|
||||
---
|
||||
|
||||
# Storage
|
||||
|
||||
All data within one collection is divided into segments.
|
||||
Each segment has its independent vector and payload storage as well as indexes.
|
||||
|
||||
@@ -79,7 +81,7 @@ This is the recommended way, in case your Qdrant instance operates with fast dis
|
||||
|
||||
There are two ways to do this:
|
||||
|
||||
1. You can set the threshold globally in the [configuration file](../configuration/). The parameter is called `memmap_threshold_kb`.
|
||||
1. You can set the threshold globally in the [configuration file](../../guides/configuration/). The parameter is called `memmap_threshold_kb`.
|
||||
2. You can set the threshold for each collection separately during [creation](../collections/#create-collection) or [update](../collections/#update-collection-parameters).
|
||||
|
||||
```http
|
||||
@@ -161,7 +163,7 @@ If you need to query vectors with some payload-based conditions - checking value
|
||||
In this scenario, we recommend creating a payload index for each field used in filtering conditions to avoid disk access.
|
||||
Once you create the field index, Qdrant will preserve all values of the indexed field in RAM regardless of the payload storage type.
|
||||
|
||||
You can specify the desired type of payload storage with [configuration file](../configuration/) or with collection parameter `on_disk_payload` during [creation](../collections/#create-collection) of the collection.
|
||||
You can specify the desired type of payload storage with [configuration file](../../guides/configuration/) or with collection parameter `on_disk_payload` during [creation](../collections/#create-collection) of the collection.
|
||||
|
||||
## Versioning
|
||||
|
||||
@@ -0,0 +1,6 @@
|
||||
---
|
||||
title: Guides
|
||||
weight: 40
|
||||
# If the index.md file is empty, the link to the section will be hidden from the sidebar
|
||||
is_empty: true
|
||||
---
|
||||
+4
-2
@@ -1,8 +1,10 @@
|
||||
---
|
||||
title: Administration
|
||||
weight: 170
|
||||
weight: 10
|
||||
---
|
||||
|
||||
# Administration
|
||||
|
||||
Qdrant exposes administration tools which enable to modify at runtime the behavior of a qdrant instance without changing its configuration manually.
|
||||
|
||||
## Locking
|
||||
@@ -40,7 +42,7 @@ example, by deleting a collection. After resolving Qdrant can be restarted
|
||||
normally to continue operation.
|
||||
|
||||
In recovery mode, collection operations are limited to
|
||||
[deleting](https://qdrant.tech/documentation/collections/#delete-collection) a
|
||||
[deleting](../../concepts/collections/#delete-collection) a
|
||||
collection. That is because only collection metadata is loaded during recovery.
|
||||
|
||||
To enable recovery mode with the Qdrant Docker image you must set the
|
||||
+2
@@ -3,6 +3,8 @@ title: Configuration
|
||||
weight: 160
|
||||
---
|
||||
|
||||
# Configuration
|
||||
|
||||
To change or correct Qdrant's behavior, default collection settings, and network interface parameters, you can use the configuration file.
|
||||
|
||||
Default configuration file is located in [config/config.yaml](https://github.com/qdrant/qdrant/blob/master/config/config.yaml).
|
||||
+4
-2
@@ -3,7 +3,9 @@ title: Distributed deployment
|
||||
weight: 100
|
||||
---
|
||||
|
||||
As of v0.8.0 Qdrant supports a distributed deployment mode.
|
||||
# Distributed deployment
|
||||
|
||||
Since version v0.8.0 Qdrant supports a distributed deployment mode.
|
||||
In this mode, multiple Qdrant services communicate with each other to distribute the data across the peers to extend the storage capabilities and increase stability.
|
||||
|
||||
To enable distributed deployment - enable the cluster mode in the [configuration](../configuration) or using the ENV variable: `QDRANT__CLUSTER__ENABLED=true`.
|
||||
@@ -108,7 +110,7 @@ Qdrant is using the [Raft](https://raft.github.io/) consensus protocol to mainta
|
||||
|
||||
Operation with points, on the other hand, are not going through the consensus infrastructure.
|
||||
Qdrant is not intended to have strong transaction guarantees, which allows it to perform point operations with low overhead.
|
||||
In practice, it means that Qdrant does not guarantee atomic distributed updates but allows you to wait until the [operation is complete](../points/#awaiting-result) to see the results of your writes.
|
||||
In practice, it means that Qdrant does not guarantee atomic distributed updates but allows you to wait until the [operation is complete](../../concepts/points/#awaiting-result) to see the results of your writes.
|
||||
|
||||
Collection operations, on the contrary, are part of the consensus which guarantees that all operations are durable and eventually executed by all nodes.
|
||||
In practice it means that a majority of node agree on what operations should be applied before the service will perform them.
|
||||
+2
@@ -3,6 +3,8 @@ title: Monitoring
|
||||
weight: 155
|
||||
---
|
||||
|
||||
# Monitoring
|
||||
|
||||
Qdrant exposes its metrics in a Prometheus format, so you can integrate them easily
|
||||
with the compatible tools and monitor Qdrant with your own monitoring system. You can
|
||||
use the `/metrics` endpoint and configure it as a scrape target.
|
||||
+2
@@ -3,6 +3,8 @@ title: Quantization
|
||||
weight: 120
|
||||
---
|
||||
|
||||
# Quantization
|
||||
|
||||
Quantization is an optional feature in Qdrant that enables efficient storage and search of high-dimensional vectors.
|
||||
By transforming original vectors into a new representations, quantization compresses data while preserving close to original relative distances between vectors.
|
||||
Different quantization methods have different mechanics and tradeoffs. We will cover them in this section.
|
||||
+2
@@ -3,6 +3,8 @@ title: Security
|
||||
weight: 165
|
||||
---
|
||||
|
||||
# Security
|
||||
|
||||
There are various ways to secure your own Qdrant instance.
|
||||
|
||||
## Authentication
|
||||
+1
@@ -3,6 +3,7 @@ title: Telemetry
|
||||
weight: 150
|
||||
---
|
||||
|
||||
# Telemetry
|
||||
|
||||
Qdrant collects anonymized usage statistics from users in order to improve the engine.
|
||||
You can [deactivate](#deactivate-telemetry) at any time, and any data that has already been collected can be [deleted on request](#request-information-deletion).
|
||||
@@ -1,9 +1,11 @@
|
||||
---
|
||||
title: Install
|
||||
title: Installation
|
||||
weight: 20
|
||||
---
|
||||
|
||||
## With Docker
|
||||
# Installation
|
||||
|
||||
## Docker
|
||||
|
||||
The easiest way to start using Qdrant is to run it from a ready-made Docker image.
|
||||
The latest versions are always available on [DockerHub](https://hub.docker.com/r/qdrant/qdrant/tags?page=1&ordering=last_updated).
|
||||
|
||||
@@ -1,295 +0,0 @@
|
||||
---
|
||||
title: Integrations
|
||||
weight: 140
|
||||
---
|
||||
Qdrant is a vector database performing an approximate nearest neighbours search on neural embeddings. It can work perfectly fine
|
||||
as a standalone system, yet, in some cases, you may find it easier to implement your semantic search application using some
|
||||
higher-level libraries. Some of such projects provide ready-to-go integrations and here is a curated list of them.
|
||||
|
||||
## LangChain
|
||||
|
||||
LangChain is a library that makes developing Large Language Models based applications much easier. It unifies the interfaces
|
||||
to different libraries, including major embedding providers and Qdrant. Using LangChain, you can focus on the business value
|
||||
instead of writing the boilerplate.
|
||||
|
||||
Langchain comes with the Qdrant integration by default. It might be installed with pip:
|
||||
|
||||
```bash
|
||||
pip install langchain
|
||||
```
|
||||
|
||||
Qdrant acts as a vector index that may store the embeddings with the documents used to generate them. There are various ways
|
||||
how to use it, but calling `Qdrant.from_texts` is probably the most straightforward way how to get started:
|
||||
|
||||
```python
|
||||
from langchain.vectorstores import Qdrant
|
||||
from langchain.embeddings import HuggingFaceEmbeddings
|
||||
|
||||
embeddings = HuggingFaceEmbeddings(
|
||||
model_name="sentence-transformers/all-mpnet-base-v2"
|
||||
)
|
||||
doc_store = Qdrant.from_texts(
|
||||
texts, embeddings, url="<qdrant-url>", api_key="<qdrant-api-key>", collection_name="texts"
|
||||
)
|
||||
```
|
||||
|
||||
Calling `Qdrant.from_documents` or `Qdrant.from_texts` will always recreate the collection and remove all the existing points.
|
||||
That's fine for some experiments, but you'll prefer not to start from scratch every single time in a real-world scenario.
|
||||
If you prefer reusing an existing collection, you can create an instance of Qdrant on your own:
|
||||
|
||||
```
|
||||
import qdrant_client
|
||||
|
||||
client = qdrant_client.QdrantClient(
|
||||
"<qdrant-url>",
|
||||
api_key="<qdrant-api-key>", # For Qdrant Cloud, None for local instance
|
||||
)
|
||||
|
||||
doc_store = Qdrant(
|
||||
client=client, collection_name="texts",
|
||||
embedding_function=embeddings.embed_query,
|
||||
)
|
||||
```
|
||||
|
||||
If you'd like to know more about running Qdrant in a LangChain-based application, please read our article
|
||||
[Question Answering with LangChain and Qdrant without boilerplate](/articles/langchain-integration/). Some more information
|
||||
might also be found in the [LangChain documentation](https://python.langchain.com/en/latest/modules/indexes/vectorstores/examples/qdrant.html).
|
||||
|
||||
## LlamaIndex (GPT Index)
|
||||
|
||||
LlamaIndex (formerly GPT Index) acts as an interface between your external data and Large Language Models. So you can bring your
|
||||
private data and augment LLMs with it. LlamaIndex simplifies data ingestion and indexing, integrating Qdrant as a vector index.
|
||||
|
||||
Installing LlamaIndex is straightforward if we use pip as a package manager:
|
||||
|
||||
```bash
|
||||
pip install llama-index
|
||||
```
|
||||
|
||||
LlamaIndex requires providing an instance of `QdrantClient`, so it can interact with Qdrant server.
|
||||
|
||||
```python
|
||||
from llama_index import GPTQdrantIndex
|
||||
|
||||
import qdrant_client
|
||||
|
||||
client = qdrant_client.QdrantClient(
|
||||
"<qdrant-url>",
|
||||
api_key="<qdrant-api-key>", # For Qdrant Cloud, None for local instance
|
||||
)
|
||||
|
||||
index = GPTQdrantIndex.from_documents(documents, client=client, collection_name="documents")
|
||||
```
|
||||
|
||||
The library [comes with a notebook](https://github.com/jerryjliu/llama_index/blob/main/docs/examples/vector_stores/QdrantIndexDemo.ipynb)
|
||||
that shows an end-to-end example of how to use Qdrant within LlamaIndex.
|
||||
|
||||
## DocArray
|
||||
You can use Qdrant natively in DocArray, where Qdrant serves as a high-performance document store to enable scalable vector search.
|
||||
|
||||
DocArray is a library from Jina AI for nested, unstructured data in transit, including text, image, audio, video, 3D mesh, etc.
|
||||
It allows deep-learning engineers to efficiently process, embed, search, recommend, store, and transfer the data with a Pythonic API.
|
||||
|
||||
|
||||
To install DocArray with Qdrant support, please do
|
||||
|
||||
```bash
|
||||
pip install "docarray[qdrant]"
|
||||
```
|
||||
|
||||
More information can be found in [DocArray's documentations](https://docarray.jina.ai/advanced/document-store/qdrant/).
|
||||
|
||||
## Haystack
|
||||
|
||||
[Haystack](https://haystack.deepset.ai/) serves as a comprehensive NLP framework, offering a modular methodology for constructing
|
||||
cutting-edge generative AI, QA, and semantic knowledge base search systems. A critical element in contemporary NLP systems is an
|
||||
efficient database for storing and retrieving extensive text data. Vector databases excel in this role, as they house vector
|
||||
representations of text and implement effective methods for swift retrieval. Thus, we are happy to announce the integration
|
||||
with Haystack - `QdrantDocumentStore`. This document store is unique, as it is maintained externally by the Qdrant team.
|
||||
|
||||
The new document store comes as a separate package and can be updated independently of Haystack:
|
||||
|
||||
```bash
|
||||
pip install qdrant-haystack
|
||||
```
|
||||
|
||||
`QdrantDocumentStore` supports [all the configuration properties](/documentation/collections/#create-collection) available in
|
||||
the Qdrant Python client. If you want to customize the default configuration of the collection used under the hood, you can
|
||||
provide that settings when you create an instance of the `QdrantDocumentStore`. For example, if you'd like to enable the
|
||||
Scalar Quantization, you'd make that in the following way:
|
||||
|
||||
```python
|
||||
from qdrant_haystack.document_stores import QdrantDocumentStore
|
||||
from qdrant_client.http import models
|
||||
|
||||
document_store = QdrantDocumentStore(
|
||||
":memory:",
|
||||
index="Document",
|
||||
embedding_dim=512,
|
||||
recreate_index=True,
|
||||
quantization_config=models.ScalarQuantization(
|
||||
scalar=models.ScalarQuantizationConfig(
|
||||
type=models.ScalarType.INT8,
|
||||
quantile=0.99,
|
||||
always_ram=True,
|
||||
),
|
||||
),
|
||||
)
|
||||
```
|
||||
|
||||
## txtai
|
||||
Qdrant might be also used as an embedding backend in [txtai](https://neuml.github.io/txtai/) semantic applications.
|
||||
|
||||
txtai simplifies building AI-powered semantic search applications using Transformers. It leverages the neural embeddings and their
|
||||
properties to encode high-dimensional data in a lower-dimensional space and allows to find similar objects based on their embeddings'
|
||||
proximity.
|
||||
|
||||
Qdrant is not built-in txtai backend and requires installing an additional dependency:
|
||||
|
||||
```bash
|
||||
pip install qdrant-txtai
|
||||
```
|
||||
|
||||
The examples and some more information might be found in [qdrant-txtai repository](https://github.com/qdrant/qdrant-txtai).
|
||||
|
||||
## FiftyOne
|
||||
|
||||
[FiftyOne](https://voxel51.com/) is an open-source toolkit designed to enhance computer vision workflows by optimizing dataset quality
|
||||
and providing valuable insights about your models. FiftyOne 0.20, which includes a native integration with Qdrant, supporting workflows
|
||||
like [image similarity search](https://docs.voxel51.com/user_guide/brain.html#image-similarity) and
|
||||
[text search](https://docs.voxel51.com/user_guide/brain.html#text-similarity).
|
||||
|
||||
Qdrant helps FiftyOne to find the most similar images in the dataset using vector embeddings.
|
||||
|
||||
FiftyOne is available as a Python package that might be installed in the following way:
|
||||
|
||||
```bash
|
||||
pip install fiftyone
|
||||
```
|
||||
|
||||
Please check out the documentation of FiftyOne on [Qdrant integration](https://docs.voxel51.com/integrations/qdrant.html).
|
||||
|
||||
## Cohere
|
||||
Qdrant is compatible with Cohere [co.embed API](https://docs.cohere.ai/reference/embed) and it's official Python SDK that
|
||||
might be installed as any other package:
|
||||
|
||||
```bash
|
||||
pip install cohere
|
||||
```
|
||||
|
||||
The embeddings returned by co.embed API might be used directly in the Qdrant client's calls:
|
||||
|
||||
```python
|
||||
import cohere
|
||||
import qdrant_client
|
||||
|
||||
from qdrant_client.http.models import Batch
|
||||
|
||||
cohere_client = cohere.Client("<< your_api_key >>")
|
||||
qdrant_client = qdrant_client.QdrantClient()
|
||||
qdrant_client.upsert(
|
||||
collection_name="MyCollection",
|
||||
points=Batch(
|
||||
ids=[1],
|
||||
vectors=cohere_client.embed(
|
||||
model="large",
|
||||
texts=["The best vector database"],
|
||||
).embeddings,
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
If you are interested in seeing an end-to-end project created with co.embed API and Qdrant, please check out the
|
||||
"[Question Answering as a Service with Cohere and Qdrant](https://qdrant.tech/articles/qa-with-cohere-and-qdrant/)" article.
|
||||
|
||||
## OpenAI
|
||||
Qdrant can also easily work with [OpenAI embeddings](https://beta.openai.com/docs/guides/embeddings/embeddings). There is an
|
||||
official OpenAI Python package that simplifies obtaining them, and it might be installed with pip:
|
||||
|
||||
```bash
|
||||
pip install openai
|
||||
```
|
||||
|
||||
Once installed, the package exposes the method allowing to retrieve the embedding for given text. OpenAI requires an API key
|
||||
that has to be provided either as an environmental variable `OPENAI_API_KEY` or set in the source code directly, as
|
||||
presented below:
|
||||
|
||||
```python
|
||||
import openai
|
||||
import qdrant_client
|
||||
|
||||
from qdrant_client.http.models import Batch
|
||||
|
||||
# Provide OpenAI API key and choose one of the available models:
|
||||
# https://beta.openai.com/docs/models/overview
|
||||
openai.api_key = "<< your_api_key >>"
|
||||
embedding_model = "text-embedding-ada-002"
|
||||
|
||||
response = openai.Embedding.create(
|
||||
input="The best vector database",
|
||||
model=embedding_model,
|
||||
)
|
||||
|
||||
qdrant_client = qdrant_client.QdrantClient()
|
||||
qdrant_client.upsert(
|
||||
collection_name="MyCollection",
|
||||
points=Batch(
|
||||
ids=[1],
|
||||
vectors=[response["data"][0]["embedding"]],
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
## Aleph Alpha
|
||||
Aleph Alpha is a multimodal and multilingual embeddings' provider. Their API allows creating the embeddings for text and images, both
|
||||
in the same latent space. They maintain an [official Python client](https://github.com/Aleph-Alpha/aleph-alpha-client) that might be
|
||||
installed with pip:
|
||||
|
||||
```bash
|
||||
pip install aleph-alpha-client
|
||||
```
|
||||
|
||||
There is both synchronous and asynchronous client available. Obtaining the embeddings for an image and storing it into Qdrant might
|
||||
be done in the following way:
|
||||
|
||||
```python
|
||||
import qdrant_client
|
||||
|
||||
from aleph_alpha_client import (
|
||||
Prompt,
|
||||
AsyncClient,
|
||||
SemanticEmbeddingRequest,
|
||||
SemanticRepresentation,
|
||||
ImagePrompt
|
||||
)
|
||||
from qdrant_client.http.models import Batch
|
||||
|
||||
aa_token = "<< your_token >>"
|
||||
model = "luminous-base"
|
||||
|
||||
qdrant_client = qdrant_client.QdrantClient()
|
||||
async with AsyncClient(token=aa_token) as client:
|
||||
prompt = ImagePrompt.from_file("./path/to/the/image.jpg")
|
||||
prompt = Prompt.from_image(prompt)
|
||||
|
||||
query_params = {
|
||||
"prompt": prompt,
|
||||
"representation": SemanticRepresentation.Symmetric,
|
||||
"compress_to_size": 128,
|
||||
}
|
||||
query_request = SemanticEmbeddingRequest(**query_params)
|
||||
query_response = await client.semantic_embed(
|
||||
request=query_request, model=model
|
||||
)
|
||||
|
||||
qdrant_client.upsert(
|
||||
collection_name="MyCollection",
|
||||
points=Batch(
|
||||
ids=[1],
|
||||
vectors=[query_response.embedding],
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
If we wanted to create text embeddings with the same model, we wouldn't use `ImagePrompt.from_file`, but simply provide the input
|
||||
text into the `Prompt.from_text` method.
|
||||
@@ -0,0 +1,12 @@
|
||||
---
|
||||
title: Integrations
|
||||
weight: 60
|
||||
# If the index.md file is empty, the link to the section will be hidden from the sidebar
|
||||
is_empty: false
|
||||
---
|
||||
|
||||
# Integrations
|
||||
|
||||
Qdrant is a vector database performing an approximate nearest neighbours search on neural embeddings. It can work perfectly fine
|
||||
as a standalone system, yet, in some cases, you may find it easier to implement your semantic search application using some
|
||||
higher-level libraries. Some of such projects provide ready-to-go integrations and here is a curated list of them.
|
||||
@@ -0,0 +1,57 @@
|
||||
---
|
||||
title: Aleph Alpha
|
||||
weight: 900
|
||||
---
|
||||
|
||||
Aleph Alpha is a multimodal and multilingual embeddings' provider. Their API allows creating the embeddings for text and images, both
|
||||
in the same latent space. They maintain an [official Python client](https://github.com/Aleph-Alpha/aleph-alpha-client) that might be
|
||||
installed with pip:
|
||||
|
||||
```bash
|
||||
pip install aleph-alpha-client
|
||||
```
|
||||
|
||||
There is both synchronous and asynchronous client available. Obtaining the embeddings for an image and storing it into Qdrant might
|
||||
be done in the following way:
|
||||
|
||||
```python
|
||||
import qdrant_client
|
||||
|
||||
from aleph_alpha_client import (
|
||||
Prompt,
|
||||
AsyncClient,
|
||||
SemanticEmbeddingRequest,
|
||||
SemanticRepresentation,
|
||||
ImagePrompt
|
||||
)
|
||||
from qdrant_client.http.models import Batch
|
||||
|
||||
aa_token = "<< your_token >>"
|
||||
model = "luminous-base"
|
||||
|
||||
qdrant_client = qdrant_client.QdrantClient()
|
||||
async with AsyncClient(token=aa_token) as client:
|
||||
prompt = ImagePrompt.from_file("./path/to/the/image.jpg")
|
||||
prompt = Prompt.from_image(prompt)
|
||||
|
||||
query_params = {
|
||||
"prompt": prompt,
|
||||
"representation": SemanticRepresentation.Symmetric,
|
||||
"compress_to_size": 128,
|
||||
}
|
||||
query_request = SemanticEmbeddingRequest(**query_params)
|
||||
query_response = await client.semantic_embed(
|
||||
request=query_request, model=model
|
||||
)
|
||||
|
||||
qdrant_client.upsert(
|
||||
collection_name="MyCollection",
|
||||
points=Batch(
|
||||
ids=[1],
|
||||
vectors=[query_response.embedding],
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
If we wanted to create text embeddings with the same model, we wouldn't use `ImagePrompt.from_file`, but simply provide the input
|
||||
text into the `Prompt.from_text` method.
|
||||
@@ -0,0 +1,39 @@
|
||||
---
|
||||
title: Cohere
|
||||
weight: 700
|
||||
---
|
||||
|
||||
# Cohere
|
||||
|
||||
Qdrant is compatible with Cohere [co.embed API](https://docs.cohere.ai/reference/embed) and it's official Python SDK that
|
||||
might be installed as any other package:
|
||||
|
||||
```bash
|
||||
pip install cohere
|
||||
```
|
||||
|
||||
The embeddings returned by co.embed API might be used directly in the Qdrant client's calls:
|
||||
|
||||
```python
|
||||
import cohere
|
||||
import qdrant_client
|
||||
|
||||
from qdrant_client.http.models import Batch
|
||||
|
||||
cohere_client = cohere.Client("<< your_api_key >>")
|
||||
qdrant_client = qdrant_client.QdrantClient()
|
||||
qdrant_client.upsert(
|
||||
collection_name="MyCollection",
|
||||
points=Batch(
|
||||
ids=[1],
|
||||
vectors=cohere_client.embed(
|
||||
model="large",
|
||||
texts=["The best vector database"],
|
||||
).embeddings,
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
If you are interested in seeing an end-to-end project created with co.embed API and Qdrant, please check out the
|
||||
"[Question Answering as a Service with Cohere and Qdrant](https://qdrant.tech/articles/qa-with-cohere-and-qdrant/)" article.
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
---
|
||||
title: DocArray
|
||||
weight: 300
|
||||
---
|
||||
|
||||
# DocArray
|
||||
You can use Qdrant natively in DocArray, where Qdrant serves as a high-performance document store to enable scalable vector search.
|
||||
|
||||
DocArray is a library from Jina AI for nested, unstructured data in transit, including text, image, audio, video, 3D mesh, etc.
|
||||
It allows deep-learning engineers to efficiently process, embed, search, recommend, store, and transfer the data with a Pythonic API.
|
||||
|
||||
|
||||
To install DocArray with Qdrant support, please do
|
||||
|
||||
```bash
|
||||
pip install "docarray[qdrant]"
|
||||
```
|
||||
|
||||
More information can be found in [DocArray's documentations](https://docarray.jina.ai/advanced/document-store/qdrant/).
|
||||
@@ -0,0 +1,22 @@
|
||||
---
|
||||
title: FiftyOne
|
||||
weight: 600
|
||||
---
|
||||
|
||||
# FiftyOne
|
||||
|
||||
[FiftyOne](https://voxel51.com/) is an open-source toolkit designed to enhance computer vision workflows by optimizing dataset quality
|
||||
and providing valuable insights about your models. FiftyOne 0.20, which includes a native integration with Qdrant, supporting workflows
|
||||
like [image similarity search](https://docs.voxel51.com/user_guide/brain.html#image-similarity) and
|
||||
[text search](https://docs.voxel51.com/user_guide/brain.html#text-similarity).
|
||||
|
||||
Qdrant helps FiftyOne to find the most similar images in the dataset using vector embeddings.
|
||||
|
||||
FiftyOne is available as a Python package that might be installed in the following way:
|
||||
|
||||
```bash
|
||||
pip install fiftyone
|
||||
```
|
||||
|
||||
Please check out the documentation of FiftyOne on [Qdrant integration](https://docs.voxel51.com/integrations/qdrant.html).
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
---
|
||||
title: Haystack
|
||||
weight: 400
|
||||
---
|
||||
|
||||
# Haystack
|
||||
|
||||
[Haystack](https://haystack.deepset.ai/) serves as a comprehensive NLP framework, offering a modular methodology for constructing
|
||||
cutting-edge generative AI, QA, and semantic knowledge base search systems. A critical element in contemporary NLP systems is an
|
||||
efficient database for storing and retrieving extensive text data. Vector databases excel in this role, as they house vector
|
||||
representations of text and implement effective methods for swift retrieval. Thus, we are happy to announce the integration
|
||||
with Haystack - `QdrantDocumentStore`. This document store is unique, as it is maintained externally by the Qdrant team.
|
||||
|
||||
The new document store comes as a separate package and can be updated independently of Haystack:
|
||||
|
||||
```bash
|
||||
pip install qdrant-haystack
|
||||
```
|
||||
|
||||
`QdrantDocumentStore` supports [all the configuration properties](/documentation/collections/#create-collection) available in
|
||||
the Qdrant Python client. If you want to customize the default configuration of the collection used under the hood, you can
|
||||
provide that settings when you create an instance of the `QdrantDocumentStore`. For example, if you'd like to enable the
|
||||
Scalar Quantization, you'd make that in the following way:
|
||||
|
||||
```python
|
||||
from qdrant_haystack.document_stores import QdrantDocumentStore
|
||||
from qdrant_client.http import models
|
||||
|
||||
document_store = QdrantDocumentStore(
|
||||
":memory:",
|
||||
index="Document",
|
||||
embedding_dim=512,
|
||||
recreate_index=True,
|
||||
quantization_config=models.ScalarQuantization(
|
||||
scalar=models.ScalarQuantizationConfig(
|
||||
type=models.ScalarType.INT8,
|
||||
quantile=0.99,
|
||||
always_ram=True,
|
||||
),
|
||||
),
|
||||
)
|
||||
```
|
||||
@@ -0,0 +1,53 @@
|
||||
---
|
||||
title: LangChain
|
||||
weight: 100
|
||||
---
|
||||
|
||||
# LangChain
|
||||
|
||||
LangChain is a library that makes developing Large Language Models based applications much easier. It unifies the interfaces
|
||||
to different libraries, including major embedding providers and Qdrant. Using LangChain, you can focus on the business value
|
||||
instead of writing the boilerplate.
|
||||
|
||||
Langchain comes with the Qdrant integration by default. It might be installed with pip:
|
||||
|
||||
```bash
|
||||
pip install langchain
|
||||
```
|
||||
|
||||
Qdrant acts as a vector index that may store the embeddings with the documents used to generate them. There are various ways
|
||||
how to use it, but calling `Qdrant.from_texts` is probably the most straightforward way how to get started:
|
||||
|
||||
```python
|
||||
from langchain.vectorstores import Qdrant
|
||||
from langchain.embeddings import HuggingFaceEmbeddings
|
||||
|
||||
embeddings = HuggingFaceEmbeddings(
|
||||
model_name="sentence-transformers/all-mpnet-base-v2"
|
||||
)
|
||||
doc_store = Qdrant.from_texts(
|
||||
texts, embeddings, url="<qdrant-url>", api_key="<qdrant-api-key>", collection_name="texts"
|
||||
)
|
||||
```
|
||||
|
||||
Calling `Qdrant.from_documents` or `Qdrant.from_texts` will always recreate the collection and remove all the existing points.
|
||||
That's fine for some experiments, but you'll prefer not to start from scratch every single time in a real-world scenario.
|
||||
If you prefer reusing an existing collection, you can create an instance of Qdrant on your own:
|
||||
|
||||
```
|
||||
import qdrant_client
|
||||
|
||||
client = qdrant_client.QdrantClient(
|
||||
"<qdrant-url>",
|
||||
api_key="<qdrant-api-key>", # For Qdrant Cloud, None for local instance
|
||||
)
|
||||
|
||||
doc_store = Qdrant(
|
||||
client=client, collection_name="texts",
|
||||
embedding_function=embeddings.embed_query,
|
||||
)
|
||||
```
|
||||
|
||||
If you'd like to know more about running Qdrant in a LangChain-based application, please read our article
|
||||
[Question Answering with LangChain and Qdrant without boilerplate](/articles/langchain-integration/). Some more information
|
||||
might also be found in the [LangChain documentation](https://python.langchain.com/en/latest/modules/indexes/vectorstores/examples/qdrant.html).
|
||||
@@ -0,0 +1,33 @@
|
||||
---
|
||||
title: LlamaIndex
|
||||
weight: 200
|
||||
---
|
||||
|
||||
# LlamaIndex (GPT Index)
|
||||
|
||||
LlamaIndex (formerly GPT Index) acts as an interface between your external data and Large Language Models. So you can bring your
|
||||
private data and augment LLMs with it. LlamaIndex simplifies data ingestion and indexing, integrating Qdrant as a vector index.
|
||||
|
||||
Installing LlamaIndex is straightforward if we use pip as a package manager:
|
||||
|
||||
```bash
|
||||
pip install llama-index
|
||||
```
|
||||
|
||||
LlamaIndex requires providing an instance of `QdrantClient`, so it can interact with Qdrant server.
|
||||
|
||||
```python
|
||||
from llama_index import GPTQdrantIndex
|
||||
|
||||
import qdrant_client
|
||||
|
||||
client = qdrant_client.QdrantClient(
|
||||
"<qdrant-url>",
|
||||
api_key="<qdrant-api-key>", # For Qdrant Cloud, None for local instance
|
||||
)
|
||||
|
||||
index = GPTQdrantIndex.from_documents(documents, client=client, collection_name="documents")
|
||||
```
|
||||
|
||||
The library [comes with a notebook](https://github.com/jerryjliu/llama_index/blob/main/docs/examples/vector_stores/QdrantIndexDemo.ipynb)
|
||||
that shows an end-to-end example of how to use Qdrant within LlamaIndex.
|
||||
@@ -0,0 +1,44 @@
|
||||
---
|
||||
title: OpenAI
|
||||
weight: 800
|
||||
---
|
||||
|
||||
# OpenAI
|
||||
|
||||
Qdrant can also easily work with [OpenAI embeddings](https://beta.openai.com/docs/guides/embeddings/embeddings). There is an
|
||||
official OpenAI Python package that simplifies obtaining them, and it might be installed with pip:
|
||||
|
||||
```bash
|
||||
pip install openai
|
||||
```
|
||||
|
||||
Once installed, the package exposes the method allowing to retrieve the embedding for given text. OpenAI requires an API key
|
||||
that has to be provided either as an environmental variable `OPENAI_API_KEY` or set in the source code directly, as
|
||||
presented below:
|
||||
|
||||
```python
|
||||
import openai
|
||||
import qdrant_client
|
||||
|
||||
from qdrant_client.http.models import Batch
|
||||
|
||||
# Provide OpenAI API key and choose one of the available models:
|
||||
# https://beta.openai.com/docs/models/overview
|
||||
openai.api_key = "<< your_api_key >>"
|
||||
embedding_model = "text-embedding-ada-002"
|
||||
|
||||
response = openai.Embedding.create(
|
||||
input="The best vector database",
|
||||
model=embedding_model,
|
||||
)
|
||||
|
||||
qdrant_client = qdrant_client.QdrantClient()
|
||||
qdrant_client.upsert(
|
||||
collection_name="MyCollection",
|
||||
points=Batch(
|
||||
ids=[1],
|
||||
vectors=[response["data"][0]["embedding"]],
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
---
|
||||
title: txtai
|
||||
weight: 500
|
||||
---
|
||||
|
||||
# txtai
|
||||
|
||||
Qdrant might be also used as an embedding backend in [txtai](https://neuml.github.io/txtai/) semantic applications.
|
||||
|
||||
txtai simplifies building AI-powered semantic search applications using Transformers. It leverages the neural embeddings and their
|
||||
properties to encode high-dimensional data in a lower-dimensional space and allows to find similar objects based on their embeddings'
|
||||
proximity.
|
||||
|
||||
Qdrant is not built-in txtai backend and requires installing an additional dependency:
|
||||
|
||||
```bash
|
||||
pip install qdrant-txtai
|
||||
```
|
||||
|
||||
The examples and some more information might be found in [qdrant-txtai repository](https://github.com/qdrant/qdrant-txtai).
|
||||
|
||||
|
||||
+1
-1
@@ -1,5 +1,5 @@
|
||||
---
|
||||
title: API Clients
|
||||
title: Python Client
|
||||
weight: 40
|
||||
draft: true
|
||||
---
|
||||
@@ -1,8 +1,10 @@
|
||||
---
|
||||
title: Quick Start
|
||||
title: Quickstart
|
||||
weight: 10
|
||||
---
|
||||
|
||||
# Getting started with Qdrant
|
||||
|
||||
## Installation
|
||||
|
||||
The easiest way to use Qdrant is to run a pre-built image. To do this, make sure Docker is installed on your system.
|
||||
|
||||
@@ -1,5 +0,0 @@
|
||||
---
|
||||
title: Quick Start - Python
|
||||
weight: 12
|
||||
draft: true
|
||||
---
|
||||
@@ -0,0 +1,10 @@
|
||||
---
|
||||
title: Tutorials
|
||||
weight: 50
|
||||
# If the index.md file is empty, the link to the section will be hidden from the sidebar
|
||||
is_empty: true
|
||||
---
|
||||
|
||||
# Tutorials
|
||||
|
||||
This section contains a collection of how-to guides and tutorials for different use cases of Qdrant.
|
||||
+2
-4
@@ -1,11 +1,9 @@
|
||||
---
|
||||
title: How To
|
||||
weight: 130
|
||||
title: How to
|
||||
weight: 100
|
||||
---
|
||||
|
||||
|
||||
This section contains a collection of how-to guides and tutorials for different use cases of Qdrant.
|
||||
|
||||
## Optimize qdrant
|
||||
|
||||
Different use cases have different requirements for balancing between memory, speed, and precision.
|
||||
@@ -1,6 +1,6 @@
|
||||
---
|
||||
image: "content/images/logos/open-ai-logo"
|
||||
origin_url: "https://qdrant.tech/documentation/integrations/#openai"
|
||||
origin_url: "https://qdrant.tech/documentation/integrations/openai"
|
||||
name: "OpenAI"
|
||||
sitemapExclude: True
|
||||
---
|
||||
@@ -0,0 +1,7 @@
|
||||
---
|
||||
#Delimiter files are used to separate the list of documentation pages into sections.
|
||||
title: "{{ replace .Name "-" " " | title }}"
|
||||
type: delimiter
|
||||
weight: 0 # Change this weight to change order of sections
|
||||
sitemapExclude: True
|
||||
---
|
||||
@@ -0,0 +1,7 @@
|
||||
---
|
||||
# External link template
|
||||
title: "{{ replace .Name "-" " " | title }}"
|
||||
type: external-link
|
||||
external_url: https://github.com/qdrant/qdrant # Change this link to your external link
|
||||
sitemapExclude: True
|
||||
---
|
||||
@@ -1,13 +1,11 @@
|
||||
{{- partial "header.html" . -}}
|
||||
|
||||
{{- partial "second_header.html" . -}}
|
||||
|
||||
|
||||
<section>
|
||||
<div class="auto-container pt-4 pt-md-5">
|
||||
<section class="documentation">
|
||||
<div class="container-xxl pt-4 pt-md-5">
|
||||
<div class="row clearfix">
|
||||
{{- partial "docs_sidebar.html" . -}}
|
||||
{{ block "main" . }}{{ end }}
|
||||
{{- partial "table_of_content.html" . -}}
|
||||
</div>
|
||||
</div>
|
||||
</section>
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
{{ define "main" }}
|
||||
<article class="documentation col-12 col-md-8 col-11 pb-5">
|
||||
<article class="documentation__article col-12 col-md-9 col-lg-8 pb-5">
|
||||
|
||||
{{ .Content }}
|
||||
|
||||
|
||||
@@ -1,15 +1,16 @@
|
||||
{{ define "main" }}
|
||||
{{ $isCloud := cond (in .Parent.RelPermalink "cloud") true false }}
|
||||
|
||||
<div class="col-12 col-md-8 col-11 pb-5">
|
||||
<div class="col-12 col-md-9 col-lg-8 pb-5">
|
||||
|
||||
<article class="documentation__article">
|
||||
{{ .Content }}
|
||||
</article>
|
||||
|
||||
<div style="overflow: auto">
|
||||
<span style="float: right">
|
||||
<a href="{{if $isCloud}}{{ .Site.Params.githubCloudDocPrefix }}{{else}}{{ .Site.Params.githubDocPrefix }}{{end}}{{ .File.LogicalName }}" target="_blank">Edit on GitHub <i class="flaticon-github"></i></a>
|
||||
</span>
|
||||
</div>
|
||||
|
||||
<article class="documentation">
|
||||
{{ .Content }}
|
||||
</article>
|
||||
</div>
|
||||
{{ end }}
|
||||
@@ -1,37 +1,96 @@
|
||||
{{$currentNode := .}}
|
||||
<nav class="col-lg-3 col-md-3 col-12 mt-0 mt-md-3 pr-4 mr-3 mb-4 position-relative">
|
||||
<nav class="documentation__sidebar-inner col-lg-2 col-md-3 col-12 pr-4 mb-4 position-relative">
|
||||
|
||||
{{ $sectionLink := cond .IsSection .RelPermalink .Parent.RelPermalink }}
|
||||
{{ $subject := site.GetPage $sectionLink }}
|
||||
<div id="accordion" class="sticky-top documentation__sidebar">
|
||||
{{ $subject := site.GetPage "documentation" }}
|
||||
|
||||
<div id="accordion" class="sticky-top documentation__sidebar-inner">
|
||||
|
||||
<!-- buttons for Qdrant/Cloud doc switching-->
|
||||
{{ $isCloud := cond (in $sectionLink "cloud") true false }}
|
||||
<div class="btn-group btn-group-sm w-100 mb-3" role="group" aria-label="...">
|
||||
<a class="btn {{if $isCloud}}btn--light{{else}}btn--mid-grey font-weight-bold{{end}}" href="/documentation/">Qdrant</a>
|
||||
<a class="btn {{if $isCloud}}btn--mid-grey font-weight-bold{{else}}btn--light{{end}}" href="/documentation/cloud/">Cloud</a>
|
||||
</div>
|
||||
<!-- Button triggers search modal -->
|
||||
<button type="button" class="qdr-search-input-btn w-100 mb-2 w-100" data-target="#searchModal">
|
||||
<i class="fa fa-search mr-2"></i>Search...
|
||||
</button>
|
||||
|
||||
<div class="documentation__sidebar-list">
|
||||
{{ range $subject.RegularPages }}
|
||||
<div class="link-group {{ if eq .File.UniqueID $currentNode.File.UniqueID }}active {{ end }}">
|
||||
<div class="link-group-heading" id="headingOne">
|
||||
<span class="mb-0">
|
||||
<a class="btn btn-link collapsed pl-0" href="{{ .Permalink }}">
|
||||
{{ .Title }}
|
||||
</a>
|
||||
</span>
|
||||
</div>
|
||||
{{ if eq .File.UniqueID $currentNode.File.UniqueID }}
|
||||
<div class="pl-4">
|
||||
{{ .TableOfContents }}
|
||||
</div>
|
||||
<div class="documentation__sidebar-list">
|
||||
{{ range $subject.Pages }}
|
||||
|
||||
{{ if (eq .Params.type "delimiter") }}
|
||||
<h5 class="documentation__sidebar-delimiter">{{ .Title }}</h5>
|
||||
{{ else }}
|
||||
|
||||
{{ if .IsPage }}
|
||||
|
||||
<div class="link-group {{ if eq .File.UniqueID $currentNode.File.UniqueID }}active {{ end }}">
|
||||
<div class="link-group-heading">
|
||||
<span class="mb-0">
|
||||
<a class="btn btn-link text-left collapsed pl-0"
|
||||
href="{{ if (eq .Params.type "external-link") }}{{ .Params.external_url }}{{ else }}{{ .Permalink }}{{ end }}"
|
||||
{{ if (eq .Params.type "external-link") }}target="_blank"{{ end }}
|
||||
>
|
||||
{{ .Title }}
|
||||
{{ if (eq .Params.type "external-link") }}
|
||||
<i class="link-group__icon fas fa-external-link-alt"></i>
|
||||
{{ end }}
|
||||
</a>
|
||||
</span>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{{ end }}
|
||||
</div>
|
||||
|
||||
{{ if .IsSection }}
|
||||
|
||||
<!-- current page is active or any sub-page is active -->
|
||||
{{ $isActive := false }}
|
||||
{{ range .RegularPages }}
|
||||
{{ if eq .File.UniqueID $currentNode.File.UniqueID }}
|
||||
{{ $isActive = true }}
|
||||
{{ end }}
|
||||
{{ end }}
|
||||
|
||||
{{ if eq .File.UniqueID $currentNode.File.UniqueID }}
|
||||
{{ $isActive = true }}
|
||||
{{ end }}
|
||||
|
||||
{{ $sectionLink := .Permalink }}
|
||||
|
||||
{{ if .Params.is_empty }}
|
||||
{{ $sectionLink = (index (.RegularPages) 0).Permalink }}
|
||||
{{ end }}
|
||||
|
||||
<details class="documentation__sidebar-link-group link-group {{ if $isActive }}active {{ end }}"
|
||||
{{ if $isActive }}open{{ end }}>
|
||||
<summary class="link-group-heading">
|
||||
<span class="mb-0">
|
||||
<a class="btn btn-link text-left collapsed pl-0" href="{{ $sectionLink }}">
|
||||
{{ .Title }}
|
||||
</a>
|
||||
</span>
|
||||
</summary>
|
||||
|
||||
<div class="pl-4">
|
||||
<nav id="TableOfContents">
|
||||
<ul>
|
||||
{{ range .RegularPages }}
|
||||
{{ if (eq .Params.type "external-link") }}
|
||||
<li>
|
||||
<a href="{{ .Params.external_url }}" target="_blank">
|
||||
{{ .Title }}
|
||||
<i class="link-group__icon fas fa-external-link-alt"></i>
|
||||
</a>
|
||||
</li>
|
||||
{{ else }}
|
||||
<li class="{{ if eq .File.UniqueID $currentNode.File.UniqueID }}active {{ end }}"><a href="{{ .Permalink }}">{{ .Title }}</a></li>
|
||||
{{ end }}
|
||||
{{ end }}
|
||||
</ul>
|
||||
</nav>
|
||||
</div>
|
||||
</details>
|
||||
|
||||
{{ end }}
|
||||
|
||||
{{ end }}
|
||||
|
||||
{{ end }}
|
||||
|
||||
<br/>
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
<!-- Main Footer -->
|
||||
<footer class="main-footer">
|
||||
<div class="auto-container">
|
||||
<div class="{{ if (eq $.Section "documentation") }}container-xxl{{ else }}auto-container{{ end }}">
|
||||
<!--Widgets Section-->
|
||||
<div class="widgets-section">
|
||||
<div class="row clearfix">
|
||||
|
||||
@@ -85,13 +85,14 @@
|
||||
<!-- Main Header-->
|
||||
|
||||
{{ if (eq $.Section "documentation") }}
|
||||
<header class="main-header">
|
||||
<header class="main-header position-static">
|
||||
{{ else }}
|
||||
<header class="main-header main-sticky-header{{ if .IsHome }} main-header__home{{ end }}">
|
||||
{{ end }}
|
||||
|
||||
<!--Header-Upper-->
|
||||
<div class="auto-container d-flex justify-content-between">
|
||||
<div class="{{ if (eq $.Section "documentation") }}container-xxl
|
||||
{{ else }}auto-container{{ end }} d-flex justify-content-between">
|
||||
|
||||
<div class="logo-box">
|
||||
<div class="logo logo_with-text {{ if .IsHome }}logo_home{{ end }}">
|
||||
|
||||
@@ -0,0 +1,15 @@
|
||||
{{$currentNode := .}}
|
||||
<nav class="tableOfContent col-lg-2 col-12 d-lg-block d-none mt-0 pr-4 mb-4">
|
||||
|
||||
{{ $sectionLink := cond .IsSection .RelPermalink .Parent.RelPermalink }}
|
||||
{{ $subject := site.GetPage $sectionLink }}
|
||||
<div class="sticky-top">
|
||||
{{ range $subject.RegularPages }}
|
||||
{{ if eq .File.UniqueID $currentNode.File.UniqueID }}
|
||||
<h5 class="tableOfContent__header">Table of contents</h5>
|
||||
{{ .TableOfContents }}
|
||||
{{ end }}
|
||||
{{ end }}
|
||||
<br/>
|
||||
</div>
|
||||
</nav>
|
||||
@@ -0,0 +1 @@
|
||||
*.map
|
||||
@@ -2,39 +2,46 @@
|
||||
@use "../helpers/functions" as lib;
|
||||
|
||||
.documentation {
|
||||
@include mixins.content-reset;
|
||||
padding-top: 2rem;
|
||||
margin-top: 0.5rem;
|
||||
border-top: 1px solid var(--brand-light-grey);
|
||||
|
||||
h1 {
|
||||
font-size: 2.25rem;
|
||||
&__article {
|
||||
@include mixins.content-reset;
|
||||
|
||||
h1 {
|
||||
font-size: 2.25rem;
|
||||
}
|
||||
|
||||
h2 {
|
||||
font-size: 1.75rem;
|
||||
}
|
||||
|
||||
h3 {
|
||||
font-size: 1.25rem;
|
||||
}
|
||||
|
||||
h4 {
|
||||
font-size: 1.25rem;
|
||||
}
|
||||
|
||||
h5 {
|
||||
font-size: 1.25rem;
|
||||
}
|
||||
|
||||
h6 {
|
||||
font-size: 1.25rem;
|
||||
}
|
||||
|
||||
p {
|
||||
margin-bottom: 0.7rem;
|
||||
}
|
||||
}
|
||||
|
||||
h2 {
|
||||
font-size: 1.75rem;
|
||||
&__sidebar-inner {
|
||||
top: lib.rem(8);
|
||||
}
|
||||
|
||||
h3 {
|
||||
font-size: 1.25rem;
|
||||
}
|
||||
|
||||
h4 {
|
||||
font-size: 1.25rem;
|
||||
}
|
||||
|
||||
h5 {
|
||||
font-size: 1.25rem;
|
||||
}
|
||||
|
||||
h6 {
|
||||
font-size: 1.25rem;
|
||||
}
|
||||
|
||||
p {
|
||||
margin-bottom: 0.7rem;
|
||||
}
|
||||
|
||||
&__sidebar {
|
||||
top: lib.rem(30);
|
||||
}
|
||||
&__sidebar-list {
|
||||
max-height: calc(100vh - 90px);
|
||||
overflow-y: scroll;
|
||||
@@ -45,6 +52,7 @@
|
||||
&::-webkit-scrollbar {
|
||||
display: none;
|
||||
}
|
||||
|
||||
&::-webkit-scrollbar {
|
||||
width: 1px;
|
||||
}
|
||||
@@ -52,6 +60,7 @@
|
||||
&::-webkit-scrollbar-thumb {
|
||||
background: var(--brand-gray);
|
||||
}
|
||||
|
||||
&:hover {
|
||||
scrollbar-width: thin; /* Firefox */
|
||||
scrollbar-color: var(--brand-gray) var(--brand-mid-grey) !important;
|
||||
@@ -60,6 +69,7 @@
|
||||
display: block;
|
||||
}
|
||||
}
|
||||
|
||||
@media (hover: none) {
|
||||
scrollbar-width: thin; /* Firefox */
|
||||
scrollbar-color: var(--brand-gray) var(--brand-mid-grey) !important;
|
||||
@@ -69,4 +79,40 @@
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
&__sidebar-link-group {
|
||||
& > summary {
|
||||
list-style: none;
|
||||
margin-right: 10px;
|
||||
position: relative;
|
||||
|
||||
&:after {
|
||||
display: block;
|
||||
width: lib.rem(10);
|
||||
height: lib.rem(20);
|
||||
position: absolute;
|
||||
top: 6px;
|
||||
right: 0;
|
||||
content: "\f0da";
|
||||
text-align: center;
|
||||
color: #272727;
|
||||
font-family: 'Font Awesome 5 Free';
|
||||
font-size: lib.rem(14);
|
||||
font-weight: 600;
|
||||
}
|
||||
}
|
||||
|
||||
&[open] > summary {
|
||||
&:after {
|
||||
content: "\f0d7";
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
&__sidebar-delimiter {
|
||||
font-size: 1rem;
|
||||
font-weight: 600;
|
||||
padding-top: 1rem;
|
||||
padding-bottom: 0.3rem;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,33 @@
|
||||
.tableOfContent {
|
||||
top: 0;
|
||||
right: 0;
|
||||
|
||||
&__header {
|
||||
margin-bottom: 0.8rem;
|
||||
margin-left: -5px;
|
||||
}
|
||||
|
||||
& ul {
|
||||
margin: 0;
|
||||
}
|
||||
|
||||
& ul li {
|
||||
padding-left: 0;
|
||||
list-style: none;
|
||||
margin: 0;
|
||||
&:not(nav > ul > li) {
|
||||
list-style: '· ';
|
||||
margin-left: 0.5rem;
|
||||
}
|
||||
}
|
||||
|
||||
& a {
|
||||
line-height: 2.2;
|
||||
color: var(--brand-secondary);
|
||||
|
||||
&:hover {
|
||||
color: var(--brand-primary);
|
||||
text-decoration: underline;
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
.container-xxl {
|
||||
max-width: 1440px;
|
||||
padding-right: 15px;
|
||||
padding-left: 15px;
|
||||
margin-right: auto;
|
||||
margin-left: auto;
|
||||
}
|
||||
File diff suppressed because one or more lines are too long
@@ -1,4 +1,5 @@
|
||||
@use "helpers/functions";
|
||||
@use "helpers/utils";
|
||||
|
||||
@import "colors";
|
||||
@import "../../../../node_modules/bootstrap/scss/bootstrap.scss";
|
||||
@@ -95,12 +96,31 @@ code {
|
||||
|
||||
.link-group .btn,
|
||||
.link-group a {
|
||||
display: flex;
|
||||
align-items: center;
|
||||
font-family: $font-family-non-content;
|
||||
text-decoration: none;
|
||||
color: var(--brand-secondary);
|
||||
.link-group__icon {
|
||||
color: var(--brand-dark-gray);
|
||||
font-size: 10px;
|
||||
margin-left: 6px;
|
||||
margin-bottom: 1px;
|
||||
}
|
||||
}
|
||||
|
||||
.link-group a:hover {
|
||||
color: var(--brand-primary);
|
||||
.link-group__icon {
|
||||
color: var(--brand-primary);
|
||||
}
|
||||
}
|
||||
|
||||
.link-group.active nav li.active > a {
|
||||
color: var(--brand-primary);
|
||||
&:hover {
|
||||
color: var(--brand-secondary);
|
||||
}
|
||||
}
|
||||
|
||||
.installation-section p {
|
||||
@@ -147,6 +167,7 @@ blockquote {
|
||||
@import "blocks/article-card";
|
||||
@import "blocks/breadcrumbs";
|
||||
@import "blocks/share-post";
|
||||
@import "blocks/table_of_content";
|
||||
@import "blocks/footer";
|
||||
@import "responsive";
|
||||
|
||||
|
||||
Reference in New Issue
Block a user