krzysztof/landing_page
1
0
Fork 0
mirror of https://github.com/qdrant/landing_page.git synced 2026-10-05 19:08:32 +02:00
Code Issues Packages Projects Releases Wiki Activity

Initial docs for Qdrant Edge (#2076)

Browse Source 
* Initial docs for Qdrant Edge

* Qdrant Edge is not python library

* Anush' feedback

* Talk about 'loading a shard from disk' instead of 'creating a new shard from existing data'

* Talk about 'Edge Shards' instead of 'shards' and use EdgeShard class in code snippets

* Remove unnecessary styling

* review suggestions

* link to github examples + link to SA

---------

Co-authored-by: generall <andrey@vasnetsov.com>
This commit is contained in:
Abdon Pijpelink
2026-01-19 18:33:05 +01:00
committed by GitHub
co-authored by generall
parent 63e10ea16c
commit a9681af722
3 changed files with 364 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
qdrant-landing/content/documentation/edge/_index.md
+41
View File
@@ -0,0 +1,41 @@
---
title: "Qdrant Edge"
weight: 13
partition: qdrant
---
<aside role="status">Qdrant Edge is in beta. The API and functionality may change in future releases.</aside>
# What Is Qdrant Edge?
Qdrant Edge is a lightweight, embedded vector search engine for AI on devices like robots, kiosks, home assistants, and mobile phones. Designed for real-time vector search on edge devices with limited computational resources, Qdrant Edge allows applications to use Qdrant's functionality even with intermittent or no internet connectivity.
Qdrant Edge does not run as a separate process. Instead, it runs inside an application process. Data is stored and queried locally on the device, ensuring low-latency access and enhanced privacy since data does not need to be transmitted to an external server. That said, Qdrant Edge provides APIs to [synchronize data with a Qdrant server](/documentation/edge/edge-synchronizing-with-the-cloud/). This enables you to offload heavy computations such as indexing to more powerful server instances, back up and restore data, and centrally aggregate data from multiple edge devices.
## Qdrant Edge Shard
Qdrant Edge is built around the concept of an **Edge Shard**: a self-contained storage unit that can operate independently on edge devices. Each Edge Shard manages its own data, including vector and payload storage, and can perform local search and retrieval operations.
To work with a Qdrant Edge Shard from a Python application, use the [Python Bindings for Qdrant Edge](https://pypi.org/project/qdrant-edge-py/) package. This package provides an `EdgeShard` class with methods to manage data, query it, and restore snapshots:
- `update`: Updates the data.
- `query`: Queries the data.
- `scroll`: Returns all points.
- `count`: Returns the number of points.
- `retrieve`: Retrieves points with the given IDs.
- `flush`: Flushes the data to ensure that all writes have been persisted to disk.
- `close`: Cleanly destroys the shard instance, ensuring the data is flushed. The data is persisted on disk and can be used to create another shard.
- `info`: Returns metadata information about the shard.
- `unpack_snapshot`: Unpacks a snapshot on disk.
- `snapshot_manifest`: Returns the current shard’s snapshot manifest.
- `update_from_snapshot`: Applies a snapshot to the shard.
## Using Qdrant Edge
To get started with Qdrant Edge, refer to the [Qdrant Edge Quickstart Guide](/documentation/edge/edge-quickstart/).
## More Exmamples
For more examples and advanced usage of Qdrant Edge API can be found in [GitHub repository](https://github.com/qdrant/qdrant/tree/master/lib/edge/python/examples).
qdrant-landing/content/documentation/edge/edge-quickstart.md
+127
View File
@@ -0,0 +1,127 @@
---
title: "Quickstart"
weight: 10
---
# Qdrant Edge Quickstart
## Install Qdrant Edge
First, install the [Python Bindings for Qdrant Edge](https://pypi.org/project/qdrant-edge-py/):
```python
pip install qdrant-edge-py
```
## Create a Storage Directory
A Qdrant Edge Shard stores its data in a local directory on disk. Create the directory if it doesn't exist yet:
```python
from pathlib import Path
SHARD_DIRECTORY = "./qdrant-edge-directory"
Path(SHARD_DIRECTORY).mkdir(parents=True, exist_ok=True)
```
## Configure the Edge Shard
An Edge Shard is configured with a definition of the dense and sparse vectors that can be stored in the Edge Shard, similar to how you would configure a Qdrant collection. Set up a configuration by creating an instance of `EdgeConfig`:
```python
from qdrant_edge import (
Distance,
PayloadStorageType,
PlainIndexConfig,
EdgeConfig,
VectorDataConfig,
VectorStorageType
)
VECTOR_NAME="my-vector"
VECTOR_DIMENSION=4
config = EdgeConfig(
vector_data={
VECTOR_NAME: VectorDataConfig(
size=VECTOR_DIMENSION,
distance=Distance.Cosine,
)
}
)
```
## Initialize the Edge Shard
Now you can create an instance of `EdgeShard` with the storage directory and the configuration:
```python
from qdrant_edge import EdgeShard
edge_shard = EdgeShard(SHARD_DIRECTORY, config)
```
## Work with Points
An Edge Shard has several methods to work with points. To add points, use the `update` method:
```python
from qdrant_edge import ( Point, UpdateOperation )
point = Point(
id=1,
vector={VECTOR_NAME: [0.1, 0.2, 0.3, 0.4]},
payload={"color": "red"}
)
edge_shard.update(UpdateOperation.upsert_points([point]))
```
To retrieve a point by ID, use the `retrieve` method:
```python
point = edge_shard.retrieve(
point_ids=[1],
with_payload=True,
with_vector=False
)
```
## Query Points
To query points in the Edge Shard, use the `query` method:
```python
from qdrant_edge import Query, QueryRequest
results = edge_shard.query(
QueryRequest(
query=Query.Nearest([0.2, 0.1, 0.9, 0.7], using=VECTOR_NAME),
limit=10,
with_vector=False,
with_payload=True
)
)
```
## Close the Edge Shard
When shutting down your application, close the Edge Shard to ensure all data is flushed to disk. The data is persisted on disk and can be used to reopen the Edge Shard.
```python
edge_shard.close()
```
## Load Existing Edge Shard from Disk
After closing an Edge Shard, you can reopen it by loading its data and configuration from disk. Create a new `EdgeShard` instance with the storage directory and provide `None` for the configuration:
```python
edge_shard = EdgeShard(SHARD_DIRECTORY)
```
## More Exmamples
For more examples and advanced usage of Qdrant Edge API can be found in [GitHub repository](https://github.com/qdrant/qdrant/tree/master/lib/edge/python/examples).
qdrant-landing/content/documentation/edge/edge-synchronizing-with-the-cloud.md
+196
View File
@@ -0,0 +1,196 @@
---
title: "Synchronizing with a Server"
weight: 20
---
# Synchronizing Qdrant Edge with a Server
A Qdrant Edge Shard can be synchronized with a collection from an external Qdrant server to support use cases like:
- **Offload indexing**: Indexing is a computationally expensive operation. By synchronizing an Edge Shard with a server collection, you can offload the indexing process to a more powerful server instance. The indexed data can then be synchronized back to the Edge Shard.
- **Back up and Restore**: Regularly back up your Edge Shard data to a central Qdrant instance to prevent data loss. In case of hardware failure or data corruption on the edge device, you can restore the data from the central instance.
- **Data Aggregation**: Collect data from multiple Edge Shards deployed in different locations and aggregate it into a central Qdrant instance for comprehensive analysis and reporting.
- **Synchronization between devices**: Keep data consistent across multiple edge devices by synchronizing their Edge Shards with a central Qdrant instance.
For an example implementation of the patterns described in this guide, refer to the [Qdrant Edge Demo GitHub repository](https://github.com/qdrant/qdrant-edge-demo).
## Initialize Edge Shard from existing Qdrant Collection
First, create a snapshot on the server:
```python
import requests
snapshot_url = f"{QDRANT_URL}/collections/{COLLECTION_NAME}/shards/0/snapshot"
```
Note, that Qdrant Edge operates on a single shard. Therefore, when creating a snapshot for synchronization, specify shard `0` in the snapshot URL (assuming the collection has a single shard).
This allows single collection to serve multiple independent users or devices, each with its own Edge Shard. Read more about qdrant sharding strategy in the [Tiered Multitenancy Documentation](/documentation/guides/multitenancy/#tiered-multitenancy).
Using the snapshot URL, you can download the snapshot, as shown in this helper function:
```python
def download_snapshot(url: str, target_path: Path):
with requests.get(url, headers={"api-key": QDRANT_API_KEY}, stream=True) as r:
r.raise_for_status()
with open(target_path, "wb") as f:
for chunk in r.iter_content(chunk_size=8192):
f.write(chunk)
```
Finally, you can use this function to download the snapshot to the local disk and use the snapshot's data to initialize a new Edge Shard:
```python
import tempfile
import shutil
STORAGE_DIRECTORY = "./qdrant-edge-directory"
data_dir = Path(STORAGE_DIRECTORY)
with tempfile.TemporaryDirectory(dir=data_dir.parent) as restore_dir:
snapshot_path = Path(restore_dir) / "shard.snapshot"
download_snapshot(snapshot_url, snapshot_path)
edge_shard = None
if data_dir.exists():
shutil.rmtree(data_dir)
data_dir.mkdir(parents=True, exist_ok=True)
EdgeShard.unpack_snapshot(str(snapshot_path), str(data_dir))
edge_shard = EdgeShard(str(data_dir))
```
This code first downloads the snapshot to a temporary directory. Next, the current instance of `EdgeShard` (if it existed) is destroyed by setting it to `None` and deleting its data directory. Finally, `EdgeShard.unpack_snapshot` unpacks the downloaded snapshot into the data directory, and a new instance of `EdgeShard` is created using the unpacked snapshot's data and configuration.
While restoring a snapshot, you may want to pause and buffer any ongoing data updates on the Edge Shard. Before taking the snapshot, ensure all queued data has been written to the server. After the restoration is complete, you can resume normal operations. Refer to the [Qdrant Edge Demo GitHub repository](https://github.com/qdrant/qdrant-edge-demo) for an example implementation.
The `edge_shard` will use same configuration and same file structure as the source collection from which the snapshot was created, including vector and payload indexes.
<!-- ToDO -->
<!-- ## Synchronize Server-Side Changes with an Edge Shard -->
<!-- Talk about partial snapshots here -->
## Update a Server Collection from an Edge Shard
To synchronize data from an Edge Shard to a server collection, implement a dual-write mechanism in your application. When you add or update a point in the Edge Shard, simultaneously store it in a server collection using the Qdrant client.
Instead of writing to the server collection directly, you may want to set up a background job or a message queue that handles the synchronization asynchronously. The device running the Edge Shard may not always have a stable internet connection, so queuing updates ensures that data is eventually synchronized when connectivity is restored.
First, initialize:
- an Edge Shard from scratch or from server-side snapshot
- Qdrant server connection.
<details>
<summary>Details</summary>
Initialize an Edge Shard:
```python
from pathlib import Path
from qdrant_edge import (
Distance,
EdgeShard,
PayloadStorageType,
PlainIndexConfig,
EdgeConfig,
VectorDataConfig,
VectorStorageType
)
SHARD_DIRECTORY = "./qdrant-edge-directory"
VECTOR_NAME="my-vector"
VECTOR_DIMENSION=4
Path(SHARD_DIRECTORY).mkdir(parents=True, exist_ok=True)
config = EdgeConfig(
vector_data={
VECTOR_NAME: VectorDataConfig(
size=VECTOR_DIMENSION,
distance=Distance.Cosine,
)
}
)
edge_shard = EdgeShard(SHARD_DIRECTORY, config)
```
Initialize a Qdrant client connection to the server and create the target collection if it does not exist:
```python
from qdrant_client import QdrantClient, models
server_client = QdrantClient(url=QDRANT_URL, api_key=QDRANT_API_KEY)
COLLECTION_NAME="edge-collection"
if not server_client.collection_exists(collection_name=COLLECTION_NAME):
server_client.create_collection(
collection_name=COLLECTION_NAME,
vectors_config={VECTOR_NAME: models.VectorParams(size=VECTOR_DIMENSION, distance=models.Distance.COSINE)}
)
```
</details>
Next, instantiate the queue that will hold the points that need to be synchronized with the server:
```python
from queue import Empty, Queue
# This is in-memory queue
# For production use cases consider persisting changes
upload_queue = Queue()
```
When adding or updating points in the Edge Shard, also enqueue the point for synchronization with the server.
```python
from qdrant_edge import ( Point, UpdateOperation )
from qdrant_client import models
id=1
vector=[0.1, 0.2, 0.3, 0.4]
payload={"color": "red"}
point = Point(
id=1,
vector={VECTOR_NAME: vector},
payload={"color": "red"}
)
edge_shard.update(UpdateOperation.upsert_points([point]))
rest_point = models.PointStruct(id=id, vector={VECTOR_NAME: vector}, payload=payload)
upload_queue.put(rest_point)
```
A background worker can process the upload queue and synchronize points with the server collection.
This example uploads points in batches of up to 10 points at a time:
```python
BATCH_SIZE = 10
points_to_upload = []
while len(points_to_upload) < BATCH_SIZE:
try:
points_to_upload.append(upload_queue.get_nowait())
except Empty:
break
if points_to_upload:
server_client.upsert(
collection_name=COLLECTION_NAME, points=points_to_upload
)
```
Make sure to properly handle errors and retries in case of network issues or server unavailability.
## Support
For explicit support in implementing Qdrant Edge in your project, please contact [Qdrant Sales](https://qdrant.tech/contact-us/).