Merge pull request #430 from qdrant/shard-transfer-method

Document shard transfer method

qdrant-landing/content/documentation/guides/distributed_deployment.md

@@ -219,8 +219,11 @@ POST /collections/{collection_name}/cluster
 }
 ```
 
-After the transfer is initiated, the service will keep both copies of the shard in sync until the transfer is complete.
-It will also make sure the transferred shard indexing process is keeping up before performing a final switch. This way, Qdrant ensures that there will be no degradation in performance at the end of the transfer. Once the transfer is completed, the old shard is deleted from the original node.
+<aside role="status">You likely want to select a specific <a href="#shard-transfer-method">shard transfer method</a> to get desired performance and guarantees.</aside>
+
+After the transfer is initiated, the service will process it based on the used
+[transfer method](#shard-transfer-method) keeping both shards in sync. Once the
+transfer is completed, the old shard is deleted from the source node.
 
 In case you want to downscale the cluster, you can move all shards away from a peer and then remove the peer using the [remove peer API](https://qdrant.github.io/qdrant/redoc/index.html#tag/cluster/operation/remove_peer).
 
@@ -230,6 +233,85 @@ DELETE /cluster/peer/{peer_id}
 
 After that, Qdrant will exclude the node from the consensus, and the instance will be ready for shutdown.
 
+### Shard transfer method
+
+*Available as of v1.7.0*
+
+There are different methods for transferring, such as moving or replicating, a
+shard to another node. Depending on what performance and guarantees you'd like
+to have and how you'd like to manage your cluster, you likely want to choose a
+specific method. Each method has its own pros and cons. Which is fastest depends
+on the size and state of a shard.
+
+Available shard transfer methods are:
+
+- `stream_records`: _(default)_ transfer shard by streaming just its records to the target node in batches.
+- `snapshot`: transfer shard including its index and quantized data by utilizing a [snapshot](../../concepts/snapshots) automatically.
+
+Each has pros, cons and specific requirements, which are:
+
+| Method: | Stream records | Snapshot |
+|:---|:---|:---|
+| **Connection** | <ul><li>Requires internal gRPC API <small>(port 6335)</small></li></ul> | <ul><li>Requires internal gRPC API <small>(port 6335)</small></li><li>Requires REST API <small>(port 6333)</small></li></ul> |
+| **HNSW index** | <ul><li>Doesn't transfer index</li><li>Will reindex on target node</li></ul> | <ul><li>Index is transferred with a snapshot</li><li>Immediately ready on target node</li></ul> |
+| **Quantization** | <ul><li>Doesn't transfer quantized data</li><li>Will re-quantize on target node</li></ul> | <ul><li>Quantized data is transferred with a snapshot</li><li>Immediately ready on target node</li></ul> |
+| **Consistency** | <ul><li>Weak data consistency</li><li>Unordered updates on target node[^unordered]</li></ul> | <ul><li>Strong data consistency</li><li>Ordered updates on target node[^ordered]</li></ul> |
+| **Disk space** | <ul><li>No extra disk space required</li></ul> | <ul><li>Extra disk space required for snapshot on both nodes</li></ul> |
+
+[^unordered]: Weak data consistency and unordered updates: All records are streamed to the target node in order.
+    New updates are received on the target node in parallel, while the transfer
+    of records is still happening. We therefore have `weak` ordering, regardless
+    of what [ordering](#write-ordering) is used for updates.
+[^ordered]: Strong data consistency and ordered updates: A snapshot of the shard
+    is created, it is transferred and recovered on the target node. That ensures
+    the state of the shard is kept consistent. New updates are queued on the
+    source node, and transferred in order to the target node. Updates therefore
+    have the same [ordering](#write-ordering) as the user selects, making
+    `strong` ordering possible.
+
+To select a shard transfer method, specify the `method` like:
+
+```http
+POST /collections/{collection_name}/cluster
+{
+    "move_shard": {
+        "shard_id": 0,
+        "from_peer_id": 381894127,
+        "to_peer_id": 467122995,
+        "method": "snapshot"
+    }
+}
+```
+
+The `stream_records` transfer method is the simplest available. It simply
+transfers all shard records in batches to the target node until it has
+transferred all of them, keeping both shards in sync. It will also make sure the
+transferred shard indexing process is keeping up before performing a final
+switch. The method has two common disadvantages: 1. It does not transfer index
+or quantization data, meaning that the shard has to be optimized again on the
+new node, which can be very expensive. 2. The consistency and ordering
+guarantees are `weak`[^unordered], which is not suitable for some applications.
+Because it is so simple, it's also very robust, making it a reliable choice if
+the above cons are acceptable in your use case. If your cluster is unstable and
+out of resources, it's probably best to use the `stream_records` transfer
+method, because it is unlikely to fail.
+
+The `snapshot` transfer method utilizes [snapshots](../../concepts/snapshots) to
+transfer a shard. A snapshot is created automatically. It is then transferred
+and restored on the target node. After this is done, the snapshot is removed
+from both nodes. While the snapshot/transfer/restore operation is happening, the
+source node queues up all new operations. All queued updates are then sent in
+order to the target shard to bring it into the same state as the source. There
+are two important benefits: 1. It transfers index and quantization data, so that
+the shard does not have to be optimized again on the target node, making them
+immediately available. This way, Qdrant ensures that there will be no
+degradation in performance at the end of the transfer. Especially on large
+shards, this can give a huge performance improvement. 2. The consistency and
+ordering guarantees can be `strong`[^ordered], required for some applications.
+
+The `stream_records` method is currently used as default. This may change in the
+future.
+
 ## Replication
 
 *Available as of v0.11.0*
@@ -332,6 +414,8 @@ POST /collections/{collection_name}/cluster
 }
 ```
 
+<aside role="status">You likely want to select a specific <a href="#shard-transfer-method">shard transfer method</a> to get desired performance and guarantees.</aside>
+
 And a replica can be removed on a specific peer.
 
 ```http
@@ -617,10 +701,9 @@ Write `ordering` can be specified for any write request to serialize it through
 which ensures that all write operations (issued with the same `ordering`) are performed and observed
 sequentially.
 
-- `weak` ordering does not provide any additional guarantees, so write operations can be freely reordered
-- `medium` ordering serializes all write operations through a dynamically elected leader, which might cause minor inconsistencies in case of leader change
-- `strong` ordering serializes all write operations through the permanent leader, which provides strong consistency, but write operations may be unavailable if the leader is down
-- default ordering is `weak`
+- `weak` _(default)_ ordering does not provide any additional guarantees, so write operations can be freely reordered.
+- `medium` ordering serializes all write operations through a dynamically elected leader, which might cause minor inconsistencies in case of leader change.
+- `strong` ordering serializes all write operations through the permanent leader, which provides strong consistency, but write operations may be unavailable if the leader is down.
 
 ```http
 PUT /collections/{collection_name}/points?ordering=strong

qdrant-landing/themes/qdrant/static/css/blocks/_documentation.scss

@@ -39,6 +39,11 @@
     h6 {
       font-size: 1rem;
     }
+
+    // Correct spacing of unordered list in table cells
+    & table tr td ul {
+        margin: 0;
+    }
   }
 
   &__sidebar-inner {