Improve hybrid cloud docs

* Adds screenshots * Restructures setup for clarity * Configurable cpu and memory resource reservations * Configurable control plane node selector and tolerations * Reference new cleanup script
2026-09-25 14:08:30 +02:00 · 2024-11-18 11:12:26 +01:00
parent 33fac0ee24
commit 2792ff9d6a
13 changed files with 143 additions and 64 deletions
@@ -7,6 +7,8 @@ weight: 2

 Once you have created a Hybrid Cloud Environment, you can create a Qdrant cluster in that enviroment. Use the same process to [Create a cluster](/documentation/cloud/create-cluster/). Make sure to select your Hybrid Cloud Environment as the target.

+![Create Hybrid Cloud Cluster](/documentation/cloud/hybrid_cloud_create_cluster.png)
+
 Note that in the "Kubernetes Configuration" section you can additionally configure:

 * Node selectors for the Qdrant database pods
@@ -16,12 +18,20 @@ Note that in the "Kubernetes Configuration" section you can additionally configu

 These settings can also be changed after the cluster is created on the cluster detail page.

+![Create Hybrid Cloud Cluster - Kubernetes Configuration](/documentation/cloud/hybrid_cloud_kubernetes_configuration.png)
+
+### Scheduling Configuration
+
+When creating or editing a cluster, you can configure how the database Pods get scheduled in your Kubernetes cluster. This can be useful to ensure that the Qdrant databases will run on dedicated nodes. You can configure the necessary node selectors and tolerations in the "Kubernetes Configuration" section during cluster creation, or on the cluster detail page.
+
 ### Authentication to your Qdrant clusters

 In Hybrid Cloud the authentication information is provided by Kubernetes secrets.

 You can configure authentication for your Qdrant clusters in the "Configuration" section of the Qdrant Cluster detail page. There you can configure the Kubernetes secret name and key to be used as an API key and/or read-only API key.

+![Hybrid Cloud cluster configuration](/documentation/cloud/hybrid_cloud_detail_cluster_configuration.png)
+
 One way to create a secret is with kubectl:

 ```shell
@@ -133,6 +143,8 @@ If you want to configure TLS for accessing your Qdrant database in Hybrid Cloud,
 * You can offload TLS at the ingress or loadbalancer level.
 * You can configure TLS directly in the Qdrant database.

+If you want to offload TLS at the ingress or loadbancer level, please refer to their respective documentation.
+
 If you want to configure TLS directly in the Qdrant database, you can reference a secret containing the TLS certificate and key in the "Configuration" section of the Qdrant Cluster detail page.

 To create such a secret, you can use `kubectl`:
@@ -155,4 +167,16 @@ metadata:
 type: kubernetes.io/tls
 ```

-With this command the secret name to enter into the UI would be `qdrant-tls` and the keys would be `tls.crt` and `tls.key`.
+With this command the secret name to enter into the UI would be `qdrant-tls` and the keys would be `tls.crt` and `tls.key`.
+
+### Configuring CPU and memory resource reservations
+
+If you create a Qdrant database cluster with a certain amount of CPU and memory resources, the Qdrant Cloud platform will schedule Pods that have these resources configured as requests and limits. Keeping requests and limits the same is necessary to guarantee optimal performance and stable operations. It is recommended to create Kubernetes nodes with the same size, so that one database Pod runs per VM. To accomodate the additional resources your need for the operating system, Kubernetes and Kubernetes system components, the Qdrant Cloud platform will automatically reserve some resources for these components by requesting less resources. By default, Qdrant will not use 20% of the available CPU and memory. While it is a conservative setting, this may not be correct for your setup. E.g. on smaller nodes you may need a higher percentage reserved for the system, on larger nodes less. You can change the resource reservation in the "Configuration" section of the Qdrant Cluster detail page.
+
+If you want to check how much resources are availabe on an empty Kubernetes node, you can use the following command:
+
+```shell
+kubectl describe node <node-name>
+```
+
+This will give you a breakdown of the available resources to Kubernetes and how much is already reserved and used for system Pods.
@@ -12,10 +12,11 @@ To learn how Hybrid Cloud works, [read the overview document](/documentation/hyb
 ## Prerequisites

 - **Kubernetes cluster:** To create a Hybrid Cloud Environment, you need a [standard compliant](https://www.cncf.io/training/certification/software-conformance/) Kubernetes cluster. You can run this cluster in any cloud, on-premise or edge environment, with distributions that range from AWS EKS to VMWare vSphere. See [Deployment Platforms](/documentation/hybrid-cloud/platform-deployment-options/) for more information.
- **Storage:** For storage, you need to set up the Kubernetes cluster with a Container Storage Interface (CSI) driver that provides block storage. For vertical scaling, the CSI driver needs to support volume expansion. For backups and restores, the driver needs to support CSI snapshots and restores.
+- **Storage:** For storage, you need to set up the Kubernetes cluster with a Container Storage Interface (CSI) driver that provides block storage. For vertical scaling, the CSI driver needs to support volume expansion. The `StorageClass` needs to be created beforehand. For backups and restores, the driver needs to support CSI snapshots and restores. The `VolumeSnapshotClass` needs to be created beforehand. See [Deployment Platforms](/documentation/hybrid-cloud/platform-deployment-options/) for more information.

 <aside role="status">Network storage systems like NFS or object storage systems such as S3 are not supported.</aside>

+- **Kubernetes nodes:** You need enough CPU and memory capacity for the Qdrant database clusters that you create. A small amount of resources is also needed for the Hybrid Cloud control plane components. Qdrant Hybrid Cloud supports x86_64 and ARM64 architectures.
 - **Permissions:** To install the Qdrant Kubernetes Operator you need to have `cluster-admin` access in your Kubernetes cluster.
 - **Connection:** The Qdrant Kubernetes Operator in your cluster needs to be able to connect to Qdrant Cloud. It will create an outgoing connection to `cloud.qdrant.io` on port `443`.
 - **Locations:** By default, the Qdrant Cloud Agent and Operator pulls Helm charts and container images from `registry.cloud.qdrant.io`. The Qdrant database container image is pulled from `docker.io`.
@@ -31,13 +32,69 @@ During the onboarding, you will need to deploy the Qdrant Kubernetes Operator an

 You will need to have access to the Kubernetes cluster with `kubectl` and `helm` configured to connect to it. Please refer the documentation of your Kubernetes distribution for more information.

-### Required artifacts
+## Installation
+
+1. To set up Hybrid Cloud, open the Qdrant Cloud Console at [cloud.qdrant.io](https://cloud.qdrant.io). On the dashboard, select **Hybrid Cloud**.
+
+2. Before creating your first Hybrid Cloud Environment, you have to provide billing information and accept the Hybrid Cloud license agreement. The installation wizard will guide you through the process.
+
+> **Note:** You will only be charged for the Qdrant cluster you create in a Hybrid Cloud Environment, but not for the environment itself.
+
+3. Now you can specify the following:
+
+- **Name:** A name for the Hybrid Cloud Environment
+- **Kubernetes Namespace:** The Kubernetes namespace for the operator and agent. Once you select a namespace, you can't change it.
+
+You can also configure the StorageClass and VolumeSnapshotClass to use for the Qdrant databases, if you want to deviate from the default settings of your cluster.
+
+![Create Hybrid Cloud Environment](/documentation/cloud/hybrid_cloud_env_create.png)
+
+4. You can then enter the YAML configuration for your Kubernetes operator. Qdrant supports a specific list of configuration options, as described in the [Qdrant Operator configuration](/documentation/hybrid-cloud/operator-configuration/) section.
+
+5. (Optional) If you have special requirements for any of the following, activate the **Show advanced configuration** option:
+
+- If you use a proxy to connect from your infrastructure to the Qdrant Cloud API, you can specify the proxy URL, credentials and cetificates.
+- Container registry URL for Qdrant Operator and Agent images. The default is <https://registry.cloud.qdrant.io/qdrant/>.
+- Helm chart repository URL for the Qdrant Operator and Agent. The default is <oci://registry.cloud.qdrant.io/qdrant-charts>.
+- An optional secret with credentials to access your own container registry.
+- Log level for the operator and agent
+- Node selectors and tolerations for the operater, agent and monitoring stack
+
+![Create Hybrid Cloud Environment - Advanced Configuration](/documentation/cloud/hybrid_cloud_advanced_configuration.png)
+
+6. Once complete, click **Create**.
+
+> **Note:** All settings but the Kubernetes namespace can be changed later.
+
+### Generate Installation Command
+
+After creating your Hybrid Cloud, select **Generate Installation Command** to generate a script that you can run in your Kubernetes cluster which will perform the initial installation of the Kubernetes operator and agent. 
+
+![Rotate Hybrid Cloud Secrets](/documentation/cloud/hybrid_cloud_create_command.png)
+
+It will:
+
+- Create the Kubernetes namespace, if not present
+- Set up the necessary secrets with credentials to access the Qdrant container registry and the Qdrant Cloud API.
+- Sign in to the Helm registry at `registry.cloud.qdrant.io`
+- Install the Qdrant cloud agent and Kubernetes operator chart
+
+You need this command only for the initial installation. After that, you can update the agent and operator using the Qdrant Cloud Console.
+
+> **Note:** If you generate the installation command a second time, it will re-generate the included secrets, and you will have to apply the command again to update them.
+
+## Advanced configuration
+
+### Mirroring images and charts
+
+#### Required artifacts

 Container images:

- `docker.io/qdrant/qdrant`
+- `registry.cloud.qdrant.io/qdrant/qdrant`
 - `registry.cloud.qdrant.io/qdrant/qdrant-cloud-agent`
 - `registry.cloud.qdrant.io/qdrant/qdrant-operator`
+- `registry.cloud.qdrant.io/qdrant/operator`
 - `registry.cloud.qdrant.io/qdrant/cluster-manager`
 - `registry.cloud.qdrant.io/qdrant/prometheus`
 - `registry.cloud.qdrant.io/qdrant/prometheus-config-reloader`
@@ -47,30 +104,32 @@ Open Containers Initiative (OCI) Helm charts:

 - `registry.cloud.qdrant.io/qdrant-charts/qdrant-cloud-agent`
 - `registry.cloud.qdrant.io/qdrant-charts/qdrant-operator`
+- `registry.cloud.qdrant.io/qdrant-charts/operator`
 - `registry.cloud.qdrant.io/qdrant-charts/qdrant-cluster-manager`
 - `registry.cloud.qdrant.io/qdrant-charts/prometheus`

-### Rate limits at `docker.io`
+To mirror all necessary container images and Helm charts into your own registry, we recommend to use an automatic replication feature that your registry provides, so that you have new image versions available automatically. Alternatively you can manually sync the images with tools like [Skopeo](https://github.com/containers/skopeo). When syncing images manually, make sure that you sync then with all, or with the right CPU architecture.

-By default, the Qdrant database image will be fetched from Docker Hub, which is the main source of truth. Docker Hub has rate limits for anonymous users. If you have larger setups and also fetch other images from their, you may run into these limits. To solve this, you can provide authentication information for Docker Hub.
+##### Automatic replication

-First, create a secret with your Docker Hub credentials into your `the-qdrant-namespace` namespace:
+Ensure that you have both the container images in the `/qdrant/` repository, and the helm charts in the `/qdrant-charts/` repository synced. Then go to the advanced section of your Hybrid Cloud Environment and configure your registry locations:
+
+* Container registry URL: `your-registry.example.com/qdrant` (this will for example result in `your-registry.example.com/qdrant/qdrant-cloud-agent`)
+* Chart repository URL: `oci://your-registry.example.com/qdrant-charts` (this will for example result in `oci://your-registry.example.com/qdrant-charts/qdrant-cloud-agent`)
+
+If you registry requires authentication, you have to create your own secrets with authentication information into your `the-qdrant-namespace` namespace.
+
+Example:

 ```shell
-kubectl create secret docker-registry dockerhub-registry-secret --namespace the-qdrant-namespace --docker-server=https://index.docker.io/v1/ --docker-username=<your-name> --docker-password=<your-pword> --docker-email=<your-email>
+kubectl --namespace the-qdrant-namespace create secret docker-registry my-creds --docker-server='your-registry.example.com' --docker-username='your-username' --docker-password='your-password'
 ```

-Then, you can reference this secret by adding the following configuration in the operator configuration YAML editor in the advanced section of the Hybrid Cloud Environment:
+You can then reference they secret in the advanced section of your Hybrid Cloud Environment.

-```yaml
-qdrant:
-  image:
-    pull_secret: "dockerhub-registry-secret"
-```
+##### Manual replication

-### Mirroring images and charts
-
-To mirror all necessary container images and Helm charts into your own registry, you can either use a replication feature that your registry provides, or you can manually sync the images with [Skopeo](https://github.com/containers/skopeo):
+This example uses Skopeo.

 You can find your personal credentials for the Qdrant Cloud registry in the onboarding command, or you can fetch them with `kubectl`:

@@ -94,6 +153,7 @@ To sync all container images:

 ```shell
 skopeo sync --all --src docker --dest docker registry.cloud.qdrant.io/qdrant/qdrant-operator your-registry.example.com/qdrant/qdrant-operator
+skopeo sync --all --src docker --dest docker registry.cloud.qdrant.io/qdrant/operator your-registry.example.com/qdrant/operator
 skopeo sync --all --src docker --dest docker registry.cloud.qdrant.io/qdrant/qdrant-cloud-agent your-registry.example.com/qdrant/qdrant-cloud-agent
 skopeo sync --all --src docker --dest docker registry.cloud.qdrant.io/qdrant/prometheus your-registry.example.com/qdrant/prometheus
 skopeo sync --all --src docker --dest docker registry.cloud.qdrant.io/qdrant/prometheus-config-reloader your-registry.example.com/qdrant/prometheus-config-reloader
@@ -107,6 +167,7 @@ To sync all helm charts:

 ```shell
 skopeo sync --all --src docker --dest docker registry.cloud.qdrant.io/qdrant-charts/prometheus your-registry.example.com/qdrant-charts/prometheus
+skopeo sync --all --src docker --dest docker registry.cloud.qdrant.io/qdrant-charts/operator your-registry.example.com/qdrant-charts/operator
 skopeo sync --all --src docker --dest docker registry.cloud.qdrant.io/qdrant-charts/qdrant-operator your-registry.example.com/qdrant-charts/qdrant-operator
 skopeo sync --all --src docker --dest docker registry.cloud.qdrant.io/qdrant-charts/qdrant-kubernetes-api your-registry.example.com/qdrant-charts/qdrant-kubernetes-api
 skopeo sync --all --src docker --dest docker registry.cloud.qdrant.io/qdrant-charts/qdrant-cloud-agent your-registry.example.com/qdrant-charts/qdrant-cloud-agent
@@ -118,68 +179,47 @@ With the above configuration, you can add the following values to the advanced s
 * Container registry URL: `your-registry.example.com/qdrant`
 * Chart repository URL: `oci://your-registry.example.com/qdrant-charts`

-If you registry requires authentication, you have to create your own secrets with authentication information into your `the-qdrant-namespace` namespace.
+If your registry requires authentication, you can create and reference the secret the same way as described above.

-You can then reference they secret by 
+### Rate limits at `docker.io`

-## Installation
+By default, the Qdrant database image will be fetched from Docker Hub, which is the main source of truth. Docker Hub has rate limits for anonymous users. If you have larger setups and also fetch other images from their, you may run into these limits. To solve this, you can provide authentication information for Docker Hub.

-1. To set up Hybrid Cloud, open the Qdrant Cloud Console at [cloud.qdrant.io](https://cloud.qdrant.io). On the dashboard, select **Hybrid Cloud**.
+First, create a secret with your Docker Hub credentials into your `the-qdrant-namespace` namespace:

-2. Before creating your first Hybrid Cloud Environment, you have to provide billing information and accept the Hybrid Cloud license agreement. The installation wizard will guide you through the process.
+```shell
+kubectl create secret docker-registry dockerhub-registry-secret --namespace the-qdrant-namespace --docker-server=https://index.docker.io/v1/ --docker-username=<your-name> --docker-password=<your-pword> --docker-email=<your-email>
+```

-> **Note:** You will only be charged for the Qdrant cluster you create in a Hybrid Cloud Environment, but not for the environment itself.
+Then, you can reference this secret by adding the following configuration in the operator configuration YAML editor in the advanced section of the Hybrid Cloud Environment:

-3. Now you can specify the following:
+```yaml
+qdrant:
+  image:
+    pull_secret: "dockerhub-registry-secret"
+```

- **Name:** A name for the Hybrid Cloud Environment
- **Kubernetes Namespace:** The Kubernetes namespace for the operator and agent. Once you select a namespace, you can't change it.
+## Rotating Secrets

-You can also configure the StorageClass and VolumeSnapshotClass to use for the Qdrant databases, if you want to deviate from the default settings of your cluster.
+If you need to rotate the secrets to pull container images and charts from the Qdrant registry and to authenticate at the Qdrant Cloud API, you can do so by following these steps:

-4. You can then enter the YAML configuration for your Kubernetes operator. Qdrant supports a specific list of configuration options, as described in the [Qdrant Operator configuration](/documentation/hybrid-cloud/operator-configuration/) section.
+* Go to the Hybrid Cloud environment list or the detail page of the environment.
+* In the actions menu, choose "Rotate Secrets"
+* Confirm the action
+* You will receive a new installation command that you can run in your Kubernetes cluster to update the secrets.

-5. (Optional) If you have special requirements for any of the following, activate the **Show advanced configuration** option:
+If you don't run the installation command, the secrets will not be updated and the communication between your Hybrid Cloud Environment and the Qdrant Cloud API will not work.

- If you use a proxy to connect from your infrastructure to the Qdrant Cloud API, you can specify the proxy URL, credentials and cetificates.
- Container registry URL for Qdrant Operator and Agent images. The default is <https://registry.cloud.qdrant.io/qdrant/>.
- Helm chart repository URL for the Qdrant Operator and Agent. The default is <oci://registry.cloud.qdrant.io/qdrant-charts>.
- Log level for the operator and agent
-
-6. Once complete, click **Create**.
-
-> **Note:** All settings but the Kubernetes namespace can be changed later.
-
-### Generate Installation Command
-
-After creating your Hybrid Cloud, select **Generate Installation Command** to generate a script that you can run in your Kubernetes cluster which will perform the initial installation of the Kubernetes operator and agent. It will:
-
- Create the Kubernetes namespace, if not present
- Set up the necessary secrets with credentials to access the Qdrant container registry and the Qdrant Cloud API.
- Sign in to the Helm registry at `registry.cloud.qdrant.io`
- Install the Qdrant cloud agent and Kubernetes operator chart
-
-You need this command only for the initial installation. After that, you can update the agent and operator using the Qdrant Cloud Console.
-
-> **Note:** If you generate the installation command a second time, it will re-generate the included secrets, and you will have to apply the command again to update them.
+![Rotate Hybrid Cloud Secrets](/documentation/cloud/hybrid_cloud_rotate_secrets.png)

 ## Deleting a Hybrid Cloud Environment

 To delete a Hybrid Cloud Environment, first delete all Qdrant database clusters in it. Then you can delete the environment itself.

-To clean up your Kubernetes cluster, after deleting the Hybrid Cloud Environment, you can use the following command:
+To clean up your Kubernetes cluster, after deleting the Hybrid Cloud Environment, you can use the following script to remove all Qdrant related resources:
+
+Download the script at https://github.com/qdrant/qdrant-cloud-support-tools/tree/main/hybrid-cloud-cleanup. Then run the following command while being connected to your Kubernetes cluster. The script requires `kubectl` and `helm` to be installed.

 ```shell
-helm -n the-qdrant-namespace delete qdrant-cloud-agent
-helm -n the-qdrant-namespace delete qdrant-prometheus
-helm -n the-qdrant-namespace delete qdrant-operator
-kubectl -n the-qdrant-namespace patch HelmRelease.cd.qdrant.io qdrant-cloud-agent -p '{"metadata":{"finalizers":null}}' --type=merge
-kubectl -n the-qdrant-namespace patch HelmRelease.cd.qdrant.io qdrant-prometheus -p '{"metadata":{"finalizers":null}}' --type=merge
-kubectl -n the-qdrant-namespace patch HelmRelease.cd.qdrant.io qdrant-operator -p '{"metadata":{"finalizers":null}}' --type=merge
-kubectl -n the-qdrant-namespace patch HelmChart.cd.qdrant.io the-qdrant-namespace-qdrant-cloud-agent -p '{"metadata":{"finalizers":null}}' --type=merge
-kubectl -n the-qdrant-namespace patch HelmChart.cd.qdrant.io the-qdrant-namespace-qdrant-prometheus -p '{"metadata":{"finalizers":null}}' --type=merge
-kubectl -n the-qdrant-namespace patch HelmChart.cd.qdrant.io the-qdrant-namespace-qdrant-operator -p '{"metadata":{"finalizers":null}}' --type=merge
-kubectl -n the-qdrant-namespace patch HelmRepository.cd.qdrant.io qdrant-cloud -p '{"metadata":{"finalizers":null}}' --type=merge
-kubectl delete namespace the-qdrant-namespace
-kubectl get crd -o name | grep qdrant | xargs -n 1 kubectl delete
+./hybrid-cloud-cleanup.sh your-qdrant-namespace
 ```
@@ -33,6 +33,11 @@ At the time of writing, Linode [does not support CSI Volume Snapshots](https://g

 First, consult AWS' managed Kubernetes instructions below. Then, **to set up Qdrant Hybrid Cloud on AWS**, follow our [step-by-step documentation](/documentation/hybrid-cloud/hybrid-cloud-setup/). 

+For a good balance between peformance and cost, we recommend:
+
+* Depending on your cluster resource onfiguration either general purpuse (m6*, m7*, or m8*), memory optimized (r6*, r7*, or r8*) or cpu optimized (c6*, c7*, or c8*) instance types. Qdrant Hybrid Cloud also supports AWS Graviton ARM64 instances.
+* At least gp3 EBS volumes for storage
+
 ### More on Amazon Elastic Kubernetes Service

 - [Getting Started with Amazon EKS](https://docs.aws.amazon.com/eks/)
@@ -131,6 +136,11 @@ First, consult Gcore's managed Kubernetes instructions below. Then, **to set up

 First, consult GCP's managed Kubernetes instructions below. Then, **to set up Qdrant Hybrid Cloud on GCP**, follow our [step-by-step documentation](/documentation/hybrid-cloud/hybrid-cloud-setup/). 

+For a good balance between peformance and cost, we recommend:
+
+* Depending on your cluster resource onfiguration either general purpuse (standard), memory optimized (highmem) or cpu optimized (highcpu) instance types of at least 2nd generation. Qdrant Hybrid Cloud also supports ARM64 instances.
+* At least pd-balanced disks for storage
+
 ### More on the Google Kubernetes Engine

 - [Getting Started with GKE](https://cloud.google.com/kubernetes-engine/docs/quickstart)
@@ -157,6 +167,11 @@ With [Azure Kubernetes Service (AKS)](https://azure.microsoft.com/en-in/products

 First, consult Azure's managed Kubernetes instructions below. Then, **to set up Qdrant Hybrid Cloud on Azure**, follow our [step-by-step documentation](/documentation/hybrid-cloud/hybrid-cloud-setup/). 

+For a good balance between peformance and cost, we recommend:
+
+* Depending on your cluster resource onfiguration either general purpuse (D-family), memory optimized (E-family) or cpu optimized (F-family) instance types. Qdrant Hybrid Cloud also supports Azure Cobalt ARM64 instances.
+* At least Premium SSD v2 disks for storage
+
 ### More on Azure Kubernetes Service

 - [Getting Started with AKS](https://learn.microsoft.com/en-us/azure/architecture/reference-architectures/containers/aks-start-here)
@@ -34,4 +34,4 @@ You can also choose a severity, when creating a ticket. This helps us prioritize

 If you are opening a ticket for a Hybrid Cloud or Private Cloud environment, we may ask for additional information about your environment, such as detailed logs of the Qdrant databases or operator and the state of your Kubernetes cluster.

-We have prepared a support bundle script that can help you with collecting all this information. A support bundle will not contain any user data or sensitive information like api keys. It will contain the names and configuration of Qdrant collections though. For more information see the [support bundle documentation](https://github.com/qdrant/qdrant-cloud-support-tools/tree/main/hybrid-cloud-support-bundle).
+We have prepared a support bundle script that can help you with collecting all this information. A support bundle will not contain any user data or sensitive information like api keys. It will contain the names and configuration of Qdrant collections though. For more information see the [support bundle documentation](https://github.com/qdrant/qdrant-cloud-support-tools/tree/main/hybrid-cloud-support-bundle). We recommend creating one and attaching it to your support ticket, so that we can help you faster.