krzysztof/landing_page
1
0
Fork 0
mirror of https://github.com/qdrant/landing_page.git synced 2026-09-28 15:38:33 +02:00
Code Issues Packages Projects Releases Wiki Activity

Merge pull request #597 from qdrant/feat/bashofmann/hybrid-cloud

Browse Source 
WIP: Add documentation page for Hybrid Cloud
This commit is contained in:
Bastian Hofmann
2024-04-11 10:05:53 +02:00
committed by GitHub
parent c983603cfe af79d6bfa5
commit 9e93dcc499
4 changed files with 420 additions and 21 deletions
Download Patch File Download Diff File Expand all files Collapse all files
qdrant-landing/content/documentation/cloud/_index.md
+38 -17
View File
@@ -7,41 +7,62 @@ aliases:
# About Qdrant Cloud
Qdrant Cloud is our SaaS (software-as-a-service) solution, providing managed Qdrant instances on the cloud.
We provide you with the same fast and reliable similarity search engine, but without the need to maintain your own infrastructure.
Qdrant Cloud is our SaaS (software-as-a-service) solution, providing managed
Qdrant instances on the cloud. We provide you the same fast and reliable
similarity search engine, but without the need to maintain your own infrastructure.
Transitioning from on-premise to the cloud version of Qdrant does not require changing anything in the way you interact with the service. All you have to do is [create a Qdrant Cloud account](https://qdrant.to/cloud/) and [provide a new API key](/documentation/cloud/authentication/) to each request.
Transitioning from on-premise to the cloud version of Qdrant does not change
how you interact with the service. All you need is a [Qdrant Cloud account](https://qdrant.to/cloud/)
and an [API key](/documentation/cloud/authentication/) for each request.
The transition is even easier if you use the official client libraries. For example, the [Python Client](https://github.com/qdrant/qdrant-client/) has the support of the API key already built-in, so you only need to provide it once, when the QdrantClient instance is created.
Our official [client libraries](/documentation/interfaces/#client-libraries/)
can help. For example, if you use the [Python Client](https://github.com/qdrant/qdrant-client/)
you can take advantage of the built-in API key. With that client, you provide
the API key only once, when the QdrantClient instance is created.
### Cluster configuration
*Available as of v1.8.2* <!-- MUST CONFIRM -->
Each instance comes pre-configured with the following tools, features and support services:
You can also attach your own infrastructure as a private region on the Hybrid
Cloud. Once attached, you can control this cloud using the same tools and UI
that you use for other cloud providers. For details, see our
[Hybrid Cloud](/documentation/cloud/hybrid-cloud/) documentation.
- Automatically created with the latest available version of Qdrant.
- Upgradeable to later versions of Qdrant as they are released.
- Equipped with monitoring and logging to observe the health of each cluster.
- Accessible through the Qdrant Cloud Console.
## Cluster configuration
Each instance comes pre-configured with the following tools, features, and
support services:
- Uses the latest available version of Qdrant.
- Supports upgrades to later versions of Qdrant as they are released.
- Includes monitoring and logging to observe the health of each cluster.
- Configurable through the Qdrant Cloud Console.
- Vertically scalable.
- Offered on AWS and GCP, with Azure currently in development.
- Available natively on AWS and GCP, and Azure.
- Available on other providers if you use the Hybrid Cloud.
### Getting started with Qdrant Cloud
## Getting started with Qdrant Cloud
To use Qdrant Cloud, you will need to create at least one cluster. There are two ways to start:
To use Qdrant Cloud, you need at least one cluster. You can create one in the
following ways:
1. [**Create a Free Tier cluster**](/documentation/cloud/quickstart-cloud/) with
1 node and a default configuration (1 GB RAM, 0.5 CPU and 4 GB Disk). This
one node and a default configuration (1 GB RAM, 0.5 CPU and 4 GB Disk). This
option is perfect for prototyping. You don't need a credit card to join.
2. [**Configure a custom cluster**](/documentation/cloud/create-cluster/) with additional nodes and more resources. For this option, you will have to provide billing information.
2. [**Configure a custom cluster**](/documentation/cloud/create-cluster/) with
additional nodes and resources. For this option, you need billing information.
If you're testing Qdrant, We recommend the Free Tier cluster. The capacity
should be enough to serve up to 1 M vectors of 768 dimensions. To calculate
your needs, refer to our documentation on [Capacity and sizing](/documentation/cloud/capacity-sizing/).
We recommend that you use the Free Tier cluster for testing purposes. The
capacity should be enough to serve up to 1 M vectors of 768 dimensions. To
calculate your needs, refer to our documentation on [Capacity and sizing](/documentation/cloud/capacity-sizing/).
### Support & Troubleshooting
## Support & Troubleshooting
All Qdrant Cloud users are welcome to join our [Discord community](https://qdrant.to/discord/).
Our Support Engineers are available to help you anytime.
Additionally, paid customers can also contact support through channels provided during cluster
Paid customers can also contact support through channels provided during cluster
creation and/or on-boarding.
qdrant-landing/content/documentation/cloud/hybrid-cloud.md
+378
View File
@@ -0,0 +1,378 @@
---
title: Hybrid Cloud
weight: 90
---
# Hybrid Cloud
Qdrant Hybrid Cloud allows you to attach your own infrastructure as a private environment to the Qdrant Cloud. You can use Qdrant Cloud to manage your clusters, and run them in your own infrastructure.
## How it works
When you onboard a Kubernetes cluster as a Hybrid Cloud Environment, you can deploy the Qdrant Kubernetes Operator into this cluster. This operator manages the Qdrant databases within your Kubernetes cluster. The operator creates an outgoing connection to the Qdrant cloud at `cloud.qdrant.io` on port `443`. You can then have the same cloud management features and transport telemetry as is available with any managed Qdrant Cloud cluster.
Qdrant Cloud does not need access to the API of your Kubernetes cluster, or to any cloud provider, or other platform APIs.
The Qdrant databases operate solely within your network, using your storage and compute resources.
## Signing up for Hybrid Cloud
To activate Hybrid Cloud, go to the Hybrid Cloud section, enter you company and billing information and request access.
## Creating a Hybrid Cloud Environment
The following sections specify prerequisites and required artifacts to set up a Qdrant cluster in your Hybrid Cloud Environment.
### Prerequisites
To create a Hybrid Cloud Environment, you need a [standard compliant](https://www.cncf.io/training/certification/software-conformance/)Kubernetes cluster. You can run this cluster in any cloud, on-premise or edge environment, with distributions that range from AWS EKS to VMWare vSphere.
For storage, you need to set up the Kubernetes cluster with a Container Storage Interface (CSI) driver that provides block storage. For vertical scaling, the CSI driver needs to support volume expansion. For backups and restores, the driver needs to support CSI snapshots and restores.
<aside role="status">Network storage systems like NFS or object storage systems such as S3 are not supported.</aside>
To install the Qdrant Kubernetes Operator you need to have `cluster-admin` access in your Kubernetes cluster.
The Qdrant Kubernetes operator in your cluster needs to be able to connect to the Qdrant Cloud. It will create an outgoing connection to `cloud.qdrant.io` on port `443`.
By default, the Qdrant Cloud Agent and Operator pulls Helm charts and container images from `registry.cloud.qdrant.io`. The Qdrant database container image is pulled from `docker.io`.
You can also mirror these images and charts into your own registry and pull them from there.
### Required artifacts
To set up Hybrid Cloud, you need the following artifacts:
Container images
- `docker.io/qdrant/qdrant`
- `registry.cloud.qdrant.io/qdrant/qdrant-cloud-agent`
- `registry.cloud.qdrant.io/qdrant/qdrant-operator`
- `registry.cloud.qdrant.io/qdrant/qdrant-cloud-cluster-manager`
- `registry.cloud.qdrant.io/qdrant/prometheus`
- `registry.cloud.qdrant.io/qdrant/prometheus-config-reloader`
- `registry.cloud.qdrant.io/qdrant/kube-state-metrics`
Open Containers Initiative (OCI) Helm charts
- `registry.cloud.qdrant.io/qdrant-charts/qdrant-cloud-agent`
- `registry.cloud.qdrant.io/qdrant-charts/qdrant-operator`
- `registry.cloud.qdrant.io/qdrant-charts/prometheus`
-
## Installation
To set up Hybrid Cloud, open the Qdrant Cloud Console at [cloud.qdrant.io](https://cloud.qdrant.io). On the dashboard, select **Hybrid Cloud**.
Before creating your first Hybrid Cloud Environment, you have to provide billing information and accept the Hybrid Cloud license agreement. The installation wizard will guide you through the process. You will only be charged for the Qdrant cluster you create in a Hybrid Cloud Environment, not for the environment itself.
You can then enter:
- Name: A name for the Hybrid Cloud Environment
- Kubernetes Namespace: The Kubernetes namespace for the operator and agent. Once you select a namespace, you can't change it.
You can then enter the YAML configuration for your Kubernetes operator. Qdrant supports a specific list of configuration options, as described in the [Operator Configuration](#operator-configuration) section.
If you have special requirements for any of the following, activate the **Show advanced configuration** option:
- Proxy server
- Container registry URL for Qdrant Operator and Agent images. The default is <https://registry.cloud.qdrant.io/qdrant/>.
- Helm chart repository URL for the Qdrant Operator and Agent. The default is <oci://registry.cloud.qdrant.io/qdrant-charts>.
- CA certificate
- Log level for the operator and agent
Once complete, select Create.
All settings but the Kubernetes namespace can be changed later.
### Generate Installation Command
After creating your Hybrid Cloud, select **Generate Installation Command** to generate a script that you can run in your Kubernetes cluster which will perform the initial installation of the Kubernetes operator and agent. It will:
- Create the Kubernetes namespace
- Set up the necessary secrets with credentials to access the Qdrant container registry and the Qdrant Cloud API.
- Sign in to the Helm registry at `registry.cloud.qdrant.io`
- Install the Qdrant cloud agent and Kubernetes operator chart
You need this command only for the initial installation. After that, you can update the agent and operator using the Qdrant Cloud Console.
## Creating a Qdrant cluster
Once you have created a Hybrid Cloud Environment, you can create a Qdrant cluster in that enviroment. Use the same process to [Create a cluster](/documentation/cloud/create-cluster/). Make sure to select your Hybrid Cloud Environment as the target.
### Authentication at your Qdrant clusters
In Hybrid Cloud the authentication information is provided with Kubernetes secrets.
You can configure authentication for your Qdrant clusters in the "Configuration" section of the Qdrant Cluster detail page. There you can configure the Kubernetes secret name and key to be used as an API key and/or read-only API key.
One way to create a secret is with kubectl:
```
kubectl create secret generic qdrant-api-key --from-literal=api-key=your-secret-api-key
```
With this command the secret name would be `qdrant-api-key` and the key would be `api-key`.
### Exposing Qdrant clusters to your client applications
You can expose your Qdrant clusters to your client applications using Kubernetes services and ingresses. By default, a `ClusterIP` service is created for each Qdrant cluster.
Within your Kubernetes cluster, you can access the Qdrant cluster using the service name and port:
```
http://qdrant-9a9f48c7-bb90-4fb2-816f-418a46a74b24.qdrant-namespace.svc:6333
```
This endpoint is also visible on the cluster detail page.
If you want to access the database from your local developer machine, you can use `kubectl port-forward` to forward the service port to your local machine:
```
kubectl -n qdrant-namespace port-forward service/qdrant-9a9f48c7-bb90-4fb2-816f-418a46a74b24 6333:6333
```
You can also expose the database outside the Kubernetes cluster with a `LoadBalancer` (if supported in your Kubernetes environment) or `NodePort` service or an ingress.
A simple Loadbalancer service could look like this:
```yaml
apiVersion: v1
kind: Service
metadata:
name: qdrant-9a9f48c7-bb90-4fb2-816f-418a46a74b24-lb
namespace: qdrant-namespace
spec:
type: LoadBalancer
ports:
- name: http
port: 6333
- name: grpc
port: 6334
selector:
app: qdrant
cluster-id: 9a9f48c7-bb90-4fb2-816f-418a46a74b24
```
An ingress could look like this:
```yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: qdrant-9a9f48c7-bb90-4fb2-816f-418a46a74b24
namespace: qdrant-namespace
spec:
rules:
- host: qdrant-9a9f48c7-bb90-4fb2-816f-418a46a74b24.your-domain.com
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: qdrant-9a9f48c7-bb90-4fb2-816f-418a46a74b24
port:
number: 6333
```
Please refer to the Kubernetes, ingress controller and cloud provider documention for more details.
### Network policies
For security reasons, each database cluster is secured with tight network policies. By default, the database pods do only allow egress traffic between themselves and only allow ingress traffic from the operator for monitoring.
To allow additional ingress or egress traffic, you can either deploy additionial network policieson your own
```yaml
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: qdrant-9a9f48c7-bb90-4fb2-816f-418a46a74b24
namespace: qdrant-namespace
spec:
podSelector:
matchLabels:
app: qdrant
cluster-id: 9a9f48c7-bb90-4fb2-816f-418a46a74b24
policyTypes:
- Ingress
ingress:
- from:
- ipBlock:
cidr: 192.168.0.0/22
- podSelector:
matchLabels:
app: client-app
namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: client-namespace
- podSelector:
matchLabels:
app: traefik
namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
ports:
- protocol: TCP
port: 6333
- protocol: TCP
port: 6334
```
Or you can modify the default network policies in the Hybrid Cloud environment configuration:
```yaml
qdrant:
networkPolicies:
ingress:
- from:
- ipBlock:
cidr: 192.168.0.0/22
- podSelector:
matchLabels:
app: client-app
namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: client-namespace
- podSelector:
matchLabels:
app: traefik
namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
ports:
- port: 6333
protocol: TCP
- port: 6334
protocol: TCP
```
## Logging
You can access the logs with kubectl or the Kubernetes log management tool of your choice.
Example:
```bash
kubectl -n qdrant-namespace logs -l app=qdrant,cluster-id=9a9f48c7-bb90-4fb2-816f-418a46a74b24
```
### Log levels
You can configure log levels for the databases individually in the configuration section of the Qdrant Cluster detail page.
The log level for the Qdrant Cloud Agent and Operator can be set in the Hybrid Cloud Environment configuration.
## Monitoring
The Qdrant Cloud console provides you access to basic metrics about CPU, memory and disk usage of your Qdrant clusters. You can also access the Prometheus metrics endpoint of the Qdrant databases. And use the Kubernetes workload monitoring tool of your choice to monitor your Qdrant clusters.
## Operator configuration
You should configure the Qdrant Operator with the configuration for the hybrid cloud. Use the following options, in YAML format:
```yaml
# Configuration for the Qdrant operator
settings:
# Does the operator run inside a Kubernetes cluster (kubernetes) or outside (local)
app_environment: kubernetes
# Retention for the backup history of Qdrant clusters
backupHistoryRetentionDays: 2
# Timeout configuration for the Qdrant operator operations
operationTimeout: 7200 # 2 hours
handlerTimeout: 21600 # 6 hours
backupTimeout: 12600 # 3.5 hours
# Incremental backoff configuration for the Qdrant operator operations
backOff:
minDelay: 5
maxDelay: 300
increment: 5
# Cluster-manager configuration for a Qdrant cluster (experimental)
clusterManager:
image:
repository: qdrant/qdrant-cloud-cluster-manager
tag: 0.1.2
pullInterval: 10
logSize: 10
debug: false
# node_selector: {}
# tolerations: []
# Default ingress configuration for a Qdrant cluster
ingress:
enabled: false
provider: KubernetesIngress # or NginxIngress
# kubernetesIngress:
# ingressClassName: ""
# Default storage configuration for a Qdrant cluster
# storage:
# # Default VolumeSnapshotClass for a Qdrant cluster
# snapshot_class: "csi-snapclass"
# # Default StorageClass for a Qdrant cluster, uses cluster default StorageClass if not set
# default_storage_class_names:
# # StorageClass for DB volumes
# db: ""
# # StorageClass for snapshot volumes
# snapshot: ""
# Default scheduling configuration for a Qdrant cluster
# scheduling:
# default_topology_spread_constraints: []
# default_pod_disruption_budget: {}
qdrant:
# Default security context for Qdrant cluster
# securityContext:
# enabled: false
# user: ""
# fsGroup: ""
# group: ""
# Default Qdrant image configuration
# image:
# pull_secret: ""
# pull_policy: IfNotPresent
# repository: qdrant/qdrant
# Default Qdrant log_level
# log_level: INFO
# Default network policies to create for a qdrant cluster
networkPolicies:
# ingress:
# - from:
# - podSelector:
# matchLabels:
# app.kubernetes.io/name: traefik
# namespaceSelector:
# matchLabels:
# kubernetes.io/metadata.name: kube-system
# ports:
# - protocol: TCP
# port: 6333
# - protocol: TCP
# port: 6334
# - protocol: TCP
# port: 6335
# Allow DNS resolution from qdrant pods at Kubernetes internal DNS server
egress:
- to:
- namespaceSelector:
matchLabels:
kubernetes.io/metadata.name: kube-system
ports:
- protocol: UDP
port: 53
```
## Deleting a Hybrid Cloud Environment
To delete a Hybrid Cloud Environment, first delete all Qdrant database clusters in it, then open a support ticket with the id of the environment you want to delete.
## Roadmap
We plan to introduce the following configuration options directly in the Qdrant Cloud Console in the future. If you need any of them beforehand, please contact our Support team.
* Self-service environment deletion
* Node selectors
* Tolerations
* Affinities and anti-affinities
* Service types and annotations
* Ingresses
* Network policies
* Storage classes
* Volume snapshot classes