krzysztof/landing_page
1
0
Fork 0
mirror of https://github.com/qdrant/landing_page.git synced 2026-09-28 07:28:30 +02:00
Code Issues Packages Projects Releases Wiki Activity

First draft of the Oracle Cloud / Cohere / Airbyte tutorial

Browse Source
This commit is contained in:
Kacper Łukawski
2024-03-28 21:33:22 +01:00
parent c6ae37487b
commit 504891e280
5 changed files with 252 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files
qdrant-landing/content/documentation/tutorials/_index.md
+17 -16
View File
@@ -12,19 +12,20 @@ aliases:
These tutorials demonstrate different ways you can build vector search into your applications.
| Tutorial | Description | Stack |
|----------------------------------------------------------------------------|-------------------------------------------------------------------|---------------------------------------------|
| [Configure Optimal Use](../tutorials/optimize/) | Configure Qdrant collections for best resource use. | Qdrant |
| [Separate Partitions](../tutorials/multiple-partitions/) | Serve vectors for many independent users. | Qdrant |
| [Bulk Upload Vectors](../tutorials/bulk-upload/) | Upload a large scale dataset. | Qdrant |
| [Create Dataset Snapshots](../tutorials/create-snapshot/) | Turn a dataset into a snapshot by exporting it from a collection. | Qdrant |
| [Semantic Search for Beginners](../tutorials/search-beginners/) | Create a simple search engine locally in minutes. | Qdrant |
| [Simple Neural Search](../tutorials/neural-search/) | Build and deploy a neural search that browses startup data. | Qdrant, BERT, FastAPI |
| [Aleph Alpha Search](../tutorials/aleph-alpha-search/) | Build a multimodal search that combines text and image data. | Qdrant, Aleph Alpha |
| [Mighty Semantic Search](../tutorials/mighty/) | Build a simple semantic search with an on-demand NLP service. | Qdrant, Mighty |
| [Asynchronous API](../tutorials/async-api/) | Communicate with Qdrant server asynchronously with Python SDK. | Qdrant, Python |
| [Multitenancy with LlamaIndex](../tutorials/llama-index-multitenancy/) | Handle data coming from multiple users in LlamaIndex. | Qdrant, Python, LlamaIndex |
| [HuggingFace datasets](../tutorials/huggingface-datasets/) | Load a Hugging Face dataset to Qdrant | Qdrant, Python, datasets |
| [Measure retrieval quality](../tutorials/retrieval-quality/) | Measure and fine-tune the retrieval quality | Qdrant, Python, datasets |
| [Use semantic search to navigate your codebase](../tutorials/code-search/) | Implement semantic search application for code search task | Qdrant, Python, sentence-transformers, Jina |
| [Troubleshooting](../tutorials/common-errors/) | Solutions to common errors and fixes | Qdrant |
| Tutorial | Description | Stack |
|--------------------------------------------------------------------------------|------------------------------------------------------------------------------------------|--------------------------------------------------|
| [Configure Optimal Use](../tutorials/optimize/) | Configure Qdrant collections for best resource use. | Qdrant |
| [Separate Partitions](../tutorials/multiple-partitions/) | Serve vectors for many independent users. | Qdrant |
| [Bulk Upload Vectors](../tutorials/bulk-upload/) | Upload a large scale dataset. | Qdrant |
| [Create Dataset Snapshots](../tutorials/create-snapshot/) | Turn a dataset into a snapshot by exporting it from a collection. | Qdrant |
| [Semantic Search for Beginners](../tutorials/search-beginners/) | Create a simple search engine locally in minutes. | Qdrant |
| [Simple Neural Search](../tutorials/neural-search/) | Build and deploy a neural search that browses startup data. | Qdrant, BERT, FastAPI |
| [Aleph Alpha Search](../tutorials/aleph-alpha-search/) | Build a multimodal search that combines text and image data. | Qdrant, Aleph Alpha |
| [Mighty Semantic Search](../tutorials/mighty/) | Build a simple semantic search with an on-demand NLP service. | Qdrant, Mighty |
| [Asynchronous API](../tutorials/async-api/) | Communicate with Qdrant server asynchronously with Python SDK. | Qdrant, Python |
| [Multitenancy with LlamaIndex](../tutorials/llama-index-multitenancy/) | Handle data coming from multiple users in LlamaIndex. | Qdrant, Python, LlamaIndex |
| [HuggingFace datasets](../tutorials/huggingface-datasets/) | Load a Hugging Face dataset to Qdrant | Qdrant, Python, datasets |
| [Measure retrieval quality](../tutorials/retrieval-quality/) | Measure and fine-tune the retrieval quality | Qdrant, Python, datasets |
| [Use semantic search to navigate your codebase](../tutorials/code-search/) | Implement semantic search application for code search task | Qdrant, Python, sentence-transformers, Jina |
| [Automate customer support](../tutorials/customer-support-oci-cohere-airbyte/) | Unleash your customer support team from answering the same questions over and over again | Qdrant, Cohere, Command-R, Oracle Cloud, Airbyte |
| [Troubleshooting](../tutorials/common-errors/) | Solutions to common errors and fixes | Qdrant |
qdrant-landing/content/documentation/tutorials/customer-support-oci-cohere-airbyte.md
+235
View File
@@ -0,0 +1,235 @@
---
title: Automate customer support
weight: 24
---
# Unleash your customer support from repetitive tasks
| Time: 120 min | Level: Advanced | |
| --- | ----------- | ----------- |----------- |
Maintaining a proper level of service plays an important role in the success of any business. The customer support team
is the first line of defense when it comes to addressing customer queries and concerns. However, as the business grows,
the volume of customer queries also increases, making it difficult for the support team to handle them efficiently. On
the other hand, the majority of the queries is repetitive and if someone in the team has already answered a similar
question, automating the response can save a lot of time and effort. In times of Large Language Models that does not
sound like a science fiction anymore.
The know-how of the customer support team is usually a proprietary knowledge base that is not available to the public.
You never want this data to leave your infrastructure. However, you can still leverage the power of AI to automate the
responses, thanks to private deployments of the state-of-the-art tools. Cohere’s powerful models [might be deployed to
Oracle Cloud](https://cohere.com/deployment-options/oracle) and used together with Qdrant Hybrid Cloud to build a fully
private customer support system. One missing piece is the data synchronization, and this is where
[Airbyte](https://airbyte.com/) comes into play.
[//]: # (TODO: add a link to the corresponding Qdrant Hybrid Cloud documentation: deployment on OCI)
TODO: add a diagram presenting all the components
## System design
The history of past interactions with your customers is not a static dataset. It is constantly evolving, as new
questions are coming in. You probably have a ticketing system that stores all the interactions, or use a different way
to communicate with your customers. No matter what is the communication channel, you need to bring the correct answers
to the selected Large Language Model, and have an established way to do it in a continuous manner. Thus, we will build
an ingestion pipeline and then a Retrieval Augmented Generation application that will use the data.
- **Dataset:** a [set of Frequently Asked Questions from Qdrant
users](https://qdrant.tech/documentation/faq/qdrant-fundamentals/) as an incrementally updated Excel sheet
- **Embedding model:** Cohere `embed-multilingual-v3.0`, to support different languages with the same pipeline
- **Knowledge base:** Qdrant, running in Hybrid Cloud mode
- **Ingestion pipeline:** [Airbyte](https://airbyte.com/), loading the data into Qdrant
- **Large Language Model:** Cohere [Command-R](https://docs.cohere.com/docs/command-r)
- **RAG:** Cohere [RAG](https://docs.cohere.com/docs/retrieval-augmented-generation-rag) using our knowledge base
through a custom connector
[//]: # (TODO: the dataset has to be published somewhere)
All the selected components might be running on [Oracle Cloud](https://www.oracle.com/cloud/) infrastructure only. Thanks to the availability of
the Cohere models on OCI, you can build a fully private customer support system that does not require any data to leave
your infrastructure.
### Data ingestion
Building a RAG starts with a well-curated dataset. In your specific case you may prefer loading the data directly from
a ticketing system, such as [Zendesk Support](https://airbyte.com/connectors/zendesk-support),
[Freshdesk](https://airbyte.com/connectors/freshdesk), or maybe integrate it with a shared inbox. However, in case of
customer questions quality over quantity is the key. There should be a conscious decision on what data to include in the
knowledge base, so we do not confuse the model with possibly irrelevant information. We'll assume there is an [Excel
sheet](https://docs.airbyte.com/integrations/sources/file) available over HTTP/FTP that Airbyte can access and load into
Qdrant in an incremental manner.
### Cohere <> Qdrant Connector for RAG
Cohere RAG relies on [connectors](https://docs.cohere.com/docs/connectors) which brings additional context to the model.
The connector is a web service that implements a specific interface, and exposes its data through HTTP API. With that
setup, the Large Language Model becomes responsible for communicating with the connectors, so building a prompt with the
context is not needed anymore.
### Answering bot
Finally, we want to automate the responses and send them automatically when we are sure that the model is confident
enough. Again, the way such an application should be created strongly depends on the system you are using within the
customer support team. If it exposes a way to set up a webhook whenever a new question is coming in, you can create a
web service and use it to automate the responses. In general, our bot should be created specifically for the platform
you use, so we'll just cover the general idea here and build a simple CLI tool.
## Prerequisites
### Qdrant Hybrid Cloud on OCI
Our documentation covers the deployment of Qdrant on Oracle Cloud, so you can follow the steps described there to set up
your own instance. The deployment process is quite straightforward, and you can have your Qdrant cluster up and running
in a few minutes.
[//]: # (TODO: refer to the documentation on how to deploy Qdrant on Oracle Cloud)
Once you perform all the steps, your Qdrant cluster should be running on a specific URL. You will need this URL and the
API key to interact with Qdrant, so let's store them both in the environment variables:
```shell
export QDRANT_URL="https://qdrant.example.com"
export QDRANT_API_KEY="your-api-key"
```
```python
import os
os.environ["QDRANT_URL"] = "https://qdrant.example.com"
os.environ["QDRANT_API_KEY"] = "your-api-key"
```
### Airbyte Open Source
Airbyte is an open-source data integration platform that helps you replicate your data in your warehouses, lakes, and
databases. You can install it on your infrastructure and use it to load the data into Qdrant. The installation process
for Oracle Cloud is described in the [official documentation](https://docs.airbyte.com/deploying-airbyte/on-oci-vm).
Please follow the instructions to set up your own instance.
#### Setting up the connection
Once you have an Airbyte up and running, you can configure the connection to load the data from the respective source
into Qdrant. The configuration will require setting up the source and destination connectors. In this tutorial we will
use the following connectors:
- **Source:** [File](https://docs.airbyte.com/integrations/sources/file) to load the data from an Excel sheet
- **Destination:** [Qdrant](https://docs.airbyte.com/integrations/destinations/qdrant) to load the data into Qdrant
Airbyte UI will guide you through the process of setting up the source and destination and connecting them. Here is how
the configuration of the source might look like:
![Airbyte source configuration](/documentation/tutorials/customer-support-oci-cohere-airbyte/airbyte-excel-source.png)
Qdrant is our target destination, so we need to set up the connection to it. We need to specify which fields should be
included to generate the embeddings. In our case it makes complete sense to embed just the questions, as we are going
to look for similar questions asked in the past and provide the answers.
![Airbyte destination configuration](/documentation/tutorials/customer-support-oci-cohere-airbyte/airbyte-qdrant-destination.png)
Once we have the destination set up, we can finally configure a connection. The connection will define the schedule
of the data synchronization.
![Airbyte connection configuration](/documentation/tutorials/customer-support-oci-cohere-airbyte/airbyte-connection.png)
Airbyte should now be ready to accept any data updates from the source and load them into Qdrant. You can monitor the
progress of the synchronization in the UI.
## RAG connector
One of our previous tutorials, guides you step-by-step on [implementing custom connector for Cohere
RAG](../tutorials/cohere-rag-connector/) with Cohere Embed v3 and Qdrant. You can just point it to use your Hybrid Cloud
Qdrant instance running on OCI. Created connector might be deployed to Oracle Cloud in various ways, even in a
[Serverless](https://developer.oracle.com/learn/use-cases.html#serverless) manner using [Oracle Cloud Infrastructure
Functions](https://docs.oracle.com/en-us/iaas/Content/Functions/home.htm#top).
In general, RAG connector has to expose a single endpoint that will accept POST requests with `query` parameter and
return the matching documents as JSON document with a specific structure. Our FastAPI implementation created [in the
related tutorial](../tutorials/cohere-rag-connector/) is a perfect fit for this task. The only difference is that you
should point it to the Cohere models and Qdrant running on Oracle Cloud infrastructure.
> Our connector is a lightweight web service that exposes a single endpoint and glues the Cohere embedding model with
> our Qdrant Hybrid Cloud instance. Thus, it perfectly fits the serverless architecture, requiring no additional
> infrastructure to run.
You can also run the connector as another service within your [Kubernetes cluster running on Oracle Cloud
(OKE)](https://www.oracle.com/cloud/cloud-native/container-engine-kubernetes/). This step is dependent on the way you
deploy your other services, so we'll leave it to you to decide how to run the connector.
Eventually, the web service should be available under a specific URL, and it's a good practice to store it in the
environment variable, so the other services can easily access it.
```shell
export RAG_CONNECTOR_URL="https://rag-connector.example.com/search"
```
```python
os.environ["RAG_CONNECTOR_URL"] = "https://rag-connector.example.com/search"
```
[//]: # (TODO: refer to the tutorial on a custom RAG connector for Cohere)
[//]: # (See: https://github.com/qdrant/landing_page/pull/761)
## Customer interface
At this part we have all the data loaded into Qdrant, and the RAG connector is ready to serve the relevant context. The
last missing piece is the customer interface, that will call the Command-R model to create the answer. Such a system
should be built specifically for the platform you use and integrated into its workflow, but we will build the strong
foundation for it and show how to use it in a simple CLI tool.
> Our application does not have to connect to Qdrant anymore, as the model will connect to the RAG connector directly.
First of all, we have to create a connection to Cohere services through the Cohere SDK.
```python
import cohere
# Create a Cohere client pointing to the Oracle Cloud instance
cohere_client = cohere.Client(...)
```
Next, our connector should be registered. **Please make sure to do it once, and store the id of the connector in the
environment variable or in any other way that will be accessible to the application.**
```python
import os
connector_response = cohere_client.connectors.create(
name="customer-support",
url=os.environ["RAG_CONNECTOR_URL"],
)
# The id returned by the API should be stored for future use
connector_id = connector_response.connector.id
```
Finally, we can create a prompt and get the answer from the model. Additionally, we define which of the connectors
should be used to provide the context, as we may have multiple connectors and want to use specific ones, depending on
some conditions. Let's start with asking a question.
```python
query = "Why Qdrant does not return my vectors?"
```
Now we can send the query to the model, get the response, and possibly send it back to the customer.
```python
response = cohere_client.chat(
message=query,
connectors=[
cohere.ChatConnector(id=connector_id),
],
model="command-r",
)
print(response.text)
```
Customer support should not be fully automated, as some completely new issues might require human intervention. We
should play with prompt engineering and expect the model to provide the answer with a certain confidence level. If the
confidence is too low, we should not send the answer automatically but present it to the support team for review.
## Wrapping up
This tutorial shows how to build a fully private customer support system using Cohere models, Qdrant Hybrid Cloud, and
Airbyte, which runs on Oracle Cloud infrastructure. You can ensure your data does not leave your premises and focus on
providing the best customer support experience without bothering your team with repetitive tasks.
qdrant-landing/static/documentation/tutorials/customer-support-oci-cohere-airbyte/airbyte-connection.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 93 KiB

qdrant-landing/static/documentation/tutorials/customer-support-oci-cohere-airbyte/airbyte-excel-source.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 44 KiB

qdrant-landing/static/documentation/tutorials/customer-support-oci-cohere-airbyte/airbyte-qdrant-destination.png
BIN
View File
Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 98 KiB