mirror of
https://github.com/qdrant/landing_page.git
synced 2026-09-29 07:58:31 +02:00
Merge pull request #765 from qdrant/hybrid-cloud/tutorial/customer-support-oci-cohere-airbyte
HC AWS / Cohere / Airbyte tutorial
This commit is contained in:
David Myriel
@@ -0,0 +1,232 @@
|
||||
---
|
||||
title: Automate customer support tasks
|
||||
weight: 26
|
||||
---
|
||||
|
||||
# Build a RAG system to answer customer support queries
|
||||
|
||||
| Time: 120 min | Level: Advanced | |
|
||||
| --- | ----------- | ----------- |----------- |
|
||||
|
||||
Maintaining top-notch customer service is vital to business success. As your operation expands, so does the influx of customer queries. Many of these queries are repetitive, making automation a time-saving solution.
|
||||
Your support team's expertise is typically kept private, but you can still use AI to automate responses securely.
|
||||
|
||||
In this tutorial we will setup a private AI service that answers customer support queries with high accuracy and effectiveness. By leveraging Cohere's powerful models (deployed to [AWS](https://cohere.com/deployment-options/aws)) with Qdrant Hybrid Cloud, you can create a fully private customer support system. Data synchronization, facilitated by [Airbyte](https://airbyte.com/), will complete the setup.
|
||||
|
||||
[//]: # (TODO: add a link to the corresponding Qdrant Hybrid Cloud documentation: deployment on AWS)
|
||||
|
||||
|
||||
|
||||
## System design
|
||||
|
||||
The history of past interactions with your customers is not a static dataset. It is constantly evolving, as new
|
||||
questions are coming in. You probably have a ticketing system that stores all the interactions, or use a different way
|
||||
to communicate with your customers. No matter what is the communication channel, you need to bring the correct answers
|
||||
to the selected Large Language Model, and have an established way to do it in a continuous manner. Thus, we will build
|
||||
an ingestion pipeline and then a Retrieval Augmented Generation application that will use the data.
|
||||
|
||||
- **Dataset:** a [set of Frequently Asked Questions from Qdrant
|
||||
users](https://qdrant.tech/documentation/faq/qdrant-fundamentals/) as an incrementally updated Excel sheet
|
||||
- **Embedding model:** Cohere `embed-multilingual-v3.0`, to support different languages with the same pipeline
|
||||
- **Knowledge base:** Qdrant, running in Hybrid Cloud mode
|
||||
- **Ingestion pipeline:** [Airbyte](https://airbyte.com/), loading the data into Qdrant
|
||||
- **Large Language Model:** Cohere [Command-R](https://docs.cohere.com/docs/command-r)
|
||||
- **RAG:** Cohere [RAG](https://docs.cohere.com/docs/retrieval-augmented-generation-rag) using our knowledge base
|
||||
through a custom connector
|
||||
|
||||
All the selected components are compatible with the [AWS](https://aws.amazon.com/) infrastructure. Thanks to Cohere
|
||||
models' availability, you can build a fully private customer support system completely isolates data within your
|
||||
infrastructure. Also, if you have AWS credits, you can now use them without spending additional money on the models or
|
||||
semantic search layer.
|
||||
|
||||
### Data ingestion
|
||||
|
||||
Building a RAG starts with a well-curated dataset. In your specific case you may prefer loading the data directly from
|
||||
a ticketing system, such as [Zendesk Support](https://airbyte.com/connectors/zendesk-support),
|
||||
[Freshdesk](https://airbyte.com/connectors/freshdesk), or maybe integrate it with a shared inbox. However, in case of
|
||||
customer questions quality over quantity is the key. There should be a conscious decision on what data to include in the
|
||||
knowledge base, so we do not confuse the model with possibly irrelevant information. We'll assume there is an [Excel
|
||||
sheet](https://docs.airbyte.com/integrations/sources/file) available over HTTP/FTP that Airbyte can access and load into
|
||||
Qdrant in an incremental manner.
|
||||
|
||||
### Cohere <> Qdrant Connector for RAG
|
||||
|
||||
Cohere RAG relies on [connectors](https://docs.cohere.com/docs/connectors) which brings additional context to the model.
|
||||
The connector is a web service that implements a specific interface, and exposes its data through HTTP API. With that
|
||||
setup, the Large Language Model becomes responsible for communicating with the connectors, so building a prompt with the
|
||||
context is not needed anymore.
|
||||
|
||||
### Answering bot
|
||||
|
||||
Finally, we want to automate the responses and send them automatically when we are sure that the model is confident
|
||||
enough. Again, the way such an application should be created strongly depends on the system you are using within the
|
||||
customer support team. If it exposes a way to set up a webhook whenever a new question is coming in, you can create a
|
||||
web service and use it to automate the responses. In general, our bot should be created specifically for the platform
|
||||
you use, so we'll just cover the general idea here and build a simple CLI tool.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
### Cohere models on AWS
|
||||
|
||||
One of the possible ways to deploy Cohere models on AWS is to use AWS SageMaker. Cohere's website has [a detailed
|
||||
guide on how to deploy the models in that way](https://docs.cohere.com/docs/amazon-sagemaker-setup-guide), so you can
|
||||
follow the steps described there to set up your own instance.
|
||||
|
||||
### Qdrant Hybrid Cloud on AWS
|
||||
|
||||
Our documentation covers the deployment of Qdrant on AWS in your private region, so you can follow the steps described
|
||||
there to set up your own instance. The deployment process is quite straightforward, and you can have your Qdrant cluster
|
||||
up and running in a few minutes.
|
||||
|
||||
[//]: # (TODO: refer to the documentation on how to deploy Qdrant on AWS)
|
||||
|
||||
Once you perform all the steps, your Qdrant cluster should be running on a specific URL. You will need this URL and the
|
||||
API key to interact with Qdrant, so let's store them both in the environment variables:
|
||||
|
||||
```shell
|
||||
export QDRANT_URL="https://qdrant.example.com"
|
||||
export QDRANT_API_KEY="your-api-key"
|
||||
```
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
os.environ["QDRANT_URL"] = "https://qdrant.example.com"
|
||||
os.environ["QDRANT_API_KEY"] = "your-api-key"
|
||||
```
|
||||
|
||||
### Airbyte Open Source
|
||||
|
||||
Airbyte is an open-source data integration platform that helps you replicate your data in your warehouses, lakes, and
|
||||
databases. You can install it on your infrastructure and use it to load the data into Qdrant. The installation process
|
||||
for AWS EC2 is described in the [official documentation](https://docs.airbyte.com/deploying-airbyte/on-aws-ec2).
|
||||
Please follow the instructions to set up your own instance.
|
||||
|
||||
#### Setting up the connection
|
||||
|
||||
Once you have an Airbyte up and running, you can configure the connection to load the data from the respective source
|
||||
into Qdrant. The configuration will require setting up the source and destination connectors. In this tutorial we will
|
||||
use the following connectors:
|
||||
|
||||
- **Source:** [File](https://docs.airbyte.com/integrations/sources/file) to load the data from an Excel sheet
|
||||
- **Destination:** [Qdrant](https://docs.airbyte.com/integrations/destinations/qdrant) to load the data into Qdrant
|
||||
|
||||
Airbyte UI will guide you through the process of setting up the source and destination and connecting them. Here is how
|
||||
the configuration of the source might look like:
|
||||
|
||||
|
||||
|
||||
Qdrant is our target destination, so we need to set up the connection to it. We need to specify which fields should be
|
||||
included to generate the embeddings. In our case it makes complete sense to embed just the questions, as we are going
|
||||
to look for similar questions asked in the past and provide the answers.
|
||||
|
||||
|
||||
|
||||
Once we have the destination set up, we can finally configure a connection. The connection will define the schedule
|
||||
of the data synchronization.
|
||||
|
||||
|
||||
|
||||
Airbyte should now be ready to accept any data updates from the source and load them into Qdrant. You can monitor the
|
||||
progress of the synchronization in the UI.
|
||||
|
||||
## RAG connector
|
||||
|
||||
One of our previous tutorials, guides you step-by-step on [implementing custom connector for Cohere
|
||||
RAG](../cohere-rag-connector/) with Cohere Embed v3 and Qdrant. You can just point it to use your Hybrid Cloud
|
||||
Qdrant instance running on AWS. Created connector might be deployed to Amazon Web Services in various ways, even in a
|
||||
[Serverless](https://aws.amazon.com/serverless/) manner using [AWS
|
||||
Lambda](https://aws.amazon.com/lambda/?c=ser&sec=srv).
|
||||
|
||||
In general, RAG connector has to expose a single endpoint that will accept POST requests with `query` parameter and
|
||||
return the matching documents as JSON document with a specific structure. Our FastAPI implementation created [in the
|
||||
related tutorial](../cohere-rag-connector/) is a perfect fit for this task. The only difference is that you
|
||||
should point it to the Cohere models and Qdrant running on AWS infrastructure.
|
||||
|
||||
> Our connector is a lightweight web service that exposes a single endpoint and glues the Cohere embedding model with
|
||||
> our Qdrant Hybrid Cloud instance. Thus, it perfectly fits the serverless architecture, requiring no additional
|
||||
> infrastructure to run.
|
||||
|
||||
You can also run the connector as another service within your [Kubernetes cluster running on AWS
|
||||
(EKS)](https://aws.amazon.com/eks/), or by launching an [EC2](https://aws.amazon.com/ec2/) compute instance. This step
|
||||
is dependent on the way you deploy your other services, so we'll leave it to you to decide how to run the connector.
|
||||
|
||||
Eventually, the web service should be available under a specific URL, and it's a good practice to store it in the
|
||||
environment variable, so the other services can easily access it.
|
||||
|
||||
```shell
|
||||
export RAG_CONNECTOR_URL="https://rag-connector.example.com/search"
|
||||
```
|
||||
|
||||
```python
|
||||
os.environ["RAG_CONNECTOR_URL"] = "https://rag-connector.example.com/search"
|
||||
```
|
||||
|
||||
## Customer interface
|
||||
|
||||
At this part we have all the data loaded into Qdrant, and the RAG connector is ready to serve the relevant context. The
|
||||
last missing piece is the customer interface, that will call the Command model to create the answer. Such a system
|
||||
should be built specifically for the platform you use and integrated into its workflow, but we will build the strong
|
||||
foundation for it and show how to use it in a simple CLI tool.
|
||||
|
||||
> Our application does not have to connect to Qdrant anymore, as the model will connect to the RAG connector directly.
|
||||
|
||||
First of all, we have to create a connection to Cohere services through the Cohere SDK.
|
||||
|
||||
```python
|
||||
import cohere
|
||||
|
||||
# Create a Cohere client pointing to the AWS instance
|
||||
cohere_client = cohere.Client(...)
|
||||
```
|
||||
|
||||
Next, our connector should be registered. **Please make sure to do it once, and store the id of the connector in the
|
||||
environment variable or in any other way that will be accessible to the application.**
|
||||
|
||||
```python
|
||||
import os
|
||||
|
||||
connector_response = cohere_client.connectors.create(
|
||||
name="customer-support",
|
||||
url=os.environ["RAG_CONNECTOR_URL"],
|
||||
)
|
||||
|
||||
# The id returned by the API should be stored for future use
|
||||
connector_id = connector_response.connector.id
|
||||
```
|
||||
|
||||
Finally, we can create a prompt and get the answer from the model. Additionally, we define which of the connectors
|
||||
should be used to provide the context, as we may have multiple connectors and want to use specific ones, depending on
|
||||
some conditions. Let's start with asking a question.
|
||||
|
||||
```python
|
||||
query = "Why Qdrant does not return my vectors?"
|
||||
```
|
||||
|
||||
Now we can send the query to the model, get the response, and possibly send it back to the customer.
|
||||
|
||||
```python
|
||||
response = cohere_client.chat(
|
||||
message=query,
|
||||
connectors=[
|
||||
cohere.ChatConnector(id=connector_id),
|
||||
],
|
||||
model="command-r",
|
||||
)
|
||||
|
||||
print(response.text)
|
||||
```
|
||||
|
||||
The output should be the answer to the question, generated by the model, for example:
|
||||
|
||||
> Qdrant is set up by default to minimize network traffic and therefore doesn't return vectors in search results. However, you can make Qdrant return your vectors by setting the 'with_vector' parameter of the Search/Scroll function to true.
|
||||
|
||||
Customer support should not be fully automated, as some completely new issues might require human intervention. We
|
||||
should play with prompt engineering and expect the model to provide the answer with a certain confidence level. If the
|
||||
confidence is too low, we should not send the answer automatically but present it to the support team for review.
|
||||
|
||||
## Wrapping up
|
||||
|
||||
This tutorial shows how to build a fully private customer support system using Cohere models, Qdrant Hybrid Cloud, and
|
||||
Airbyte, which runs on AWS infrastructure. You can ensure your data does not leave your premises and focus on providing
|
||||
the best customer support experience without bothering your team with repetitive tasks.
|
||||
Reference in New Issue
Block a user