---
title: Automate customer support
weight: 26
---

# Unleash your customer support from repetitive tasks

| Time: 120 min | Level: Advanced |  |
| --- | ----------- | ----------- |----------- |

Maintaining a proper level of service plays an important role in the success of any business. The customer support team 
is the first line of defense when it comes to addressing customer queries and concerns. However, as the business grows, 
the volume of customer queries also increases, making it difficult for the support team to handle them efficiently. On
the other hand, the majority of the queries is repetitive and if someone in the team has already answered a similar 
question, automating the response can save a lot of time and effort. In times of Large Language Models that does not 
sound like a science fiction anymore.

The know-how of the customer support team is usually a proprietary knowledge base that is not available to the public.
You never want this data to leave your infrastructure. However, you can still leverage the power of AI to automate the
responses, thanks to private deployments of the state-of-the-art tools. Cohere’s powerful models [might be deployed to 
Oracle Cloud](https://cohere.com/deployment-options/oracle) and used together with Qdrant Hybrid Cloud to build a fully 
private customer support system. One missing piece is the data synchronization, and this is where 
[Airbyte](https://airbyte.com/) comes into play.

[//]: # (TODO: add a link to the corresponding Qdrant Hybrid Cloud documentation: deployment on OCI)

TODO: add a diagram presenting all the components

## System design

The history of past interactions with your customers is not a static dataset. It is constantly evolving, as new 
questions are coming in. You probably have a ticketing system that stores all the interactions, or use a different way
to communicate with your customers. No matter what is the communication channel, you need to bring the correct answers
to the selected Large Language Model, and have an established way to do it in a continuous manner. Thus, we will build
an ingestion pipeline and then a Retrieval Augmented Generation application that will use the data.

- **Dataset:** a [set of Frequently Asked Questions from Qdrant 
  users](https://qdrant.tech/documentation/faq/qdrant-fundamentals/) as an incrementally updated Excel sheet
- **Embedding model:** Cohere `embed-multilingual-v3.0`, to support different languages with the same pipeline
- **Knowledge base:** Qdrant, running in Hybrid Cloud mode
- **Ingestion pipeline:** [Airbyte](https://airbyte.com/), loading the data into Qdrant
- **Large Language Model:** Cohere [Command-R](https://docs.cohere.com/docs/command-r)
- **RAG:** Cohere [RAG](https://docs.cohere.com/docs/retrieval-augmented-generation-rag) using our knowledge base 
  through a custom connector

All the selected components might be running on [Oracle Cloud](https://www.oracle.com/cloud/) infrastructure only. Thanks to the availability of 
the Cohere models on OCI, you can build a fully private customer support system that does not require any data to leave
your infrastructure.

### Data ingestion

Building a RAG starts with a well-curated dataset. In your specific case you may prefer loading the data directly from
a ticketing system, such as [Zendesk Support](https://airbyte.com/connectors/zendesk-support), 
[Freshdesk](https://airbyte.com/connectors/freshdesk), or maybe integrate it with a shared inbox. However, in case of 
customer questions quality over quantity is the key. There should be a conscious decision on what data to include in the
knowledge base, so we do not confuse the model with possibly irrelevant information. We'll assume there is an [Excel
sheet](https://docs.airbyte.com/integrations/sources/file) available over HTTP/FTP that Airbyte can access and load into 
Qdrant in an incremental manner.

### Cohere <> Qdrant Connector for RAG

Cohere RAG relies on [connectors](https://docs.cohere.com/docs/connectors) which brings additional context to the model. 
The connector is a web service that implements a specific interface, and exposes its data through HTTP API. With that 
setup, the Large Language Model becomes responsible for communicating with the connectors, so building a prompt with the 
context is not needed anymore.

### Answering bot

Finally, we want to automate the responses and send them automatically when we are sure that the model is confident
enough. Again, the way such an application should be created strongly depends on the system you are using within the
customer support team. If it exposes a way to set up a webhook whenever a new question is coming in, you can create a 
web service and use it to automate the responses. In general, our bot should be created specifically for the platform 
you use, so we'll just cover the general idea here and build a simple CLI tool.

## Prerequisites

### Qdrant Hybrid Cloud on OCI

Our documentation covers the deployment of Qdrant on Oracle Cloud, so you can follow the steps described there to set up
your own instance. The deployment process is quite straightforward, and you can have your Qdrant cluster up and running
in a few minutes.

[//]: # (TODO: refer to the documentation on how to deploy Qdrant on Oracle Cloud)

Once you perform all the steps, your Qdrant cluster should be running on a specific URL. You will need this URL and the 
API key to interact with Qdrant, so let's store them both in the environment variables:

```shell
export QDRANT_URL="https://qdrant.example.com"
export QDRANT_API_KEY="your-api-key"
```

```python
import os

os.environ["QDRANT_URL"] = "https://qdrant.example.com"
os.environ["QDRANT_API_KEY"] = "your-api-key"
```

### Airbyte Open Source

Airbyte is an open-source data integration platform that helps you replicate your data in your warehouses, lakes, and
databases. You can install it on your infrastructure and use it to load the data into Qdrant. The installation process
for Oracle Cloud is described in the [official documentation](https://docs.airbyte.com/deploying-airbyte/on-oci-vm).
Please follow the instructions to set up your own instance.

#### Setting up the connection

Once you have an Airbyte up and running, you can configure the connection to load the data from the respective source 
into Qdrant. The configuration will require setting up the source and destination connectors. In this tutorial we will 
use the following connectors:

- **Source:** [File](https://docs.airbyte.com/integrations/sources/file) to load the data from an Excel sheet
- **Destination:** [Qdrant](https://docs.airbyte.com/integrations/destinations/qdrant) to load the data into Qdrant

Airbyte UI will guide you through the process of setting up the source and destination and connecting them. Here is how
the configuration of the source might look like:

![Airbyte source configuration](/documentation/tutorials/customer-support-cohere-airbyte/airbyte-excel-source.png)

Qdrant is our target destination, so we need to set up the connection to it. We need to specify which fields should be
included to generate the embeddings. In our case it makes complete sense to embed just the questions, as we are going
to look for similar questions asked in the past and provide the answers. 

![Airbyte destination configuration](/documentation/tutorials/customer-support-cohere-airbyte/airbyte-qdrant-destination.png)

Once we have the destination set up, we can finally configure a connection. The connection will define the schedule
of the data synchronization. 

![Airbyte connection configuration](/documentation/tutorials/customer-support-cohere-airbyte/airbyte-connection.png)

Airbyte should now be ready to accept any data updates from the source and load them into Qdrant. You can monitor the
progress of the synchronization in the UI.

## RAG connector

One of our previous tutorials, guides you step-by-step on [implementing custom connector for Cohere 
RAG](../cohere-rag-connector/) with Cohere Embed v3 and Qdrant. You can just point it to use your Hybrid Cloud
Qdrant instance running on OCI. Created connector might be deployed to Oracle Cloud in various ways, even in a 
[Serverless](https://developer.oracle.com/learn/use-cases.html#serverless) manner using [Oracle Cloud Infrastructure 
Functions](https://docs.oracle.com/en-us/iaas/Content/Functions/home.htm#top).

In general, RAG connector has to expose a single endpoint that will accept POST requests with `query` parameter and 
return the matching documents as JSON document with a specific structure. Our FastAPI implementation created [in the
related tutorial](../tutorials/cohere-rag-connector/) is a perfect fit for this task. The only difference is that you
should point it to the Cohere models and Qdrant running on Oracle Cloud infrastructure.

> Our connector is a lightweight web service that exposes a single endpoint and glues the Cohere embedding model with 
> our Qdrant Hybrid Cloud instance. Thus, it perfectly fits the serverless architecture, requiring no additional 
> infrastructure to run.

You can also run the connector as another service within your [Kubernetes cluster running on Oracle Cloud 
(OKE)](https://www.oracle.com/cloud/cloud-native/container-engine-kubernetes/). This step is dependent on the way you
deploy your other services, so we'll leave it to you to decide how to run the connector.

Eventually, the web service should be available under a specific URL, and it's a good practice to store it in the
environment variable, so the other services can easily access it.

```shell
export RAG_CONNECTOR_URL="https://rag-connector.example.com/search"
```

```python
os.environ["RAG_CONNECTOR_URL"] = "https://rag-connector.example.com/search"
```

[//]: # (TODO: refer to the tutorial on a custom RAG connector for Cohere)
[//]: # (See: https://github.com/qdrant/landing_page/pull/761)

## Customer interface

At this part we have all the data loaded into Qdrant, and the RAG connector is ready to serve the relevant context. The 
last missing piece is the customer interface, that will call the Command-R model to create the answer. Such a system
should be built specifically for the platform you use and integrated into its workflow, but we will build the strong 
foundation for it and show how to use it in a simple CLI tool.

> Our application does not have to connect to Qdrant anymore, as the model will connect to the RAG connector directly.

First of all, we have to create a connection to Cohere services through the Cohere SDK.

```python
import cohere

# Create a Cohere client pointing to the Oracle Cloud instance
cohere_client = cohere.Client(...)
```

Next, our connector should be registered. **Please make sure to do it once, and store the id of the connector in the
environment variable or in any other way that will be accessible to the application.**

```python
import os

connector_response = cohere_client.connectors.create(
    name="customer-support",
    url=os.environ["RAG_CONNECTOR_URL"],
)

# The id returned by the API should be stored for future use
connector_id = connector_response.connector.id
```

Finally, we can create a prompt and get the answer from the model. Additionally, we define which of the connectors 
should be used to provide the context, as we may have multiple connectors and want to use specific ones, depending on 
some conditions. Let's start with asking a question.

```python
query = "Why Qdrant does not return my vectors?"
```

Now we can send the query to the model, get the response, and possibly send it back to the customer.

```python
response = cohere_client.chat(
    message=query,
    connectors=[
        cohere.ChatConnector(id=connector_id),
    ],
    model="command-r",
)

print(response.text)
```

The output should be the answer to the question, generated by the model, for example:

> Qdrant is set up by default to minimize network traffic and therefore doesn't return vectors in search results. However, you can make Qdrant return your vectors by setting the 'with_vector' parameter of the Search/Scroll function to true.

Customer support should not be fully automated, as some completely new issues might require human intervention. We 
should play with prompt engineering and expect the model to provide the answer with a certain confidence level. If the 
confidence is too low, we should not send the answer automatically but present it to the support team for review.

## Wrapping up

This tutorial shows how to build a fully private customer support system using Cohere models, Qdrant Hybrid Cloud, and 
Airbyte, which runs on Oracle Cloud infrastructure. You can ensure your data does not leave your premises and focus on 
providing the best customer support experience without bothering your team with repetitive tasks.