diff --git a/qdrant-landing/content/documentation/tutorials/customer-support-cohere-airbyte.md b/qdrant-landing/content/documentation/tutorials/customer-support-cohere-airbyte.md new file mode 100644 index 000000000..404549a3f --- /dev/null +++ b/qdrant-landing/content/documentation/tutorials/customer-support-cohere-airbyte.md @@ -0,0 +1,232 @@ +--- +title: Automate customer support tasks +weight: 26 +--- + +# Build a RAG system to answer customer support queries + +| Time: 120 min | Level: Advanced | | +| --- | ----------- | ----------- |----------- | + +Maintaining top-notch customer service is vital to business success. As your operation expands, so does the influx of customer queries. Many of these queries are repetitive, making automation a time-saving solution. +Your support team's expertise is typically kept private, but you can still use AI to automate responses securely. + +In this tutorial we will setup a private AI service that answers customer support queries with high accuracy and effectiveness. By leveraging Cohere's powerful models (deployed to [AWS](https://cohere.com/deployment-options/aws)) with Qdrant Hybrid Cloud, you can create a fully private customer support system. Data synchronization, facilitated by [Airbyte](https://airbyte.com/), will complete the setup. + +[//]: # (TODO: add a link to the corresponding Qdrant Hybrid Cloud documentation: deployment on AWS) + +![Architecture diagram](/documentation/tutorials/customer-support-cohere-airbyte/architecture-diagram.png) + +## System design + +The history of past interactions with your customers is not a static dataset. It is constantly evolving, as new +questions are coming in. You probably have a ticketing system that stores all the interactions, or use a different way +to communicate with your customers. No matter what is the communication channel, you need to bring the correct answers +to the selected Large Language Model, and have an established way to do it in a continuous manner. Thus, we will build +an ingestion pipeline and then a Retrieval Augmented Generation application that will use the data. + +- **Dataset:** a [set of Frequently Asked Questions from Qdrant + users](https://qdrant.tech/documentation/faq/qdrant-fundamentals/) as an incrementally updated Excel sheet +- **Embedding model:** Cohere `embed-multilingual-v3.0`, to support different languages with the same pipeline +- **Knowledge base:** Qdrant, running in Hybrid Cloud mode +- **Ingestion pipeline:** [Airbyte](https://airbyte.com/), loading the data into Qdrant +- **Large Language Model:** Cohere [Command-R](https://docs.cohere.com/docs/command-r) +- **RAG:** Cohere [RAG](https://docs.cohere.com/docs/retrieval-augmented-generation-rag) using our knowledge base + through a custom connector + +All the selected components are compatible with the [AWS](https://aws.amazon.com/) infrastructure. Thanks to Cohere +models' availability, you can build a fully private customer support system completely isolates data within your +infrastructure. Also, if you have AWS credits, you can now use them without spending additional money on the models or +semantic search layer. + +### Data ingestion + +Building a RAG starts with a well-curated dataset. In your specific case you may prefer loading the data directly from +a ticketing system, such as [Zendesk Support](https://airbyte.com/connectors/zendesk-support), +[Freshdesk](https://airbyte.com/connectors/freshdesk), or maybe integrate it with a shared inbox. However, in case of +customer questions quality over quantity is the key. There should be a conscious decision on what data to include in the +knowledge base, so we do not confuse the model with possibly irrelevant information. We'll assume there is an [Excel +sheet](https://docs.airbyte.com/integrations/sources/file) available over HTTP/FTP that Airbyte can access and load into +Qdrant in an incremental manner. + +### Cohere <> Qdrant Connector for RAG + +Cohere RAG relies on [connectors](https://docs.cohere.com/docs/connectors) which brings additional context to the model. +The connector is a web service that implements a specific interface, and exposes its data through HTTP API. With that +setup, the Large Language Model becomes responsible for communicating with the connectors, so building a prompt with the +context is not needed anymore. + +### Answering bot + +Finally, we want to automate the responses and send them automatically when we are sure that the model is confident +enough. Again, the way such an application should be created strongly depends on the system you are using within the +customer support team. If it exposes a way to set up a webhook whenever a new question is coming in, you can create a +web service and use it to automate the responses. In general, our bot should be created specifically for the platform +you use, so we'll just cover the general idea here and build a simple CLI tool. + +## Prerequisites + +### Cohere models on AWS + +One of the possible ways to deploy Cohere models on AWS is to use AWS SageMaker. Cohere's website has [a detailed +guide on how to deploy the models in that way](https://docs.cohere.com/docs/amazon-sagemaker-setup-guide), so you can +follow the steps described there to set up your own instance. + +### Qdrant Hybrid Cloud on AWS + +Our documentation covers the deployment of Qdrant on AWS in your private region, so you can follow the steps described +there to set up your own instance. The deployment process is quite straightforward, and you can have your Qdrant cluster +up and running in a few minutes. + +[//]: # (TODO: refer to the documentation on how to deploy Qdrant on AWS) + +Once you perform all the steps, your Qdrant cluster should be running on a specific URL. You will need this URL and the +API key to interact with Qdrant, so let's store them both in the environment variables: + +```shell +export QDRANT_URL="https://qdrant.example.com" +export QDRANT_API_KEY="your-api-key" +``` + +```python +import os + +os.environ["QDRANT_URL"] = "https://qdrant.example.com" +os.environ["QDRANT_API_KEY"] = "your-api-key" +``` + +### Airbyte Open Source + +Airbyte is an open-source data integration platform that helps you replicate your data in your warehouses, lakes, and +databases. You can install it on your infrastructure and use it to load the data into Qdrant. The installation process +for AWS EC2 is described in the [official documentation](https://docs.airbyte.com/deploying-airbyte/on-aws-ec2). +Please follow the instructions to set up your own instance. + +#### Setting up the connection + +Once you have an Airbyte up and running, you can configure the connection to load the data from the respective source +into Qdrant. The configuration will require setting up the source and destination connectors. In this tutorial we will +use the following connectors: + +- **Source:** [File](https://docs.airbyte.com/integrations/sources/file) to load the data from an Excel sheet +- **Destination:** [Qdrant](https://docs.airbyte.com/integrations/destinations/qdrant) to load the data into Qdrant + +Airbyte UI will guide you through the process of setting up the source and destination and connecting them. Here is how +the configuration of the source might look like: + +![Airbyte source configuration](/documentation/tutorials/customer-support-cohere-airbyte/airbyte-excel-source.png) + +Qdrant is our target destination, so we need to set up the connection to it. We need to specify which fields should be +included to generate the embeddings. In our case it makes complete sense to embed just the questions, as we are going +to look for similar questions asked in the past and provide the answers. + +![Airbyte destination configuration](/documentation/tutorials/customer-support-cohere-airbyte/airbyte-qdrant-destination.png) + +Once we have the destination set up, we can finally configure a connection. The connection will define the schedule +of the data synchronization. + +![Airbyte connection configuration](/documentation/tutorials/customer-support-cohere-airbyte/airbyte-connection.png) + +Airbyte should now be ready to accept any data updates from the source and load them into Qdrant. You can monitor the +progress of the synchronization in the UI. + +## RAG connector + +One of our previous tutorials, guides you step-by-step on [implementing custom connector for Cohere +RAG](../cohere-rag-connector/) with Cohere Embed v3 and Qdrant. You can just point it to use your Hybrid Cloud +Qdrant instance running on AWS. Created connector might be deployed to Amazon Web Services in various ways, even in a +[Serverless](https://aws.amazon.com/serverless/) manner using [AWS +Lambda](https://aws.amazon.com/lambda/?c=ser&sec=srv). + +In general, RAG connector has to expose a single endpoint that will accept POST requests with `query` parameter and +return the matching documents as JSON document with a specific structure. Our FastAPI implementation created [in the +related tutorial](../cohere-rag-connector/) is a perfect fit for this task. The only difference is that you +should point it to the Cohere models and Qdrant running on AWS infrastructure. + +> Our connector is a lightweight web service that exposes a single endpoint and glues the Cohere embedding model with +> our Qdrant Hybrid Cloud instance. Thus, it perfectly fits the serverless architecture, requiring no additional +> infrastructure to run. + +You can also run the connector as another service within your [Kubernetes cluster running on AWS +(EKS)](https://aws.amazon.com/eks/), or by launching an [EC2](https://aws.amazon.com/ec2/) compute instance. This step +is dependent on the way you deploy your other services, so we'll leave it to you to decide how to run the connector. + +Eventually, the web service should be available under a specific URL, and it's a good practice to store it in the +environment variable, so the other services can easily access it. + +```shell +export RAG_CONNECTOR_URL="https://rag-connector.example.com/search" +``` + +```python +os.environ["RAG_CONNECTOR_URL"] = "https://rag-connector.example.com/search" +``` + +## Customer interface + +At this part we have all the data loaded into Qdrant, and the RAG connector is ready to serve the relevant context. The +last missing piece is the customer interface, that will call the Command model to create the answer. Such a system +should be built specifically for the platform you use and integrated into its workflow, but we will build the strong +foundation for it and show how to use it in a simple CLI tool. + +> Our application does not have to connect to Qdrant anymore, as the model will connect to the RAG connector directly. + +First of all, we have to create a connection to Cohere services through the Cohere SDK. + +```python +import cohere + +# Create a Cohere client pointing to the AWS instance +cohere_client = cohere.Client(...) +``` + +Next, our connector should be registered. **Please make sure to do it once, and store the id of the connector in the +environment variable or in any other way that will be accessible to the application.** + +```python +import os + +connector_response = cohere_client.connectors.create( + name="customer-support", + url=os.environ["RAG_CONNECTOR_URL"], +) + +# The id returned by the API should be stored for future use +connector_id = connector_response.connector.id +``` + +Finally, we can create a prompt and get the answer from the model. Additionally, we define which of the connectors +should be used to provide the context, as we may have multiple connectors and want to use specific ones, depending on +some conditions. Let's start with asking a question. + +```python +query = "Why Qdrant does not return my vectors?" +``` + +Now we can send the query to the model, get the response, and possibly send it back to the customer. + +```python +response = cohere_client.chat( + message=query, + connectors=[ + cohere.ChatConnector(id=connector_id), + ], + model="command-r", +) + +print(response.text) +``` + +The output should be the answer to the question, generated by the model, for example: + +> Qdrant is set up by default to minimize network traffic and therefore doesn't return vectors in search results. However, you can make Qdrant return your vectors by setting the 'with_vector' parameter of the Search/Scroll function to true. + +Customer support should not be fully automated, as some completely new issues might require human intervention. We +should play with prompt engineering and expect the model to provide the answer with a certain confidence level. If the +confidence is too low, we should not send the answer automatically but present it to the support team for review. + +## Wrapping up + +This tutorial shows how to build a fully private customer support system using Cohere models, Qdrant Hybrid Cloud, and +Airbyte, which runs on AWS infrastructure. You can ensure your data does not leave your premises and focus on providing +the best customer support experience without bothering your team with repetitive tasks. \ No newline at end of file diff --git a/qdrant-landing/static/documentation/frameworks/Langchain4J-social-preview.png b/qdrant-landing/static/documentation/frameworks/Langchain4J-social-preview.png deleted file mode 100644 index 177cc270d..000000000 Binary files a/qdrant-landing/static/documentation/frameworks/Langchain4J-social-preview.png and /dev/null differ diff --git a/qdrant-landing/static/documentation/frameworks/langchain4j-social-preview.png b/qdrant-landing/static/documentation/frameworks/langchain4j-social-preview.png deleted file mode 100644 index 177cc270d..000000000 Binary files a/qdrant-landing/static/documentation/frameworks/langchain4j-social-preview.png and /dev/null differ diff --git a/qdrant-landing/static/documentation/tutorials/customer-support-cohere-airbyte/airbyte-connection.png b/qdrant-landing/static/documentation/tutorials/customer-support-cohere-airbyte/airbyte-connection.png new file mode 100644 index 000000000..509d7fcdc Binary files /dev/null and b/qdrant-landing/static/documentation/tutorials/customer-support-cohere-airbyte/airbyte-connection.png differ diff --git a/qdrant-landing/static/documentation/tutorials/customer-support-cohere-airbyte/airbyte-excel-source.png b/qdrant-landing/static/documentation/tutorials/customer-support-cohere-airbyte/airbyte-excel-source.png new file mode 100644 index 000000000..0a2f1a955 Binary files /dev/null and b/qdrant-landing/static/documentation/tutorials/customer-support-cohere-airbyte/airbyte-excel-source.png differ diff --git a/qdrant-landing/static/documentation/tutorials/customer-support-cohere-airbyte/airbyte-qdrant-destination.png b/qdrant-landing/static/documentation/tutorials/customer-support-cohere-airbyte/airbyte-qdrant-destination.png new file mode 100644 index 000000000..4fca53524 Binary files /dev/null and b/qdrant-landing/static/documentation/tutorials/customer-support-cohere-airbyte/airbyte-qdrant-destination.png differ diff --git a/qdrant-landing/static/documentation/tutorials/customer-support-cohere-airbyte/architecture-diagram.png b/qdrant-landing/static/documentation/tutorials/customer-support-cohere-airbyte/architecture-diagram.png new file mode 100644 index 000000000..2d0b7f8dc Binary files /dev/null and b/qdrant-landing/static/documentation/tutorials/customer-support-cohere-airbyte/architecture-diagram.png differ