Move assets to documentation/examples
@@ -29,7 +29,7 @@ Our application will consist of two main processes: indexing and searching. Lang
|
|||||||
as we will use a few components, including Cohere and Qdrant, as well as some OCI services. Here is a high-level
|
as we will use a few components, including Cohere and Qdrant, as well as some OCI services. Here is a high-level
|
||||||
overview of the architecture:
|
overview of the architecture:
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
### Prerequisites
|
### Prerequisites
|
||||||
|
|
||||||
@@ -122,7 +122,7 @@ Service, so we can easily access the models.
|
|||||||
Our dataset will be fairly simple, as it will consist of the questions and answers from the [Oracle Cloud Free Tier
|
Our dataset will be fairly simple, as it will consist of the questions and answers from the [Oracle Cloud Free Tier
|
||||||
FAQ page](https://www.oracle.com/cloud/free/faq/).
|
FAQ page](https://www.oracle.com/cloud/free/faq/).
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
Questions and answers are presented in an HTML format, but we don't want to manually extract the text and adapt it for
|
Questions and answers are presented in an HTML format, but we don't want to manually extract the text and adapt it for
|
||||||
each subpage. Instead, we will use the `WebBaseLoader` that just loads the HTML content from given URL and converts it
|
each subpage. Instead, we will use the `WebBaseLoader` that just loads the HTML content from given URL and converts it
|
||||||
|
|||||||
@@ -25,7 +25,7 @@ this setup as a knowledge base providing the relevant pieces of documents for a
|
|||||||
Hybrid Cloud mode on Vultr. The last missing piece, the DSPy application will be also running in the same environment.
|
Hybrid Cloud mode on Vultr. The last missing piece, the DSPy application will be also running in the same environment.
|
||||||
If you work in a regulated industry, or just need to keep your data private, this tutorial is for you.
|
If you work in a regulated industry, or just need to keep your data private, this tutorial is for you.
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
## Configuring the environment
|
## Configuring the environment
|
||||||
|
|
||||||
|
|||||||
@@ -16,7 +16,7 @@ Companies want their data to be kept and processed within specific geographical
|
|||||||
|
|
||||||
[//]: # (TODO: add link to Qdrant Hybrid Cloud above)
|
[//]: # (TODO: add link to Qdrant Hybrid Cloud above)
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
## Components
|
## Components
|
||||||
|
|
||||||
@@ -157,7 +157,7 @@ documents = {
|
|||||||
|
|
||||||
This is how the documents might look like:
|
This is how the documents might look like:
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
Each has to be split into chunks first; there is no silver bullet. Our chunking algorithm will be simple and based on
|
Each has to be split into chunks first; there is no silver bullet. Our chunking algorithm will be simple and based on
|
||||||
recursive splitting, with the maximum chunk size of 500 characters and the overlap of 100 characters.
|
recursive splitting, with the maximum chunk size of 500 characters and the overlap of 100 characters.
|
||||||
|
|||||||
@@ -17,7 +17,7 @@ In this tutorial we will setup a private AI service that answers customer suppor
|
|||||||
|
|
||||||
[//]: # (TODO: add a link to the corresponding Qdrant Hybrid Cloud documentation: deployment on AWS)
|
[//]: # (TODO: add a link to the corresponding Qdrant Hybrid Cloud documentation: deployment on AWS)
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
## System design
|
## System design
|
||||||
|
|
||||||
@@ -116,18 +116,18 @@ use the following connectors:
|
|||||||
Airbyte UI will guide you through the process of setting up the source and destination and connecting them. Here is how
|
Airbyte UI will guide you through the process of setting up the source and destination and connecting them. Here is how
|
||||||
the configuration of the source might look like:
|
the configuration of the source might look like:
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
Qdrant is our target destination, so we need to set up the connection to it. We need to specify which fields should be
|
Qdrant is our target destination, so we need to set up the connection to it. We need to specify which fields should be
|
||||||
included to generate the embeddings. In our case it makes complete sense to embed just the questions, as we are going
|
included to generate the embeddings. In our case it makes complete sense to embed just the questions, as we are going
|
||||||
to look for similar questions asked in the past and provide the answers.
|
to look for similar questions asked in the past and provide the answers.
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
Once we have the destination set up, we can finally configure a connection. The connection will define the schedule
|
Once we have the destination set up, we can finally configure a connection. The connection will define the schedule
|
||||||
of the data synchronization.
|
of the data synchronization.
|
||||||
|
|
||||||

|

|
||||||
|
|
||||||
Airbyte should now be ready to accept any data updates from the source and load them into Qdrant. You can monitor the
|
Airbyte should now be ready to accept any data updates from the source and load them into Qdrant. You can monitor the
|
||||||
progress of the synchronization in the UI.
|
progress of the synchronization in the UI.
|
||||||
|
|||||||
|
Before Width: | Height: | Size: 99 KiB After Width: | Height: | Size: 99 KiB |
|
Before Width: | Height: | Size: 163 KiB After Width: | Height: | Size: 163 KiB |
|
Before Width: | Height: | Size: 93 KiB After Width: | Height: | Size: 93 KiB |
|
Before Width: | Height: | Size: 44 KiB After Width: | Height: | Size: 44 KiB |
|
Before Width: | Height: | Size: 98 KiB After Width: | Height: | Size: 98 KiB |
|
Before Width: | Height: | Size: 105 KiB After Width: | Height: | Size: 105 KiB |
|
Before Width: | Height: | Size: 106 KiB After Width: | Height: | Size: 106 KiB |
|
Before Width: | Height: | Size: 54 KiB After Width: | Height: | Size: 54 KiB |
|
Before Width: | Height: | Size: 96 KiB After Width: | Height: | Size: 96 KiB |
|
Before Width: | Height: | Size: 152 KiB After Width: | Height: | Size: 152 KiB |