Add tutorial visuals

This commit is contained in:
Kacper Łukawski
2024-04-10 09:19:46 +02:00
parent 24ece2cf91
commit a1a704404d
3 changed files with 6 additions and 4 deletions
@@ -14,7 +14,7 @@ Companies want their data to be kept and processed within specific geographical
[//]: # (TODO: add link to Qdrant Hybrid Cloud above) [//]: # (TODO: add link to Qdrant Hybrid Cloud above)
TODO: add a system diagram with all the components ![Architecture diagram](/documentation/tutorials/contract-management-stackit-aleph-alpha/architecture-diagram.png)
## Components ## Components
@@ -25,9 +25,7 @@ Directory.
> **Note:** In this tutorial, we are going to build a solid foundation for such a system. However, it is up to your organization's setup to implement the entire solution. > **Note:** In this tutorial, we are going to build a solid foundation for such a system. However, it is up to your organization's setup to implement the entire solution.
[//]: # (TODO: upload the dataset to GCP and link it below) - **Dataset** - a collection of documents, using different formats, such as PDF or DOCx, scraped from internet
- **Dataset** - a collection of documents, using different formats, such as PDF or DOCx
- **Asymmetric semantic embeddings** - [Aleph Alpha embedding](https://docs.aleph-alpha.com/api/semantic-embed/) to - **Asymmetric semantic embeddings** - [Aleph Alpha embedding](https://docs.aleph-alpha.com/api/semantic-embed/) to
convert the queries and the documents into vectors convert the queries and the documents into vectors
- **Large Language Model** - the [Luminous-extended-control - **Large Language Model** - the [Luminous-extended-control
@@ -155,6 +153,10 @@ documents = {
} }
``` ```
This is how the documents might look like:
![Example of the indexed document](/documentation/tutorials/contract-management-stackit-aleph-alpha/indexed-document.png)
Each has to be split into chunks first; there is no silver bullet. Our chunking algorithm will be simple and based on Each has to be split into chunks first; there is no silver bullet. Our chunking algorithm will be simple and based on
recursive splitting, with the maximum chunk size of 500 characters and the overlap of 100 characters. recursive splitting, with the maximum chunk size of 500 characters and the overlap of 100 characters.
Binary file not shown.

After

Width:  |  Height:  |  Size: 99 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 163 KiB