Add tutorial visuals

This commit is contained in:
Kacper Łukawski
2024-04-10 09:19:46 +02:00
parent 24ece2cf91
commit a1a704404d
3 changed files with 6 additions and 4 deletions
@@ -14,7 +14,7 @@ Companies want their data to be kept and processed within specific geographical
[//]: # (TODO: add link to Qdrant Hybrid Cloud above)
TODO: add a system diagram with all the components
![Architecture diagram](/documentation/tutorials/contract-management-stackit-aleph-alpha/architecture-diagram.png)
## Components
@@ -25,9 +25,7 @@ Directory.
> **Note:** In this tutorial, we are going to build a solid foundation for such a system. However, it is up to your organization's setup to implement the entire solution.
[//]: # (TODO: upload the dataset to GCP and link it below)
- **Dataset** - a collection of documents, using different formats, such as PDF or DOCx
- **Dataset** - a collection of documents, using different formats, such as PDF or DOCx, scraped from internet
- **Asymmetric semantic embeddings** - [Aleph Alpha embedding](https://docs.aleph-alpha.com/api/semantic-embed/) to
convert the queries and the documents into vectors
- **Large Language Model** - the [Luminous-extended-control
@@ -155,6 +153,10 @@ documents = {
}
```
This is how the documents might look like:
![Example of the indexed document](/documentation/tutorials/contract-management-stackit-aleph-alpha/indexed-document.png)
Each has to be split into chunks first; there is no silver bullet. Our chunking algorithm will be simple and based on
recursive splitting, with the maximum chunk size of 500 characters and the overlap of 100 characters.