mirror of
https://github.com/qdrant/landing_page.git
synced 2026-09-29 16:08:32 +02:00
docs: Sycamore Integration (#1228)
This commit is contained in:
@@ -24,5 +24,6 @@ weight: 20
|
|||||||
| [Pandas-AI](/documentation/frameworks/pandas-ai/) | Python library to query/visualize your data (CSV, XLSX, PostgreSQL, etc.) in natural language |
|
| [Pandas-AI](/documentation/frameworks/pandas-ai/) | Python library to query/visualize your data (CSV, XLSX, PostgreSQL, etc.) in natural language |
|
||||||
| [Semantic Router](/documentation/frameworks/semantic-router/) | Python library to build a decision-making layer for AI applications using vector search. |
|
| [Semantic Router](/documentation/frameworks/semantic-router/) | Python library to build a decision-making layer for AI applications using vector search. |
|
||||||
| [Spring AI](/documentation/frameworks/spring-ai/) | Java AI framework for building with Spring design principles such as portability and modular design. |
|
| [Spring AI](/documentation/frameworks/spring-ai/) | Java AI framework for building with Spring design principles such as portability and modular design. |
|
||||||
|
| [Sycamore](/documentation/frameworks/sycamore/) | Document processing engine for ETL, RAG, LLM-based applications, and analytics on unstructured data. |
|
||||||
| [txtai](/documentation/frameworks/txtai/) | Python library for semantic search, LLM orchestration and language model workflows. |
|
| [txtai](/documentation/frameworks/txtai/) | Python library for semantic search, LLM orchestration and language model workflows. |
|
||||||
| [Vanna AI](/documentation/frameworks/vanna-ai/) | Python RAG framework for SQL generation and querying. |
|
| [Vanna AI](/documentation/frameworks/vanna-ai/) | Python RAG framework for SQL generation and querying. |
|
||||||
|
|||||||
@@ -0,0 +1,62 @@
|
|||||||
|
---
|
||||||
|
title: Sycamore
|
||||||
|
---
|
||||||
|
|
||||||
|
## Sycamore
|
||||||
|
|
||||||
|
[Sycamore](https://sycamore.readthedocs.io/en/stable/) is an LLM-powered data preparation, processing, and analytics system for complex, unstructured documents like PDFs, HTML, presentations, and more. With Aryn, you can prepare data for GenAI and RAG applications, power high-quality document processing workflows, and run analytics on large document collections with natural language.
|
||||||
|
|
||||||
|
You can use the Qdrant connector to write into and read documents from Qdrant collections.
|
||||||
|
|
||||||
|
<aside role="status">You can find an end-to-end example usage of the Qdrant connector <a a target="_blank" href="https://github.com/aryn-ai/sycamore/blob/main/examples/simple_qdrant.py">here.</a></aside>
|
||||||
|
|
||||||
|
## Writing to Qdrant
|
||||||
|
|
||||||
|
To write a Docset to a Qdrant collection in Sycamore, use the `docset.write.qdrant(....)` function. The Qdrant writer accepts the following arguments:
|
||||||
|
|
||||||
|
- `client_params`: Parameters that are passed to the Qdrant client constructor. See more information in the [Client API Reference](https://python-client.qdrant.tech/qdrant_client.qdrant_client).
|
||||||
|
- `collection_params`: Parameters that are passed into the `qdrant_client.QdrantClient.create_collection` method. See more information in the [Client API Reference](https://python-client.qdrant.tech/_modules/qdrant_client/qdrant_client#QdrantClient.create_collection).
|
||||||
|
- `vector_name`: The name of the vector in the Qdrant collection. Defaults to `None`.
|
||||||
|
- `execute`: Execute the pipeline and write to Qdrant on adding this operator. If `False`, will return a `DocSet` with this write in the plan. Defaults to `True`.
|
||||||
|
- `kwargs`: Keyword arguments to pass to the underlying execution engine.
|
||||||
|
|
||||||
|
```python
|
||||||
|
ds.write.qdrant(
|
||||||
|
{
|
||||||
|
"url": "http://localhost:6333",
|
||||||
|
"timeout": 50,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"collection_name": "{collection_name}",
|
||||||
|
"vectors_config": {
|
||||||
|
"size": 384,
|
||||||
|
"distance": "Cosine",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
## Reading from Qdrant
|
||||||
|
|
||||||
|
To read a Docset from a Qdrant collection in Sycamore, use the `docset.read.qdrant(....)` function. The Qdrant reader accepts the following arguments:
|
||||||
|
|
||||||
|
- `client_params`: Parameters that are passed to the Qdrant client constructor. See more information in the[Client API Reference](https://python-client.qdrant.tech/qdrant_client.qdrant_client).
|
||||||
|
- `query_params`: Parameters that are passed into the `qdrant_client.QdrantClient.query_points` method. See more information in the [Client API Reference](https://python-client.qdrant.tech/_modules/qdrant_client/qdrant_client#QdrantClient.query_points).
|
||||||
|
- `kwargs`: Keyword arguments to pass to the underlying execution engine.
|
||||||
|
|
||||||
|
```python
|
||||||
|
docs = ctx.read.qdrant(
|
||||||
|
{
|
||||||
|
"url": "https://xyz-example.eu-central.aws.cloud.qdrant.io:6333",
|
||||||
|
"api_key": "<paste-your-api-key-here>",
|
||||||
|
},
|
||||||
|
{"collection_name": "{collection_name}", "limit": 100, "using": "{optional_vector_name}"},
|
||||||
|
).take_all()
|
||||||
|
|
||||||
|
```
|
||||||
|
|
||||||
|
## 📚 Further Reading
|
||||||
|
|
||||||
|
- [Sycamore Reference](https://sycamore.readthedocs.io/en/stable/)
|
||||||
|
- [Sycamore](https://github.com/aryn-ai/sycamore/tree/main/examples)
|
||||||
Reference in New Issue
Block a user