docs: Clean up integration docs (#1840)

* docs: Clean up integration docs Signed-off-by: Anush008 <anushshetty90@gmail.com> * chore: More cleanup Signed-off-by: Anush008 <anushshetty90@gmail.com> * chore: Deleted image artifacts Signed-off-by: Anush008 <anushshetty90@gmail.com> * chore: Updated redirects Signed-off-by: Anush008 <anushshetty90@gmail.com> --------- Signed-off-by: Anush008 <anushshetty90@gmail.com>
2026-10-04 02:18:29 +02:00 · 2025-08-08 22:24:51 +05:30
parent 03455c7e46
commit 9968ce4842
43 changed files with 125 additions and 1209 deletions
@@ -14,7 +14,7 @@ Qdrant can be used as an ingestion destination in Unstructured.
 Install Unstructured with the `qdrant` extra.

 ```bash
-pip install "unstructured[qdrant]"
+pip install "unstructured-ingest[qdrant]"
 ```

 ## Usage
@@ -25,21 +25,21 @@ Depending on the use case you can prefer the command line or using it within you
 ### CLI

 ```bash
-EMBEDDING_PROVIDER=${EMBEDDING_PROVIDER:-"langchain-huggingface"}
-
 unstructured-ingest \
  local \
-  --input-path example-docs/book-war-and-peace-1225p.txt \
-  --output-dir local-output-to-qdrant \
-  --strategy fast \
-  --chunk-elements \
-  --embedding-provider "$EMBEDDING_PROVIDER" \
-  --num-processes 2 \
-  --verbose \
-  qdrant \
-  --collection-name "test" \
-  --url "http://localhost:6333" \
-  --batch-size 80
+    --input-path $LOCAL_FILE_INPUT_DIR \
+    --chunking-strategy by_title \
+    --embedding-provider huggingface \
+    --partition-by-api \
+    --api-key $UNSTRUCTURED_API_KEY \
+    --partition-endpoint $UNSTRUCTURED_API_URL \
+    --additional-partition-args="{\"split_pdf_page\":\"true\", \"split_pdf_allow_failed\":\"true\", \"split_pdf_concurrency_level\": 15}" \
+  qdrant-cloud \
+    --url $QDRANT_URL \
+    --api-key $QDRANT_API_KEY \
+    --collection-name $QDRANT_COLLECTION \
+    --batch-size 50 \
+    --num-processes 1
 ```

 For a full list of the options the CLI accepts, run `unstructured-ingest <upstream connector> qdrant --help`
@@ -47,54 +47,63 @@ For a full list of the options the CLI accepts, run `unstructured-ingest <upstre
 ### Programmatic usage

 ```python
-from unstructured.ingest.connector.local import SimpleLocalConfig
-from unstructured.ingest.connector.qdrant import (
-    QdrantWriteConfig,
-    SimpleQdrantConfig,
-)
-from unstructured.ingest.interfaces import (
-    ChunkingConfig,
-    EmbeddingConfig,
-    PartitionConfig,
-    ProcessorConfig,
-    ReadConfig,
-)
-from unstructured.ingest.runner import LocalRunner
-from unstructured.ingest.runner.writers.base_writer import Writer
-from unstructured.ingest.runner.writers.qdrant import QdrantWriter
+import os

-def get_writer() -> Writer:
-    return QdrantWriter(
-        connector_config=SimpleQdrantConfig(
-            url="http://localhost:6333",
-            collection_name="test",
-        ),
-        write_config=QdrantWriteConfig(batch_size=80),
-    )
+from unstructured_ingest.pipeline.pipeline import Pipeline
+from unstructured_ingest.interfaces import ProcessorConfig
+
+from unstructured_ingest.processes.connectors.local import (
+    LocalIndexerConfig,
+    LocalDownloaderConfig,
+    LocalConnectionConfig
+)
+from unstructured_ingest.processes.partitioner import PartitionerConfig
+from unstructured_ingest.processes.chunker import ChunkerConfig
+from unstructured_ingest.processes.embedder import EmbedderConfig
+
+from unstructured_ingest.processes.connectors.qdrant.cloud import (
+    CloudQdrantConnectionConfig,
+    CloudQdrantAccessConfig,
+    CloudQdrantUploadStagerConfig,
+    CloudQdrantUploaderConfig
+)

 if __name__ == "__main__":
-    writer = get_writer()
-    runner = LocalRunner(
-        processor_config=ProcessorConfig(
-            verbose=True,
-            output_dir="local-output-to-qdrant",
-            num_processes=2,
+    Pipeline.from_configs(
+        context=ProcessorConfig(),
+        indexer_config=LocalIndexerConfig(input_path=os.getenv("LOCAL_FILE_INPUT_DIR")),
+        downloader_config=LocalDownloaderConfig(),
+        source_connection_config=LocalConnectionConfig(),
+        partitioner_config=PartitionerConfig(
+            partition_by_api=True,
+            api_key=os.getenv("UNSTRUCTURED_API_KEY"),
+            partition_endpoint=os.getenv("UNSTRUCTURED_API_URL"),
+            additional_partition_args={
+                "split_pdf_page": True,
+                "split_pdf_allow_failed": True,
+                "split_pdf_concurrency_level": 15
+            }
        ),
-        connector_config=SimpleLocalConfig(
-            input_path="example-docs/book-war-and-peace-1225p.txt",
+        chunker_config=ChunkerConfig(chunking_strategy="by_title"),
+        embedder_config=EmbedderConfig(embedding_provider="huggingface"),
+
+        destination_connection_config=CloudQdrantConnectionConfig(
+            access_config=CloudQdrantAccessConfig(
+                api_key=os.getenv("QDRANT_API_KEY")
+            ),
+            url=os.getenv("QDRANT_URL")
        ),
-        read_config=ReadConfig(),
-        partition_config=PartitionConfig(),
-        chunking_config=ChunkingConfig(chunk_elements=True),
-        embedding_config=EmbeddingConfig(provider="langchain-huggingface"),
-        writer=writer,
-        writer_kwargs={},
-    )
-    runner.run()
+        stager_config=CloudQdrantUploadStagerConfig(),
+        uploader_config=CloudQdrantUploaderConfig(
+            collection_name=os.getenv("QDRANT_COLLECTION"),
+            batch_size=50,
+            num_processes=1
+        )
+    ).run()
 ```

 ## Next steps

 - Unstructured API [reference](https://unstructured-io.github.io/unstructured/api.html).
- Qdrant ingestion destination [reference](https://unstructured-io.github.io/unstructured/ingest/destination_connectors/qdrant.html).
- [Source Code](https://github.com/Unstructured-IO/unstructured-ingest/blob/main/unstructured_ingest/connector/qdrant.py)
+- Qdrant ingestion destination [reference](https://docs.unstructured.io/ui/destinations/qdrant).
+- [Source Code](https://github.com/Unstructured-IO/unstructured-ingest/tree/main/unstructured_ingest/processes/connectors/qdrant)