docs: Restructured integrations section (#1089)

* docs: Reorder integrations

* docs: Formatting langchain-go.md

* docs: Title for index

* docs: Redpanda docs (#1092)
This commit is contained in:
Anush
2024-08-15 08:50:37 +05:30
committed by GitHub
parent 590dfa807d
commit bdfcb44d23
46 changed files with 171 additions and 117 deletions
@@ -0,0 +1,19 @@
---
title: Platforms
weight: 15
---
## Platform Integrations
| Platform | Description |
| ------------------------------------- | ---------------------------------------------------------------------------------------------------- |
| [Apify](./apify/) | Platform to build web scrapers and automate web browser tasks. |
| [Bubble](./bubble) | Development platform for application development with a no-code interface |
| [BuildShip](./buildship) | Low-code visual builder to create APIs, scheduled jobs, and backend workflows. |
| [DocsGPT](./docsgpt/) | Tool for ingesting documentation sources and enabling conversations and queries. |
| [Make](./make/) | Cloud platform to build low-code workflows by integrating various software applications. |
| [N8N](./n8n/) | Platform for node-based, low-code workflow automation. |
| [Pipedream](./pipedream/) | Platform for connecting apps and developing event-driven automation. |
| [Portable.io](./portable/) | Cloud platform for developing and deploying ELT transformations. |
| [PrivateGPT](./privategpt/) | Tool to ask questions about your documents using local LLMs emphasising privacy. |
| [Rivet](./rivet/) | A visual programming environment for building AI agents with LLMs. |
@@ -0,0 +1,82 @@
---
title: Apify
aliases: [ ../frameworks/apify/ ]
---
# Apify
[Apify](https://apify.com/) is a web scraping and browser automation platform featuring an [app store](https://apify.com/store) with over 1,500 pre-built micro-apps known as Actors. These serverless cloud programs, which are essentially dockers under the hood, are designed for various web automation applications, including data collection.
One such Actor, built especially for AI and RAG applications, is [Website Content Crawler](https://apify.com/apify/website-content-crawler).
It's ideal for this purpose because it has built-in HTML processing and data-cleaning functions. That means you can easily remove fluff, duplicates, and other things on a web page that aren't relevant, and provide only the necessary data to the language model.
The Markdown can then be used to feed Qdrant to train AI models or supply them with fresh web content.
Qdrant is available as an [official integration](https://apify.com/apify/qdrant-integration) to load Apify datasets into a collection.
You can refer to the [Apify documentation](https://docs.apify.com/platform/integrations/qdrant) to set up the integration via the Apify UI.
## Programmatic Usage
Apify also supports programmatic access to integrations via the [Apify Python SDK](https://docs.apify.com/sdk/python/).
1. Install the Apify Python SDK by running the following command:
```sh
pip install apify-client
```
2. Create a Python script and import all the necessary modules:
```python
from apify_client import ApifyClient
APIFY_API_TOKEN = "YOUR-APIFY-TOKEN"
OPENAI_API_KEY = "YOUR-OPENAI-API-KEY"
# COHERE_API_KEY = "YOUR-COHERE-API-KEY"
QDRANT_URL = "YOUR-QDRANT-URL"
QDRANT_API_KEY = "YOUR-QDRANT-API-KEY"
client = ApifyClient(APIFY_API_TOKEN)
```
3. Call the [Website Content Crawler](https://apify.com/apify/website-content-crawler) Actor to crawl the Qdrant documentation and extract text content from the web pages:
```python
actor_call = client.actor("apify/website-content-crawler").call(
run_input={"startUrls": [{"url": "https://qdrant.tech/documentation/"}]}
)
```
4. Call the Qdrant integration and store all data in the Qdrant Vector Database:
```python
qdrant_integration_inputs = {
"qdrantUrl": QDRANT_URL,
"qdrantApiKey": QDRANT_API_KEY,
"qdrantCollectionName": "apify",
"qdrantAutoCreateCollection": True,
"datasetId": actor_call["defaultDatasetId"],
"datasetFields": ["text"],
"enableDeltaUpdates": True,
"deltaUpdatesPrimaryDatasetFields": ["url"],
"expiredObjectDeletionPeriodDays": 30,
"embeddingsProvider": "OpenAI", # "Cohere"
"embeddingsApiKey": OPENAI_API_KEY,
"performChunking": True,
"chunkSize": 1000,
"chunkOverlap": 0,
}
actor_call = client.actor("apify/qdrant-integration").call(run_input=qdrant_integration_inputs)
```
Upon running the script, the data from <https://qdrant.tech/documentation/> will be scraped, transformed into vector embeddings and stored in the Qdrant collection.
## Further Reading
- Apify [Documentation](https://docs.apify.com/)
- Apify [Templates](https://apify.com/templates)
- Integration [Source Code](https://github.com/apify/actor-vector-database-integrations)
@@ -0,0 +1,36 @@
---
title: Bubble
aliases: [ ../frameworks/bubble/ ]
---
# Bubble
[Bubble](https://bubble.io/) is a software development platform that enables anyone to build and launch fully functional web applications without writing code.
You can use the [Qdrant Bubble plugin](https://bubble.io/plugin/qdrant-1716804374179x344999530386685950) to interface with Qdrant in your workflows.
## Prerequisites
1. A Qdrant instance to connect to. You can get a free cloud instance at [cloud.qdrant.io](https://cloud.qdrant.io/).
2. An account at [Bubble.io](https://bubble.io/) and an app set up.
## Setting up the plugin
Navigate to your app's workflows. Select `"Install more plugins actions"`.
![Install New Plugin](/documentation/frameworks/bubble/install-bubble-plugin.png)
You can now search for the Qdrant plugin and install it. Ensure all the categories are selected to perform a full search.
![Qdrant Plugin Search](/documentation/frameworks/bubble/qdrant-plugin-search.png)
The Qdrant plugin can now be found in the installed plugins section of your workflow. Enter the API key of your Qdrant instance for authentication.
![Qdrant Plugin Home](/documentation/frameworks/bubble/qdrant-plugin-home.png)
The plugin provides actions for upserting, searching, updating and deleting points from your Qdrant collection with dynamic and static values from your Bubble workflow.
## Further Reading
- [Bubble Academy](https://bubble.io/academy).
- [Bubble Manual](https://manual.bubble.io/)
@@ -0,0 +1,42 @@
---
title: BuildShip
aliases: [ ../frameworks/buildship/ ]
---
# BuildShip
[BuildShip](https://buildship.com/) is a low-code visual builder to create APIs, scheduled jobs, and backend workflows with AI assitance.
You can use the [Qdrant integration](https://buildship.com/integrations/qdrant) to development workflows with semantic-search capabilites.
## Prerequisites
1. A Qdrant instance to connect to. You can get a free cloud instance at [cloud.qdrant.io](https://cloud.qdrant.io/).
2. A [BuildsShip](https://buildship.app/) for developing workflows.
## Nodes
Nodes are are fundamental building blocks of BuildShip. Each responsible for an operation in your workflow.
The Qdrant integration includes the following nodes with extensibility if required.
### Add Point
![Add Point](/documentation/frameworks/buildship/add.png)
### Retrieve Points
![Retrieve Points](/documentation/frameworks/buildship/get.png)
### Delete Points
![Delete Points](/documentation/frameworks/buildship/delete.png)
### Search Points
![Search Points](/documentation/frameworks/buildship/search.png)
## Further Reading
- [BuildShip Docs](https://docs.buildship.com/basics/node).
- [BuildShip Integrations](https://buildship.com/integrations)
@@ -0,0 +1,28 @@
---
title: DocsGPT
aliases: [ ../frameworks/docsgpt/ ]
---
# DocsGPT
[DocsGPT](https://docsgpt.arc53.com/) is an open-source documentation assistant that enables you to build conversational user experiences on top of your data.
Qdrant is supported as a vectorstore in DocsGPT to ingest and semantically retrieve documents.
## Configuration
Learn how to setup DocsGPT in their [Quickstart guide](https://docs.docsgpt.co.uk/Deploying/Quickstart).
You can configure DocsGPT with environment variables in a `.env` file.
To configure DocsGPT to use Qdrant as the vector store, set `VECTOR_STORE` to `"qdrant"`.
```bash
echo "VECTOR_STORE=qdrant" >> .env
```
DocsGPT includes a list of the Qdrant configuration options that you can set as environment variables [here](https://github.com/arc53/DocsGPT/blob/00dfb07b15602319bddb95089e3dab05fac56240/application/core/settings.py#L46-L59).
## Further reading
- [DocsGPT Reference](https://github.com/arc53/DocsGPT)
@@ -0,0 +1,45 @@
---
title: Make.com
aliases: [ ../frameworks/make/ ]
---
# Make.com
[Make](https://www.make.com/) is a platform for anyone to design, build, and automate anything—from tasks and workflows to apps and systems without code.
Find the comprehensive list of available Make apps [here](https://www.make.com/en/integrations).
Qdrant is available as an [app](https://www.make.com/en/integrations/qdrant) within Make to add to your scenarios.
![Qdrant Make hero](/documentation/frameworks/make/hero-page.png)
## Prerequisites
Before you start, make sure you have the following:
1. A Qdrant instance to connect to. You can get free cloud instance [cloud.qdrant.io](https://cloud.qdrant.io/).
2. An account at Make.com. You can register yourself [here](https://www.make.com/en/register).
## Setting up a connection
Navigate to your scenario on the Make dashboard and select a Qdrant app module to start a connection.
![Qdrant Make connection](/documentation/frameworks/make/connection.png)
You can now establish a connection to Qdrant using your [instance credentials](/documentation/cloud/authentication/).
![Qdrant Make form](/documentation/frameworks/make/connection-form.png)
## Modules
Modules represent actions that Make performs with an app.
The Qdrant Make app enables you to trigger the following app modules.
![Qdrant Make modules](/documentation/frameworks/make/modules.png)
The modules support mapping to connect the data retrieved by one module to another module to perform the desired action. You can read more about the data processing options available for the modules in the [Make reference](https://www.make.com/en/help/modules).
## Next steps
- Find a list of Make workflow templates to connect with Qdrant [here](https://www.make.com/en/templates).
- Make scenario reference docs can be found [here](https://www.make.com/en/help/scenarios).
@@ -0,0 +1,41 @@
---
title: N8N
aliases: [ ../frameworks/n8n/ ]
---
# N8N
[N8N](https://n8n.io/) is an automation platform that allows you to build flexible workflows focused on deep data integration.
Qdrant is available as a vectorstore node in N8N for building AI-powered functionality within your workflows.
## Prerequisites
1. A Qdrant instance to connect to. You can get a free cloud instance at [cloud.qdrant.io](https://cloud.qdrant.io/).
2. A running N8N instance. You can learn more about using the N8N cloud or self-hosting [here](https://docs.n8n.io/choose-n8n/).
## Setting up the vectorstore
Select the Qdrant vectorstore from the list of nodes in your workflow editor.
![Qdrant n8n node](/documentation/frameworks/n8n/node.png)
You can now configure the vectorstore node according to your workflow requirements. The configuration options reference can be found [here](https://docs.n8n.io/integrations/builtin/cluster-nodes/root-nodes/n8n-nodes-langchain.vectorstoreqdrant/#node-parameters).
![Qdrant Config](/documentation/frameworks/n8n/config.png)
Create a connection to Qdrant using your [instance credentials](/documentation/cloud/authentication/).
![Qdrant Credentials](/documentation/frameworks/n8n/credentials.png)
The vectorstore supports the following operations:
- Get Many - Get the top-ranked documents for a query.
- Insert documents - Add documents to the vectorstore.
- Retrieve documents - Retrieve documents for use with AI nodes.
## Further Reading
- N8N vectorstore [reference](https://docs.n8n.io/integrations/builtin/cluster-nodes/root-nodes/n8n-nodes-langchain.vectorstoreqdrant/).
- N8N AI-based workflows [reference](https://n8n.io/integrations/basic-llm-chain/).
- [Source Code](https://github.com/n8n-io/n8n/tree/master/packages/@n8n/nodes-langchain/nodes/vector_store/VectorStoreQdrant)
@@ -0,0 +1,43 @@
---
title: Pipedream
aliases: [ ../frameworks/pipedream/ ]
---
# Pipedream
[Pipedream](https://pipedream.com/) is a development platform that allows developers to connect many different applications, data sources, and APIs in order to build automated cross-platform workflows. It also offers code-level control with Node.js, Python, Go, or Bash if required.
You can use the [Qdrant app](https://pipedream.com/apps/qdrant) in Pipedream to add vector search capabilities to your workflows.
## Prerequisites
1. A Qdrant instance to connect to. You can get a free cloud instance at [cloud.qdrant.io](https://cloud.qdrant.io/).
2. A [Pipedream project](https://pipedream.com/) to develop your workflows.
## Setting Up
Search for the Qdrant app in your workflow apps.
![Qdrant Pipedream App](/documentation/frameworks/pipedream/qdrant-app.png)
The Qdrant app offers extensible API interface and pre-built actions.
![Qdrant App Features](/documentation/frameworks/pipedream/app-features.png)
Select any of the actions of the app to set up a connection.
![Qdrant Connect Account](/documentation/frameworks/pipedream/app-upsert-action.png)
Configure connection with the credentials of your Qdrant instance.
![Qdrant Connection Credentials](/documentation/frameworks/pipedream/app-connection.png)
You can verify your credentials using the "Test Connection" button.
Once a connection is set up, you can use the app to build workflows with the [2000+ apps supported by Pipedream](https://pipedream.com/apps/).
## Further Reading
- [Pipedream Documentation](https://pipedream.com/docs).
- [Qdrant Cloud Authentication](https://qdrant.tech/documentation/cloud/authentication/).
- [Source Code](https://github.com/PipedreamHQ/pipedream/tree/master/components/qdrant)
@@ -0,0 +1,34 @@
---
title: Portable.io
aliases: [ ../frameworks/portable/ ]
---
# Portable
[Portable](https://portable.io/) is an ELT platform that builds connectors on-demand for data teams. It enables connecting applications to your data warehouse with no code.
You can avail the [Qdrant connector](https://portable.io/connectors/qdrant) to build data pipelines from your collections.
![Qdrant Connector](/documentation/frameworks/portable/home.png)
## Prerequisites
1. A Qdrant instance to connect to. You can get a free cloud instance at [cloud.qdrant.io](https://cloud.qdrant.io/).
2. A [Portable account](https://app.portable.io/).
## Setting up the connector
Navigate to the Portable dashboard. Search for `"Qdrant"` in the sources section.
![Install New Source](/documentation/frameworks/portable/install.png)
Configure the connector with your Qdrant instance credentials.
![Configure connector](/documentation/frameworks/portable/configure.png)
You can now build your flows using data from Qdrant by selecting a [destination](https://app.portable.io/destinations) and scheduling it.
## Further Reading
- [Portable API Reference](https://developer.portable.io/api-reference/introduction).
- [Portable Academy](https://portable.io/learn)
@@ -0,0 +1,41 @@
---
title: PrivateGPT
aliases: [ ../integrations/privategpt/, ../frameworks/privategpt/ ]
---
# PrivateGPT
[PrivateGPT](https://docs.privategpt.dev/) is a production-ready AI project that allows you to inquire about your documents using Large Language Models (LLMs) with offline support.
PrivateGPT uses Qdrant as the default vectorstore for ingesting and retrieving documents.
## Configuration
Qdrant settings can be configured by setting values to the qdrant property in the `settings.yaml` file. By default, Qdrant tries to connect to an instance at http://localhost:3000.
Example:
```yaml
qdrant:
url: "https://xyz-example.eu-central.aws.cloud.qdrant.io:6333"
api_key: "<your-api-key>"
```
The available [configuration options](https://docs.privategpt.dev/manual/storage/vector-stores#qdrant-configuration) are:
| Field | Description |
|--------------|-------------|
| location | If `:memory:` - use in-memory Qdrant instance.<br>If `str` - use it as a `url` parameter.|
| url | Either host or str of `Optional[scheme], host, Optional[port], Optional[prefix]`.<br> Eg. `http://localhost:6333` |
| port | Port of the REST API interface. Default: `6333` |
| grpc_port | Port of the gRPC interface. Default: `6334` |
| prefer_grpc | If `true` - use gRPC interface whenever possible in custom methods. |
| https | If `true` - use HTTPS(SSL) protocol.|
| api_key | API key for authentication in Qdrant Cloud.|
| prefix | If set, add `prefix` to the REST URL path.<br>Example: `service/v1` will result in `http://localhost:6333/service/v1/{qdrant-endpoint}` for REST API.|
| timeout | Timeout for REST and gRPC API requests.<br>Default: 5.0 seconds for REST and unlimited for gRPC |
| host | Host name of Qdrant service. If url and host are not set, defaults to 'localhost'.|
| path | Persistence path for QdrantLocal. Eg. `local_data/private_gpt/qdrant`|
| force_disable_check_same_thread | Force disable check_same_thread for QdrantLocal sqlite connection.|
## Next steps
Find the PrivateGPT docs [here](https://docs.privategpt.dev/).
@@ -0,0 +1,34 @@
---
title: Ironclad Rivet
aliases: [ ../frameworks/rivet/ ]
---
# Ironclad Rivet
[Rivet](https://rivet.ironcladapp.com/) is an Integrated Development Environment (IDE) and library designed for creating AI agents using a visual, graph-based interface.
Qdrant is available as a [plugin](https://github.com/qdrant/rivet-plugin-qdrant) for building vector-search powered workflows in Rivet.
## Installation
- Open the plugins overlay at the top of the screen.
- Search for the official Qdrant plugin.
- Click the "Add" button to install it in your current project.
![Rivet plugin installation](/documentation/frameworks/rivet/installation.png)
## Setting up the connection
You can configure your Qdrant instance credentials in the Rivet settings after installing the plugin.
![Rivet plugin connection](/documentation/frameworks/rivet/connection.png)
Once you've configured your credentials, you can right-click on your workspace to add nodes from the plugin and get building!
![Rivet plugin nodes](/documentation/frameworks/rivet/node.png)
## Further Reading
- Rivet [Tutorial](https://rivet.ironcladapp.com/docs/tutorial).
- Rivet [Documentation](https://rivet.ironcladapp.com/docs).
- Plugin [Source Code](https://github.com/qdrant/rivet-plugin-qdrant)