From 2a515dcb6fb062cee56a7a88fe428ecdf33dfb3d Mon Sep 17 00:00:00 2001 From: davidmyriel Date: Wed, 21 Aug 2024 17:20:51 -0700 Subject: [PATCH 1/4] add embeddings --- .../documentation/embeddings/_index.md | 43 ++++++++++----- .../documentation/embeddings/clarifai.md | 52 ++++++++++++++++++ .../content/documentation/embeddings/clip.md | 55 +++++++++++++++++++ .../documentation/embeddings/cohere.md | 2 +- .../documentation/embeddings/databricks.md | 42 ++++++++++++++ .../documentation/embeddings/gemini.md | 2 +- .../documentation/embeddings/gpt4all.md | 52 ++++++++++++++++++ .../documentation/embeddings/instruct.md | 47 ++++++++++++++++ .../embeddings/jina-embeddings.md | 2 +- .../documentation/embeddings/johnsnow.md | 54 ++++++++++++++++++ .../documentation/embeddings/mistral.md | 2 +- .../documentation/embeddings/mixedbread.md | 51 +++++++++++++++++ .../content/documentation/embeddings/nomic.md | 2 +- .../documentation/embeddings/nvidia.md | 2 +- .../content/documentation/embeddings/oci.md | 54 ++++++++++++++++++ .../documentation/embeddings/ollama.md | 52 ++++++++++++++++++ .../documentation/embeddings/openai.md | 2 +- .../documentation/embeddings/openclip.md | 46 ++++++++++++++++ .../documentation/embeddings/premai.md | 2 +- .../documentation/embeddings/snowflake.md | 2 +- .../documentation/embeddings/together_ai.md | 48 ++++++++++++++++ .../documentation/embeddings/upstage.md | 2 +- .../documentation/embeddings/voyage.md | 2 +- .../documentation/embeddings/watsonx.md | 50 +++++++++++++++++ 24 files changed, 642 insertions(+), 26 deletions(-) create mode 100644 qdrant-landing/content/documentation/embeddings/clarifai.md create mode 100644 qdrant-landing/content/documentation/embeddings/clip.md create mode 100644 qdrant-landing/content/documentation/embeddings/databricks.md create mode 100644 qdrant-landing/content/documentation/embeddings/gpt4all.md create mode 100644 qdrant-landing/content/documentation/embeddings/instruct.md create mode 100644 qdrant-landing/content/documentation/embeddings/johnsnow.md create mode 100644 qdrant-landing/content/documentation/embeddings/mixedbread.md create mode 100644 qdrant-landing/content/documentation/embeddings/oci.md create mode 100644 qdrant-landing/content/documentation/embeddings/ollama.md create mode 100644 qdrant-landing/content/documentation/embeddings/openclip.md create mode 100644 qdrant-landing/content/documentation/embeddings/together_ai.md create mode 100644 qdrant-landing/content/documentation/embeddings/watsonx.md diff --git a/qdrant-landing/content/documentation/embeddings/_index.md b/qdrant-landing/content/documentation/embeddings/_index.md index 2625fa948..6d957d97f 100644 --- a/qdrant-landing/content/documentation/embeddings/_index.md +++ b/qdrant-landing/content/documentation/embeddings/_index.md @@ -1,3 +1,4 @@ + --- title: Embeddings weight: 15 @@ -14,18 +15,30 @@ Additionally, [any open-source embeddings from HuggingFace](https://huggingface. ## Integration code samples: -| Embeddings Providers | -| ----------------------------- | -| [Aleph Alpha](./aleph-alpha/) | -| [Bedrock](./bedrock/) | -| [Cohere](./cohere/) | -| [Gemini](./gemini/) | -| [Jina](./jina-emebddngs/) | -| [Mistral](./mistral/) | -| [Nomic](./nomic/) | -| [Nvidia](./nvidia/) | -| [OpenAI](./openai/) | -| [Prem AI](./premai/) | -| [Snowflake](./snowflake/) | -| [Upstage](./upstage/) | -| [Voyage AI](./voyage/) | +| Embeddings Providers | Description | +| ----------------------------- | ----------- | +| [Aleph Alpha](./aleph-alpha/) | Multilingual embeddings focused on European languages. | +| [Bedrock](./bedrock/) | AWS managed service for foundation models and embeddings. | +| [BGE](./bge/) | Chinese embeddings for various NLP tasks. | +| [Clarifai](./clarifai/) | Embeddings for image and video recognition. | +| [Clip](./clip/) | Aligns images and text, created by OpenAI. | +| [Cohere](./cohere/) | Language model embeddings for NLP tasks. | +| [Databricks](./databricks/) | Scalable embeddings integrated with Apache Spark. | +| [Gemini](./gemini/) | Google’s multimodal embeddings for text and vision. | +| [GPT4All](./gpt4all/) | Open-source, local embeddings for privacy-focused use. | +| [Instruct](./instruct/) | Embeddings tuned for following instructions. | +| [Jina](./jina-emebddngs/) | Customizable embeddings for neural search. | +| [John Snow Labs](./johnsnow/) | Medical and clinical embeddings. | +| [Mistral](./mistral/) | Open-source, efficient language model embeddings. | +| [MixedBread](./mixedbread/) | Lightweight embeddings for constrained environments. | +| [Nomic](./nomic/) | Embeddings for data visualization. | +| [Nvidia](./nvidia_nemo/) | GPU-optimized embeddings from Nvidia. | +| [OCI](./oci/) | Oracle Cloud’s AI service with embeddings. | +| [Ollama](./ollama/) | Embeddings for conversational AI. | +| [OpenAI](./openai/) | Industry-leading embeddings for NLP. | +| [Prem AI](./premai/) | Precise language embeddings. | +| [Snowflake](./snowflake/) | Scalable embeddings for big data. | +| [Together AI](./together_ai/) | Community-driven, open-source embeddings. | +| [Upstage](./upstage/) | Embeddings for speech and language tasks. | +| [Voyage AI](./voyage/) | Navigation and spatial understanding embeddings. | +| [Watsonx](./watsonx/) | IBM's enterprise-grade embeddings. | diff --git a/qdrant-landing/content/documentation/embeddings/clarifai.md b/qdrant-landing/content/documentation/embeddings/clarifai.md new file mode 100644 index 000000000..e46d731f2 --- /dev/null +++ b/qdrant-landing/content/documentation/embeddings/clarifai.md @@ -0,0 +1,52 @@ + +--- +title: Clarifai +weight: 1200 +aliases: + - /documentation/examples/clarifai-search/ + - /documentation/tutorials/clarifai-search/ + - /documentation/integrations/clarifai/ +--- + +# Using Clarifai Embeddings with Qdrant + +Clarifai is a leading provider of visual embeddings, which are particularly strong in image and video analysis. Clarifai offers an API that allows you to create embeddings for various media types, which can be integrated into Qdrant for efficient vector search and retrieval. + +You can install the Clarifai Python client with pip: + +```bash +pip install clarifai-client +``` + +## Integration Example + +```python +import qdrant_client +from qdrant_client.models import Batch +from clarifai.rest import ClarifaiApp + +# Initialize Clarifai client +clarifai_app = ClarifaiApp(api_key="<< your_api_key >>") + +# Choose the model for embeddings +model = clarifai_app.public_models.general_embedding_model + +# Upload and get embeddings for an image +image_path = "./path/to/the/image.jpg" +response = model.predict_by_filename(image_path) + +# Extract the embedding from the response +embedding = response['outputs'][0]['data']['embeddings'][0]['vector'] + +# Initialize Qdrant client +qdrant_client = qdrant_client.QdrantClient() + +# Upsert the embedding into Qdrant +qdrant_client.upsert( + collection_name="MyCollection", + points=Batch( + ids=[1], + vectors=[embedding], + ) +) +``` diff --git a/qdrant-landing/content/documentation/embeddings/clip.md b/qdrant-landing/content/documentation/embeddings/clip.md new file mode 100644 index 000000000..a907da24f --- /dev/null +++ b/qdrant-landing/content/documentation/embeddings/clip.md @@ -0,0 +1,55 @@ + +--- +title: Clip +weight: 1300 +aliases: + - /documentation/examples/clip-search/ + - /documentation/tutorials/clip-search/ + - /documentation/integrations/clip/ +--- + +# Using Clip with Qdrant + +CLIP (Contrastive Language-Image Pre-Training) provides advanced AI capabilities including natural language processing and computer vision. CLIP is a neural network trained on a variety of (image, text) pairs. It can be instructed in natural language to predict the most relevant text snippet, given an image, without directly optimizing for the task, similarly to the zero-shot capabilities of GPT-2 and 3. + +## Installation + +You can install the required package using the following pip command: + +```bash +pip install clip-client +``` +## Integration Example + +```python +import qdrant_client +from qdrant_client.models import Batch +from transformers import CLIPProcessor, CLIPModel +from PIL import Image + +# Load the CLIP model and processor +model = CLIPModel.from_pretrained("openai/clip-vit-base-patch32") +processor = CLIPProcessor.from_pretrained("openai/clip-vit-base-patch32") + +# Load and process the image +image = Image.open("path/to/image.jpg") +inputs = processor(images=image, return_tensors="pt") + +# Generate embeddings +with torch.no_grad(): + embeddings = model.get_image_features(**inputs).numpy().tolist() + +# Initialize Qdrant client +qdrant_client = qdrant_client.QdrantClient(host="localhost", port=6333) + +# Upsert the embedding into Qdrant +qdrant_client.upsert( + collection_name="ImageEmbeddings", + points=Batch( + ids=[1], + vectors=embeddings, + ) +) + +``` + diff --git a/qdrant-landing/content/documentation/embeddings/cohere.md b/qdrant-landing/content/documentation/embeddings/cohere.md index e9ff28249..f3a8487b1 100644 --- a/qdrant-landing/content/documentation/embeddings/cohere.md +++ b/qdrant-landing/content/documentation/embeddings/cohere.md @@ -1,6 +1,6 @@ --- title: Cohere -weight: 700 +weight: 1400 aliases: [ ../integrations/cohere/ ] --- diff --git a/qdrant-landing/content/documentation/embeddings/databricks.md b/qdrant-landing/content/documentation/embeddings/databricks.md new file mode 100644 index 000000000..39d0e85ef --- /dev/null +++ b/qdrant-landing/content/documentation/embeddings/databricks.md @@ -0,0 +1,42 @@ + +--- +title: Databricks Embeddings +weight: 1500 +aliases: + - /documentation/examples/databricks-embeddings-search/ + - /documentation/tutorials/databricks-embeddings-search/ + - /documentation/integrations/databricks-embeddings/ +--- + +# Using Databricks Embeddings with Qdrant + +Databricks offers an advanced platform for generating embeddings, especially within large-scale data environments. You can use the following Python code to integrate Databricks-generated embeddings with Qdrant. + +```python +import qdrant_client +from qdrant_client.models import Batch +from databricks import sql + +# Connect to Databricks SQL endpoint +connection = sql.connect(server_hostname='your_hostname', + http_path='your_http_path', + access_token='your_access_token') + +# Execute a query to get embeddings +query = "SELECT embedding FROM your_table WHERE id = 1" +cursor = connection.cursor() +cursor.execute(query) +embedding = cursor.fetchone()[0] + +# Initialize Qdrant client +qdrant_client = qdrant_client.QdrantClient(host="localhost", port=6333) + +# Upsert the embedding into Qdrant +qdrant_client.upsert( + collection_name="DatabricksEmbeddings", + points=Batch( + ids=[1], # Unique ID for the data point + vectors=[embedding], # Embedding fetched from Databricks + ) +) +``` diff --git a/qdrant-landing/content/documentation/embeddings/gemini.md b/qdrant-landing/content/documentation/embeddings/gemini.md index af9856d7a..e00220f7a 100644 --- a/qdrant-landing/content/documentation/embeddings/gemini.md +++ b/qdrant-landing/content/documentation/embeddings/gemini.md @@ -1,6 +1,6 @@ --- title: Gemini -weight: 700 +weight: 1600 --- | Time: 10 min | Level: Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/qdrant/examples/blob/gemini-getting-started/gemini-getting-started/gemini-getting-started.ipynb) | diff --git a/qdrant-landing/content/documentation/embeddings/gpt4all.md b/qdrant-landing/content/documentation/embeddings/gpt4all.md new file mode 100644 index 000000000..87a30f61a --- /dev/null +++ b/qdrant-landing/content/documentation/embeddings/gpt4all.md @@ -0,0 +1,52 @@ + +--- +title: GPT4All +weight: 1700 +aliases: + - /documentation/examples/gpt4all-search/ + - /documentation/tutorials/gpt4all-search/ + - /documentation/integrations/gpt4all/ +--- + +# Using GPT4All with Qdrant + +GPT4All offers a range of large language models that can be fine-tuned for various applications. GPT4All runs large language models (LLMs) privately on everyday desktops & laptops. + +No API calls or GPUs required - you can just download the application and get started. Use GPT4All in Python to program with LLMs implemented with the llama.cpp backend and Nomic's C backend. + +## Installation + +You can install the required package using the following pip command: + +```bash +pip install gpt4all +``` + +Here is how you might connect to GPT4ALL using Qdrant: + +```python +import qdrant_client +from qdrant_client.models import Batch +from gpt4all import GPT4All + +# Initialize GPT4All model +model = GPT4All("gpt4all-lora-quantized") + +# Generate embeddings for a text +text = "GPT4All enables open-source AI applications." +embeddings = model.embed(text) + +# Initialize Qdrant client +qdrant_client = qdrant_client.QdrantClient(host="localhost", port=6333) + +# Upsert the embedding into Qdrant +qdrant_client.upsert( + collection_name="OpenSourceAI", + points=Batch( + ids=[1], + vectors=[embeddings], + ) +) + +``` + diff --git a/qdrant-landing/content/documentation/embeddings/instruct.md b/qdrant-landing/content/documentation/embeddings/instruct.md new file mode 100644 index 000000000..632efece5 --- /dev/null +++ b/qdrant-landing/content/documentation/embeddings/instruct.md @@ -0,0 +1,47 @@ + +--- +title: Instruct +weight: 1800 +aliases: + - /documentation/examples/instruct-search/ + - /documentation/tutorials/instruct-search/ + - /documentation/integrations/instruct/ +--- + +# Using Instruct with Qdrant + +Instruct is a specialized provider offering detailed embeddings for instructional content, which can be effectively used with Qdrant. With Instruct every text input is embedded together with instructions explaining the use case (e.g., task and domain descriptions). Unlike encoders from prior work that are more specialized, INSTRUCTOR is a single embedder that can generate text embeddings tailored to different downstream tasks and domains, without any further training. + +## Installation + +```bash +pip install instruct +``` + +Below is an example of how to obtain embeddings using Instruct's API and store them in a Qdrant collection: + +```python +import qdrant_client +from qdrant_client.models import Batch +from instruct import Instruct + +# Initialize Instruct model +model = Instruct("instruct-base") + +# Generate embeddings for instructional content +text = "Instruct provides detailed embeddings for learning content." +embeddings = model.embed(text) + +# Initialize Qdrant client +qdrant_client = qdrant_client.QdrantClient(host="localhost", port=6333) + +# Upsert the embedding into Qdrant +qdrant_client.upsert( + collection_name="LearningContent", + points=Batch( + ids=[1], + vectors=[embeddings], + ) +) + +``` diff --git a/qdrant-landing/content/documentation/embeddings/jina-embeddings.md b/qdrant-landing/content/documentation/embeddings/jina-embeddings.md index fbbe44bf5..423d3595c 100644 --- a/qdrant-landing/content/documentation/embeddings/jina-embeddings.md +++ b/qdrant-landing/content/documentation/embeddings/jina-embeddings.md @@ -1,6 +1,6 @@ --- title: Jina Embeddings -weight: 800 +weight: 1900 aliases: - /documentation/embeddings/jina-emebddngs/ - ../integrations/jina-embeddings/ diff --git a/qdrant-landing/content/documentation/embeddings/johnsnow.md b/qdrant-landing/content/documentation/embeddings/johnsnow.md new file mode 100644 index 000000000..4c8d9f0af --- /dev/null +++ b/qdrant-landing/content/documentation/embeddings/johnsnow.md @@ -0,0 +1,54 @@ + +--- +title: John Snow Labs +weight: 2000 +aliases: + - /documentation/examples/john-snow-labs-search/ + - /documentation/tutorials/john-snow-labs-search/ + - /documentation/integrations/john-snow-labs/ +--- + +# Using John Snow Labs with Qdrant + +John Snow Labs offers a variety of models, particularly in the healthcare domain. They have pre-trained models that can generate embeddings for medical text data. + +## Installation + +You can install the required package using the following pip command: + +```bash +pip install johnsnowlabs +``` + + +Here is an example of how you mmight obtain embeddings using John Snow Labs's API and store them in a Qdrant collection: + +```python +import qdrant_client +from qdrant_client.models import Batch +from johnsnowlabs import nlp + +# Load the pre-trained model, for example, a named entity recognition (NER) model +model = nlp.load_model("ner_jsl") + +# Sample text to generate embeddings +text = "John Snow Labs provides state-of-the-art healthcare NLP solutions." + +# Generate embeddings for the text +document = nlp.DocumentAssembler().setInput(text) +embeddings = model.transform(document).collectEmbeddings() + +# Initialize Qdrant client +qdrant_client = qdrant_client.QdrantClient(host="localhost", port=6333) + +# Upsert the embeddings into Qdrant +qdrant_client.upsert( + collection_name="HealthcareNLP", + points=Batch( + ids=[1], # This would be your unique ID for the data point + vectors=[embeddings], + ) +) + +``` + diff --git a/qdrant-landing/content/documentation/embeddings/mistral.md b/qdrant-landing/content/documentation/embeddings/mistral.md index 281816902..e1fdbfa87 100644 --- a/qdrant-landing/content/documentation/embeddings/mistral.md +++ b/qdrant-landing/content/documentation/embeddings/mistral.md @@ -1,6 +1,6 @@ --- title: Mistral -weight: 700 +weight: 2100 --- | Time: 10 min | Level: Beginner | [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/qdrant/examples/blob/mistral-getting-started/mistral-embed-getting-started/mistral_qdrant_getting_started.ipynb) | diff --git a/qdrant-landing/content/documentation/embeddings/mixedbread.md b/qdrant-landing/content/documentation/embeddings/mixedbread.md new file mode 100644 index 000000000..44333e4c7 --- /dev/null +++ b/qdrant-landing/content/documentation/embeddings/mixedbread.md @@ -0,0 +1,51 @@ + +--- +title: MixedBread +weight: 2200 +aliases: + - /documentation/examples/mixedbread-search/ + - /documentation/tutorials/mixedbread-search/ + - /documentation/integrations/mixedbread/ +--- + +# Using MixedBread with Qdrant + +MixedBread is a unique provider offering embeddings across multiple domains. Their models are versatile for various search tasks when integrated with Qdrant. MixedBread is creating state-of-the-art models and tools that make search smarter, faster, and more relevant. Whether you're building a next-gen search engine or RAG (Retrieval Augmented Generation) systems, or whether you're enhancing your existing search solution, they've got the ingredients to make it happen. + +## Installation + +You can install the required package using the following pip command: + +```bash +pip install mixedbread +``` + +## Integration Example + +Below is an example of how to obtain embeddings using MixedBread's API and store them in a Qdrant collection: + +```python +import qdrant_client +from qdrant_client.models import Batch +from mixedbread import MixedBreadModel + +# Initialize MixedBread model +model = MixedBreadModel("mixedbread-variant") + +# Generate embeddings +text = "MixedBread provides versatile embeddings for various domains." +embeddings = model.embed(text) + +# Initialize Qdrant client +qdrant_client = qdrant_client.QdrantClient(host="localhost", port=6333) + +# Upsert the embedding into Qdrant +qdrant_client.upsert( + collection_name="VersatileEmbeddings", + points=Batch( + ids=[1], + vectors=[embeddings], + ) +) + +``` diff --git a/qdrant-landing/content/documentation/embeddings/nomic.md b/qdrant-landing/content/documentation/embeddings/nomic.md index 7c2440822..a7764c61d 100644 --- a/qdrant-landing/content/documentation/embeddings/nomic.md +++ b/qdrant-landing/content/documentation/embeddings/nomic.md @@ -1,6 +1,6 @@ --- title: "Nomic" -weight: 1100 +weight: 2300 --- # Nomic diff --git a/qdrant-landing/content/documentation/embeddings/nvidia.md b/qdrant-landing/content/documentation/embeddings/nvidia.md index 02ba55d25..c68647044 100644 --- a/qdrant-landing/content/documentation/embeddings/nvidia.md +++ b/qdrant-landing/content/documentation/embeddings/nvidia.md @@ -1,6 +1,6 @@ --- title: Nvidia -weight: 1200 +weight: 2400 --- # Nvidia diff --git a/qdrant-landing/content/documentation/embeddings/oci.md b/qdrant-landing/content/documentation/embeddings/oci.md new file mode 100644 index 000000000..934904254 --- /dev/null +++ b/qdrant-landing/content/documentation/embeddings/oci.md @@ -0,0 +1,54 @@ + +--- +title: OCI (Oracle Cloud Infrastructure) +weight: 2500 +aliases: + - /documentation/examples/oci-(oracle-cloud-infrastructure)-search/ + - /documentation/tutorials/oci-(oracle-cloud-infrastructure)-search/ + - /documentation/integrations/oci-(oracle-cloud-infrastructure)/ +--- + +# Using OCI (Oracle Cloud Infrastructure) with Qdrant + +OCI provides robust cloud-based embeddings for various media types. The Generative AI Embedding Models convert textual input - ranging from phrases and sentences to entire paragraphs - into a structured format known as embeddings. Each piece of text input is transformed into a numerical array consisting of 1024 distinct numbers. + +## Installation + +You can install the required package using the following pip command: + +```bash +pip install oci +``` + +## Code Example + +Below is an example of how to obtain embeddings using OCI (Oracle Cloud Infrastructure)'s API and store them in a Qdrant collection: + +```python +import qdrant_client +from qdrant_client.models import Batch +import oci + +# Initialize OCI client +config = oci.config.from_file() +ai_client = oci.ai_language.AIServiceLanguageClient(config) + +# Generate embeddings using OCI's AI service +text = "OCI provides cloud-based AI services." +response = ai_client.batch_detect_language_entities(text) +embeddings = response.data[0].entities[0].embedding + +# Initialize Qdrant client +qdrant_client = qdrant_client.QdrantClient(host="localhost", port=6333) + +# Upsert the embedding into Qdrant +qdrant_client.upsert( + collection_name="CloudAI", + points=Batch( + ids=[1], + vectors=[embeddings], + ) +) + +``` + diff --git a/qdrant-landing/content/documentation/embeddings/ollama.md b/qdrant-landing/content/documentation/embeddings/ollama.md new file mode 100644 index 000000000..5d329dc66 --- /dev/null +++ b/qdrant-landing/content/documentation/embeddings/ollama.md @@ -0,0 +1,52 @@ + +--- +title: Ollama +weight: 2600 +aliases: + - /documentation/examples/ollama-search/ + - /documentation/tutorials/ollama-search/ + - /documentation/integrations/ollama/ +--- + +# Using Ollama with Qdrant + +Ollama provides specialized embeddings for niche applications. Ollama supports a variety of embedding models, making it possible to build retrieval augmented generation (RAG) applications that combine text prompts with existing documents or other data in specialized areas. + + + +## Installation + +You can install the required package using the following pip command: + +```bash +pip install ollama +``` +## Integration Example + + +```python +import qdrant_client +from qdrant_client.models import Batch +from ollama import Ollama + +# Initialize Ollama model +model = Ollama("ollama-unique") + +# Generate embeddings for niche applications +text = "Ollama excels in niche applications with specific embeddings." +embeddings = model.embed(text) + +# Initialize Qdrant client +qdrant_client = qdrant_client.QdrantClient(host="localhost", port=6333) + +# Upsert the embedding into Qdrant +qdrant_client.upsert( + collection_name="NicheApplications", + points=Batch( + ids=[1], + vectors=[embeddings], + ) +) + +``` + diff --git a/qdrant-landing/content/documentation/embeddings/openai.md b/qdrant-landing/content/documentation/embeddings/openai.md index cc958479c..3f7074bed 100644 --- a/qdrant-landing/content/documentation/embeddings/openai.md +++ b/qdrant-landing/content/documentation/embeddings/openai.md @@ -1,6 +1,6 @@ --- title: OpenAI -weight: 800 +weight: 2700 aliases: [ ../integrations/openai/ ] --- diff --git a/qdrant-landing/content/documentation/embeddings/openclip.md b/qdrant-landing/content/documentation/embeddings/openclip.md new file mode 100644 index 000000000..863419a29 --- /dev/null +++ b/qdrant-landing/content/documentation/embeddings/openclip.md @@ -0,0 +1,46 @@ + +--- +title: OpenCLIP +weight: 2750 +aliases: + - /documentation/examples/openclip-search/ + - /documentation/tutorials/openclip-search/ + - /documentation/integrations/openclip/ +--- + +# Using OpenCLIP with Qdrant + +OpenCLIP is an open-source implementation of the CLIP model, allowing for open source generation of multimodal embeddings that link text and images. + +```python +import qdrant_client +from qdrant_client.models import Batch +import open_clip + +# Load the OpenCLIP model and tokenizer +model, preprocess = open_clip.create_model_and_transforms('ViT-B-32', pretrained='openai') +tokenizer = open_clip.get_tokenizer('ViT-B-32') + +# Generate embeddings for a text +text = "A photo of a cat" +text_inputs = tokenizer([text]) + +with torch.no_grad(): + text_features = model.encode_text(text_inputs) + +# Convert tensor to a list +embeddings = text_features[0].cpu().numpy().tolist() + +# Initialize Qdrant client +qdrant_client = qdrant_client.QdrantClient(host="localhost", port=6333) + +# Upsert the embedding into Qdrant +qdrant_client.upsert( + collection_name="OpenCLIPEmbeddings", + points=Batch( + ids=[1], + vectors=[embeddings], + ) +) +``` + diff --git a/qdrant-landing/content/documentation/embeddings/premai.md b/qdrant-landing/content/documentation/embeddings/premai.md index df7868be5..cda863568 100644 --- a/qdrant-landing/content/documentation/embeddings/premai.md +++ b/qdrant-landing/content/documentation/embeddings/premai.md @@ -1,6 +1,6 @@ --- title: Prem AI -weight: 1600 +weight: 2800 --- # Prem AI diff --git a/qdrant-landing/content/documentation/embeddings/snowflake.md b/qdrant-landing/content/documentation/embeddings/snowflake.md index f732c44b2..22aa6a06e 100644 --- a/qdrant-landing/content/documentation/embeddings/snowflake.md +++ b/qdrant-landing/content/documentation/embeddings/snowflake.md @@ -1,6 +1,6 @@ --- title: Snowflake Models -weight: 1500 +weight: 2900 --- # Snowflake diff --git a/qdrant-landing/content/documentation/embeddings/together_ai.md b/qdrant-landing/content/documentation/embeddings/together_ai.md new file mode 100644 index 000000000..ba7f932b2 --- /dev/null +++ b/qdrant-landing/content/documentation/embeddings/together_ai.md @@ -0,0 +1,48 @@ + +--- +title: Together AI +weight: 3000 +aliases: + - /documentation/examples/together-ai-search/ + - /documentation/tutorials/together-ai-search/ + - /documentation/integrations/together-ai/ +--- + +# Using Together AI with Qdrant + +Together AI focuses on collaborative AI embeddings that enhance multi-user search scenarios when integrated with Qdrant. + +## Installation + +You can install the required package using the following pip command: + +```bash +pip install togetherai +``` +## Integration Example + +```python +import qdrant_client +from qdrant_client.models import Batch +from togetherai import TogetherAI + +# Initialize Together AI model +model = TogetherAI("togetherai-collab") + +# Generate embeddings for collaborative content +text = "Together AI enhances collaborative content search." +embeddings = model.embed(text) + +# Initialize Qdrant client +qdrant_client = qdrant_client.QdrantClient(host="localhost", port=6333) + +# Upsert the embedding into Qdrant +qdrant_client.upsert( + collection_name="CollaborativeContent", + points=Batch( + ids=[1], + vectors=[embeddings], + ) +) + +``` diff --git a/qdrant-landing/content/documentation/embeddings/upstage.md b/qdrant-landing/content/documentation/embeddings/upstage.md index 40905f410..2a24d52c3 100644 --- a/qdrant-landing/content/documentation/embeddings/upstage.md +++ b/qdrant-landing/content/documentation/embeddings/upstage.md @@ -1,6 +1,6 @@ --- title: Upstage -weight: 1700 +weight: 3100 --- # Upstage diff --git a/qdrant-landing/content/documentation/embeddings/voyage.md b/qdrant-landing/content/documentation/embeddings/voyage.md index a63108b14..024804467 100644 --- a/qdrant-landing/content/documentation/embeddings/voyage.md +++ b/qdrant-landing/content/documentation/embeddings/voyage.md @@ -1,6 +1,6 @@ --- title: Voyage AI -weight: 1300 +weight: 3200 --- # Voyage AI diff --git a/qdrant-landing/content/documentation/embeddings/watsonx.md b/qdrant-landing/content/documentation/embeddings/watsonx.md new file mode 100644 index 000000000..5ceaafc98 --- /dev/null +++ b/qdrant-landing/content/documentation/embeddings/watsonx.md @@ -0,0 +1,50 @@ + +--- +title: Watsonx +weight: 3000 +aliases: + - /documentation/examples/watsonx-search/ + - /documentation/tutorials/watsonx-search/ + - /documentation/integrations/watsonx/ +--- + +# Using Watsonx with Qdrant + +Watsonx is IBM's platform for AI embeddings, focusing on enterprise-level text and data analytics. These embeddings are suitable for high-precision vector searches in Qdrant. + +## Installation + +You can install the required package using the following pip command: + +```bash +pip install watsonx +``` + +## Code Example + + +```python +import qdrant_client +from qdrant_client.models import Batch +from watsonx import Watsonx + +# Initialize Watsonx AI model +model = Watsonx("watsonx-model") + +# Generate embeddings for enterprise data +text = "Watsonx provides enterprise-level NLP solutions." +embeddings = model.embed(text) + +# Initialize Qdrant client +qdrant_client = qdrant_client.QdrantClient(host="localhost", port=6333) + +# Upsert the embedding into Qdrant +qdrant_client.upsert( + collection_name="EnterpriseData", + points=Batch( + ids=[1], + vectors=[embeddings], + ) +) + +``` From 828979ab094a0c8a7e3786d15b630c840fb0d5a0 Mon Sep 17 00:00:00 2001 From: davidmyriel Date: Wed, 21 Aug 2024 17:38:33 -0700 Subject: [PATCH 2/4] fix list --- .../content/documentation/embeddings/_index.md | 14 +++++--------- .../content/documentation/embeddings/clarifai.md | 5 ----- .../content/documentation/embeddings/clip.md | 5 ----- .../content/documentation/embeddings/databricks.md | 5 ----- .../content/documentation/embeddings/gpt4all.md | 5 ----- .../content/documentation/embeddings/instruct.md | 7 +------ .../content/documentation/embeddings/johnsnow.md | 7 +------ .../content/documentation/embeddings/mixedbread.md | 5 ----- .../content/documentation/embeddings/oci.md | 5 ----- .../content/documentation/embeddings/ollama.md | 5 ----- .../content/documentation/embeddings/openclip.md | 5 ----- .../embeddings/{together_ai.md => togetherai.md} | 5 ----- 12 files changed, 7 insertions(+), 66 deletions(-) rename qdrant-landing/content/documentation/embeddings/{together_ai.md => togetherai.md} (86%) diff --git a/qdrant-landing/content/documentation/embeddings/_index.md b/qdrant-landing/content/documentation/embeddings/_index.md index 6d957d97f..36fbb46c6 100644 --- a/qdrant-landing/content/documentation/embeddings/_index.md +++ b/qdrant-landing/content/documentation/embeddings/_index.md @@ -7,19 +7,14 @@ weight: 15 Qdrant supports all available text and multimodal dense vector embedding models as well as vector embedding services without any limitations. -## The following have been tested and verified: - -SentenceTransformers, BERT, SBERT, Clip, OpenClip, Open AI, Vertex AI, Azure AI, AWS Bedrock, Jina AI, Upstage AI, Mistral AI, Cohere AI, Voyage AI, Aleph Alpha, Baidu Qianfan, BGE, Instruct, Watsonx Embeddings, Snowflake Embeddings, NVIDIA NeMo, Nomic, OCI Embeddings, Ollama Embeddings, MixedBread, Together AI, Clarifai, Databricks Embeddings, GPT4All Embeddings, John Snow Labs Embeddings. - Additionally, [any open-source embeddings from HuggingFace](https://huggingface.co/spaces/mteb/leaderboard) can be used with Qdrant. -## Integration code samples: +## The following providers have been tested and verified: | Embeddings Providers | Description | | ----------------------------- | ----------- | | [Aleph Alpha](./aleph-alpha/) | Multilingual embeddings focused on European languages. | | [Bedrock](./bedrock/) | AWS managed service for foundation models and embeddings. | -| [BGE](./bge/) | Chinese embeddings for various NLP tasks. | | [Clarifai](./clarifai/) | Embeddings for image and video recognition. | | [Clip](./clip/) | Aligns images and text, created by OpenAI. | | [Cohere](./cohere/) | Language model embeddings for NLP tasks. | @@ -27,18 +22,19 @@ Additionally, [any open-source embeddings from HuggingFace](https://huggingface. | [Gemini](./gemini/) | Google’s multimodal embeddings for text and vision. | | [GPT4All](./gpt4all/) | Open-source, local embeddings for privacy-focused use. | | [Instruct](./instruct/) | Embeddings tuned for following instructions. | -| [Jina](./jina-emebddngs/) | Customizable embeddings for neural search. | +| [Jina AI](./jina-embeddings/) | Customizable embeddings for neural search. | | [John Snow Labs](./johnsnow/) | Medical and clinical embeddings. | | [Mistral](./mistral/) | Open-source, efficient language model embeddings. | | [MixedBread](./mixedbread/) | Lightweight embeddings for constrained environments. | | [Nomic](./nomic/) | Embeddings for data visualization. | -| [Nvidia](./nvidia_nemo/) | GPU-optimized embeddings from Nvidia. | +| [Nvidia](./nvidia/) | GPU-optimized embeddings from Nvidia. | | [OCI](./oci/) | Oracle Cloud’s AI service with embeddings. | | [Ollama](./ollama/) | Embeddings for conversational AI. | | [OpenAI](./openai/) | Industry-leading embeddings for NLP. | +| [OpenCLIP](./openclip/) | OS implementation of CLIP for image and text. | | [Prem AI](./premai/) | Precise language embeddings. | | [Snowflake](./snowflake/) | Scalable embeddings for big data. | -| [Together AI](./together_ai/) | Community-driven, open-source embeddings. | +| [Together AI](./togetherai/) | Community-driven, open-source embeddings. | | [Upstage](./upstage/) | Embeddings for speech and language tasks. | | [Voyage AI](./voyage/) | Navigation and spatial understanding embeddings. | | [Watsonx](./watsonx/) | IBM's enterprise-grade embeddings. | diff --git a/qdrant-landing/content/documentation/embeddings/clarifai.md b/qdrant-landing/content/documentation/embeddings/clarifai.md index e46d731f2..6a15107f7 100644 --- a/qdrant-landing/content/documentation/embeddings/clarifai.md +++ b/qdrant-landing/content/documentation/embeddings/clarifai.md @@ -1,11 +1,6 @@ - --- title: Clarifai weight: 1200 -aliases: - - /documentation/examples/clarifai-search/ - - /documentation/tutorials/clarifai-search/ - - /documentation/integrations/clarifai/ --- # Using Clarifai Embeddings with Qdrant diff --git a/qdrant-landing/content/documentation/embeddings/clip.md b/qdrant-landing/content/documentation/embeddings/clip.md index a907da24f..e5374ed92 100644 --- a/qdrant-landing/content/documentation/embeddings/clip.md +++ b/qdrant-landing/content/documentation/embeddings/clip.md @@ -1,11 +1,6 @@ - --- title: Clip weight: 1300 -aliases: - - /documentation/examples/clip-search/ - - /documentation/tutorials/clip-search/ - - /documentation/integrations/clip/ --- # Using Clip with Qdrant diff --git a/qdrant-landing/content/documentation/embeddings/databricks.md b/qdrant-landing/content/documentation/embeddings/databricks.md index 39d0e85ef..47f41efa6 100644 --- a/qdrant-landing/content/documentation/embeddings/databricks.md +++ b/qdrant-landing/content/documentation/embeddings/databricks.md @@ -1,11 +1,6 @@ - --- title: Databricks Embeddings weight: 1500 -aliases: - - /documentation/examples/databricks-embeddings-search/ - - /documentation/tutorials/databricks-embeddings-search/ - - /documentation/integrations/databricks-embeddings/ --- # Using Databricks Embeddings with Qdrant diff --git a/qdrant-landing/content/documentation/embeddings/gpt4all.md b/qdrant-landing/content/documentation/embeddings/gpt4all.md index 87a30f61a..0501568ba 100644 --- a/qdrant-landing/content/documentation/embeddings/gpt4all.md +++ b/qdrant-landing/content/documentation/embeddings/gpt4all.md @@ -1,11 +1,6 @@ - --- title: GPT4All weight: 1700 -aliases: - - /documentation/examples/gpt4all-search/ - - /documentation/tutorials/gpt4all-search/ - - /documentation/integrations/gpt4all/ --- # Using GPT4All with Qdrant diff --git a/qdrant-landing/content/documentation/embeddings/instruct.md b/qdrant-landing/content/documentation/embeddings/instruct.md index 632efece5..992a17d78 100644 --- a/qdrant-landing/content/documentation/embeddings/instruct.md +++ b/qdrant-landing/content/documentation/embeddings/instruct.md @@ -1,11 +1,6 @@ - --- title: Instruct -weight: 1800 -aliases: - - /documentation/examples/instruct-search/ - - /documentation/tutorials/instruct-search/ - - /documentation/integrations/instruct/ +weight: 1800 --- # Using Instruct with Qdrant diff --git a/qdrant-landing/content/documentation/embeddings/johnsnow.md b/qdrant-landing/content/documentation/embeddings/johnsnow.md index 4c8d9f0af..a799dcf5b 100644 --- a/qdrant-landing/content/documentation/embeddings/johnsnow.md +++ b/qdrant-landing/content/documentation/embeddings/johnsnow.md @@ -1,11 +1,6 @@ - --- title: John Snow Labs weight: 2000 -aliases: - - /documentation/examples/john-snow-labs-search/ - - /documentation/tutorials/john-snow-labs-search/ - - /documentation/integrations/john-snow-labs/ --- # Using John Snow Labs with Qdrant @@ -21,7 +16,7 @@ pip install johnsnowlabs ``` -Here is an example of how you mmight obtain embeddings using John Snow Labs's API and store them in a Qdrant collection: +Here is an example of how you might obtain embeddings using John Snow Labs's API and store them in a Qdrant collection: ```python import qdrant_client diff --git a/qdrant-landing/content/documentation/embeddings/mixedbread.md b/qdrant-landing/content/documentation/embeddings/mixedbread.md index 44333e4c7..c30d705ba 100644 --- a/qdrant-landing/content/documentation/embeddings/mixedbread.md +++ b/qdrant-landing/content/documentation/embeddings/mixedbread.md @@ -1,11 +1,6 @@ - --- title: MixedBread weight: 2200 -aliases: - - /documentation/examples/mixedbread-search/ - - /documentation/tutorials/mixedbread-search/ - - /documentation/integrations/mixedbread/ --- # Using MixedBread with Qdrant diff --git a/qdrant-landing/content/documentation/embeddings/oci.md b/qdrant-landing/content/documentation/embeddings/oci.md index 934904254..5e9c69ded 100644 --- a/qdrant-landing/content/documentation/embeddings/oci.md +++ b/qdrant-landing/content/documentation/embeddings/oci.md @@ -1,11 +1,6 @@ - --- title: OCI (Oracle Cloud Infrastructure) weight: 2500 -aliases: - - /documentation/examples/oci-(oracle-cloud-infrastructure)-search/ - - /documentation/tutorials/oci-(oracle-cloud-infrastructure)-search/ - - /documentation/integrations/oci-(oracle-cloud-infrastructure)/ --- # Using OCI (Oracle Cloud Infrastructure) with Qdrant diff --git a/qdrant-landing/content/documentation/embeddings/ollama.md b/qdrant-landing/content/documentation/embeddings/ollama.md index 5d329dc66..382054557 100644 --- a/qdrant-landing/content/documentation/embeddings/ollama.md +++ b/qdrant-landing/content/documentation/embeddings/ollama.md @@ -1,11 +1,6 @@ - --- title: Ollama weight: 2600 -aliases: - - /documentation/examples/ollama-search/ - - /documentation/tutorials/ollama-search/ - - /documentation/integrations/ollama/ --- # Using Ollama with Qdrant diff --git a/qdrant-landing/content/documentation/embeddings/openclip.md b/qdrant-landing/content/documentation/embeddings/openclip.md index 863419a29..b9ab44862 100644 --- a/qdrant-landing/content/documentation/embeddings/openclip.md +++ b/qdrant-landing/content/documentation/embeddings/openclip.md @@ -1,11 +1,6 @@ - --- title: OpenCLIP weight: 2750 -aliases: - - /documentation/examples/openclip-search/ - - /documentation/tutorials/openclip-search/ - - /documentation/integrations/openclip/ --- # Using OpenCLIP with Qdrant diff --git a/qdrant-landing/content/documentation/embeddings/together_ai.md b/qdrant-landing/content/documentation/embeddings/togetherai.md similarity index 86% rename from qdrant-landing/content/documentation/embeddings/together_ai.md rename to qdrant-landing/content/documentation/embeddings/togetherai.md index ba7f932b2..12fe1b622 100644 --- a/qdrant-landing/content/documentation/embeddings/together_ai.md +++ b/qdrant-landing/content/documentation/embeddings/togetherai.md @@ -1,11 +1,6 @@ - --- title: Together AI weight: 3000 -aliases: - - /documentation/examples/together-ai-search/ - - /documentation/tutorials/together-ai-search/ - - /documentation/integrations/together-ai/ --- # Using Together AI with Qdrant From dbfb2fff8fb2b72d570ef64273d0b2126fff4270 Mon Sep 17 00:00:00 2001 From: davidmyriel Date: Wed, 21 Aug 2024 17:44:48 -0700 Subject: [PATCH 3/4] add title --- qdrant-landing/content/documentation/embeddings/_index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/qdrant-landing/content/documentation/embeddings/_index.md b/qdrant-landing/content/documentation/embeddings/_index.md index 36fbb46c6..838cede8e 100644 --- a/qdrant-landing/content/documentation/embeddings/_index.md +++ b/qdrant-landing/content/documentation/embeddings/_index.md @@ -9,7 +9,7 @@ Qdrant supports all available text and multimodal dense vector embedding models Additionally, [any open-source embeddings from HuggingFace](https://huggingface.co/spaces/mteb/leaderboard) can be used with Qdrant. -## The following providers have been tested and verified: +## The following 25 providers have been tested and verified: | Embeddings Providers | Description | | ----------------------------- | ----------- | From efb3a7f402229905437bb469cfb458966d1d8260 Mon Sep 17 00:00:00 2001 From: davidmyriel Date: Thu, 22 Aug 2024 17:59:02 -0700 Subject: [PATCH 4/4] add more embeddings --- .../documentation/embeddings/_index.md | 8 +- .../content/documentation/embeddings/azure.md | 77 +++++++++++++++++++ .../documentation/embeddings/gradientai.md | 62 +++++++++++++++ 3 files changed, 146 insertions(+), 1 deletion(-) create mode 100644 qdrant-landing/content/documentation/embeddings/azure.md create mode 100644 qdrant-landing/content/documentation/embeddings/gradientai.md diff --git a/qdrant-landing/content/documentation/embeddings/_index.md b/qdrant-landing/content/documentation/embeddings/_index.md index 838cede8e..fc8b85617 100644 --- a/qdrant-landing/content/documentation/embeddings/_index.md +++ b/qdrant-landing/content/documentation/embeddings/_index.md @@ -7,13 +7,18 @@ weight: 15 Qdrant supports all available text and multimodal dense vector embedding models as well as vector embedding services without any limitations. +## The following have been tested and verified: + +SentenceTransformers, BERT, SBERT, Clip, OpenClip, Open AI, Vertex AI, Azure AI, AWS Bedrock, Jina AI, Upstage AI, Mistral AI, Cohere AI, Voyage AI, Aleph Alpha, Baidu Qianfan, BGE, Instruct, Watsonx Embeddings, Snowflake Embeddings, NVIDIA NeMo, Nomic, OCI Embeddings, Ollama Embeddings, MixedBread, Together AI, Clarifai, Databricks Embeddings, GPT4All Embeddings, John Snow Labs Embeddings. + Additionally, [any open-source embeddings from HuggingFace](https://huggingface.co/spaces/mteb/leaderboard) can be used with Qdrant. -## The following 25 providers have been tested and verified: +## Verified integration code samples: | Embeddings Providers | Description | | ----------------------------- | ----------- | | [Aleph Alpha](./aleph-alpha/) | Multilingual embeddings focused on European languages. | +| [Azure](./azure/) | Microsoft's embedding model selection. | | [Bedrock](./bedrock/) | AWS managed service for foundation models and embeddings. | | [Clarifai](./clarifai/) | Embeddings for image and video recognition. | | [Clip](./clip/) | Aligns images and text, created by OpenAI. | @@ -21,6 +26,7 @@ Additionally, [any open-source embeddings from HuggingFace](https://huggingface. | [Databricks](./databricks/) | Scalable embeddings integrated with Apache Spark. | | [Gemini](./gemini/) | Google’s multimodal embeddings for text and vision. | | [GPT4All](./gpt4all/) | Open-source, local embeddings for privacy-focused use. | +| [GradientAI](./gradient/) | AI Models for custom enterprise tasks.| | [Instruct](./instruct/) | Embeddings tuned for following instructions. | | [Jina AI](./jina-embeddings/) | Customizable embeddings for neural search. | | [John Snow Labs](./johnsnow/) | Medical and clinical embeddings. | diff --git a/qdrant-landing/content/documentation/embeddings/azure.md b/qdrant-landing/content/documentation/embeddings/azure.md new file mode 100644 index 000000000..a6ef7cc95 --- /dev/null +++ b/qdrant-landing/content/documentation/embeddings/azure.md @@ -0,0 +1,77 @@ +--- +title: Azure OpenAI +weight: 950 +--- + +# Using Azure OpenAI with Qdrant + +Azure OpenAI is Microsoft's platform for AI embeddings, focusing on powerful text and data analytics. These embeddings are suitable for high-precision vector searches in Qdrant. + +## Installation + +You can install the required packages using the following pip command: + +```bash +pip install openai azure-identity python-dotenv qdrant-client +``` + +## Code Example + +```python +import os +import openai +import dotenv +import qdrant_client +from qdrant_client.models import Batch +from azure.identity import DefaultAzureCredential, get_bearer_token_provider + +dotenv.load_dotenv() + +# Set to True if using Azure Active Directory for authentication +use_azure_active_directory = False + +# Qdrant client setup +qdrant_client = qdrant_client.QdrantClient(url="http://localhost:6333") + +# Azure OpenAI Authentication +if not use_azure_active_directory: + endpoint = os.environ["AZURE_OPENAI_ENDPOINT"] + api_key = os.environ["AZURE_OPENAI_API_KEY"] + + client = openai.AzureOpenAI( + azure_endpoint=endpoint, + api_key=api_key, + api_version="2023-09-01-preview" + ) +else: + endpoint = os.environ["AZURE_OPENAI_ENDPOINT"] + client = openai.AzureOpenAI( + azure_endpoint=endpoint, + azure_ad_token_provider=get_bearer_token_provider(DefaultAzureCredential(), "https://cognitiveservices.azure.com/.default"), + api_version="2023-09-01-preview" + ) + +# Deployment name of the model in Azure OpenAI Studio +deployment = "your-deployment-name" # Replace with your deployment name + +# Generate embeddings using the Azure OpenAI client +text_input = "The food was delicious and the waiter..." +embeddings_response = client.embeddings.create( + model=deployment, + input=text_input +) + +# Extract the embedding vector from the response +embedding_vector = embeddings_response.data[0].embedding + +# Insert the embedding into Qdrant +qdrant_client.upsert( + collection_name="MyCollection", + points=Batch( + ids=[1], # This ID can be dynamically assigned or managed + vectors=[embedding_vector], + ) +) + +print("Embedding successfully upserted into Qdrant.") +``` \ No newline at end of file diff --git a/qdrant-landing/content/documentation/embeddings/gradientai.md b/qdrant-landing/content/documentation/embeddings/gradientai.md new file mode 100644 index 000000000..06ffb2e67 --- /dev/null +++ b/qdrant-landing/content/documentation/embeddings/gradientai.md @@ -0,0 +1,62 @@ +--- +title: GradientAI +weight: 1750 +--- + +# Using GradientAI with Qdrant + +GradientAI provides state-of-the-art models for generating embeddings, which are highly effective for vector search tasks in Qdrant. + +## Installation + +You can install the required packages using the following pip command: + +```bash +pip install gradientai python-dotenv qdrant-client +``` + +## Code Example + +```python +from dotenv import load_dotenv +import qdrant_client +from qdrant_client.models import Batch +from gradientai import Gradient + +load_dotenv() + +def main() -> None: + # Initialize GradientAI client + gradient = Gradient() + + # Retrieve the embeddings model + embeddings_model = gradient.get_embeddings_model(slug="bge-large") + + # Generate embeddings for your data + generate_embeddings_response = embeddings_model.generate_embeddings( + inputs=[ + "Multimodal brain MRI is the preferred method to evaluate for acute ischemic infarct and ideally should be obtained within 24 hours of symptom onset, and in most centers will follow a NCCT", + "CTA has a higher sensitivity and positive predictive value than magnetic resonance angiography (MRA) for detection of intracranial stenosis and occlusion and is recommended over time-of-flight (without contrast) MRA", + "Echocardiographic strain imaging has the advantage of detecting early cardiac involvement, even before thickened walls or symptoms are apparent", + ], + ) + + # Initialize Qdrant client + client = qdrant_client.QdrantClient(url="http://localhost:6333") + + # Upsert the embeddings into Qdrant + for i, embedding in enumerate(generate_embeddings_response.embeddings): + client.upsert( + collection_name="MedicalRecords", + points=Batch( + ids=[i + 1], # Unique ID for each embedding + vectors=[embedding.embedding], + ) + ) + + print("Embeddings successfully upserted into Qdrant.") + gradient.close() + +if __name__ == "__main__": + main() +``` \ No newline at end of file