mirror of
https://github.com/qdrant/landing_page.git
synced 2026-09-28 23:48:31 +02:00
Add Gemini Embedding Model 001 (#457)
* * feat(gemini.md): add documentation for integrating Gemini embeddings with Qdrant * * refactor(integrations): move cohere.md to embeddings folder * refactor(integrations): move openai.md to embeddings folder * refactor(integrations): move autogen.md to frameworks folder * refactor(integrations): move langchain.md to frameworks folder * blacken * * feat(embedding, frameworks): reorganise integrations into embedding and frameworks, add _index.md to both * * chore(gemini.md): remove old Gemini integration documentation * * chore(embedding/_index.md): update weight from 24 to 23 and set is_empty to false * chore(frameworks/_index.md): update weight from 24 to 23 and set is_empty to false * Split integrations into embedding and frameworks * Update heading level for embedding a document * Update Gemini embedding documentation * Update titles for embedding and frameworks sections * Try again with nesting * Add documentation for integrated frameworks and embedding options * Delete integrations documentation file * Add Delimiter; unknown weights * Change all weights to 3x * Delimiter reorg * Update qdrant-landing/content/documentation/embedding/gemini.md Co-authored-by: Atita Arora <atarora@users.noreply.github.com> * Update qdrant-landing/content/documentation/embedding/gemini.md Co-authored-by: Atita Arora <atarora@users.noreply.github.com> * Update qdrant-landing/content/documentation/embedding/gemini.md Co-authored-by: Atita Arora <atarora@users.noreply.github.com> * Update qdrant-landing/content/documentation/embedding/gemini.md Co-authored-by: Atita Arora <atarora@users.noreply.github.com> * Update qdrant-landing/content/documentation/embedding/gemini.md Co-authored-by: Atita Arora <atarora@users.noreply.github.com> * Update qdrant-landing/content/documentation/embedding/gemini.md Co-authored-by: Atita Arora <atarora@users.noreply.github.com> * Update qdrant-landing/content/documentation/embedding/gemini.md Co-authored-by: Atita Arora <atarora@users.noreply.github.com> * Update qdrant-landing/content/documentation/embedding/gemini.md Co-authored-by: Atita Arora <atarora@users.noreply.github.com> * Update qdrant-landing/content/documentation/embedding/gemini.md Co-authored-by: Atita Arora <atarora@users.noreply.github.com> * * docs(embedding/gemini.md): update Gemini Embedding Model API documentation * * - Add information about the new Gemini Embedding Model and its compatibility with Qdrant * - Clarify the usage of the `task_type` parameter in the API call * - Provide a list of supported task types and * * docs(embedding): update list of embedding integrations * * refactor(fifty-one.md): Rename file from embedding/fifty-one.md to frameworks/fifty-one.md * refactor(txtai.md): Rename file from embedding/txtai.md to frameworks/txtai.md * * chore(embedding): update is_empty value to true in _index.md * chore(embedding): remove Fifty One from embedding/_index.md * embedding -> embeddings --------- Co-authored-by: Atita Arora <atarora@users.noreply.github.com>
This commit is contained in:
Nirant
@@ -0,0 +1,14 @@
|
||||
---
|
||||
title: Embeddings
|
||||
weight: 33
|
||||
# If the index.md file is empty, the link to the section will be hidden from the sidebar
|
||||
is_empty: true
|
||||
---
|
||||
|
||||
| Embedding |
|
||||
|---|
|
||||
| [Gemini](./gemini/) |
|
||||
| [Aleph Alpha](./aleph-alpha/) |
|
||||
| [Cohere](./cohere/) |
|
||||
| [Jina](./jina-emebddngs/) |
|
||||
| [OpenAI](./openai/) |
|
||||
@@ -0,0 +1,57 @@
|
||||
---
|
||||
title: Aleph Alpha
|
||||
weight: 900
|
||||
---
|
||||
|
||||
Aleph Alpha is a multimodal and multilingual embeddings' provider. Their API allows creating the embeddings for text and images, both
|
||||
in the same latent space. They maintain an [official Python client](https://github.com/Aleph-Alpha/aleph-alpha-client) that might be
|
||||
installed with pip:
|
||||
|
||||
```bash
|
||||
pip install aleph-alpha-client
|
||||
```
|
||||
|
||||
There is both synchronous and asynchronous client available. Obtaining the embeddings for an image and storing it into Qdrant might
|
||||
be done in the following way:
|
||||
|
||||
```python
|
||||
import qdrant_client
|
||||
|
||||
from aleph_alpha_client import (
|
||||
Prompt,
|
||||
AsyncClient,
|
||||
SemanticEmbeddingRequest,
|
||||
SemanticRepresentation,
|
||||
ImagePrompt
|
||||
)
|
||||
from qdrant_client.http.models import Batch
|
||||
|
||||
aa_token = "<< your_token >>"
|
||||
model = "luminous-base"
|
||||
|
||||
qdrant_client = qdrant_client.QdrantClient()
|
||||
async with AsyncClient(token=aa_token) as client:
|
||||
prompt = ImagePrompt.from_file("./path/to/the/image.jpg")
|
||||
prompt = Prompt.from_image(prompt)
|
||||
|
||||
query_params = {
|
||||
"prompt": prompt,
|
||||
"representation": SemanticRepresentation.Symmetric,
|
||||
"compress_to_size": 128,
|
||||
}
|
||||
query_request = SemanticEmbeddingRequest(**query_params)
|
||||
query_response = await client.semantic_embed(
|
||||
request=query_request, model=model
|
||||
)
|
||||
|
||||
qdrant_client.upsert(
|
||||
collection_name="MyCollection",
|
||||
points=Batch(
|
||||
ids=[1],
|
||||
vectors=[query_response.embedding],
|
||||
)
|
||||
)
|
||||
```
|
||||
|
||||
If we wanted to create text embeddings with the same model, we wouldn't use `ImagePrompt.from_file`, but simply provide the input
|
||||
text into the `Prompt.from_text` method.
|
||||
@@ -0,0 +1,91 @@
|
||||
---
|
||||
title: Cohere
|
||||
weight: 700
|
||||
---
|
||||
|
||||
# Cohere
|
||||
|
||||
Qdrant is compatible with Cohere [co.embed API](https://docs.cohere.ai/reference/embed) and its official Python SDK that
|
||||
might be installed as any other package:
|
||||
|
||||
```bash
|
||||
pip install cohere
|
||||
```
|
||||
|
||||
The embeddings returned by co.embed API might be used directly in the Qdrant client's calls:
|
||||
|
||||
```python
|
||||
import cohere
|
||||
import qdrant_client
|
||||
|
||||
from qdrant_client.http.models import Batch
|
||||
|
||||
cohere_client = cohere.Client("<< your_api_key >>")
|
||||
qdrant_client = qdrant_client.QdrantClient()
|
||||
qdrant_client.upsert(
|
||||
collection_name="MyCollection",
|
||||
points=Batch(
|
||||
ids=[1],
|
||||
vectors=cohere_client.embed(
|
||||
model="large",
|
||||
texts=["The best vector database"],
|
||||
).embeddings,
|
||||
),
|
||||
)
|
||||
```
|
||||
|
||||
If you are interested in seeing an end-to-end project created with co.embed API and Qdrant, please check out the
|
||||
"[Question Answering as a Service with Cohere and Qdrant](https://qdrant.tech/articles/qa-with-cohere-and-qdrant/)" article.
|
||||
|
||||
## Embed v3
|
||||
|
||||
Embed v3 is a new family of Cohere models, released in November 2023. The new models require passing an additional
|
||||
parameter to the API call: `input_type`. It determines the type of task you want to use the embeddings for.
|
||||
|
||||
- `input_type="search_document"` - for documents to store in Qdrant
|
||||
- `input_type="search_query"` - for search queries to find the most relevant documents
|
||||
- `input_type="classification"` - for classification tasks
|
||||
- `input_type="clustering"` - for text clustering
|
||||
|
||||
While implementing semantic search applications, such as RAG, you should use `input_type="search_document"` for the
|
||||
indexed documents and `input_type="search_query"` for the search queries. The following example shows how to index
|
||||
documents with the Embed v3 model:
|
||||
|
||||
```python
|
||||
import cohere
|
||||
import qdrant_client
|
||||
|
||||
from qdrant_client.http.models import Batch
|
||||
|
||||
cohere_client = cohere.Client("<< your_api_key >>")
|
||||
qdrant_client = qdrant_client.QdrantClient()
|
||||
qdrant_client.upsert(
|
||||
collection_name="MyCollection",
|
||||
points=Batch(
|
||||
ids=[1],
|
||||
vectors=cohere_client.embed(
|
||||
model="embed-english-v3.0", # New Embed v3 model
|
||||
input_type="search_document", # Input type for documents
|
||||
texts=["Qdrant is the a vector database written in Rust"],
|
||||
).embeddings,
|
||||
),
|
||||
)
|
||||
```
|
||||
|
||||
Once the documents are indexed, you can search for the most relevant documents using the Embed v3 model:
|
||||
|
||||
```python
|
||||
qdrant_client.search(
|
||||
collection_name="MyCollection",
|
||||
query=cohere_client.embed(
|
||||
model="embed-english-v3.0", # New Embed v3 model
|
||||
input_type="search_query", # Input type for search queries
|
||||
texts=["The best vector database"],
|
||||
).embeddings[0],
|
||||
)
|
||||
```
|
||||
|
||||
<aside role="status">
|
||||
According to Cohere's documentation, all v3 models can use dot product, cosine similarity,
|
||||
and Euclidean distance as the similarity metric, as all metrics return identical rankings.
|
||||
</aside>
|
||||
@@ -0,0 +1,94 @@
|
||||
---
|
||||
title: Gemini
|
||||
weight: 700
|
||||
---
|
||||
|
||||
# Gemini
|
||||
|
||||
Qdrant is compatible with Gemini Embedding Model API and its official Python SDK that can be installed as any other package:
|
||||
|
||||
Gemini is a new family of Google PaLM models, released in December 2023. The new embedding models succeed the previous Gecko Embedding Model.
|
||||
|
||||
In the latest models, an additional parameter, `task_type`, can be passed to the API call. This parameter serves to designate the intended purpose for the embeddings utilized.
|
||||
|
||||
The Embedding Model API supports various task types, outlined as follows:
|
||||
|
||||
1. `retrieval_query`: Specifies the given text is a query in a search/retrieval setting.
|
||||
2. `retrieval_document`: Specifies the given text is a document from the corpus being searched.
|
||||
3. `semantic_similarity`: Specifies the given text will be used for Semantic Text Similarity.
|
||||
4. `classification`: Specifies that the given text will be classified.
|
||||
5. `clustering`: Specifies that the embeddings will be used for clustering.
|
||||
6. `task_type_unspecified`: Unset value, which will default to one of the other values.
|
||||
|
||||
|
||||
If you're building a semantic search application, such as RAG, you should use `task_type="retrieval_document"` for the indexed documents and `task_type="retrieval_query"` for the search queries.
|
||||
|
||||
The following example shows how to do this with Qdrant:
|
||||
|
||||
## Setup
|
||||
|
||||
```bash
|
||||
pip install google-generativeai
|
||||
```
|
||||
|
||||
Let's see how to use the Embedding Model API to embed a document for retrieval.
|
||||
|
||||
The following example shows how to embed a document with the `models/embedding-001` with the `retrieval_document` task type:
|
||||
|
||||
## Embedding a document
|
||||
|
||||
```python
|
||||
import pathlib
|
||||
import google.generativeai as genai
|
||||
import qdrant_client
|
||||
|
||||
GEMINI_API_KEY = "YOUR GEMINI API KEY" # add your key here
|
||||
|
||||
genai.configure(api_key=GEMINI_API_KEY)
|
||||
|
||||
result = genai.embed_content(
|
||||
model="models/embedding-001",
|
||||
content="Qdrant is the best vector search engine to use with Gemini",
|
||||
task_type="retrieval_document",
|
||||
title="Qdrant x Gemini",
|
||||
)
|
||||
```
|
||||
|
||||
The returned result is a dictionary with a key: `embedding`. The value of this key is a list of floats representing the embedding of the document.
|
||||
|
||||
## Indexing documents with Qdrant
|
||||
|
||||
```python
|
||||
from qdrant_client.http.models import Batch
|
||||
|
||||
qdrant_client = qdrant_client.QdrantClient()
|
||||
qdrant_client.upsert(
|
||||
collection_name="GeminiCollection",
|
||||
points=Batch(
|
||||
ids=[1],
|
||||
vectors=genai.embed_content(
|
||||
model="models/embedding-001",
|
||||
content="Qdrant is the best vector search engine to use with Gemini",
|
||||
task_type="retrieval_document",
|
||||
title="Qdrant x Gemini",
|
||||
)["embedding"],
|
||||
),
|
||||
)
|
||||
```
|
||||
|
||||
## Searching for documents with Qdrant
|
||||
|
||||
Once the documents are indexed, you can search for the most relevant documents using the same model with the `retrieval_query` task type:
|
||||
|
||||
```python
|
||||
qdrant_client.search(
|
||||
collection_name="GeminiCollection",
|
||||
query=genai.embed_content(
|
||||
model="models/embedding-001",
|
||||
content="What is the best vector database to use with Gemini?",
|
||||
task_type="retrieval_query",
|
||||
)["embedding"],
|
||||
)
|
||||
```
|
||||
|
||||
That's it! You can now use Gemini Embedding Models with Qdrant.
|
||||
@@ -0,0 +1,59 @@
|
||||
---
|
||||
title: Jina Embeddings
|
||||
weight: 800
|
||||
---
|
||||
|
||||
# Jina Embeddings
|
||||
|
||||
Qdrant can also easily work with [Jina embeddings](https://jina.ai/embeddings/) which allow for model input lengths of up to 8192 tokens.
|
||||
|
||||
To call their endpoint, all you need is an API key obtainable [here](https://jina.ai/embeddings/).
|
||||
|
||||
```python
|
||||
import qdrant_client
|
||||
import requests
|
||||
|
||||
from qdrant_client.http.models import Distance, VectorParams
|
||||
from qdrant_client.http.models import Batch
|
||||
|
||||
# Provide Jina API key and choose one of the available models.
|
||||
# You can get a free trial key here: https://jina.ai/embeddings/
|
||||
JINA_API_KEY = "jina_xxxxxxxxxxx"
|
||||
MODEL = "jina-embeddings-v2-base-en" # or "jina-embeddings-v2-base-en"
|
||||
EMBEDDING_SIZE = 768 # 512 for small variant
|
||||
|
||||
# Get embeddings from the API
|
||||
url = "https://api.jina.ai/v1/embeddings"
|
||||
|
||||
headers = {
|
||||
"Content-Type": "application/json",
|
||||
"Authorization": f"Bearer {JINA_API_KEY}",
|
||||
}
|
||||
|
||||
data = {
|
||||
"input": ["Your text string goes here", "You can send multiple texts"],
|
||||
"model": MODEL,
|
||||
}
|
||||
|
||||
response = requests.post(url, headers=headers, json=data)
|
||||
embeddings = [d["embedding"] for d in response.json()["data"]]
|
||||
|
||||
|
||||
# Index the embeddings into Qdrant
|
||||
qdrant_client = qdrant_client.QdrantClient(":memory:")
|
||||
qdrant_client.create_collection(
|
||||
collection_name="MyCollection",
|
||||
vectors_config=VectorParams(size=EMBEDDING_SIZE, distance=Distance.DOT),
|
||||
)
|
||||
|
||||
|
||||
qdrant_client.upsert(
|
||||
collection_name="MyCollection",
|
||||
points=Batch(
|
||||
ids=list(range(len(embeddings))),
|
||||
vectors=embeddings,
|
||||
),
|
||||
)
|
||||
|
||||
```
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
---
|
||||
title: OpenAI
|
||||
weight: 800
|
||||
---
|
||||
|
||||
# OpenAI
|
||||
|
||||
Qdrant can also easily work with [OpenAI embeddings](https://beta.openai.com/docs/guides/embeddings/embeddings).
|
||||
|
||||
There is an official OpenAI Python package that simplifies obtaining them, and it might be installed with pip:
|
||||
|
||||
```bash
|
||||
pip install openai
|
||||
```
|
||||
|
||||
Once installed, the package exposes the method allowing to retrieve the embedding for given text. OpenAI requires an API key that has to be provided either as an environmental variable `OPENAI_API_KEY` or set in the source code directly, as presented below:
|
||||
|
||||
```python
|
||||
import openai
|
||||
import qdrant_client
|
||||
|
||||
from qdrant_client.http.models import Batch
|
||||
|
||||
# Provide OpenAI API key and choose one of the available models:
|
||||
# https://beta.openai.com/docs/models/overview
|
||||
openai.api_key = "<< your_api_key >>"
|
||||
embedding_model = "text-embedding-ada-002"
|
||||
|
||||
response = openai.Embedding.create(
|
||||
input="The best vector database",
|
||||
model=embedding_model,
|
||||
)
|
||||
|
||||
qdrant_client = qdrant_client.QdrantClient()
|
||||
qdrant_client.upsert(
|
||||
collection_name="MyCollection",
|
||||
points=Batch(
|
||||
ids=[1],
|
||||
vectors=[response["data"][0]["embedding"]],
|
||||
),
|
||||
)
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user