diff --git a/qdrant-landing/content/documentation/embeddings/jina-embeddings.md b/qdrant-landing/content/documentation/embeddings/jina-embeddings.md index 423d3595c..94e73d4c8 100644 --- a/qdrant-landing/content/documentation/embeddings/jina-embeddings.md +++ b/qdrant-landing/content/documentation/embeddings/jina-embeddings.md @@ -8,21 +8,57 @@ aliases: # Jina Embeddings -Qdrant can also easily work with [Jina embeddings](https://jina.ai/embeddings/) which allow for model input lengths of up to 8192 tokens. +Qdrant is compatible with [Jina AI](https://jina.ai/) embeddings. You can get a free trial key from [Jina Embeddings](https://jina.ai/embeddings/) to get embeddings. + +Qdrant users can receive a 10% discount on Jina AI APIs by using the code **QDRANT**. + +## Technical Summary + +| Model | Dimension | Language | MRL (matryoshka) | Context | +|:----------------------:|:---------:|:---------:|:-----------:|:---------:| +| jina-embeddings-v3 | 1024 | Multilingual (89 languages) | Yes | 8192 | +| jina-embeddings-v2-base-en | 768 | English | No | 8192 | +| jina-embeddings-v2-base-de | 768 | German & English | No | 8192 | +| jina-embeddings-v2-base-es | 768 | Spanish & English | No | 8192 | +| jina-embeddings-v2-base-zh | 768 | Chinese & English | No | 8192 | + +> Jina recommends using `jina-embeddings-v3` as it is the latest and most performant embedding model released by Jina AI. + +On top of the backbone, `jina-embeddings-v3` has been trained with 5 task-specific adapters for different embedding uses. Include `task` in your request to optimize your downstream application: + ++ **retrieval.query**: Used to encode user queries or questions in retrieval tasks. ++ **retrieval.passage**: Used to encode large documents in retrieval tasks at indexing time. ++ **classification**: Used to encode text for text classification tasks. ++ **text-matching**: Used to encode text for similarity matching, such as measuring similarity between two sentences. ++ **separation**: Used for clustering or reranking tasks. + +`jina-embeddings-v3` supports **Matryoshka Representation Learning**, allowing users to control the embedding dimension with minimal performance loss. +Include `dimensions` in your request to select the desired dimension. +By default, **dimensions** is set to 1024, and a number between 256 and 1024 is recommended. +You can reference the table below for hints on dimension vs. performance: + + +| Dimension | 32 | 64 | 128 | 256 | 512 | 768 | 1024 | +|:----------------------:|:---------:|:---------:|:-----------:|:---------:|:----------:|:---------:|:---------:| +| Average Retrieval Performance (nDCG@10) | 52.54 | 58.54 | 61.64 | 62.72 | 63.16 | 63.3 | 63.35 | + + +## Example + +The code below demonstrate how to use `jina-embeddings-v3` together with Qdrant: -To call their endpoint, all you need is an API key obtainable [here](https://jina.ai/embeddings/). By the way, our friends from **Jina AI** provided us with a code (**QDRANT**) that will grant you a **10% discount** if you plan to use Jina Embeddings in production. ```python -import qdrant_client import requests +import qdrant_client from qdrant_client.models import Distance, VectorParams, Batch # Provide Jina API key and choose one of the available models. -# You can get a free trial key here: https://jina.ai/embeddings/ JINA_API_KEY = "jina_xxxxxxxxxxx" -MODEL = "jina-embeddings-v2-base-en" # or "jina-embeddings-v2-base-en" -EMBEDDING_SIZE = 768 # 512 for small variant +MODEL = "jina-embeddings-v3" +DIMENSIONS = 1024 # Or choose your desired output vector dimensionality. +TASK = 'retrieval.passage' # For indexing, or set to retrieval.query for quering # Get embeddings from the API url = "https://api.jina.ai/v1/embeddings" @@ -35,6 +71,8 @@ headers = { data = { "input": ["Your text string goes here", "You can send multiple texts"], "model": MODEL, + "dimensions": DIMENSIONS, + "task": TASK, } response = requests.post(url, headers=headers, json=data) @@ -45,7 +83,7 @@ embeddings = [d["embedding"] for d in response.json()["data"]] client = qdrant_client.QdrantClient(":memory:") client.create_collection( collection_name="MyCollection", - vectors_config=VectorParams(size=EMBEDDING_SIZE, distance=Distance.DOT), + vectors_config=VectorParams(size= DIMENSIONS, distance=Distance.DOT), ) @@ -58,4 +96,3 @@ qdrant_client.upsert( ) ``` -