krzysztof/landing_page
1
0
Fork 0
mirror of https://github.com/qdrant/landing_page.git synced 2026-09-28 23:48:31 +02:00
Code Issues Packages Projects Releases Wiki Activity

docs: update jina embeddings integration (#1164)

Browse Source 
* docs: add jina-embeddings-v3 integration guide

* docs: fix model list table

* docs: fix broken line in jina page

* docs: fix model name in table

Co-authored-by: CatStark <susana.guzman@jina.ai>

* docs: update wording

Co-authored-by: Anush  <anushshetty90@gmail.com>

* docs: update wording

Co-authored-by: Anush  <anushshetty90@gmail.com>

* docs: update working

Co-authored-by: Anush  <anushshetty90@gmail.com>

* docs: rename task_type as task

* docs: set guide to example

Co-authored-by: Anush  <anushshetty90@gmail.com>

* docs: improve wording

Co-authored-by: Anush  <anushshetty90@gmail.com>

---------

Co-authored-by: CatStark <susana.guzman@jina.ai>
Co-authored-by: Anush <anushshetty90@gmail.com>
This commit is contained in:
Wang Bo
2024-09-16 14:45:43 +05:30
committed by GitHub
co-authored by CatStark Anush
parent 1e807801ff
commit f44604109b
1 changed files with 45 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files
qdrant-landing/content/documentation/embeddings/jina-embeddings.md
+45 -8
View File
@@ -8,21 +8,57 @@ aliases:
# Jina Embeddings
Qdrant can also easily work with [Jina embeddings](https://jina.ai/embeddings/) which allow for model input lengths of up to 8192 tokens.
Qdrant is compatible with [Jina AI](https://jina.ai/) embeddings. You can get a free trial key from [Jina Embeddings](https://jina.ai/embeddings/) to get embeddings.
Qdrant users can receive a 10% discount on Jina AI APIs by using the code **QDRANT**.
## Technical Summary
| Model | Dimension | Language | MRL (matryoshka) | Context |
|:----------------------:|:---------:|:---------:|:-----------:|:---------:|
| jina-embeddings-v3 | 1024 | Multilingual (89 languages) | Yes | 8192 |
| jina-embeddings-v2-base-en | 768 | English | No | 8192 |
| jina-embeddings-v2-base-de | 768 | German & English | No | 8192 |
| jina-embeddings-v2-base-es | 768 | Spanish & English | No | 8192 |
| jina-embeddings-v2-base-zh | 768 | Chinese & English | No | 8192 |
> Jina recommends using `jina-embeddings-v3` as it is the latest and most performant embedding model released by Jina AI.
On top of the backbone, `jina-embeddings-v3` has been trained with 5 task-specific adapters for different embedding uses. Include `task` in your request to optimize your downstream application:
+ **retrieval.query**: Used to encode user queries or questions in retrieval tasks.
+ **retrieval.passage**: Used to encode large documents in retrieval tasks at indexing time.
+ **classification**: Used to encode text for text classification tasks.
+ **text-matching**: Used to encode text for similarity matching, such as measuring similarity between two sentences.
+ **separation**: Used for clustering or reranking tasks.
`jina-embeddings-v3` supports **Matryoshka Representation Learning**, allowing users to control the embedding dimension with minimal performance loss.
Include `dimensions` in your request to select the desired dimension.
By default, **dimensions** is set to 1024, and a number between 256 and 1024 is recommended.
You can reference the table below for hints on dimension vs. performance:
| Dimension | 32 | 64 | 128 | 256 | 512 | 768 | 1024 |
|:----------------------:|:---------:|:---------:|:-----------:|:---------:|:----------:|:---------:|:---------:|
| Average Retrieval Performance (nDCG@10) | 52.54 | 58.54 | 61.64 | 62.72 | 63.16 | 63.3 | 63.35 |
## Example
The code below demonstrate how to use `jina-embeddings-v3` together with Qdrant:
To call their endpoint, all you need is an API key obtainable [here](https://jina.ai/embeddings/). By the way, our friends from **Jina AI** provided us with a code (**QDRANT**) that will grant you a **10% discount** if you plan to use Jina Embeddings in production.
```python
import qdrant_client
import requests
import qdrant_client
from qdrant_client.models import Distance, VectorParams, Batch
# Provide Jina API key and choose one of the available models.
# You can get a free trial key here: https://jina.ai/embeddings/
JINA_API_KEY = "jina_xxxxxxxxxxx"
MODEL = "jina-embeddings-v2-base-en" # or "jina-embeddings-v2-base-en"
EMBEDDING_SIZE = 768 # 512 for small variant
MODEL = "jina-embeddings-v3"
DIMENSIONS = 1024 # Or choose your desired output vector dimensionality.
TASK = 'retrieval.passage' # For indexing, or set to retrieval.query for quering
# Get embeddings from the API
url = "https://api.jina.ai/v1/embeddings"
@@ -35,6 +71,8 @@ headers = {
data = {
"input": ["Your text string goes here", "You can send multiple texts"],
"model": MODEL,
"dimensions": DIMENSIONS,
"task": TASK,
}
response = requests.post(url, headers=headers, json=data)
@@ -45,7 +83,7 @@ embeddings = [d["embedding"] for d in response.json()["data"]]
client = qdrant_client.QdrantClient(":memory:")
client.create_collection(
collection_name="MyCollection",
vectors_config=VectorParams(size=EMBEDDING_SIZE, distance=Distance.DOT),
vectors_config=VectorParams(size= DIMENSIONS, distance=Distance.DOT),
)
@@ -58,4 +96,3 @@ qdrant_client.upsert(
)
```