mirror of
https://github.com/qdrant/landing_page.git
synced 2026-10-09 21:08:31 +02:00
Add Fusion Embedding 2 to the embeddings integrations (#2548)
Co-authored-by: Abdul Basit Tonmoy <skopintelligence@gmail.com>
This commit is contained in:
co-authored by
Abdul Basit Tonmoy
parent
c87038c0aa
commit
45106e2b6f
@@ -17,6 +17,7 @@ Qdrant supports all available text and multimodal dense vector embedding models
|
||||
| [Aleph Alpha](/documentation/embeddings/aleph-alpha/) | Multilingual embeddings focused on European languages. |
|
||||
| [Bedrock](/documentation/embeddings/bedrock/) | AWS managed service for foundation models and embeddings. |
|
||||
| [Cohere](/documentation/embeddings/cohere/) | Language model embeddings for NLP tasks. |
|
||||
| [Fusion Embedding 2](/documentation/embeddings/fusion-embedding-2/) | Open-weight multimodal embeddings across text, image, video, and audio. |
|
||||
| [Gemini](/documentation/embeddings/gemini/) | Google Gemini embeddings for semantic search, classification. |
|
||||
| [Jina AI](/documentation/embeddings/jina-embeddings/) | Customizable embeddings for neural search. |
|
||||
| [Mistral](/documentation/embeddings/mistral/) | Open-source, efficient language model embeddings. |
|
||||
|
||||
@@ -0,0 +1,78 @@
|
||||
---
|
||||
title: Fusion Embedding 2
|
||||
short_description: "Embed text, image, video, and audio into one vector space with the open-weight Fusion Embedding 2 model and search it with Qdrant."
|
||||
description: "Use EximiusLabs Fusion Embedding 2, an open-weight multimodal embedding model, with Qdrant to index and search content in a shared vector space. This guide covers the text path."
|
||||
---
|
||||
|
||||
# Fusion Embedding 2
|
||||
|
||||
[Fusion Embedding 2](https://huggingface.co/EximiusLabs/fusion-embedding-2-2b-preview) is an open-weight multimodal embedding model from Eximius Labs. It maps text, image, video, and audio into a single shared vector space, so content of different types is directly comparable. The model runs on your own hardware and its weights are on Hugging Face.
|
||||
|
||||
We'll look at how to generate Fusion Embedding 2 text vectors and index them in Qdrant with the Python SDK. The lightweight text encoder loads only the base and the trained text head, so it does not pull the audio tower.
|
||||
|
||||
### Installing the dependencies
|
||||
|
||||
```python
|
||||
$ pip install "git+https://github.com/Eximius-Labs/fusion-embedding" qdrant-client
|
||||
```
|
||||
|
||||
### Loading the model
|
||||
|
||||
```python
|
||||
from fusion_embedding import FusionTextEmbedder
|
||||
from qdrant_client import QdrantClient
|
||||
|
||||
model = FusionTextEmbedder.from_pretrained("EximiusLabs/fusion-embedding-2-2b-preview")
|
||||
client = QdrantClient(url="http://localhost:6333/")
|
||||
```
|
||||
|
||||
### Encoding data
|
||||
|
||||
```python
|
||||
texts = [
|
||||
"a dog running on the beach",
|
||||
"a slow piano melody",
|
||||
"a red bicycle leaning on a wall",
|
||||
]
|
||||
embeddings = model.encode(texts) # numpy array of shape [len(texts), dim]
|
||||
```
|
||||
|
||||
### Creating a collection and upserting the vectors
|
||||
|
||||
```python
|
||||
from qdrant_client.models import VectorParams, Distance, PointStruct
|
||||
|
||||
collection_name = "fusion_embedding_2"
|
||||
|
||||
client.create_collection(
|
||||
collection_name,
|
||||
vectors_config=VectorParams(
|
||||
size=embeddings.shape[1],
|
||||
distance=Distance.COSINE,
|
||||
),
|
||||
)
|
||||
|
||||
client.upsert(
|
||||
collection_name,
|
||||
points=[
|
||||
PointStruct(id=idx, vector=vector.tolist(), payload={"text": texts[idx]})
|
||||
for idx, vector in enumerate(embeddings)
|
||||
],
|
||||
)
|
||||
```
|
||||
|
||||
### Searching
|
||||
|
||||
```python
|
||||
query = model.encode("something to ride") # a single string returns one vector
|
||||
|
||||
client.query_points(
|
||||
collection_name=collection_name,
|
||||
query=query.tolist(),
|
||||
)
|
||||
```
|
||||
|
||||
## Further reading
|
||||
|
||||
- [Fusion Embedding 2 on Hugging Face](https://huggingface.co/EximiusLabs/fusion-embedding-2-2b-preview)
|
||||
- [Eximius Labs on GitHub](https://github.com/Eximius-Labs)
|
||||
Reference in New Issue
Block a user