Merge pull request #2090 from qdrant/docs-inference-openrouter

Inference docs: add OpenRouter
This commit is contained in:
Daniel Boros
2026-01-26 15:16:39 +01:00
committed by GitHub
31 changed files with 668 additions and 2 deletions
@@ -9,7 +9,7 @@ aliases:
Inference is the process of using a machine learning model to create vector embeddings from text, images, or other data types. While you can create embeddings on the client side, you can also let Qdrant generate them while storing or querying data.
![Inference.](/docs/inference.png)
![Inference](/docs/inference.png)
There are several advantages to generating embeddings with Qdrant:
@@ -175,14 +175,17 @@ This flexibility allows you to develop and test your applications locally or in
## External Embedding Model Providers
Qdrant Cloud can act as a proxy for the APIs of three external embedding model providers:
Qdrant Cloud can act as a proxy for the APIs of external embedding model providers:
- OpenAI
- Cohere
- Jina AI
- OpenRouter
This enables you to access any of the embedding models provided by these providers through the Qdrant API.
![Inference with an external embedding model provider](/docs/inference-external-provider.png)
To use an external provider's embedding model, you need an API key from that provider. For example, to access OpenAI models, you need an OpenAI API key. Qdrant does not store or cache your API keys; they must be provided with each inference request.
When using an external embedding model, ensure that your collection has been configured for vectors with the correct dimensionality. Refer to the model's documentation for details on the output dimensions.
@@ -241,6 +244,20 @@ At query time, you can use the same model by prepending the model name with `jin
Note that, because Qdrant does not store or cache your Jina AI API key, you need to provide it with each inference request
### OpenRouter
OpenRouter is a platform that provides [several embedding models](https://openrouter.ai/models?fmt=cards&output_modalities=embeddings). To use one of the models provided by the [OpenRouter Embeddings API](https://openrouter.ai/docs/api/reference/embeddings), prepend the model name with `openrouter/`.
For example, to use the `mistralai/mistral-embed-2312` model when ingesting data, prepend the model name with `openrouter/` and provide your OpenRouter API key in the `options` object.
{{< code-snippet path="/documentation/headless/snippets/inference/openrouter-upsert/" >}}
At query time, you can use the same model by prepending the model name with `openrouter/` and providing your OpenRouter API key in the `options` object:
{{< code-snippet path="/documentation/headless/snippets/inference/openrouter-query/" >}}
Note that, because Qdrant does not store or cache your OpenRouter API key, you need to provide it with each inference request.
## Multiple Inference Operations
You can run multiple inference operations within a single request, even when models are hosted in different locations. This example generates three different named vectors for a single point: image embeddings using `jina-clip-v2` hosted by Jina AI, text embeddings using `all-minilm-l6-v2` hosted by Qdrant Cloud, and BM25 embeddings using the `bm25` model executed locally by the Qdrant cluster:
@@ -0,0 +1 @@
This code snippet illustrates how to use the OpenRouter API for query-time inference on Qdrant Cloud. Instead of supplying an explicit query vector, the query provides text, along with the name of the `openrouter/mistralai/mistral-embed-2312` model. When the model name is prepended with `openrouter/`, the Qdrant Cloud Inference proxy uses the OpenRouter API to infer embeddings out of the provided text. Qdrant will search with the resulting vector. The request also shows how to pass OpenRouter-specific parameters to the API. In this case, the request provides the OpenRouter API key and the `dimensions` parameter.
@@ -0,0 +1,25 @@
using Qdrant.Client;
using Qdrant.Client.Grpc;
public class Snippet
{
public static async Task Run()
{
var client = new QdrantClient(
host: "xyz-example.qdrant.io",
port: 6334,
https: true,
apiKey: "<your-openrouter-key>"
);
await client.QueryAsync(
collectionName: "{collection_name}",
query: new Document()
{
Model = "openrouter/mistralai/mistral-embed-2312",
Text = "How to bake cookies?",
Options = { ["openrouter-api-key"] = "<YOUR_OPENROUTER_API_KEY>" },
}
);
}
}
@@ -0,0 +1,21 @@
```csharp
using Qdrant.Client;
using Qdrant.Client.Grpc;
var client = new QdrantClient(
host: "xyz-example.qdrant.io",
port: 6334,
https: true,
apiKey: "<your-openrouter-key>"
);
await client.QueryAsync(
collectionName: "{collection_name}",
query: new Document()
{
Model = "openrouter/mistralai/mistral-embed-2312",
Text = "How to bake cookies?",
Options = { ["openrouter-api-key"] = "<YOUR_OPENROUTER_API_KEY>" },
}
);
```
@@ -0,0 +1,27 @@
```go
import (
"context"
"github.com/qdrant/go-client/qdrant"
)
client, err := qdrant.NewClient(&qdrant.Config{
Host: "xyz-example.qdrant.io",
Port: 6334,
APIKey: "<paste-your-openrouter-key-here>",
UseTLS: true,
})
client.Query(context.Background(), &qdrant.QueryPoints{
CollectionName: "{collection_name}",
Query: qdrant.NewQueryNearest(
qdrant.NewVectorInputDocument(&qdrant.Document{
Model: "openrouter/mistralai/mistral-embed-2312",
Text: "How to bake cookies?",
Options: qdrant.NewValueMap(map[string]any{
"openrouter-api-key": "<YOUR_OPENROUTER_API_KEY>",
}),
}),
),
})
```
@@ -0,0 +1,32 @@
```java
import static io.qdrant.client.QueryFactory.nearest;
import static io.qdrant.client.ValueFactory.value;
import io.qdrant.client.QdrantClient;
import io.qdrant.client.QdrantGrpcClient;
import io.qdrant.client.grpc.Points.Document;
import io.qdrant.client.grpc.Points.QueryPoints;
import java.util.Map;
QdrantClient client =
new QdrantClient(
QdrantGrpcClient.newBuilder("xyz-example.qdrant.io", 6334, true)
.withApiKey("<your-openrouter-key>")
.build());
client
.queryAsync(
QueryPoints.newBuilder()
.setCollectionName("{collection_name}")
.setQuery(
nearest(
Document.newBuilder()
.setModel("openrouter/mistralai/mistral-embed-2312")
.setText("How to bake cookies?")
.putAllOptions(
Map.of(
"openrouter-api-key",
value("<YOUR_OPENROUTER_API_KEY>")))
.build()))
.build())
.get();
```
@@ -0,0 +1,20 @@
```python
from qdrant_client import QdrantClient, models
client = QdrantClient(
url="https://xyz-example.qdrant.io:6333",
api_key="<your-api-key>",
cloud_inference=True
)
client.query_points(
collection_name="{collection_name}",
query=models.Document(
text="How to bake cookies?",
model="openrouter/mistralai/mistral-embed-2312",
options={
"openrouter-api-key": "<your_openrouter_api_key>"
}
)
)
```
@@ -0,0 +1,24 @@
```rust
use qdrant_client::{
Qdrant,
qdrant::{Document, Query, QueryPointsBuilder, Value},
};
use std::collections::HashMap;
let client = Qdrant::from_url("<your-qdrant-url>").build().unwrap();
let mut options = HashMap::<String, Value>::new();
options.insert("openrouter-api-key".to_string(), "<YOUR_OPENROUTER_API_KEY>".into());
client
.query(
QueryPointsBuilder::new("{collection_name}")
.query(Query::new_nearest(Document {
text: "How to bake cookies?".into(),
model: "openrouter/mistralai/mistral-embed-2312".into(),
options,
}))
.build(),
)
.await?;
```
@@ -0,0 +1,15 @@
```typescript
import { QdrantClient } from "@qdrant/js-client-rest";
const client = new QdrantClient({ host: "localhost", port: 6333 });
client.query("{collection_name}", {
query: {
text: 'How to bake cookies?',
model: 'openrouter/mistralai/mistral-embed-2312',
options: {
'openrouter-api-key': '<your_openrouter_api_key>'
},
},
});
```
@@ -0,0 +1,31 @@
package snippet
import (
"context"
"github.com/qdrant/go-client/qdrant"
)
func Main() {
client, err := qdrant.NewClient(&qdrant.Config{
Host: "xyz-example.qdrant.io",
Port: 6334,
APIKey: "<paste-your-openrouter-key-here>",
UseTLS: true,
})
if err != nil { panic(err) } // @hide
client.Query(context.Background(), &qdrant.QueryPoints{
CollectionName: "{collection_name}",
Query: qdrant.NewQueryNearest(
qdrant.NewVectorInputDocument(&qdrant.Document{
Model: "openrouter/mistralai/mistral-embed-2312",
Text: "How to bake cookies?",
Options: qdrant.NewValueMap(map[string]any{
"openrouter-api-key": "<YOUR_OPENROUTER_API_KEY>",
}),
}),
),
})
}
@@ -0,0 +1,12 @@
```http
POST /collections/{collection_name}/points/query
{
"query": {
"text": "How to bake cookies?",
"model": "openrouter/mistralai/mistral-embed-2312",
"options": {
"openrouter-api-key": "<YOUR_OPENROUTER_API_KEY>"
}
}
}
```
@@ -0,0 +1,36 @@
package com.example.snippets_amalgamation;
import static io.qdrant.client.QueryFactory.nearest;
import static io.qdrant.client.ValueFactory.value;
import io.qdrant.client.QdrantClient;
import io.qdrant.client.QdrantGrpcClient;
import io.qdrant.client.grpc.Points.Document;
import io.qdrant.client.grpc.Points.QueryPoints;
import java.util.Map;
public class Snippet {
public static void run() throws Exception {
QdrantClient client =
new QdrantClient(
QdrantGrpcClient.newBuilder("xyz-example.qdrant.io", 6334, true)
.withApiKey("<your-openrouter-key>")
.build());
client
.queryAsync(
QueryPoints.newBuilder()
.setCollectionName("{collection_name}")
.setQuery(
nearest(
Document.newBuilder()
.setModel("openrouter/mistralai/mistral-embed-2312")
.setText("How to bake cookies?")
.putAllOptions(
Map.of(
"openrouter-api-key",
value("<YOUR_OPENROUTER_API_KEY>")))
.build()))
.build())
.get();
}
}
@@ -0,0 +1,18 @@
from qdrant_client import QdrantClient, models
client = QdrantClient(
url="https://xyz-example.qdrant.io:6333",
api_key="<your-api-key>",
cloud_inference=True
)
client.query_points(
collection_name="{collection_name}",
query=models.Document(
text="How to bake cookies?",
model="openrouter/mistralai/mistral-embed-2312",
options={
"openrouter-api-key": "<your_openrouter_api_key>"
}
)
)
@@ -0,0 +1,26 @@
use qdrant_client::{
Qdrant,
qdrant::{Document, Query, QueryPointsBuilder, Value},
};
use std::collections::HashMap;
pub async fn main() -> anyhow::Result<()> {
let client = Qdrant::from_url("<your-qdrant-url>").build().unwrap();
let mut options = HashMap::<String, Value>::new();
options.insert("openrouter-api-key".to_string(), "<YOUR_OPENROUTER_API_KEY>".into());
client
.query(
QueryPointsBuilder::new("{collection_name}")
.query(Query::new_nearest(Document {
text: "How to bake cookies?".into(),
model: "openrouter/mistralai/mistral-embed-2312".into(),
options,
}))
.build(),
)
.await?;
Ok(())
}
@@ -0,0 +1,13 @@
import { QdrantClient } from "@qdrant/js-client-rest";
const client = new QdrantClient({ host: "localhost", port: 6333 });
client.query("{collection_name}", {
query: {
text: 'How to bake cookies?',
model: 'openrouter/mistralai/mistral-embed-2312',
options: {
'openrouter-api-key': '<your_openrouter_api_key>'
},
},
});
@@ -0,0 +1 @@
This code snippet illustrates how to use the OpenRouter API for ingest-time inference on Qdrant Cloud. The example upserts a point, but instead of providing an explicit vector, the request includes text along with the name of the `openrouter/mistralai/mistral-embed-2312` model. When the model name is prepended with `openrouter/`, the Qdrant Cloud Inference proxy uses the OpenRouter API to infer embeddings out of the provided text. Qdrant will store the resulting vector. The request also shows how to pass OpenRouter-specific parameters to the API. In this case, the request provides the OpenRouter API key and the `dimensions` parameter.
@@ -0,0 +1,28 @@
using Qdrant.Client;
using Qdrant.Client.Grpc;
public class Snippet
{
public static async Task Run()
{
var client = new QdrantClient(
host: "xyz-example.qdrant.io", port: 6334, https: true, apiKey: "<your-openrouter-key>");
await client.UpsertAsync(
collectionName: "{collection_name}",
points: new List<PointStruct>
{
new()
{
Id = 1,
Vectors = new Document()
{
Model = "openrouter/mistralai/mistral-embed-2312",
Text = "Recipe for baking chocolate chip cookies",
Options = { ["openrouter-api-key"] = "<YOUR_OPENROUTER_API_KEY>" },
},
},
}
);
}
}
@@ -0,0 +1,24 @@
```csharp
using Qdrant.Client;
using Qdrant.Client.Grpc;
var client = new QdrantClient(
host: "xyz-example.qdrant.io", port: 6334, https: true, apiKey: "<your-openrouter-key>");
await client.UpsertAsync(
collectionName: "{collection_name}",
points: new List<PointStruct>
{
new()
{
Id = 1,
Vectors = new Document()
{
Model = "openrouter/mistralai/mistral-embed-2312",
Text = "Recipe for baking chocolate chip cookies",
Options = { ["openrouter-api-key"] = "<YOUR_OPENROUTER_API_KEY>" },
},
},
}
);
```
@@ -0,0 +1,30 @@
```go
import (
"context"
"github.com/qdrant/go-client/qdrant"
)
client, err := qdrant.NewClient(&qdrant.Config{
Host: "xyz-example.qdrant.io",
Port: 6334,
APIKey: "<paste-your-openrouter-key-here>",
UseTLS: true,
})
client.Upsert(context.Background(), &qdrant.UpsertPoints{
CollectionName: "{collection_name}",
Points: []*qdrant.PointStruct{
{
Id: qdrant.NewIDNum(uint64(1)),
Vectors: qdrant.NewVectorsDocument(&qdrant.Document{
Model: "openrouter/mistralai/mistral-embed-2312",
Text: "Recipe for baking chocolate chip cookies",
Options: qdrant.NewValueMap(map[string]any{
"openrouter-api-key": "<YOUR_OPENROUTER_API_KEY>",
}),
}),
},
},
})
```
@@ -0,0 +1,37 @@
```java
import static io.qdrant.client.PointIdFactory.id;
import static io.qdrant.client.ValueFactory.value;
import static io.qdrant.client.VectorsFactory.vectors;
import io.qdrant.client.QdrantClient;
import io.qdrant.client.QdrantGrpcClient;
import io.qdrant.client.grpc.Points.Document;
import io.qdrant.client.grpc.Points.PointStruct;
import java.util.List;
import java.util.Map;
QdrantClient client =
new QdrantClient(
QdrantGrpcClient.newBuilder("xyz-example.qdrant.io", 6334, true)
.withApiKey("<your-openrouter-key>")
.build());
client
.upsertAsync(
"{collection_name}",
List.of(
PointStruct.newBuilder()
.setId(id(1))
.setVectors(
vectors(
Document.newBuilder()
.setModel("openrouter/mistralai/mistral-embed-2312")
.setText("Recipe for baking chocolate chip cookies")
.putAllOptions(
Map.of(
"openrouter-api-key",
value("<YOUR_OPENROUTER_API_KEY>")))
.build()))
.build()))
.get();
```
@@ -0,0 +1,25 @@
```python
from qdrant_client import QdrantClient, models
client = QdrantClient(
url="https://xyz-example.qdrant.io:6333",
api_key="<your-api-key>",
cloud_inference=True
)
client.upsert(
collection_name="{collection_name}",
points=[
models.PointStruct(
id=1,
vector=models.Document(
text="Recipe for baking chocolate chip cookies",
model="openrouter/mistralai/mistral-embed-2312",
options={
"openrouter-api-key": "<your_openrouter_api_key>"
}
)
)
]
)
```
@@ -0,0 +1,24 @@
```rust
use qdrant_client::{
Payload, Qdrant,
qdrant::{Document, PointStruct, UpsertPointsBuilder},
};
use std::collections::HashMap;
let client = Qdrant::from_url("<your-qdrant-url>").build()?;
let mut options = HashMap::new();
options.insert("openrouter-api-key".to_string(), "<YOUR_OPENROUTER_API_KEY>".into());
client
.upsert_points(UpsertPointsBuilder::new("{collection_name}",
vec![
PointStruct::new(1,
Document {
text: "Recipe for baking chocolate chip cookies".into(),
model: "openrouter/mistralai/mistral-embed-2312".into(),
options,
},
Payload::default())
]).wait(true))
.await?;
```
@@ -0,0 +1,20 @@
```typescript
import { QdrantClient } from "@qdrant/js-client-rest";
const client = new QdrantClient({ host: "localhost", port: 6333 });
client.upsert("{collection_name}", {
points: [
{
id: 1,
vector: {
text: 'Recipe for baking chocolate chip cookies',
model: 'openrouter/mistralai/mistral-embed-2312',
options: {
'openrouter-api-key': '<your_openrouter_api_key>',
},
},
},
],
});
```
@@ -0,0 +1,34 @@
package snippet
import (
"context"
"github.com/qdrant/go-client/qdrant"
)
func Main() {
client, err := qdrant.NewClient(&qdrant.Config{
Host: "xyz-example.qdrant.io",
Port: 6334,
APIKey: "<paste-your-openrouter-key-here>",
UseTLS: true,
})
if err != nil { panic(err) } // @hide
client.Upsert(context.Background(), &qdrant.UpsertPoints{
CollectionName: "{collection_name}",
Points: []*qdrant.PointStruct{
{
Id: qdrant.NewIDNum(uint64(1)),
Vectors: qdrant.NewVectorsDocument(&qdrant.Document{
Model: "openrouter/mistralai/mistral-embed-2312",
Text: "Recipe for baking chocolate chip cookies",
Options: qdrant.NewValueMap(map[string]any{
"openrouter-api-key": "<YOUR_OPENROUTER_API_KEY>",
}),
}),
},
},
})
}
@@ -0,0 +1,17 @@
```http
PUT /collections/{collection_name}/points?wait=true
{
"points": [
{
"id": 1,
"vector": {
"text": "Recipe for baking chocolate chip cookies",
"model": "openrouter/mistralai/mistral-embed-2312",
"options": {
"openrouter-api-key": "<YOUR_OPENROUTER_API_KEY>"
}
}
}
]
}
```
@@ -0,0 +1,41 @@
package com.example.snippets_amalgamation;
import static io.qdrant.client.PointIdFactory.id;
import static io.qdrant.client.ValueFactory.value;
import static io.qdrant.client.VectorsFactory.vectors;
import io.qdrant.client.QdrantClient;
import io.qdrant.client.QdrantGrpcClient;
import io.qdrant.client.grpc.Points.Document;
import io.qdrant.client.grpc.Points.PointStruct;
import java.util.List;
import java.util.Map;
public class Snippet {
public static void run() throws Exception {
QdrantClient client =
new QdrantClient(
QdrantGrpcClient.newBuilder("xyz-example.qdrant.io", 6334, true)
.withApiKey("<your-openrouter-key>")
.build());
client
.upsertAsync(
"{collection_name}",
List.of(
PointStruct.newBuilder()
.setId(id(1))
.setVectors(
vectors(
Document.newBuilder()
.setModel("openrouter/mistralai/mistral-embed-2312")
.setText("Recipe for baking chocolate chip cookies")
.putAllOptions(
Map.of(
"openrouter-api-key",
value("<YOUR_OPENROUTER_API_KEY>")))
.build()))
.build()))
.get();
}
}
@@ -0,0 +1,23 @@
from qdrant_client import QdrantClient, models
client = QdrantClient(
url="https://xyz-example.qdrant.io:6333",
api_key="<your-api-key>",
cloud_inference=True
)
client.upsert(
collection_name="{collection_name}",
points=[
models.PointStruct(
id=1,
vector=models.Document(
text="Recipe for baking chocolate chip cookies",
model="openrouter/mistralai/mistral-embed-2312",
options={
"openrouter-api-key": "<your_openrouter_api_key>"
}
)
)
]
)
@@ -0,0 +1,26 @@
use qdrant_client::{
Payload, Qdrant,
qdrant::{Document, PointStruct, UpsertPointsBuilder},
};
use std::collections::HashMap;
pub async fn main() -> anyhow::Result<()> {
let client = Qdrant::from_url("<your-qdrant-url>").build()?;
let mut options = HashMap::new();
options.insert("openrouter-api-key".to_string(), "<YOUR_OPENROUTER_API_KEY>".into());
client
.upsert_points(UpsertPointsBuilder::new("{collection_name}",
vec![
PointStruct::new(1,
Document {
text: "Recipe for baking chocolate chip cookies".into(),
model: "openrouter/mistralai/mistral-embed-2312".into(),
options,
},
Payload::default())
]).wait(true))
.await?;
Ok(())
}
@@ -0,0 +1,18 @@
import { QdrantClient } from "@qdrant/js-client-rest";
const client = new QdrantClient({ host: "localhost", port: 6333 });
client.upsert("{collection_name}", {
points: [
{
id: 1,
vector: {
text: 'Recipe for baking chocolate chip cookies',
model: 'openrouter/mistralai/mistral-embed-2312',
options: {
'openrouter-api-key': '<your_openrouter_api_key>',
},
},
},
],
});