mirror of
https://github.com/qdrant/landing_page.git
synced 2026-10-09 04:48:30 +02:00
Abdon PijpelinkandClaude Sonnet 4.6
478b96554f
* Break Inference page into several pages * Make all inference code snippets testable and clean up * Make more snippets testable * Edits * Document automatic query and passage prefix injection in Cloud Inference Qdrant Cloud Inference silently applies model-specific prefixes (e.g. "query: "/"passage: " for E5, BGE-style instruction prefix for BGE/mxbai/ Snowflake arctic-embed) so users don't need to manage them manually. Add a section explaining this behavior, the idempotency guarantee, and the scope (Qdrant-hosted models only; external providers handle their own). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Document short query optimization in Cloud Inference Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * Update links * Expand on external provider API key usage * Add section about external provider API keys * Default to header for external API keys --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
60 lines
2.6 KiB
Markdown
60 lines
2.6 KiB
Markdown
---
|
|
title: Inference API
|
|
weight: 10
|
|
---
|
|
|
|
# Inference API
|
|
|
|
When using [Qdrant Cloud Inference](/documentation/inference/cloud-inference/) or Qdrant's [BM25 model](/documentation/inference/inference-bm25/), embeddings are generated server-side. Instead of pre-computed vectors, you pass Inference Objects when ingesting or querying data. An Inference Object tells Qdrant how to generate a vector from your input. It contains the input, such as text or an image, along with the model to use. The API supports three types of Inference Objects:
|
|
|
|
* **`Document`** object, used for text inference
|
|
|
|
```js
|
|
// Document
|
|
{
|
|
// Text input
|
|
text: "Your text",
|
|
// Name of the model, to do inference with
|
|
model: "<the-model-to-use>",
|
|
// Extra parameters for the model, Optional
|
|
options: {}
|
|
}
|
|
```
|
|
|
|
* **`Image`** object, used for image inference
|
|
|
|
```js
|
|
// Image
|
|
{
|
|
// Image input
|
|
image: "<url>", // Or base64 encoded image
|
|
// Name of the model, to do inference with
|
|
model: "<the-model-to-use>",
|
|
// Extra parameters for the model, Optional
|
|
options: {}
|
|
}
|
|
```
|
|
|
|
* **`Object`** object, reserved for other types of input, which might be implemented in the future.
|
|
|
|
For example, the following code:
|
|
|
|
{{< code-snippet path="/documentation/headless/snippets/inference/query-vector/" >}}
|
|
|
|
can be replaced with:
|
|
|
|
{{< code-snippet path="/documentation/headless/snippets/inference/query-document/" >}}
|
|
|
|
In this case, Qdrant uses the configured embedding model to create a vector from the Inference Object and then perform the search query with it. All of this happens within a low-latency network.
|
|
|
|
<aside role="status">
|
|
When using inference at ingest time, the input used for inference is not stored. If you want to persist it in Qdrant, ensure that you explicitly include it in the payload.
|
|
</aside>
|
|
|
|
## Multiple Inference Operations
|
|
|
|
You can run multiple inference operations within a single request, even when models are hosted in different locations. This example generates three different named vectors for a single point: image embeddings using `jina-clip-v2` hosted by Jina AI, text embeddings using `all-minilm-l6-v2` hosted by Qdrant Cloud, and BM25 embeddings using the `bm25` model executed locally by the Qdrant cluster:
|
|
|
|
{{< code-snippet path="/documentation/headless/snippets/inference/multiple/" >}}
|
|
|
|
When specifying multiple identical inference objects in a single request, the inference service generates embeddings only once and reuses the resulting vectors. This optimization is particularly beneficial when working with external model providers, as it reduces both latency and cost. |