mirror of
https://github.com/qdrant/landing_page.git
synced 2026-09-27 15:08:30 +02:00
fix for doc-sync (#82)
* removed .idea from git, added --force option to the push command at github action for doc-sync * docs auto-sync Co-authored-by: qdrant <qdrant@users.noreply.github.com>
This commit is contained in:
trean
@@ -1,2 +1,3 @@
|
||||
public
|
||||
node_modules
|
||||
.idea/
|
||||
|
||||
@@ -21,9 +21,8 @@ These can be:
|
||||
- Similar Image \ Audio \ Video search
|
||||
- Recommendation systems
|
||||
|
||||
|
||||
In addition to this documentation, you may be interested in looking at examples of projects made with Qdrant:
|
||||
|
||||
* [Semantic Search for startups](https://demo.qdrant.tech/) + [Source Code](https://github.com/qdrant/qdrant_demo)
|
||||
* [Visual Food Discovery](https://food-discovery.qdrant.tech/)
|
||||
* [Step-by-Step tutorial on building neural search](/articles/neural-search-tutorial/)
|
||||
- [Semantic Search for startups](https://demo.qdrant.tech/) + [Source Code](https://github.com/qdrant/qdrant_demo)
|
||||
- [Visual Food Discovery](https://food-discovery.qdrant.tech/)
|
||||
- [Step-by-Step tutorial on building neural search](/articles/neural-search-tutorial/)
|
||||
|
||||
@@ -48,11 +48,11 @@ client.recreate_collection(
|
||||
|
||||
In addition to the required options, you can also specify custom values for the following collection options:
|
||||
|
||||
- `hnsw_config` - see [indexing](../indexing/#vector-index) for details.
|
||||
- `wal_config` - Write-Ahead-Log related configuration. See more details about [WAL](../storage/#versioning)
|
||||
- `optimizers_config` - see [optimizer](../optimizer) for details.
|
||||
- `shard_number` - which defines how many shards the collection should have. See [distributed deployment](../distributed_deployment#sharding) section for details.
|
||||
- `on_disk_payload` - defines where to store payload data. If `true` - payload will be stored on disk only. Might be useful for limiting the RAM usage in case of large payload.
|
||||
* `hnsw_config` - see [indexing](../indexing/#vector-index) for details.
|
||||
* `wal_config` - Write-Ahead-Log related configuration. See more details about [WAL](../storage/#versioning)
|
||||
* `optimizers_config` - see [optimizer](../optimizer) for details.
|
||||
* `shard_number` - which defines how many shards the collection should have. See [distributed deployment](../distributed_deployment#sharding) section for details.
|
||||
* `on_disk_payload` - defines where to store payload data. If `true` - payload will be stored on disk only. Might be useful for limiting the RAM usage in case of large payload.
|
||||
|
||||
Default parameters for the optional collection parameters are defined in [configuration file](https://github.com/qdrant/qdrant/blob/master/config/config.yaml).
|
||||
|
||||
@@ -112,7 +112,6 @@ DELETE /collections/{collection_name}
|
||||
client.delete_collection(collection_name="{collection_name}")
|
||||
```
|
||||
|
||||
|
||||
### Update collection parameters
|
||||
|
||||
Dynamic parameter updates may be helpful, for example, for more efficient initial loading of vectors.
|
||||
@@ -140,14 +139,13 @@ client.update_collection(
|
||||
|
||||
This command enables indexing for segments that have more than 10000 vectors stored.
|
||||
|
||||
|
||||
## Collection aliases
|
||||
|
||||
In a production environment, it is sometimes necessary to switch different versions of vectors seamlessly.
|
||||
For example, when upgrading to a new version of the neural network.
|
||||
|
||||
There is no way to stop the service and rebuild the collection with new vectors in these situations.
|
||||
To avoid this, you can use aliases.
|
||||
To avoid this, you can use aliases.
|
||||
Aliases are additional names for existing collections.
|
||||
All queries to the collection can also be done identically, using an alias instead of the collection name.
|
||||
|
||||
@@ -212,7 +210,6 @@ POST /collections/aliases
|
||||
Multiple alias actions are performed atomically.
|
||||
For example, you can switch underlying collection with the following command:
|
||||
|
||||
|
||||
```http
|
||||
POST /collections/aliases
|
||||
|
||||
|
||||
@@ -5,7 +5,7 @@ weight: 45
|
||||
|
||||
To change or correct Qdrant's behavior, default collection settings, and network interface parameters, you can use the configuration file.
|
||||
|
||||
Default configuration file is located in [config/config.yaml](https://github.com/qdrant/qdrant/blob/master/config/config.yaml).
|
||||
Default configuration file is located in [config/config.yaml](https://github.com/qdrant/qdrant/blob/master/config/config.yaml).
|
||||
|
||||
In the production environment, you can override any value of this file by providing new values in `/qdrant/config/production.yaml` inside the docker.
|
||||
|
||||
@@ -146,4 +146,4 @@ cluster:
|
||||
# tick period may create significant network and CPU overhead.
|
||||
# We encourage you NOT to change this parameter unless you know what you are doing.
|
||||
tick_period_ms: 100
|
||||
```
|
||||
```
|
||||
|
||||
@@ -33,7 +33,7 @@ All peers should be accessible on this port from within the cluster, but make su
|
||||
Additionally, the first peer of the cluster should be provided with its URL, so it could tell other nodes how it should be reached.
|
||||
Use the `uri` CLI argument to provide the URL to the peer:
|
||||
|
||||
```
|
||||
```bash
|
||||
./qdrant --uri 'http://qdrant_node_1:6335'
|
||||
```
|
||||
|
||||
@@ -41,7 +41,7 @@ Subsequent peers in a cluster must know at least one node of the existing cluste
|
||||
|
||||
To do this, they need to be provided with a bootstrap URL:
|
||||
|
||||
```
|
||||
```bash
|
||||
./qdrant --bootstrap 'http://qdrant_node_1:6335'
|
||||
```
|
||||
|
||||
@@ -70,7 +70,7 @@ OPTIONS:
|
||||
|
||||
After a successful synchronization you can observe the state of the cluster through the [REST API](https://qdrant.github.io/qdrant/redoc/index.html?v=master#tag/cluster):
|
||||
|
||||
```
|
||||
```http
|
||||
GET /cluster
|
||||
```
|
||||
|
||||
|
||||
@@ -79,7 +79,6 @@ Filtered points would be:
|
||||
]
|
||||
```
|
||||
|
||||
|
||||
When using `must`, the clause becomes `true` only if every condition listed inside `must` is satisfied.
|
||||
In this sense, `must` is equivalent to the operator `AND`.
|
||||
|
||||
@@ -133,7 +132,6 @@ Filtered points would be:
|
||||
When using `should`, the clause becomes `true` if at least one condition listed inside `should` is satisfied.
|
||||
In this sense, `should` is equivalent to the operator `OR`.
|
||||
|
||||
|
||||
### Must Not
|
||||
|
||||
Example:
|
||||
@@ -182,7 +180,6 @@ Filtered points would be:
|
||||
When using `must_not`, the clause becomes `true` if none if the conditions listed inside `should` is satisfied.
|
||||
In this sense, `must_not` is equivalent to the expression `(NOT A) AND (NOT B) AND (NOT C)`.
|
||||
|
||||
|
||||
### Clauses combination
|
||||
|
||||
It is also possible to use several clauses simultaneously:
|
||||
@@ -329,14 +326,13 @@ models.FieldCondition(
|
||||
)
|
||||
```
|
||||
|
||||
|
||||
The simplest kind of condition is one that checks if the stored value equals the given one.
|
||||
If several values are stored, at least one of them should match the condition.
|
||||
You can apply it to [keyword](../payload/#keyword), [integer](../payload/#integer) and [bool](../payload/#bool) payloads.
|
||||
|
||||
### Full Text Match
|
||||
|
||||
*Avaliable since version 0.10.0*
|
||||
*Available since version 0.10.0*
|
||||
|
||||
A special case of the `match` condition is the `text` match condition.
|
||||
It allows you to search for a specific substring, token or phrase within the text field.
|
||||
@@ -393,7 +389,7 @@ models.FieldCondition(
|
||||
The `range` condition sets the range of possible values for stored payload values.
|
||||
If several values are stored, at least one of them should match the condition.
|
||||
|
||||
Comparisons that can be used:
|
||||
Comparisons that can be used:
|
||||
|
||||
- `gt` - greater than
|
||||
- `gte` - greater than or equal
|
||||
@@ -404,7 +400,7 @@ Can be applied to [float](../payload/#float) and [integer](../payload/#integer)
|
||||
|
||||
### Geo
|
||||
|
||||
#### Geo Bounding Box
|
||||
#### Geo Bounding Box
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -440,7 +436,6 @@ models.FieldCondition(
|
||||
|
||||
It matches with `location`s inside a rectangle with the coordinates of the upper left corner in `bottom_right` and the coordinates of the lower right corner in `top_left`.
|
||||
|
||||
|
||||
#### Geo Radius
|
||||
|
||||
```json
|
||||
@@ -469,7 +464,6 @@ models.FieldCondition(
|
||||
)
|
||||
```
|
||||
|
||||
|
||||
It matches with `location`s inside a circle with the `center` at the center and a radius of `radius` meters.
|
||||
|
||||
If several values are stored, at least one of them should match the condition.
|
||||
@@ -568,7 +562,6 @@ client.scroll(
|
||||
)
|
||||
```
|
||||
|
||||
|
||||
Filtered points would be:
|
||||
|
||||
```json
|
||||
|
||||
@@ -10,7 +10,6 @@ The indexes in the segments exist independently, but the parameters of the index
|
||||
Not all segments automatically have indexes.
|
||||
Their necessity is determined by the [optimizer](../optimizer) settings and depends, as a rule, on the number of stored points.
|
||||
|
||||
|
||||
## Payload Index
|
||||
|
||||
Payload index in Qdrant is similar to the index in conventional document-oriented databases.
|
||||
@@ -41,13 +40,13 @@ client.create_payload_index(collection_name="{collection_name}",
|
||||
field_schema="keyword")
|
||||
```
|
||||
|
||||
Available field types are:
|
||||
Available field types are:
|
||||
|
||||
* `keyword` - for [keyword](../payload/#keyword) payload, affects [Match](../filtering/#match) filtering conditions.
|
||||
* `integer` - for [integer](../payload/#integer) payload, affects [Match](../filtering/#match) and [Range](../filtering/#range) filtering conditions.
|
||||
* `float` - for [float](../payload/#float) payload, affects [Range](../filtering/#range) filtering conditions.
|
||||
* `geo` - for [geo](../payload/#geo) payload, affects [Geo Bounding Box](../filtering/#geo-bounding-box) and [Geo Radius](../filtering/#geo-radius) filtering conditions.
|
||||
* `text` - a special kind of index, available for [keyword](../payload/#keyword) / string payloads, affects [Full Text search](../filtering/#full-text-match) filtering conditions.
|
||||
* `keyword` - for [keyword](../payload/#keyword) payload, affects [Match](../filtering/#match) filtering conditions.
|
||||
* `integer` - for [integer](../payload/#integer) payload, affects [Match](../filtering/#match) and [Range](../filtering/#range) filtering conditions.
|
||||
* `float` - for [float](../payload/#float) payload, affects [Range](../filtering/#range) filtering conditions.
|
||||
* `geo` - for [geo](../payload/#geo) payload, affects [Geo Bounding Box](../filtering/#geo-bounding-box) and [Geo Radius](../filtering/#geo-radius) filtering conditions.
|
||||
* `text` - a special kind of index, available for [keyword](../payload/#keyword) / string payloads, affects [Full Text search](../filtering/#full-text-match) filtering conditions.
|
||||
|
||||
For indexing, it is recommended to choose the field that limits the search result the most.
|
||||
As a rule, the more different values a payload value has, the more efficiently the index will be used.
|
||||
@@ -140,7 +139,6 @@ storage:
|
||||
|
||||
And so in the process of creating a [collection](../collections). The `ef` parameter is configured during [the search](../search) and by default is equal to `ef_construct`.
|
||||
|
||||
|
||||
HNSW is chosen for several reasons.
|
||||
First, HNSW is well-compatible with the modification that allows Qdrant to use filters during a search.
|
||||
Second, it is one of the most accurate and fastest algorithms, according to [public benchmarks](https://github.com/erikbern/ann-benchmarks).
|
||||
@@ -158,11 +156,9 @@ On the one hand, we cannot apply a full scan on too many vectors. On the other h
|
||||
|
||||
|
||||
|
||||
|
||||
You can find more information on why this happens in our [blog post](https://blog.vasnetsov.com/posts/categorical-hnsw/).
|
||||
Qdrant solves this problem by extending the HNSW graph with additional edges based on the stored payload values.
|
||||
|
||||
Extra edges allow you to efficiently search for nearby vectors using the HNSW index and apply filters as you search in the graph.
|
||||
|
||||
This approach minimizes the overhead on condition checks since you only need to calculate the conditions for a small fraction of the points involved in the search.
|
||||
|
||||
|
||||
@@ -17,8 +17,8 @@ sudo docker info
|
||||
* If you do not see the server listed, start the Docker daemon.
|
||||
* On Linux, Docker needs `sudo` privileges. To run Docker commands without `sudo` privileges, create a docker group and add your users (see [Post-installation Steps for Linux](https://docs.docker.com/engine/install/linux-postinstall/) for details).
|
||||
|
||||
|
||||
Pull the image:
|
||||
|
||||
```bash
|
||||
docker pull qdrant/qdrant
|
||||
```
|
||||
@@ -36,7 +36,6 @@ It will store all data in `./path/to/data` directory.
|
||||
|
||||
By default, Qdrant uses port 6333, so at [localhost:6333](http://localhost:6333) you should see the welcome message.
|
||||
|
||||
|
||||
## From source
|
||||
|
||||
Qdrant is written in Rust and can be compiled into a binary executable.
|
||||
@@ -68,7 +67,6 @@ cargo build --release --bin qdrant
|
||||
|
||||
After a successful build, the binary is available at `./target/release/qdrant`.
|
||||
|
||||
|
||||
## With Kubernetes
|
||||
|
||||
You can use a ready-made [Helm Chart](https://helm.sh/docs/) to run Qdrant in your Kubeternetes cluster.
|
||||
@@ -109,7 +107,6 @@ Among other things, the configuration file allows you to specify the following s
|
||||
|
||||
See the comments in the [configuration file itself](https://github.com/qdrant/qdrant/blob/master/config/config.yaml) for details.
|
||||
|
||||
|
||||
## Python client
|
||||
|
||||
In addition to the service itself, Qdrant has a distinct python client, which has some additional features compared to clients generated from OpenAPI directly.
|
||||
@@ -121,12 +118,12 @@ pip install qdrant-client
|
||||
```
|
||||
|
||||
### DocArray
|
||||
|
||||
You can also use Qdrant natively in DocArray, where Qdrant serves as a high-performance document store to enable scalable vector search.
|
||||
|
||||
DocArray is a library from Jina AI for nested, unstructured data in transit, including text, image, audio, video, 3D mesh, etc.
|
||||
It allows deep-learning engineers to efficiently process, embed, search, recommend, store, and transfer the data with a Pythonic API.
|
||||
|
||||
|
||||
To install DocArray with Qdrant support, please do
|
||||
|
||||
```bash
|
||||
|
||||
@@ -38,7 +38,6 @@ storage:
|
||||
vacuum_min_vector_number: 1000
|
||||
```
|
||||
|
||||
|
||||
## Merge Optimizer
|
||||
|
||||
The service may require the creation of temporary segments.
|
||||
@@ -60,7 +59,6 @@ storage:
|
||||
max_segment_number: 5
|
||||
```
|
||||
|
||||
|
||||
## Indexing Optimizer
|
||||
|
||||
Qdrant allows you to choose the type of indexes and data storage methods used depending on the number of records.
|
||||
@@ -89,5 +87,4 @@ storage:
|
||||
|
||||
In addition to the configuration file, you can also set optimizer parameters separately for each [collection](../collections).
|
||||
|
||||
|
||||
Dynamic parameter updates may be useful, for example, for more efficient initial loading of points. You can disable indexing during the upload process with these settings and enable it immediately after it is finished. As a result, you will not waste extra computation resources on rebuilding the index.
|
||||
@@ -55,7 +55,6 @@ Let's look at the data types that Qdrant supports for searching:
|
||||
|
||||
`integer` - 64-bit integer in the range from `-9223372036854775808` to `9223372036854775807`.
|
||||
|
||||
|
||||
Example of single and multiple `integer` values:
|
||||
|
||||
```json
|
||||
@@ -69,7 +68,6 @@ Example of single and multiple `integer` values:
|
||||
|
||||
`float` - 64-bit floating point number.
|
||||
|
||||
|
||||
Example of single and multiple `float` values:
|
||||
|
||||
```json
|
||||
@@ -201,7 +199,6 @@ client.upsert(
|
||||
|
||||
## Update payload
|
||||
|
||||
|
||||
### Set payload
|
||||
|
||||
REST API ([Schema](https://qdrant.github.io/qdrant/redoc/index.html#operation/set_payload)):
|
||||
@@ -235,7 +232,6 @@ client.set_payload(
|
||||
|
||||
This method removes specified payload keys from specified points
|
||||
|
||||
|
||||
REST API ([Schema](https://qdrant.github.io/qdrant/redoc/index.html#operation/delete_payload)):
|
||||
|
||||
```http
|
||||
@@ -253,7 +249,7 @@ client.delete_payload(
|
||||
keys=["color", "price"],
|
||||
points=[0, 3, 100],
|
||||
)
|
||||
```
|
||||
```
|
||||
|
||||
### Clear payload
|
||||
|
||||
|
||||
@@ -18,7 +18,7 @@ After this moment, the service will not lose the data, even if the machine loses
|
||||
|
||||
## Awaiting result
|
||||
|
||||
If the API is called with the `&wait=false` parameter, or if it is not explicitly specified, the client will receive an acknowledgment of receiving data:
|
||||
If the API is called with the `&wait=false` parameter, or if it is not explicitly specified, the client will receive an acknowledgment of receiving data:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -93,7 +93,7 @@ client.upsert(
|
||||
),
|
||||
]
|
||||
)
|
||||
```
|
||||
```
|
||||
|
||||
and
|
||||
|
||||
@@ -124,7 +124,7 @@ client.upsert(
|
||||
),
|
||||
]
|
||||
)
|
||||
```
|
||||
```
|
||||
|
||||
are both possible.
|
||||
|
||||
@@ -175,7 +175,7 @@ client.upsert(
|
||||
]
|
||||
),
|
||||
)
|
||||
```
|
||||
```
|
||||
|
||||
or record-oriented equivalent:
|
||||
|
||||
@@ -306,7 +306,7 @@ Currently, you would need to re-upload the point to modify the vector.
|
||||
|
||||
The second is to modify the payload, for which there are several methods.
|
||||
|
||||
#### Set payload
|
||||
### Set payload
|
||||
|
||||
REST API ([Schema](https://qdrant.github.io/qdrant/redoc/index.html#operation/set_payload)):
|
||||
|
||||
@@ -335,7 +335,7 @@ client.set_payload(
|
||||
)
|
||||
```
|
||||
|
||||
#### Delete payload keys
|
||||
### Delete payload keys
|
||||
|
||||
REST API ([Schema](https://qdrant.github.io/qdrant/redoc/index.html#operation/delete_payload)):
|
||||
|
||||
@@ -354,9 +354,9 @@ client.delete_payload(
|
||||
keys=["color", "price"],
|
||||
points=[0, 3, 100],
|
||||
)
|
||||
```
|
||||
```
|
||||
|
||||
#### Clear payload
|
||||
### Clear payload
|
||||
|
||||
This method removes all payload keys from specified points
|
||||
|
||||
@@ -381,7 +381,6 @@ client.clear_payload(
|
||||
|
||||
## Delete points
|
||||
|
||||
|
||||
REST API ([Schema](https://qdrant.github.io/qdrant/redoc/index.html#operation/delete_points)):
|
||||
|
||||
```http
|
||||
@@ -438,12 +437,10 @@ client.delete(
|
||||
|
||||
This example removes all points with `{ "color": "red" }` from the collection.
|
||||
|
||||
|
||||
## Retrieve points
|
||||
|
||||
There is a method for retrieving points by their ids.
|
||||
|
||||
|
||||
REST API ([Schema](https://qdrant.github.io/qdrant/redoc/index.html#operation/get_points)):
|
||||
|
||||
```http
|
||||
@@ -465,7 +462,6 @@ This method has additional parameters `with_vector` and `with_payload`.
|
||||
Using these parameters, you can select parts of the point you want as a result.
|
||||
Excluding helps you not to waste traffic transmitting useless data.
|
||||
|
||||
|
||||
The single point can also be retrieved via the API:
|
||||
|
||||
REST API ([Schema](https://qdrant.github.io/qdrant/redoc/index.html#operation/get_point)):
|
||||
@@ -483,7 +479,6 @@ Python client:
|
||||
|
||||
## Scroll points
|
||||
|
||||
|
||||
Sometimes it might be necessary to get all stored points without knowing ids, or iterate over points that correspond to a filter.
|
||||
|
||||
REST API ([Schema](https://qdrant.github.io/qdrant/redoc/index.html#operation/scroll_points)):
|
||||
@@ -570,7 +565,6 @@ Among others, for example, we can highlight the following scenarios:
|
||||
* Determining the number of pages for pagination
|
||||
* Debugging the query execution speed
|
||||
|
||||
|
||||
REST API ([Schema](https://qdrant.github.io/qdrant/redoc/index.html#operation/scroll_points)):
|
||||
|
||||
```http
|
||||
|
||||
@@ -53,7 +53,9 @@ We recommend switching to it if you are already familiar with Qdrant and are try
|
||||
If you are applying Qdrant for the first time or working on a prototype, you might prefer to use REST.
|
||||
|
||||
### Create collection
|
||||
|
||||
First - let's create a collection with dot-production metric.
|
||||
|
||||
```bash
|
||||
curl -X PUT 'http://localhost:6333/collections/test_collection' \
|
||||
-H 'Content-Type: application/json' \
|
||||
@@ -66,6 +68,7 @@ curl -X PUT 'http://localhost:6333/collections/test_collection' \
|
||||
```
|
||||
|
||||
Expected response:
|
||||
|
||||
```json
|
||||
{
|
||||
"result": true,
|
||||
@@ -75,6 +78,7 @@ Expected response:
|
||||
```
|
||||
|
||||
We can ensure that collection was created:
|
||||
|
||||
```bash
|
||||
curl 'http://localhost:6333/collections/test_collection'
|
||||
```
|
||||
@@ -106,8 +110,8 @@ Expected response:
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
### Add points
|
||||
|
||||
Let's now add vectors with some payload:
|
||||
|
||||
```bash
|
||||
@@ -126,6 +130,7 @@ curl -L -X PUT 'http://localhost:6333/collections/test_collection/points?wait=tr
|
||||
```
|
||||
|
||||
Expected response:
|
||||
|
||||
```json
|
||||
{
|
||||
"result": {
|
||||
@@ -186,6 +191,7 @@ curl -L -X POST 'http://localhost:6333/collections/test_collection/points/search
|
||||
```
|
||||
|
||||
Expected response:
|
||||
|
||||
```json
|
||||
{
|
||||
"result": [
|
||||
|
||||
@@ -38,17 +38,17 @@ In this case, it becomes equivalent to dot production - a very fast operation du
|
||||
## Query planning
|
||||
|
||||
Depending on the filter used in the search - there are several possible scenarios for query execution.
|
||||
Qdrant chooses one of the query execution options depending on the available indexes, the complexity of the conditions and the cardinality of the filtering result.
|
||||
Qdrant chooses one of the query execution options depending on the available indexes, the complexity of the conditions and the cardinality of the filtering result.
|
||||
This process is called query planning.
|
||||
|
||||
The strategy selection process relies heavily on heuristics and can vary from release to release.
|
||||
However, the general principles are:
|
||||
|
||||
- planning is performed for each segment independently (see [storage](../storage) for more information about segments)
|
||||
- prefer a full scan if the amount of points is below a threshold
|
||||
- estimate the cardinality of a filtered result before selecting a strategy
|
||||
- retrieve points using payload index (see [indexing](../indexing)) if cardinality is below threshold
|
||||
- use filterable vector index if the cardinality is above a threshold
|
||||
* planning is performed for each segment independently (see [storage](../storage) for more information about segments)
|
||||
* prefer a full scan if the amount of points is below a threshold
|
||||
* estimate the cardinality of a filtered result before selecting a strategy
|
||||
* retrieve points using payload index (see [indexing](../indexing)) if cardinality is below threshold
|
||||
* use filterable vector index if the cardinality is above a threshold
|
||||
|
||||
You can adjust the threshold using a [configuration file](https://github.com/qdrant/qdrant/blob/master/config/config.yaml), as well as independently for each collection.
|
||||
|
||||
@@ -106,7 +106,7 @@ client.search(
|
||||
)
|
||||
```
|
||||
|
||||
In this example, we are looking for vectors similar to vector `[0.2, 0.1, 0.9, 0.7]`.
|
||||
In this example, we are looking for vectors similar to vector `[0.2, 0.1, 0.9, 0.7]`.
|
||||
Parameter `limit` (or its alias - `top`) specifies the amount of most similar results we would like to retrieve.
|
||||
|
||||
Values under the key `params` specify custom parameters for the search.
|
||||
@@ -337,7 +337,6 @@ The result of this API contains one array per search requests.
|
||||
}
|
||||
```
|
||||
|
||||
|
||||
## Recommendation API
|
||||
|
||||
<aside role="alert">Negative vectors is an experimental functionality that is not guaranteed to work with all kind of embeddings.</aside>
|
||||
@@ -347,7 +346,7 @@ This API uses vector search without involving the neural network encoder for alr
|
||||
|
||||
The recommendation API allows specifying several positive and negative vector IDs, which the service will combine into a certain average vector.
|
||||
|
||||
` average_vector = avg(positive_vectors) + ( avg(positive_vectors) - avg(negative_vectors) )`
|
||||
`average_vector = avg(positive_vectors) + ( avg(positive_vectors) - avg(negative_vectors) )`
|
||||
|
||||
If there is only one positive ID provided - this request is equivalent to the regular search with vector of that point.
|
||||
|
||||
@@ -439,7 +438,7 @@ client.recommend(
|
||||
)
|
||||
```
|
||||
|
||||
Parameter `using` specifies which stored vectors to use for the recommendation.
|
||||
Parameter `using` specifies which stored vectors to use for the recommendation.
|
||||
|
||||
## Batch recommendation API
|
||||
|
||||
@@ -580,7 +579,7 @@ client.search(
|
||||
)
|
||||
```
|
||||
|
||||
Is equivalent to retrieving 11th page with 10 records per page.
|
||||
Is equivalent to retrieving the 11th page with 10 records per page.
|
||||
|
||||
<aside role="alert">Large offset values may cause performance issues</aside>
|
||||
|
||||
|
||||
@@ -124,7 +124,6 @@ client.list_full_snapshots()
|
||||
GET /snapshots/{snapshot_name}
|
||||
```
|
||||
|
||||
|
||||
## Restore full storage snapshot
|
||||
|
||||
Restoring snapshots is done through the Qdrant CLI at startup time.
|
||||
|
||||
@@ -4,7 +4,7 @@ weight: 29
|
||||
---
|
||||
|
||||
All data within one collection is divided into segments.
|
||||
Each segment has its independent vector and payload storage as well as indexes.
|
||||
Each segment has its independent vector and payload storage as well as indexes.
|
||||
|
||||
Data stored in segments usually do not overlap.
|
||||
However, storing the same point in different segments will not cause problems since the search contains a deduplication mechanism.
|
||||
@@ -30,7 +30,6 @@ This scheme allows flexible use of available memory. With sufficient RAM, it is
|
||||
However, dynamically adding vectors to the mmap file is fairly complicated and is not implemented in Qdrant.
|
||||
Thus, segments using mmap storage are `non-appendable` and can only be construed by the optimizer.
|
||||
|
||||
|
||||
## Payload storage
|
||||
|
||||
Qdrant supports two types of payload storages: InMemory and OnDisk.
|
||||
@@ -41,14 +40,13 @@ This type of storage works quite fast, but it may require a lot of space to keep
|
||||
|
||||
In the case of large payload values, it might be better to use OnDisk payload storage.
|
||||
This type of storage will read and write payload directly to RocksDB, so it won't require any significant amount of RAM to store.
|
||||
The downside, however, is the access latency.
|
||||
The downside, however, is the access latency.
|
||||
If you need to query vectors with some payload-based conditions - checking values stored on disk might take too much time.
|
||||
In this scenario, we recommend creating a payload index for each field used in filtering conditions to avoid disk access.
|
||||
Once you create the field index, Qdrant will preserve all values of the indexed field in RAM regardless of the payload storage type.
|
||||
|
||||
You can specify the desired type of payload storage with [configuration file](../configuration/) or with collection parameter `on_disk_payload` during [creation](../collections/#create-collection) of the collection.
|
||||
|
||||
|
||||
## Versioning
|
||||
|
||||
To ensure data integrity, Qdrant performs all data changes in 2 stages.
|
||||
@@ -56,6 +54,6 @@ In the first step, the data is written to the Write-ahead-log(WAL), which orders
|
||||
|
||||
Once a change has been added to the WAL, it will not be lost even if power loss occurs.
|
||||
Then the changes go into the segments.
|
||||
Each segment stores the last version of the change applied to it as well as version of each individual point.
|
||||
Each segment stores the last version of the change applied to it as well as version of each individual point.
|
||||
If the new change has a sequential number less than the current version of the point, the updater will ignore the change.
|
||||
This mechanism allows Qdrant to safely and efficiently restore the storage from the WAL in case of an abnormal shutdown.
|
||||
|
||||
Reference in New Issue
Block a user