diff --git a/qdrant-landing/content/documentation/_index.md b/qdrant-landing/content/documentation/_index.md index 2f6b1ecd9..16f5b4e9c 100644 --- a/qdrant-landing/content/documentation/_index.md +++ b/qdrant-landing/content/documentation/_index.md @@ -7,8 +7,7 @@ subtitle: # Qdrant Documentation -Qdrant (read: quadrant ) is a vector similarity search engine. It provides a production-ready service with a convenient API to store, search, and manage points - vectors with an additional payload. Qdrant is tailored to extended filtering support. It makes it useful for all sorts of neural-network or semantic-based matching, faceted search, and other applications. - +Qdrant (read: quadrant ) is a vector similarity search engine. It provides a production-ready service with a convenient API to store, search, and manage points - vectors with an additional payload. Qdrant is tailored to extended filtering support. It makes it useful for all sorts of neural network or semantic-based matching, faceted search, and other applications. Qdrant is released under the open-source Apache License 2.0. Its source code is available on [GitHub](https://github.com/qdrant/qdrant). @@ -27,4 +26,4 @@ In addition to this documentation, you may be interested in looking at examples * [Semantic Search for startups](https://demo.qdrant.tech/) + [Source Code](https://github.com/qdrant/qdrant_demo) * [Visual Food Discovery](https://food-discovery.qdrant.tech/) -* [Step-byStep tutorial on building neural search](http://localhost:1313/articles/neural-search-tutorial/) \ No newline at end of file +* [Step-by-Step tutorial on building neural search](http://localhost:1313/articles/neural-search-tutorial/) \ No newline at end of file diff --git a/qdrant-landing/content/documentation/collections.md b/qdrant-landing/content/documentation/collections.md index 1a556914d..8208f18d1 100644 --- a/qdrant-landing/content/documentation/collections.md +++ b/qdrant-landing/content/documentation/collections.md @@ -18,7 +18,7 @@ Qdrant supports these most popular types of metrics: * Euclidean distance: `Euclid` - https://en.wikipedia.org/wiki/Euclidean_distance In addition to metrics and vector size, each collection uses its own set of parameters that controls collection optimization, index construction, and vacuum. -These settings can be changed at any time by a suitable request. +These settings can be changed at any time by a corresponding request. ### Create collection @@ -42,7 +42,7 @@ In addition to the required options, you can also specify custom values for the - `wal_config` - `optimizers_config` -See [schema definitions](https://qdrant.github.io/qdrant/redoc/index.html#operation/update_collections) and [configuration file](https://github.com/qdrant/qdrant/blob/master/config/config.yaml) for more information about collection parameters. +See [schema definitions](https://qdrant.github.io/qdrant/redoc/index.html#operation/update_collections) and a [configuration file](https://github.com/qdrant/qdrant/blob/master/config/config.yaml) for more information about collection parameters. All APIs in Qdrant, including point loading, are idempotent. -This means that executing the same method several times in a row is equivalent to a single execution. +It means that executing the same method several times in a row is equivalent to a single execution. -In this case it means that points with the same id will be overwritten when re-uploaded. -Idempotence property is useful if you use, for example, a message queue that doesn't provide exactly-ones guarantee. +In this case, it means that points with the same id will be overwritten when re-uploaded. + +Idempotence property is useful if you use, for example, a message queue that doesn't provide an exactly-ones guarantee. Even with such a system, Qdrant ensures data consistency. - - ## Modify points -You can modify a point in two ways. The first is to modify vector. -Currently you would need to re-upload point in order to modify vector. +You can modify a point in two ways. The first is to modify the vector. +Currently, you would need to re-upload the point to modify the vector. -The second is not modify payload, for which there are several methods. +The second is to modify the payload, for which there are several methods. #### Set payload diff --git a/qdrant-landing/content/documentation/search.md b/qdrant-landing/content/documentation/search.md index b3b6fce1f..49c6c2981 100644 --- a/qdrant-landing/content/documentation/search.md +++ b/qdrant-landing/content/documentation/search.md @@ -6,10 +6,9 @@ weight: 26 ## Similarity search -Searching for nearest vectors is at the core of many representational learning applications. +Searching for the nearest vectors is at the core of many representational learning applications. Modern neural networks are trained to transform objects into vectors so that objects close in the real world appear close in vector space. -For example texts with the similar meaning, visually similar pictures or songs of the same genre. - +It could be, for example, texts with similar meanings, visually similar pictures, or songs of the same genre. ![Embeddings](/docs/encoders.png) @@ -18,7 +17,7 @@ For example texts with the similar meaning, visually similar pictures or songs o There are many ways to estimate the similarity of vectors with each other. In Qdrant terms, these ways are called metrics. -The choice of metric depends on the way of vectors obtaining and in particular on the method of neural network encoder training. +The choice of metric depends on vectors obtaining and, in particular, on the method of neural network encoder training. Qdrant supports these most popular types of metrics: @@ -35,7 +34,7 @@ The first step is to normalize the vector when adding it to the collection. It happens only once for each vector. The second step is the comparison of vectors. -In this case it becomes equivalent to dot production - a very fast operation due to SIMD. +In this case, it becomes equivalent to dot production - a very fast operation due to SIMD. ## Query planning @@ -47,10 +46,10 @@ The strategy selection process relies heavily on heuristics and can vary from re However, the general principles are: - planning is performed for each segment independently (see [storage](../storage) for more information about segments) -- prefer a full scan if amount of points is below threshold -- estimate cardinality of a filtered result before selecting strategy +- prefer a full scan if the amount of points is below a threshold +- estimate the cardinality of a filtered result before selecting a strategy - retrieve points using payload index (see [indexing](../indexing)) if cardinality is below threshold -- use filterable vector index if cardinality is above threshold +- use filterable vector index if the cardinality is above a threshold You can adjust the threshold using a [configuration file](https://github.com/qdrant/qdrant/blob/master/config/config.yaml), as well as independently for each collection. @@ -82,17 +81,17 @@ POST /collections/{collection_name}/points/search } ``` -In this example we are looking for vectors which are similar to vector `[0.2, 0.1, 0.9, 0.7]`. +In this example, we are looking for vectors similar to vector `[0.2, 0.1, 0.9, 0.7]`. Parameter `top` specifies the amount of most similar results we would like to retrieve. -Values under the key `params` specifies custom parameters for the search. -Currently it could be: +Values under the key `params` specify custom parameters for the search. +Currently, it could be: * `hnsw_ef` - value that specifies `ef` parameter of the HNSW algorithm. Since the `filter` parameter is specified, the search is performed only among those points that satisfy the filter condition. -See details of possible filters and how they work in [filtering](../filtering) section. +See details of possible filters and their work in the [filtering](../filtering) section. Example result of this API would be @@ -119,12 +118,12 @@ The `result` contains ordered by `score` list of found point ids. ## Recommendation API -**DISCLAIMER**: Negative vectors is an experimental functionality which is not guaranteed to work with all king of embeddings. +**DISCLAIMER**: Negative vectors is an experimental functionality that is not guaranteed to work with all king of embeddings. -In addition to the regular search, Qdrant also allows you to perform a search based on multiple, already stored in the collection vectors. -This API allows to use vector search without the need to use a neural network encoder for already encoded objects. +In addition to the regular search, Qdrant also allows you to search based on multiple already stored data collection vectors. +This API allows using vector search without using a neural network encoder for already encoded objects. -The recommendation API allows to specify several positive and negative vector IDs, which will be combined into a certain average vector. +The recommendation API allows specifying several positive and negative vector IDs, which the service will combine into a certain average vector. ` average_vector = avg(positive_vectors) + ( avg(positive_vectors) - avg(negative_vectors) )` diff --git a/qdrant-landing/content/documentation/storage.md b/qdrant-landing/content/documentation/storage.md index 5cbdb9d38..f6fb56bbb 100644 --- a/qdrant-landing/content/documentation/storage.md +++ b/qdrant-landing/content/documentation/storage.md @@ -4,10 +4,10 @@ weight: 29 --- All data within one collection is divided into segments. -Each segment has its own independent vector and payload storage as well as indexes. +Each segment has its independent vector and payload storage as well as indexes. Data stored in segments usually do not overlap. -However, storing the same point in different segments will not cause problems, since the search contains a deduplication mechanism. +However, storing the same point in different segments will not cause problems since the search contains a deduplication mechanism. The segments consist of vector and payload storages, vector and payload [indexes](../indexing), and id mapper, which stores the relationship between internal and external ids. @@ -20,12 +20,13 @@ The configuration of the segments in the collection can be different and indepen ## Vector storage Depending on the requirements of the application, Qdrant can use one of the data storage options. -The choice has to be made between the search speed and the size of RAM used. +The choice has to be made between the search speed and the size of the RAM used. -**In-memory storage** - Stores all vectors in RAM, has the highest speed, since disk access is required only for persistence. +**In-memory storage** - Stores all vectors in RAM, has the highest speed since disk access is required only for persistence. -**Memmap storage** - creates a virtual address space associated with the file on disk. [Wiki](https://en.wikipedia.org/wiki/Memory-mapped_file). Mmaped files are not directly loaded into RAM, instead they use page cache to access the contents of the file. -This scheme allows a flexible use of available memory. With sufficient RAM is almost as fast as in-memory storage. +**Memmap storage** - creates a virtual address space associated with the file on disk. [Wiki](https://en.wikipedia.org/wiki/Memory-mapped_file). +Mmaped files are not directly loaded into RAM. Instead, they use page cache to access the contents of the file. +This scheme allows flexible use of available memory. With sufficient RAM, it is almost as fast as in-memory storage. However, dynamically adding vectors to the mmap file is fairly complicated and is not implemented in Qdrant. Thus, segments using mmap storage are `non-appendable` and can only be construed by the optimizer. @@ -37,11 +38,11 @@ Payload is loaded into RAM at service startup while disk and [RocksDB](https://r ## Versioning -To ensure data integrity, all data changes occur in 2 stages. +To ensure data integrity, Qdrant performs all data changes in 2 stages. In the first step, the data is written to the Write-ahead-log(WAL), which orders all operations and assigns them a sequential number. Once a change has been added to the WAL, it will not be lost even if power loss occurs. Then the changes go into the segments. Each segment stores the last version of the change applied to it. -If the new change has a sequential number less than the current version of the segment, the change will be ignored. -This mechanism allows Qdrant to safely and efficiently restore the state of the storage from the WAL in case of abnormal shutdown. +If the new change has a sequential number less than the current version of the segment, the segment will ignore the change. +This mechanism allows Qdrant to safely and efficiently restore the storage from the WAL in case of an abnormal shutdown. \ No newline at end of file