krzysztof/landing_page
1
0
Fork 0
mirror of https://github.com/qdrant/landing_page.git synced 2026-09-28 23:48:31 +02:00
Code Issues Packages Projects Releases Wiki Activity

more

Browse Source
This commit is contained in:
Arnaud Gourlay
2023-12-07 20:17:48 +01:00
parent 503da199c4
commit 58340b3e66
4 changed files with 191 additions and 32 deletions
Download Patch File Download Diff File Expand all files Collapse all files
qdrant-landing/content/documentation/concepts/collections.md
+4 -31
View File
@@ -297,41 +297,10 @@ which is suitable for ingesting a large amount of data.
*Available as of v1.7.0*
A sparse vector is an array in which most of the elements have a value of zero.
It is possible to take advantage of this property to have an optimized representation, for this reason they have a different shape than dense vectors.
They are represented as a list of `(index, value)` pairs, where `index` is an integer and `value` is a floating point number. The `index` is the position of the non-zero value in the vector. The `values` is the value of the non-zero element.
For example, the following vector:
```
[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 2.0, 0.0, 0.0]
```
can be represented as a sparse vector:
```
[(6, 1.0), (7, 2.0)]
```
Qdrat uses the following JSON representation throughout the REST API.
```json
{
"indices": [6, 7],
"values": [1.0, 2.0]
}
```
The `indices` and `values` arrays must have the same length.
And the `indices` must be unique.
Collections can contain sparse vectors as additional [named vectors](#collection-with-multiple-vectors) along side regular dense vectors in a single point.
Sparse vectors must be named, unlike dense vectors which support a single dense anonymous vector.
```http
PUT /collections/{collection_name}
{
@@ -396,6 +365,10 @@ client
.await?;
```
There are no required configuration parameters for named sparse vectors.
However, you can optionally tune the following parameters for the underlying [sparse indexing](../indexing/#sparse-vector-index).
### Delete collection
```http
qdrant-landing/content/documentation/concepts/indexing.md
+30 -1
View File
@@ -190,7 +190,7 @@ See [Full Text match](../filtering/#full-text-match) for examples of querying wi
A vector index is a data structure built on vectors through a specific mathematical model.
Through the vector index, we can efficiently query several vectors similar to the target vector.
Qdrant currently only uses HNSW as a vector index.
Qdrant currently only uses HNSW as a dense vector index.
[HNSW](https://arxiv.org/abs/1603.09320) (Hierarchical Navigable Small World Graph) is a graph-based indexing algorithm. It builds a multi-layer navigation structure for an image according to certain rules. In this structure, the upper layers are more sparse and the distances between nodes are farther. The lower layers are denser and the distances between nodes are closer. The search starts from the uppermost layer, finds the node closest to the target in this layer, and then enters the next layer to begin another search. After multiple iterations, it can quickly approach the target position.
@@ -228,6 +228,35 @@ The HNSW parameters can also be configured on a collection and named vector
level by setting [`hnsw_config`](../indexing/#vector-index) to fine-tune search
performance.
## Sparse vector index
*Available as of v1.4.0*
Qdrant supports sparse vectors, which are vectors with a large number of zeroes.
Those vectors are stored in a specialized way, which allows to save space and speed up search.
The sparse vector index resides in memory for appendable segments providing fast search and update operations.
When the segment becomes immutable, the sparse index can either be kept in memory or mmaped to disk.
For instance, to enable on-disk storage for immutable segments and full scan for segments with less than 10000 vectors:
```http
PUT /collections/{collection_name}
{
"sparse_vectors": {
"text": {
"index": {
"on_disk": true,
"full_scan_threshold": 10000
}
},
}
}
```
## Filtrable Index
Separately, payload index and vector index cannot solve the problem of search using the filter completely.
qdrant-landing/content/documentation/concepts/points.md
+151
View File
@@ -524,6 +524,157 @@ then it is inserted with just the specified vectors. In other words, the entire
point is replaced, and any unspecified vectors are set to null. To keep existing
vectors unchanged and only update specified vectors, see [update vectors](#update-vectors).
* Available as of v1.7.0*
Points can contain dense and sparse vectors.
A sparse vector is an array in which most of the elements have a value of zero.
It is possible to take advantage of this property to have an optimized representation, for this reason they have a different shape than dense vectors.
They are represented as a list of `(index, value)` pairs, where `index` is an integer and `value` is a floating point number. The `index` is the position of the non-zero value in the vector. The `values` is the value of the non-zero element.
For example, the following vector:
```
[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 2.0, 0.0, 0.0]
```
can be represented as a sparse vector:
```
[(6, 1.0), (7, 2.0)]
```
Qdrat uses the following JSON representation throughout the REST API.
```json
{
"indices": [6, 7],
"values": [1.0, 2.0]
}
```
The `indices` and `values` arrays must have the same length.
And the `indices` must be unique.
Sparse vectors must be named and can be uploaded in the same way as dense vectors.
```http
PUT /collections/{collection_name}/points
{
"points": [
{
"id": 1,
"vector": {
"text": {
"indices": [6, 7],
"values": [1.0, 2.0]
}
}
},
{
"id": 2,
"vector": {
"text": {
"indices": [1, 1, 2, 3, 4, 5],
"values": [0.1, 0.2, 0.3, 0.4, 0.5]
}
}
}
]
}
```
```python
client.upsert(
collection_name="{collection_name}",
points=[
models.PointStruct(
id=1,
vector={
"text": models.SparseVector(
indices=[6, 7],
values=[1.0, 2.0],
)
},
),
models.PointStruct(
id=2,
vector={
"text": models.SparseVector(
indices=[1, 2, 3, 4, 5],
values= [0.1, 0.2, 0.3, 0.4, 0.5],
)
},
),
],
)
```
```typescript
client.upsert("{collection_name}", {
points: [
{
id: 1,
vector: {
text: {
indices: [6, 7],
values: [1.0, 2.0]
},
},
},
{
id: 2,
vector: {
text: {
indices=[1, 2, 3, 4, 5],
values= [0.1, 0.2, 0.3, 0.4, 0.5],
},
},
},
],
});
```
```rust
use qdrant_client::qdrant::{PointStruct, Vector};
use std::collections::HashMap;
client
.upsert_points_blocking(
"{collection_name}".to_string(),
vec![
PointStruct::new(
1,
HashMap::from([
(
"text".to_string(),
Vector::from(
(vec![6, 7], vec![1.0, 2.0])
),
),
]),
HashMap::new().into(),
),
PointStruct::new(
2,
HashMap::from([
(
"text".to_string(),
Vector::from(
(vec![1, 2, 3, 4, 5], vec![0.1, 0.2, 0.3, 0.4, 0.5])
),
),
]),
HashMap::new().into(),
),
],
None,
)
.await?;
```
## Modify points
To change a point, you can modify its vectors or its payload. There are several
qdrant-landing/content/documentation/concepts/search.md
+6
View File
@@ -250,6 +250,12 @@ client
Search is processing only among vectors with the same name.
*Available as of v0.17.0*
If the collection was created with sparse vectors, the name of the sparse vector to use for searching should be provided:
TODO: add examples
### Filtering results by score
In addition to payload filtering, it might be useful to filter out results with a low similarity score.