krzysztof/landing_page
1
0
Fork 0
mirror of https://github.com/qdrant/landing_page.git synced 2026-09-29 16:08:32 +02:00
Code Issues Packages Projects Releases Wiki Activity

Optimizers configuration question added

Browse Source
This commit is contained in:
kartik-gupta-ij
2024-05-08 20:49:16 +05:30
parent 12f4a7386d
commit 6b13919e9d
1 changed files with 15 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
qdrant-landing/content/documentation/faq/database-optimization.md
+15
View File
@@ -42,3 +42,18 @@ There are several possible reasons for that:
- **Using filters without payload index** -- If you're performing a search with a filter but you don't have a payload index, Qdrant will have to load whole payload data from disk to check the filtering condition. Ensure you have adequately configured [payload indexes](../../concepts/indexing/#payload-index).
- **Usage of on-disk vector storage with slow disks** -- If you're using on-disk vector storage, ensure you have fast enough disks. We recommend using local SSDs with at least 50k IOPS. Read more about the influence of the disk speed on the search latency in the article about [Memory Consumption](../../../articles/memory-consumption/).
- **Large limit or non-optimal query parameters** -- A large limit or offset might lead to significant performance degradation. Please pay close attention to the query/collection parameters that significantly diverge from the defaults. They might be the reason for the performance issues.
### How can I optimize optimizers configuration settings for better accuracy and speed, especially for large-scale collections?
To optimize optimizers config in Qdrant for better accuracy and speed, consider the following:
- For low memory footprint with high-speed search, utilize vector quantization with disk storage for vectors and in-memory quantized vectors. Configure memmap_threshold and always_ram accordingly.
- To prioritize high precision with a low memory footprint, enable on-disk vectors and HNSW index. Adjust HNSW parameters for precision while considering disk IOPS.
- For high precision with high-speed search, focus on keeping data in RAM. Utilize quantization with re-scoring and adjust search-time parameters like hnsw_ef for accuracy and speed balance.
- Balance latency vs. throughput based on your needs. Configure the number of segments to utilize CPU cores effectively for either minimizing latency or maximizing throughput.
- Explore different quantization methods like scalar, binary, and product quantization based on accuracy, speed, and compression requirements.
- Fine-tune quantization parameters such as quantile for scalar quantization to optimize search precision and memory usage.
- Adjust storage modes to balance between memory footprint and search speed, considering factors like disk reads and storage type (RAM, SSD, or HDD).
Read more about [optimizing](../../guides/optimize/), and [quantization](../../guides/quantization/) in Qdrant.