GeorgeandAndrey Vasnetsov
faad182279
* new: add filtered search bench draft * new: update filtered search benchmark post * new: render article * refactoring: add newlines * new: add filtered search bench file structure, add chart * fix: fix plot preselect * fix: fix canvas id * debug: debug js * debug: add benchmark filter search script to header * fix: fix filter search plot selection * new: update benchmark data * refactoring: remove commented code * fix: fix bench data * new: fix labels order * new: sort dataset labels * new: add some text * fix: fix descriptions * debug: fix mapping not allowed yaml parsing error * new: update bench data * fix: fix dataset name * new: update bench data * fix: fix qdrant kw small vocab * text for the articles * upd text * fix: fix typos * new: replace bars with scatter plots * fix: fix method calls * debug: add initializer * fix: replace debug const with variable * new: remove data based split, remain only one plot * fix: remove geo data for milvus * new: remove range data for weaviate * new: remove int 2048 for milvus, as it crashes at the moment * upd graphs * link --------- Co-authored-by: Andrey Vasnetsov <andrey@vasnetsov.com>
1.4 KiB
draft, id, title, weight
| draft | id | title | weight |
|---|---|---|---|
| false | 2 | How vector search databases should be tested? | 1 |
Benchmarking Vector Search Engines
As an Open Source vector search engine, we are often compared to the competitors and asked about our performance vs the other tools. But the answer was never simple, as the world of vector databases lacked a unified open benchmark that would show the differences. So we created one, making some bold assumptions about how it should be done. Here we describe why we think that’s the best way.
That is why we perform our benchmarks on exactly the same hardware, which you can rent from any cloud provider. It does not guarantee the best performance, making the whole process affordable and reproducible, so you can easily repeat it yourself. So in our benchmarks, we focus on the relative numbers, so it is possible to compare the performance of different engines given equal resources.
The list will be updated:
- Upload & Search speed on single node - Benchmark
- Filtered search benchmark - Benchmark
- Memory consumption benchmark - TBD
- Cluster mode benchmark - TBD
Some of our experiment design decisions are described at F.A.Q Section.
Suggest your variants of what you want to test in our Discord channel!