krzysztof/landing_page
1
0
Fork 0
mirror of https://github.com/qdrant/landing_page.git synced 2026-09-26 22:48:30 +02:00
Code Issues Packages Projects Releases Wiki Activity

fix according to feedback (#1010)

Browse Source
This commit is contained in:
Andrey Vasnetsov
2024-07-05 19:15:17 +02:00
committed by GitHub
parent d018e5961e
commit 6b9e82356f
1 changed files with 13 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
qdrant-landing/content/articles/bm42.md
+13 -4
View File
@@ -318,7 +318,9 @@ client.query_points(
To prove the point further we have conducted some benchmarks to highlight the cases where BM42 outperforms BM25.
Please note, that we didn't intend to make an exhaustive evaluation, as we are presenting a new approach, not a new model.
For out experiments we choose [quora](https://huggingface.co/datasets/BeIR/quora) dataset, as a good representative of the Question-Answering task.
For out experiments we choose [quora](https://huggingface.co/datasets/BeIR/quora) dataset, which represents a question-deduplication task ~~the Question-Answering task~~.
The typical example of the dataset is the following:
```text
@@ -334,9 +336,16 @@ After encoding with BM42, the average vector size is only **5.6 elements per doc
With `datatype: uint8` available in Qdrant, the total size of the sparse vector index is about **13Mb** for ~530k documents.
| | BM25 | BM42 |
|---------------|------|----------|
|Precision @ 10 | 0.45 | **0.49** |
| | BM25 (tantivy) | BM42 |
|-------------------|------|----------|
|~~Precision @ 10~~ * | ~~0.45~~ | ~~0.49~~ |
| Recall @ 10 | 0.71 | **0.85** |
\* - values were corrected after the publication due to a mistake in the evaluation script.
To make our benchmarks transparent, we have published scripts we used for the evaluation: see [github repo](https://github.com/qdrant/bm42_eval).
Please note, that both BM25 and BM42 won't work well on their own in a production environment.