mirror of
https://github.com/qdrant/landing_page.git
synced 2026-10-05 10:58:32 +02:00
41 lines
2.4 KiB
Markdown
41 lines
2.4 KiB
Markdown
# Qdrant Multi-Vector Search Course
|
||
|
||
This document is a proposal for another Qdrant course. I suggest covering both text and multi-modal multi-vector representations, focusing on making them usable in Qdrant.
|
||
|
||
# Outline
|
||
|
||
1. **Module 0:** Setting up dependencies
|
||
2. **Module 1:** Multi-vector representations for textual data
|
||
* ColBERT basics
|
||
* Comparison to regular dense embedding models
|
||
* Examples when multi-vectors may work better than single vectors
|
||
* MaxSim distance
|
||
3. **Module 2:** Multi-vector representations for multi-modal data (image \+ text)
|
||
* ColPali family
|
||
* Inner workings of the ColPali models
|
||
* Visual interpretability of ColPali representations
|
||
* Setting up Qdrant for multi-vector embeddings
|
||
4. **Module 3:** Scalability issues caused by multi-vector representations
|
||
* The implications of high memory usage and ways to solve it
|
||
* Vector quantization: scalar, binary, \+1.5/2-bit
|
||
* Pooling techniques: row/column pooling, hierarchical token pooling
|
||
* Incompatibility with HNSW due to MaxSim asymmetry
|
||
* MUVERA
|
||
* Combining multiple optimizations in multi-stage retrieval via Universal Query API
|
||
* Evaluating different search pipelines in terms of cost and latency
|
||
|
||
## Project
|
||
|
||
I suggest having just one project. It should start with a set of PDFs or even scanned documents for which the participants are supposed to create a two-stage ColPali retrieval system using both MUVERA for fast retrieval and some memory optimization (or multiple ones, like SQ \+ token pooling) for reduced memory usage.
|
||
|
||
## Supplementary content
|
||
|
||
To keep the momentum, I suggest publishing a few related blog posts in the upcoming weeks:
|
||
|
||
1. **The asymmetry of MaxSim**
|
||
We suggest disabling HNSW for MaxSim, but it is never discussed why it doesn’t make sense. It’s not worth an article, but a short explanation with some examples would make things clearer.
|
||
2. **A review of modern multi-vector retrievers for visual data**
|
||
There’s been a lot of excitement around ModernVBERT, so it would make sense to present how to use it with Qdrant, even though we don’t support it in FastEmbed.
|
||
3. **FastEmbed: implementation of different pooling techniques for multi-vectors**
|
||
E.g., row/column pooling (if possible), hierarchical token pooling. Techniques might be implemented in the library itself, but we should also create a blog post about it.
|