mirror of
https://github.com/qdrant/landing_page.git
synced 2026-10-05 10:58:32 +02:00
Initial structure + plan
This commit is contained in:
@@ -0,0 +1,8 @@
|
|||||||
|
---
|
||||||
|
title: "Multi-Vector Search Course"
|
||||||
|
todo: Prepare all the other attributes of the course
|
||||||
|
---
|
||||||
|
|
||||||
|
# Multi-Vector Search
|
||||||
|
|
||||||
|
TBD
|
||||||
@@ -0,0 +1,39 @@
|
|||||||
|
# Qdrant Multi-Vector Search Course
|
||||||
|
|
||||||
|
This document is a proposal for another Qdrant course. I suggest covering both text and multi-modal multi-vector representations, focusing on making them usable in Qdrant.
|
||||||
|
|
||||||
|
# Outline
|
||||||
|
|
||||||
|
1. **Module 1:** Multi-vector representations for textual data
|
||||||
|
* ColBERT basics
|
||||||
|
* Comparison to regular dense embedding models
|
||||||
|
* Examples when multi-vectors may work better than single vectors
|
||||||
|
* MaxSim distance
|
||||||
|
2. **Module 2:** Multi-vector representations for multi-modal data (image \+ text)
|
||||||
|
* ColPali family
|
||||||
|
* Inner workings of the ColPali models
|
||||||
|
* Visual interpretability of ColPali representations
|
||||||
|
* Setting up Qdrant for multi-vector embeddings
|
||||||
|
3. **Module 3:** Scalability issues caused by multi-vector representations
|
||||||
|
* The implications of high memory usage and ways to solve it
|
||||||
|
* Vector quantization: scalar, binary, \+1.5/2-bit
|
||||||
|
* Pooling techniques: row/column pooling, hierarchical token pooling
|
||||||
|
* Incompatibility with HNSW due to MaxSim asymmetry
|
||||||
|
* MUVERA
|
||||||
|
* Combining multiple optimizations in multi-stage retrieval via Universal Query API
|
||||||
|
* Evaluating different search pipelines in terms of cost and latency
|
||||||
|
|
||||||
|
## Project
|
||||||
|
|
||||||
|
I suggest having just one project. It should start with a set of PDFs or even scanned documents for which the participants are supposed to create a two-stage ColPali retrieval system using both MUVERA for fast retrieval and some memory optimization (or multiple ones, like SQ \+ token pooling) for reduced memory usage.
|
||||||
|
|
||||||
|
## Supplementary content
|
||||||
|
|
||||||
|
To keep the momentum, I suggest publishing a few related blog posts in the upcoming weeks:
|
||||||
|
|
||||||
|
1. **The asymmetry of MaxSim**
|
||||||
|
We suggest disabling HNSW for MaxSim, but it is never discussed why it doesn’t make sense. It’s not worth an article, but a short explanation with some examples would make things clearer.
|
||||||
|
2. **A review of modern multi-vector retrievers for visual data**
|
||||||
|
There’s been a lot of excitement around ModernVBERT, so it would make sense to present how to use it with Qdrant, even though we don’t support it in FastEmbed.
|
||||||
|
3. **FastEmbed: implementation of different pooling techniques for multi-vectors**
|
||||||
|
E.g., row/column pooling (if possible), hierarchical token pooling. Techniques might be implemented in the library itself, but we should also create a blog post about it.
|
||||||
Reference in New Issue
Block a user