mirror of
https://github.com/qdrant/landing_page.git
synced 2026-10-08 20:38:31 +02:00
Fix SPLADE architecture order in parts 1 & 2
Log saturation (ReLU + log1p) is applied per-token before max pooling across positions, not after. Fix text descriptions and pipeline diagram. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
co-authored by
Claude Sonnet 4.6
parent
4a727882dc
commit
f4d989b1cb
@@ -62,13 +62,13 @@ The key difference: each dimension in a sparse vector corresponds to an actual w
|
||||
|
||||
## SPLADE: Learned Sparse Representations
|
||||
|
||||
SPLADE (Sparse Lexical and Expansion) is the model architecture that makes this work. It passes text through a transformer with a [masked language model](https://huggingface.co/docs/transformers/tasks/masked_language_modeling) (MLM) head, then applies max pooling and log saturation to produce sparse weights:
|
||||
SPLADE (Sparse Lexical and Expansion) is the model architecture that makes this work. It passes text through a transformer with a [masked language model](https://huggingface.co/docs/transformers/tasks/masked_language_modeling) (MLM) head, then applies log saturation and max pooling to produce sparse weights:
|
||||
|
||||
For an input like `"noise canceling headphones"`, SPLADE encodes it in four steps:
|
||||
|
||||
1. **Tokenize and encode** the input through DistilBERT with a masked language model (MLM) head
|
||||
2. **Max pool** across all token positions to get a single score per vocabulary term
|
||||
3. **Apply log saturation** — `log(1 + ReLU(x))` — a learned version of BM25's saturation curve that prevents any single term from dominating
|
||||
2. **Apply log saturation** — `log(1 + ReLU(x))` — a learned version of BM25's saturation curve that prevents any single term from dominating
|
||||
3. **Max pool** across all token positions to get a single score per vocabulary term
|
||||
4. **Output a sparse vector** with ~200 non-zero values out of 30,522 vocabulary dimensions
|
||||
|
||||

|
||||
|
||||
Reference in New Issue
Block a user