diff --git a/qdrant-landing/content/articles/sparse-embeddings-ecommerce-part-1.md b/qdrant-landing/content/articles/sparse-embeddings-ecommerce-part-1.md index 45fb0beac..b0b2c5768 100644 --- a/qdrant-landing/content/articles/sparse-embeddings-ecommerce-part-1.md +++ b/qdrant-landing/content/articles/sparse-embeddings-ecommerce-part-1.md @@ -62,13 +62,13 @@ The key difference: each dimension in a sparse vector corresponds to an actual w ## SPLADE: Learned Sparse Representations -SPLADE (Sparse Lexical and Expansion) is the model architecture that makes this work. It passes text through a transformer with a [masked language model](https://huggingface.co/docs/transformers/tasks/masked_language_modeling) (MLM) head, then applies max pooling and log saturation to produce sparse weights: +SPLADE (Sparse Lexical and Expansion) is the model architecture that makes this work. It passes text through a transformer with a [masked language model](https://huggingface.co/docs/transformers/tasks/masked_language_modeling) (MLM) head, then applies log saturation and max pooling to produce sparse weights: For an input like `"noise canceling headphones"`, SPLADE encodes it in four steps: 1. **Tokenize and encode** the input through DistilBERT with a masked language model (MLM) head -2. **Max pool** across all token positions to get a single score per vocabulary term -3. **Apply log saturation** — `log(1 + ReLU(x))` — a learned version of BM25's saturation curve that prevents any single term from dominating +2. **Apply log saturation** — `log(1 + ReLU(x))` — a learned version of BM25's saturation curve that prevents any single term from dominating +3. **Max pool** across all token positions to get a single score per vocabulary term 4. **Output a sparse vector** with ~200 non-zero values out of 30,522 vocabulary dimensions ![The SPLADE encoding pipeline from input text to sparse vector](/articles_data/sparse-embeddings-ecommerce-part-1/splade-pipeline.png) diff --git a/qdrant-landing/content/articles/sparse-embeddings-ecommerce-part-2.md b/qdrant-landing/content/articles/sparse-embeddings-ecommerce-part-2.md index c021c0978..05fa3d5a9 100644 --- a/qdrant-landing/content/articles/sparse-embeddings-ecommerce-part-2.md +++ b/qdrant-landing/content/articles/sparse-embeddings-ecommerce-part-2.md @@ -154,7 +154,7 @@ No S3 uploads, no checkpoint management code, no lost training runs. Sentence Transformers v5 introduced `SparseEncoder`, making SPLADE training straightforward. The model has two components: 1. **MLMTransformer**: A transformer with a masked language model head that outputs logits over the full vocabulary -2. **SpladePooling**: Max-pools the token-level logits and applies ReLU + log saturation +2. **SpladePooling**: Applies ReLU + log saturation to the token-level logits and max-pools across positions ```python from sentence_transformers import SparseEncoder diff --git a/qdrant-landing/static/articles_data/sparse-embeddings-ecommerce-part-1/splade-pipeline.png b/qdrant-landing/static/articles_data/sparse-embeddings-ecommerce-part-1/splade-pipeline.png index 15467b637..48c95266d 100644 Binary files a/qdrant-landing/static/articles_data/sparse-embeddings-ecommerce-part-1/splade-pipeline.png and b/qdrant-landing/static/articles_data/sparse-embeddings-ecommerce-part-1/splade-pipeline.png differ