mirror of
https://github.com/qdrant/landing_page.git
synced 2026-10-03 01:48:32 +02:00
Merge pull request #2194 from qdrant/fix/sparse-finetune-qdrant-org
update sparse-finetune references to qdrant org
This commit is contained in:
@@ -28,7 +28,7 @@ Search "iPhone 15 Pro Max 256GB" on a dense embedding system and it happily retu
|
||||
|
||||
This is the gap that sparse embeddings fill. And with fine-tuning, they fill it dramatically well - we achieved a **29% improvement over BM25** on Amazon's ESCI dataset, one of the largest public e-commerce search benchmarks.
|
||||
|
||||
In this series, we'll build the entire system: data loading, GPU training on Modal, evaluation with Qdrant, and hard negative mining. The [full code is on GitHub](https://github.com/thierrypdamiba/finetune-ecommerce-search) and the [fine-tuned models are on HuggingFace](https://huggingface.co/thierrydamiba/splade-ecommerce-esci). But first, let's understand why sparse embeddings are the right tool for e-commerce search.
|
||||
In this series, we'll build the entire system: data loading, GPU training on Modal, evaluation with Qdrant, and hard negative mining. The [full code is on GitHub](https://github.com/thierrypdamiba/finetune-ecommerce-search) and the [fine-tuned models are on HuggingFace](https://huggingface.co/thierrydamiba/splade-ecommerce-esci). If you want to skip the walkthrough and fine-tune on your own data, the [`sparse-finetune`](https://github.com/qdrant/sparse-finetune) CLI runs the entire pipeline with one command. But first, let's understand why sparse embeddings are the right tool for e-commerce search.
|
||||
|
||||
## The Problem with Dense Embeddings in E-Commerce
|
||||
|
||||
|
||||
@@ -22,7 +22,7 @@ category: practicle-examples
|
||||
|
||||
---
|
||||
|
||||
In the last article we made the case for sparse embeddings in e-commerce search. Now we write the code. All source code is available in the [GitHub repo](https://github.com/thierrypdamiba/finetune-ecommerce-search), and you can try the [fine-tuned models on HuggingFace](https://huggingface.co/thierrydamiba/splade-ecommerce-esci). By the end of this piece, you'll have a SPLADE model trained on Amazon's ESCI dataset, running on Modal's serverless GPUs, with checkpoints saved to persistent storage.
|
||||
In the last article we made the case for sparse embeddings in e-commerce search. Now we write the code. All source code is available in the [GitHub repo](https://github.com/thierrypdamiba/finetune-ecommerce-search), and you can try the [fine-tuned models on HuggingFace](https://huggingface.co/thierrydamiba/splade-ecommerce-esci). Want to skip straight to fine-tuning on your own data? See the [`sparse-finetune`](https://github.com/qdrant/sparse-finetune) CLI. By the end of this piece, you'll have a SPLADE model trained on Amazon's ESCI dataset, running on Modal's serverless GPUs, with checkpoints saved to persistent storage.
|
||||
|
||||
## The Dataset: Amazon ESCI
|
||||
|
||||
|
||||
@@ -22,7 +22,7 @@ category: practicle-examples
|
||||
|
||||
---
|
||||
|
||||
We have a trained SPLADE model sitting on a Modal volume (or grab it from [HuggingFace](https://huggingface.co/thierrydamiba/splade-ecommerce-esci)). Now comes the question that matters: is it actually better? In this article, we'll index products into Qdrant, run retrieval benchmarks, implement hard negative mining, and dig into what the model learned. Full evaluation code is in the [GitHub repo](https://github.com/thierrypdamiba/finetune-ecommerce-search).
|
||||
We have a trained SPLADE model sitting on a Modal volume (or grab it from [HuggingFace](https://huggingface.co/thierrydamiba/splade-ecommerce-esci)). Now comes the question that matters: is it actually better? In this article, we'll index products into Qdrant, run retrieval benchmarks, implement hard negative mining, and dig into what the model learned. Full evaluation code is in the [GitHub repo](https://github.com/thierrypdamiba/finetune-ecommerce-search). To run this entire pipeline on your own data, see the [`sparse-finetune`](https://github.com/qdrant/sparse-finetune) CLI.
|
||||
|
||||
## Indexing Products in Qdrant
|
||||
|
||||
|
||||
@@ -22,7 +22,7 @@ category: practicle-examples
|
||||
|
||||
---
|
||||
|
||||
We've built a SPLADE model that beats BM25 by 28% on Amazon ESCI. But here's the question that determines whether this is a lab result or a production strategy: does it work on data it wasn't trained on? Full code is on [GitHub](https://github.com/thierrypdamiba/finetune-ecommerce-search) and you can try the [fine-tuned models on HuggingFace](https://huggingface.co/thierrydamiba/splade-ecommerce-esci).
|
||||
We've built a SPLADE model that beats BM25 by 28% on Amazon ESCI. But here's the question that determines whether this is a lab result or a production strategy: does it work on data it wasn't trained on? Full code is on [GitHub](https://github.com/thierrypdamiba/finetune-ecommerce-search), you can try the [fine-tuned models on HuggingFace](https://huggingface.co/thierrydamiba/splade-ecommerce-esci), or fine-tune on your own catalog with the [`sparse-finetune`](https://github.com/qdrant/sparse-finetune) CLI.
|
||||
|
||||
In this final article, we test cross-domain generalization, train a multi-domain model, and lay out a decision framework for when to specialize vs generalize.
|
||||
|
||||
|
||||
@@ -26,7 +26,7 @@ In Parts 1 through 4, we built a SPLADE fine-tuning pipeline piece by piece: dat
|
||||
|
||||
Using it required reading four articles, cloning a repo, understanding the training loop internals, wiring up Modal volumes, and configuring Qdrant connections manually. That's fine for a series walkthrough. It's not fine for someone who has a product catalog and wants a better search model by end of day.
|
||||
|
||||
So we packaged everything into [`qdrant-sparse-finetune`](https://github.com/thierrypdamiba/qdrant-sparse-finetune): an open-source CLI and web dashboard that runs the entire pipeline (synthetic query generation, SPLADE training with ANCE, evaluation, and HuggingFace publishing) with a single command.
|
||||
So we packaged everything into [`qdrant-sparse-finetune`](https://github.com/qdrant/sparse-finetune): an open-source CLI and web dashboard that runs the entire pipeline (synthetic query generation, SPLADE training with ANCE, evaluation, and HuggingFace publishing) with a single command.
|
||||
|
||||
## The Problem We're Solving
|
||||
|
||||
@@ -47,7 +47,7 @@ Each step has its own configuration, its own failure modes, and its own set of a
|
||||
`qdrant-sparse-finetune` handles all of that:
|
||||
|
||||
```bash
|
||||
pip install qdrant-sparse-finetune
|
||||
pip install git+https://github.com/qdrant/sparse-finetune.git
|
||||
qdrant-finetune setup
|
||||
qdrant-finetune pipeline --data products.csv --gpu modal
|
||||
```
|
||||
@@ -179,8 +179,8 @@ Same ANCE loop from Part 3, same evaluation metrics. The `Trainer` class wraps t
|
||||
## Getting Started
|
||||
|
||||
```bash
|
||||
# Install
|
||||
pip install qdrant-sparse-finetune
|
||||
# Install from GitHub
|
||||
pip install git+https://github.com/qdrant/sparse-finetune.git
|
||||
|
||||
# Configure (interactive wizard)
|
||||
qdrant-finetune setup
|
||||
@@ -197,7 +197,7 @@ Or skip the CLI and launch the dashboard:
|
||||
qdrant-finetune studio
|
||||
```
|
||||
|
||||
The [source code is on GitHub](https://github.com/thierrypdamiba/qdrant-sparse-finetune). File issues, submit PRs, or fork it for your own use case.
|
||||
The [source code is on GitHub](https://github.com/qdrant/sparse-finetune). File issues, submit PRs, or fork it for your own use case.
|
||||
|
||||
## What's Actually Different
|
||||
|
||||
|
||||
Reference in New Issue
Block a user