Merge branch 'master' of https://github.com/qdrant/landing_page into camelai-discord-bot

qdrant-landing/content/community/community-features.md

@@ -6,7 +6,7 @@ resources:
   description: Qdrant Stars are our top contributors, organizers, and evangelists. Learn more about how you can become a Star.
   link:
     text: Learn More
-    url: /blog/qdrant-stars-announcement/
+    url: /stars/
   image:
     src: /img/community-features/qdrant-stars.svg
     alt: Avatar

qdrant-landing/content/documentation/advanced-tutorials/_index.md

@@ -16,3 +16,4 @@ partition: qdrant
 | [Use Collaborative Filtering to Build a Movie Recommendation System with Qdrant](/documentation/advanced-tutorials/collaborative-filtering/)      | 
 | [Build a Text/Image Multimodal Search System with Qdrant and FastEmbed](/documentation/advanced-tutorials/multimodal-search-fastembed/)        | 
 | [Navigate Your Codebase with Semantic Search and Qdrant](/documentation/advanced-tutorials/code-search/)                     | 
+| [Ensure optimal large-scale PDF Retrieval with Qdrant and ColPali/ColQwen](/documentation/advanced-tutorials/pdf-retrieval-at-scale/)                     | 

qdrant-landing/content/documentation/advanced-tutorials/multimodal-search-fastembed.md

@@ -1,201 +0,0 @@
----
-title: Setup Text/Image Multimodal Search
-aliases:
-  - /documentation/tutorials/multimodal-search-fastembed/
-weight: 1
----
-
-# Build a Multimodal Search System with Qdrant and FastEmbed
-
-| Time: 15 min | Level: Beginner |Output: [GitHub](https://github.com/qdrant/examples/blob/master/multimodal-search/Multimodal_Search_with_FastEmbed.ipynb)|[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/qdrant/examples/blob/master/multimodal-search/Multimodal_Search_with_FastEmbed.ipynb)   |
-| --- | ----------- | ----------- | ----------- |
-
-In this tutorial, you will set up a simple Multimodal Image & Text Search with Qdrant & FastEmbed.
-
-## Overview
-
-We often understand and share information more effectively when combining different types of data. For example, the taste of comfort food can trigger childhood memories. We might describe a song with just “pam pam clap” sounds. Instead of writing paragraphs. Sometimes, we may use emojis and stickers to express how we feel or to share complex ideas.
-
-Modalities of data such as **text**, **images**, **video** and **audio** in various combinations form valuable use cases for Semantic Search applications.
-
-Vector databases, being **modality-agnostic**, are perfect for building these applications.
-
-In this simple tutorial, we are working with two simple modalities: **image** and **text** data. However, you can create a Semantic Search application with any combination of modalities if you choose the right embedding model to bridge the **semantic gap**.
-
-> The **semantic gap** refers to the difference between low-level features (aka brightness) and high-level concepts (aka cuteness).
-
-For example, the [ImageBind model](https://github.com/facebookresearch/ImageBind) from Meta AI is said to bind all 4 mentioned modalities in one shared space.
-
-## Prerequisites
-
-> **Note**: The code for this tutorial can be found [here](https://github.com/qdrant/examples/multimodal-search)
-
-To complete this tutorial, you will need either Docker to run a pre-built Docker image of Qdrant and Python version ≥ 3.8 or a Google Collab Notebook if you don't want to install anything locally. 
-
-We showed how to run Qdrant in Docker in the ["Create a Simple Neural Search Service"](https://qdrant.tech/documentation/tutorials/neural-search/) Tutorial.
-
-## Setup
-
-First, install the required libraries `qdrant-client`, `fastembed` and `Pillow`.
-For example, with the `pip` package manager, it can be done in the following way.
-
-```bash
-python3 -m pip install --upgrade qdrant-client fastembed Pillow
-```
-
-<aside role="status">
-We will use <a href="https://qdrant.tech/documentation/fastembed/">FastEmbed</a> for generating multimodal embeddings and <b>Qdrant</b> for storing and retrieving them.
-</aside>
-
-## Dataset
-To make the demonstration simple, we created a tiny dataset of images and their captions for you.
-
-Images can be downloaded from [here](https://github.com/qdrant/examples/multimodal-search/images).
-It's **important** to place them in the same folder as your code/notebook, in the folder named `images`.
-
-You can check out how images look like in the following way:
-```python
-from PIL import Image
-
-Image.open('images/lizard.jpg')
-```
-## Vectorize data
-
-`FastEmbed` supports **Contrastive Language–Image Pre-training** ([CLIP](https://openai.com/index/clip/)) model, the old (2021) but gold classics of multimodal Image-Text Machine Learning. 
-**CLIP** model was one of the first models of such kind with ZERO-SHOT capabilities.
-
-When using it for semantic search, it's important to remember that the textual encoder of CLIP is trained to process no more than **77 tokens**, 
-so CLIP is good for short texts.
-
-Let's embed a very short selection of images and their captions in the **shared embedding space** with CLIP.
-
-```python
-from fastembed import TextEmbedding, ImageEmbedding
-
-documents = [{"caption": "A photo of a cute pig",
-              "image": "images/piggy.jpg"},
- {"caption": "A picture with a coffee cup",
-              "image": "images/coffee.jpg"},
- {"caption": "A photo of a colourful lizard",
-              "image": "images/lizard.jpg"}
-]
-
-text_model_name = "Qdrant/clip-ViT-B-32-text" #CLIP text encoder
-text_model = TextEmbedding(model_name=text_model_name)
-text_embeddings_size = text_model._get_model_description(text_model_name)["dim"] #dimension of text embeddings, produced by CLIP text encoder (512)
-texts_embeded = list(text_model.embed([document["caption"] for document in documents])) #embedding captions with CLIP text encoder
-
-image_model_name = "Qdrant/clip-ViT-B-32-vision" #CLIP image encoder
-image_model = ImageEmbedding(model_name=image_model_name)
-image_embeddings_size = image_model._get_model_description(image_model_name)["dim"] #dimension of image embeddings, produced by CLIP image encoder (512)
-images_embeded = list(image_model.embed([document["image"] for document in documents]))  #embedding images with CLIP image encoder
-```
-
-## Upload data to Qdrant
-
-1. **Create a client object for Qdrant**.
-
-```python
-from qdrant_client import QdrantClient, models
-
-client = QdrantClient("http://localhost:6333") #or QdrantClient(":memory:") if you're using Google Collab, this option is suitable only for simple prototypes/demos with Python client
-```
-
-2. **Create a new collection for your images with captions**.
-
-CLIP’s weights were trained to maximize the scaled **Cosine Similarity** of truly corresponding image/caption pairs,
-so that's the **Distance Metric** we will choose for our [Collection](https://qdrant.tech/documentation/concepts/collections/) of [Named Vectors](https://qdrant.tech/documentation/concepts/collections/#collection-with-multiple-vectors).
-
-Using **Named Vectors**, we can easily showcase both Text-to-Image and Image-to-Text (Image-to-Image and Text-to-Text) search.
-
-```python
-if not client.collection_exists("text_image"): #creating a Collection
- client.create_collection(
-        collection_name ="text_image",
-        vectors_config={ #Named Vectors
-            "image": models.VectorParams(size=image_embeddings_size, distance=models.Distance.COSINE),
-            "text": models.VectorParams(size=text_embeddings_size, distance=models.Distance.COSINE),
- }
- )
-```
-3. **Upload our images with captions to the Collection**.
-
-Each image with its caption will create a [Point](https://qdrant.tech/documentation/concepts/points/) in Qdrant.
-
-```python
-client.upload_points(
-    collection_name="text_image",
-    points=[
- models.PointStruct(
-            id=idx, #unique id of a point, pre-defined by the user
-            vector={
-                "text": texts_embeded[idx], #embeded caption
-                "image": images_embeded[idx] #embeded image
- },
-            payload=doc #original image and its caption
- )
-        for idx, doc in enumerate(documents)
- ]
-)
-```
-
-## Search
-
-<h3 style="font-size: 1.25em;">Text-to-Image</h3>
-
-Let's see what image we will get to the query "*What would make me energetic in the morning?*"
-
-```python
-from PIL import Image
-
-find_image = text_model.embed(["What would make me energetic in the morning?"]) #query, we embed it, so it also becomes a vector
-
-Image.open(client.search(
-    collection_name="text_image", #searching in our collection
-    query_vector=("image", list(find_image)[0]), #searching only among image vectors with our textual query
-    with_payload=["image"], #user-readable information about search results, we are interested to see which image we will find
-    limit=1 #top-1 similar to the query result
-)[0].payload['image'])
-```
-**Response:**
-
-![Coffee Image](/docs/coffee.jpg)
-
-### Image-to-Text
-Now, let's do a reverse search with an image:
-
-
-```python
-from PIL import Image
-
-Image.open('images/piglet.jpg')
-```
-![Piglet Image](/docs/piglet.jpg)
-
-Let's see what caption we will get, searching by this piglet image, which, as you can check, is not in our **Collection**.
-
-```python
-find_image = image_model.embed(['images/piglet.jpg']) #embedding our image query
-
-client.search(
-    collection_name="text_image",
-    query_vector=("text", list(find_image)[0]), #now we are searching only among text vectors with our image query
-    with_payload=["caption"], #user-readable information about search results, we are interested to see which caption we will get
-    limit=1
-)[0].payload['caption']
-```
-**Response:**
-```text
-'A photo of a cute pig'
-```
-
-## Next steps
-
-Use cases of even just Image & Text Multimodal Search are countless: E-Commerce, Media Management, Content Recommendation, Emotion Recognition Systems, Biomedical Image Retrieval, Spoken Sign Language Transcription, etc.
-
-Imagine a scenario: user wants to find a product similar to a picture they have, but they also have specific textual requirements, like "*in beige colour*".
-You can search using just texts or images and combine their embeddings in a **late fusion manner** (summing and weighting might work surprisingly well).
-
-Moreover, using [Discovery Search](https://qdrant.tech/articles/discovery-search/) with both modalities, you can provide users with information that is impossible to retrieve unimodally!
-
-Join our [Discord community](https://qdrant.to/discord), where we talk about vector search and similarity learning, experiment, and have fun!

qdrant-landing/content/documentation/advanced-tutorials/pdf-retrieval-at-scale.md

@@ -0,0 +1,343 @@
+---
+title: Scaling PDF Retrieval with Qdrant
+aliases:
+  - /documentation/tutorials/pdf-retrieval-at-scale/
+short_description: "Optimizing PDF retrieval at scale with Qdrant and Vision Large Language Models (VLLMs) such as ColPali and ColQwen."
+description: "Optimizing PDF retrieval at scale with Qdrant and Vision Large Language Models (VLLMs) such as ColPali and ColQwen. Two-stage retrieval with multivector representations mean pooling."
+weight: 4
+---
+
+# Scaling PDF Retrieval with Qdrant
+
+| Time: 30 min | Level: Intermediate |Output: [GitHub](https://github.com/qdrant/examples/blob/master/pdf-retrieval-at-scale/ColPali_ColQwen2_Tutorial.ipynb)|[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/qdrant/examples/blob/master/pdf-retrieval-at-scale/ColPali_ColQwen2_Tutorial.ipynb)   |
+| --- | ----------- | ----------- | ----------- |
+
+Efficient PDF documents retrieval is a common requirement in tasks like **(agentic) retrieval-augmented generation (RAG)** and many other search-based applications. At the same time, setting up PDF documents retrieval is rarely possible without additional challenges. 
+
+Many traditional PDF retrieval solutions rely on **optical character recognition (OCR)** together with use case-specific heuristics to handle visually complex elements like tables, images and charts. These algorithms are often non-transferable -- even within the same domain -- with their task-customized parsing and chunking strategies, labor-intensive, prone to errors, and difficult to scale.
+
+Recent advancements in **Vision Large Language Models (VLLMs)**, such as [**ColPali**](https://huggingface.co/blog/manu/colpali) and its successor [**ColQwen**](https://huggingface.co/vidore/colqwen2-v0.1), started the transformation of the PDF retrieval. These multimodal models work directly with PDF pages as inputs, no pre-processing required. Anything that can be converted into an **image** (think of PDFs as screenshots of document pages) can be effectively processed by these models. Being far simpler in use, VLLMs achieve state-of-the-art performance in PDF retrieval benchmarks like the [Visual Document Retrieval (ViDoRe) Benchmark](https://huggingface.co/spaces/vidore/vidore-leaderboard).
+
+## How VLLMs Work for PDF Retrieval
+
+VLLMs like **ColPali** and **ColQwen** generate **multivector representations** for each PDF page; the representations are stored and indexed in a vector database. During the retrieval process, models dynamically create multivector representations for (textual) user queries, and precise retrieval -- matching between PDF pages and queries -- is achieved through [late-interaction mechanism](/blog/qdrant-colpali/#how-colpali-works-under-the-hood).
+
+<aside role="status"> Qdrant supports <a href="/documentation/concepts/vectors/#multivectors">multivector representations</a>, making it well-suited for using embedding models such as ColPali, ColQwen, or <a href="/documentation/fastembed/fastembed-colbert/">ColBERT</a></aside>
+
+## Challenges of Scaling VLLMs
+
+The heavy multivector representations produced by VLLMs make PDF retrieval at scale computationally intensive. These models are inefficient for large-scale PDF retrieval tasks if used without optimization.
+
+### Math Behind the Scaling
+
+**ColPali** generates over **1,000 vectors per PDF page**, while its successor, **ColQwen**, generates slightly fewer — up to **768 vectors**, dynamically adjusted based on the image size. Typically, ColQwen produces **~700 vectors per page**.
+
+To understand the impact, consider the construction of an [**HNSW index**](/articles/what-is-a-vector-database/#1-indexing-hnsw-index-and-sending-data-to-qdrant), a common indexing algorithm for vector databases. Let's roughly estimate the number of comparisons needed to insert a new PDF page into the index.
+
+- **Vectors per page:** ~700 (ColQwen) or ~1,000 (ColPali)
+- **[ef_construct](/documentation/concepts/indexing/#vector-index):** 100 (default)
+
+The lower bound estimation for the number of vector comparisions comparisons would be:
+
+$$
+700 \times 700 \times 100 = 49 \ \text{millions}
+$$
+
+Now imagine how much it will take to build an index on **20,000 pages**!
+
+For ColPali, this number doubles. The result is **extremely slow index construction time**.
+
+### Our Solution
+
+We recommend reducing the number of vectors in a PDF page representation for the **first-stage retrieval**. After the first stage retrieval with a reduced amount of vectors, we propose to **rerank** retrieved subset with the original uncompressed representation.
+
+<aside role="status"> You might consider using <b>quantization</b> (e.g., binary quantization) to reduce computational resources. However, as you can see above, quantization does not impact the parameters that determine the number of comparisons, so it will only affect memory consumption.</aside>
+
+The reduction of vectors can be achieved by applying a **mean pooling operation** to the multivector VLLM-generated outputs. Mean pooling averages the values across all vectors within a selected subgroup, condensing multiple vectors into a single representative vector. If done right, it allows the preservation of important information from the original page while significantly reducing the number of vectors.
+
+VLLMs generate vectors corresponding to patches that represent different portions of a PDF page. These patches can be grouped in columns and rows of a PDF page.
+
+For example:
+- ColPali divides PDF page into **1,024 patches**.
+- Applying mean pooling by rows (or columns) of this patch matrix reduces the page representation to just **32 vectors**.
+
+![ColPali patching of a PDF page](/documentation/tutorials/pdf-retrieval-at-scale/pooling-by-rows.png)
+
+We tested this approach with the ColPali model, mean pooling its multivectors by PDF page rows. The results showed:
+- **Indexing time faster by an order of magnitude**
+- **Retrieval quality comparable to the original model**
+
+For details of this experiment refer to our [gitHub repository](https://github.com/qdrant/demo-colpali-optimized), [ColPali optimization blog post](/blog/colpali-qdrant-optimization/) or [webinar "PDF Retrieval at Scale"](https://www.youtube.com/watch?v=_h6SN1WwnLs)
+
+## Goal of This Tutorial
+
+In this tutorial, we will demonstrate a scalable approach to PDF retrieval using **Qdrant** and **ColPali** & **ColQwen2** VLLMs.
+The presented approach is **highly recommended** to avoid the common pitfalls of long indexing times and slow retrieval speeds.
+
+In the following sections, we will demonstrate an optimized retrieval algorithm born out of our successful experimentation:
+
+**First-Stage Retrieval with Mean-Pooled Vectors:**
+   - Construct an HNSW index using **only mean-pooled vectors**.
+   - Use them for the first-stage retrieval.
+
+**Reranking with Original Model Multivectors:**
+   - Use the original multivectors from ColPali or ColQwen2 **to rerank** the results retrieved in the first stage.
+
+## Setup
+Install & import required libraries
+
+```python
+# pip install colpali_engine>=0.3.1
+from colpali_engine.models import ColPali, ColPaliProcessor
+# pip install qdrant-client>=1.12.0
+from qdrant_client import QdrantClient, models
+```
+
+To run these experiments, we’re using a **Qdrant cluster**. If you’re just getting started, you can set up a **free-tier cluster** for testing and exploration. Follow the instructions in the documentation ["How to Create a Free-Tier Qdrant Cluster"](/documentation/cloud/create-cluster/#free-clusters)
+
+```python
+client = QdrantClient(
+    url=<YOUR CLUSTER URL>,
+    api_key=<YOUR API KEY>
+)
+```
+
+Download **ColPali** model along with its input processors. Make sure to select the backend that suits your setup.
+
+```python
+colpali_model = ColPali.from_pretrained(
+        "vidore/colpali-v1.3",
+        torch_dtype=torch.bfloat16,
+        device_map="mps",  # Use "cuda:0" for GPU, "cpu" for CPU, or "mps" for Apple Silicon
+    ).eval()
+
+colpali_processor = ColPaliProcessor.from_pretrained("vidore/colpali-v1.3")
+```
+
+<details>
+<summary> For <b>ColQwen</b> model </summary>
+
+```python
+from colpali_engine.models import ColQwen2, ColQwen2Processor
+
+colqwen_model = ColQwen2.from_pretrained(
+        "vidore/colqwen2-v0.1",
+        torch_dtype=torch.bfloat16,
+        device_map="mps", # Use "cuda:0" for GPU, "cpu" for CPU, or "mps" for Apple Silicon
+    ).eval()
+
+colqwen_processor = ColQwen2Processor.from_pretrained("vidore/colqwen2-v0.1")
+```
+</details>
+
+
+
+## Create Qdrant Collections
+
+We can now create a collection in Qdrant to store the multivector representations of PDF pages generated by **ColPali** or **ColQwen**.
+
+Collection will include **mean pooled** by rows and columns representations of a PDF page, as well as the **original** multivector representation.
+
+<aside role="status"> For the original multivectors generated by the models, we will disable HNSW index construction </aside>
+
+```python
+client.create_collection(
+    collection_name=collection_name,
+    vectors_config={
+        "original": 
+            models.VectorParams( #switch off HNSW
+                    size=128,
+                    distance=models.Distance.COSINE,
+                    multivector_config=models.MultiVectorConfig(
+                        comparator=models.MultiVectorComparator.MAX_SIM
+                    ),
+                    hnsw_config=models.HnswConfigDiff(
+                        m=0 #switching off HNSW
+                    )
+            ),
+        "mean_pooling_columns": models.VectorParams(
+                size=128,
+                distance=models.Distance.COSINE,
+                multivector_config=models.MultiVectorConfig(
+                    comparator=models.MultiVectorComparator.MAX_SIM
+                )
+            ),
+        "mean_pooling_rows": models.VectorParams(
+                size=128,
+                distance=models.Distance.COSINE,
+                multivector_config=models.MultiVectorConfig(
+                    comparator=models.MultiVectorComparator.MAX_SIM
+                )
+            )
+    }
+)
+```
+## Choose a dataset
+
+We’ll use the **UFO Dataset** by Daniel van Strien for this tutorial. It’s available on Hugging Face; you can download it directly from there. 
+
+```python
+from datasets import load_dataset
+ufo_dataset = "davanstrien/ufo-ColPali"
+dataset = load_dataset(ufo_dataset, split="train")
+```
+
+## Embedding and Mean Pooling
+
+We'll use a function that generates multivector representations and their mean pooled versions of each PDF page (aka image) in batches.
+For complete understanding, it's important to consider the following specifics of **ColPali** and **ColQwen**:
+
+**ColPali:**
+In theory, ColPali is designed to generate 1,024 vectors per PDF page, but in practice, it produces 1,030 vectors. This discrepancy is due to ColPali's pre-processor, which appends the text `<bos>Describe the image.` to each input. This additional text generates an extra 6 multivectors.
+
+**ColQwen:**
+ColQwen dynamically determines the number of patches in "rows and columns" of a PDF page based on its size. Consequently, the number of multivectors can vary between inputs. ColQwen pre-processor prepends `<|im_start|>user<|vision_start|>` and appends `<|vision_end|>Describe the image.<|im_end|><|endoftext|>`.
+
+For example, that's how ColQwen multivector output is formed.
+
+![that's how ColQwen multivector output is formed](/documentation/tutorials/pdf-retrieval-at-scale/ColQwen-preprocessing.png)
+
+The `get_patches` function is to get the number of `x_patches` (rows) and `y_patches` (columns) ColPali/ColQwen2 models will divide a PDF page into.
+For ColPali, the numbers will always be 32 by 32; ColQwen will define them dynamically based on the PDF page size.
+
+```python
+x_patches, y_patches = model_processor.get_n_patches(
+    image_size, 
+    patch_size=model.patch_size
+)
+```
+
+<details>
+<summary> For <b>ColQwen</b> model </summary>
+
+```python
+model_processor.get_n_patches(
+    image_size, 
+    patch_size=model.patch_size,
+    spatial_merge_size=model.spatial_merge_size
+)
+```
+</details>
+
+
+We choose to **preserve prefix and postfix multivectors**. Our **pooling** operation compresses the multivectors representing **the image tokens** based on the number of rows and columns determined by the model (static 32x32 for ColPali, dynamic XxY for ColQwen). Function retains and integrates the additional multivectors produced by the model back to pooled representations.
+
+
+Simplified version of pooling for **ColPali** model:
+
+(see the full version -- also applicable for **ColQwen** -- in the [tutorial notebook](https://githubtocolab.com/qdrant/examples/blob/master/pdf-retrieval-at-scale/ColPali_ColQwen2_Tutorial.ipynb))
+
+```python
+
+processed_images = model_processor.process_images(image_batch) 
+# Image embeddings of shape (batch_size, 1030, 128)
+image_embeddings = model(**processed_images)
+
+# (1030, 128)
+image_embedding = image_embeddings[0] # take the first element of the batch
+
+# Now we need to identify vectors that correspond to the image tokens
+# It can be done by selecting tokens corresponding to special `image_token_id`
+
+# (1030, ) - boolean mask (for the first element in the batch), True for image tokens 
+mask = processed_images.input_ids[0] == model_processor.image_token_id
+
+# For convenience, we now select only image tokens 
+#   and reshape them to (x_patches, y_patches, dim)
+
+# (x_patches, y_patches, 128)
+image_tokens = image_embedding[mask].view(x_patches, y_patches, model.dim)
+
+# Now we can apply mean pooling by rows and columns
+
+# (x_patches, 128)
+pooled_by_rows = image_tokens.mean(dim=0)
+
+# (y_patches, 128)
+pooled_by_columns = image_tokens.mean(dim=1)
+
+# [Optionally] we can also concatenate special tokens to the pooled representations, 
+# For ColPali, it's only postfix
+
+# (x_patches + 6, 128)
+pooled_by_rows = torch.cat([pooled_by_rows, image_embedding[~mask]])
+
+# (y_patches + 6, 128)
+pooled_by_columns = torch.cat([pooled_by_columns, image_embedding[~mask]])
+```
+
+</details>
+
+## Upload to Qdrant
+
+The upload process is trivial; the only thing to pay attention to is the compute cost for ColPali and ColQwen2 models. 
+In low-resource environments, it's recommended to use a smaller batch size for embedding and mean pooling.
+
+Full version of the upload code is available in the [tutorial notebook](https://githubtocolab.com/qdrant/examples/blob/master/pdf-retrieval-at-scale/ColPali_ColQwen2_Tutorial.ipynb)
+
+
+## Querying PDFs
+
+After indexing PDF documents, we can move on to querying them using our two-stage retrieval approach.
+
+```python
+query = "Lee Harvey Oswald's involvement in the JFK assassination"
+processed_queries = model_processor.process_queries([query]).to(model.device)
+
+# Resulting query embedding is a tensor of shape (22, 128)
+query_embedding = model(**processed_queries)[0]
+```
+
+Now let's design a function for the two-stage retrieval with multivectors produced by VLLMs:
+
+- **Step 1:** Prefetch results using a compressed multivector representation & HNSW index.
+- **Step 2:** Re-rank the prefetched results using the original multivector representation.
+
+Let's query our collections using combined mean pooled representations for the first stage of retrieval.
+
+```python
+# Final amount of results to return
+search_limit = 10
+# Amount of results to prefetch for reranking
+prefetch_limit = 100
+
+response = client.query_points(
+    collection_name=collection_name,
+    query=query_embedding,
+    prefetch=[
+        models.Prefetch(
+            query=query_embedding,
+            limit=prefetch_limit,
+            using="mean_pooling_columns"
+        ),
+        models.Prefetch(
+            query=query_embedding,
+            limit=prefetch_limit,
+            using="mean_pooling_rows"
+        ),
+    ],
+    limit=search_limit,
+    with_payload=True,
+    with_vector=False,
+    using="original"
+)
+```
+
+And check the top retrieved result to our query *"Lee Harvey Oswald's involvement in the JFK assassination"*.
+
+```python
+dataset[response.points[0].payload['index']]['image']
+```
+
+![Results, ColPali](/documentation/tutorials/pdf-retrieval-at-scale/result-VLLMs.png)
+
+
+## Conclusion
+
+In this tutorial, we demonstrated an optimized approach using **Qdrant for PDF retrieval at scale** with VLLMs producing **heavy multivector representations** like **ColPali** and **ColQwen2**.
+
+Without such optimization, the performance of retrieval systems can degrade severely, both in terms of indexing time and query latency, especially as the dataset size grows.
+
+We **strongly recommend** implementing this approach in your workflows to ensure efficient and scalable PDF retrieval. Neglecting to optimize the retrieval process could result in unacceptably slow performance, hindering the usability of your system.
+
+Start scaling your PDF retrieval today!

qdrant-landing/content/documentation/agentic-rag-langgraph.md

@@ -50,6 +50,25 @@ Ready to start building this system from the ground up? Let’s get to it!
 
 Before we dive into building our agent, let’s get everything set up. 
 
+### Imports
+
+Here’s a list of key imports required:
+
+```python
+import os
+import json
+from typing import Annotated, TypedDict
+from dotenv import load_dotenv
+from langchain.embeddings import OpenAIEmbeddings
+from langgraph import StateGraph, tool, ToolNode, ToolMessage
+from langchain.document_loaders import HuggingFaceDatasetLoader
+from langchain.text_splitter import RecursiveCharacterTextSplitter
+from langchain.llms import ChatOpenAI
+from qdrant_client import QdrantClient
+from qdrant_client.http.models import VectorParams
+from brave_search import BraveSearch
+```
+
 ### Qdrant Vector Database Setup
 
 We’ll use **Qdrant Cloud** as our vector store for document embeddings. Here’s how to set it up:

qdrant-landing/content/documentation/multimodal-search.md

@@ -0,0 +1,193 @@
+---
+title: Multilingual & Multimodal RAG with LlamaIndex
+weight: 14
+partition: build
+social_preview_image: /documentation/examples/multimodal-search/social_preview.png
+aliases:
+  - /documentation/tutorials/multimodal-search-fastembed/
+  - /documentation/advanced-tutorials/multimodal-search-fastembed/
+---
+
+# Multilingual & Multimodal Search with LlamaIndex
+
+![Snow prints](/documentation/examples/multimodal-search/image-1.png)
+
+| Time: 15 min | Level: Beginner |Output: [GitHub](https://github.com/qdrant/examples/blob/master/multimodal-search/Multimodal_Search_with_LlamaIndex.ipynb)|[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://githubtocolab.com/qdrant/examples/blob/master/multimodal-search/Multimodal_Search_with_LlamaIndex.ipynb)   |
+| --- | ----------- | ----------- | ----------- |
+
+## Overview
+
+We often understand and share information more effectively when combining different types of data. For example, the taste of comfort food can trigger childhood memories. We might describe a song with just “pam pam clap” sounds. Instead of writing paragraphs. Sometimes, we may use emojis and stickers to express how we feel or to share complex ideas.
+
+Modalities of data such as **text**, **images**, **video** and **audio** in various combinations form valuable use cases for Semantic Search applications.
+
+Vector databases, being **modality-agnostic**, are perfect for building these applications.
+
+In this simple tutorial, we are working with two simple modalities: **image** and **text** data. However, you can create a Semantic Search application with any combination of modalities if you choose the right embedding model to bridge the **semantic gap**.
+
+> The **semantic gap** refers to the difference between low-level features (aka brightness) and high-level concepts (aka cuteness).
+
+For example, the [vdr-2b-multi-v1 model](https://huggingface.co/llamaindex/vdr-2b-multi-v1) from LlamaIndex is designed for multilingual embedding, particularly effective for visual document retrieval across multiple languages and domains. It allows for searching and querying visually rich multilingual documents without the need for OCR or other data extraction pipelines.
+
+## Setup
+
+First, install the required libraries `qdrant-client` and `llama-index-embeddings-huggingface`.
+
+```bash
+pip install qdrant-client llama-index-embeddings-huggingface
+```
+
+<aside role="status">
+The code for this tutorial can be found <a href="https://github.com/qdrant/examples/multimodal-search">here</a>.
+</aside>
+
+## Dataset
+
+To make the demonstration simple, we created a tiny dataset of images and their captions for you.
+
+Images can be downloaded from [here](https://github.com/qdrant/examples/tree/master/multimodal-search/images). It's **important** to place them in the same folder as your code/notebook, in the folder named `images`.
+
+## Vectorize data
+
+`LlamaIndex`'s `vdr-2b-multi-v1` model supports cross-lingual retrieval, allowing for effective searches across languages and domains. It encodes document page screenshots into dense single-vector representations, eliminating the need for OCR and other complex data extraction processes.
+
+Let's embed the images and their captions in the **shared embedding space**.
+
+```python
+from llama_index.embeddings.huggingface import HuggingFaceEmbedding
+
+model = HuggingFaceEmbedding(
+    model_name="llamaindex/vdr-2b-multi-v1",
+    device="cpu",  # "mps" for mac, "cuda" for nvidia GPUs
+    trust_remote_code=True,
+)
+
+documents = [
+    {"caption": "An image about plane emergency safety.", "image": "images/image-1.png"},
+    {"caption": "An image about airplane components.", "image": "images/image-2.png"},
+    {"caption": "An image about COVID safety restrictions.", "image": "images/image-3.png"},
+    {"caption": "An confidential image about UFO sightings.", "image": "images/image-4.png"},
+    {"caption": "An image about unusual footprints on Aralar 2011.", "image": "images/image-5.png"},
+]
+
+text_embeddings = model.get_text_embedding_batch([doc["caption"] for doc in documents])
+image_embeddings = model.get_image_embedding_batch([doc["image"] for doc in documents])
+```
+
+## Upload data to Qdrant
+
+1. **Create a client object for Qdrant**.
+
+```python
+from qdrant_client import QdrantClient, models
+
+# docker run -p 6333:6333 qdrant/qdrant
+client = QdrantClient(url="http://localhost:6333/")
+```
+
+2. **Create a new collection for the images with captions**.
+
+```python
+COLLECTION_NAME = "llama-multi"
+
+if not client.collection_exists(COLLECTION_NAME):
+    client.create_collection(
+        collection_name=COLLECTION_NAME,
+        vectors_config={
+            "image": models.VectorParams(size=len(image_embeddings[0]), distance=models.Distance.COSINE),
+            "text": models.VectorParams(size=len(text_embeddings[0]), distance=models.Distance.COSINE),
+        }
+    )
+```
+
+3. **Upload our images with captions to the Collection**.
+
+```python
+client.upload_points(
+    collection_name=COLLECTION_NAME,
+    points=[
+        models.PointStruct(
+            id=idx,
+            vector={
+                "text": text_embeddings[idx],
+                "image": image_embeddings[idx],
+            },
+            payload=doc
+        )
+        for idx, doc in enumerate(documents)
+    ]
+)
+```
+
+## Search
+
+### Text-to-Image
+
+Let's see what image we will get to the query "*Adventures on snow hills*".
+
+```python
+from PIL import Image
+
+find_image = model.get_query_embedding("Adventures on snow hills")
+
+Image.open(client.query_points(
+    collection_name=COLLECTION_NAME,
+    query=find_image,
+    using="image",
+    with_payload=["image"],
+    limit=1
+).points[0].payload['image'])
+```
+
+Let's also run the same query in Italian and compare the results.
+
+### Multilingual Search
+
+Now, let's do a multilingual search using an Italian query:
+
+```python
+Image.open(client.query_points(
+    collection_name=COLLECTION_NAME,
+    query=model.get_query_embedding("Avventure sulle colline innevate"),
+    using="image",
+    with_payload=["image"],
+    limit=1
+).points[0].payload['image'])
+```
+
+**Response:**
+
+![Snow prints](/documentation/advanced-tutorials/snow-prints.png)
+
+### Image-to-Text
+
+Now, let's do a reverse search with the following image:
+
+![Airplane](/documentation/advanced-tutorials/airplane.png)
+
+```python
+client.query_points(
+    collection_name=COLLECTION_NAME,
+    query=model.get_image_embedding("images/image-2.png"),  
+    # Now we are searching only among text vectors with our image query
+    using="text",
+    with_payload=["caption"],
+    limit=1
+).points[0].payload['caption']
+```
+
+**Response:**
+
+```text
+'An image about plane emergency safety.'
+```
+
+## Next steps
+
+Use cases of even just Image & Text Multimodal Search are countless: E-Commerce, Media Management, Content Recommendation, Emotion Recognition Systems, Biomedical Image Retrieval, Spoken Sign Language Transcription, etc.
+
+Imagine a scenario: a user wants to find a product similar to a picture they have, but they also have specific textual requirements, like "*in beige colour*". You can search using just texts or images and combine their embeddings in a **late fusion manner** (summing and weighting might work surprisingly well).
+
+Moreover, using [Discovery Search](/articles/discovery-search/) with both modalities, you can provide users with information that is impossible to retrieve unimodally!
+
+Join our [Discord community](https://qdrant.to/discord), where we talk about vector search and similarity learning, experiment, and have fun!

qdrant-landing/content/headless/main/hero.md

@@ -5,8 +5,8 @@ startFree:
   text: Get Started
   url: https://cloud.qdrant.io/
 learnMore:
-  text: Talk to Sales
-  url: /contact-sales/
+  text: Learn More
+  url: /qdrant-vector-database/
 heroImageSources:
   - minWidth: 2881px
     srcset: /img/hero-home-illustration-x3.webp