docs: Updated FAQ and Query API usage (#1026)

Co-authored-by: Kacper Łukawski <kacperlukawski@users.noreply.github.com>
This commit is contained in:
Anush
2024-07-17 14:31:51 +05:30
committed by GitHub
co-authored by Kacper Łukawski
parent 9befc97d50
commit aec0935542
4 changed files with 9 additions and 12 deletions
@@ -65,7 +65,7 @@ from fastembed import TextEmbedding, SparseTextEmbedding
def vectorize(partition_data): def vectorize(partition_data):
# Initialize dense and sparse models # Initialize dense and sparse models
dense_model = TextEmbedding(model_name="BAAI/bge-small-en-v1.5") dense_model = TextEmbedding(model_name="BAAI/bge-small-en-v1.5")
sparse_model = SparseTextEmbedding(model_name="prithivida/Splade_PP_en_v1") sparse_model = SparseTextEmbedding(model_name="Qdrant/bm25")
for row in partition_data: for row in partition_data:
# Generate dense and sparse vectors # Generate dense and sparse vectors
@@ -81,7 +81,7 @@ def vectorize(partition_data):
] ]
``` ```
We're using the [BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5) model for dense embeddings and [prithivida/Splade_PP_en_v1](https://huggingface.co/prithivida/Splade_PP_en_v1) for sparse embeddings. We're using the [BAAI/bge-small-en-v1.5](https://huggingface.co/BAAI/bge-small-en-v1.5) model for dense embeddings and [BM25](https://huggingface.co/Qdrant/bm25) for sparse embeddings.
#### Applying the UDF on our dataframe #### Applying the UDF on our dataframe
@@ -42,7 +42,7 @@ This command generates all of the project files you need to run Airflow locally.
To use Qdrant within Airflow, install the Qdrant Airflow provider by adding the following to the `requirements.txt` file To use Qdrant within Airflow, install the Qdrant Airflow provider by adding the following to the `requirements.txt` file
```text ```text
apache-airflow-providers-qdrant==1.1.0 apache-airflow-providers-qdrant
``` ```
### Configure credentials ### Configure credentials
@@ -178,12 +178,12 @@ def recommend_book():
) -> None: ) -> None:
hook = QdrantHook(conn_id=QDRANT_CONNECTION_ID) hook = QdrantHook(conn_id=QDRANT_CONNECTION_ID)
result = hook.conn.search( result = hook.conn.query_points(
collection_name=COLLECTION_NAME, collection_name=COLLECTION_NAME,
query_vector=preference_embedding, query=preference_embedding,
limit=1, limit=1,
with_payload=True, with_payload=True,
) ).points
print("Book recommendation: " + result[0].payload["title"]) print("Book recommendation: " + result[0].payload["title"])
print("Description: " + result[0].payload["description"]) print("Description: " + result[0].payload["description"])
@@ -36,11 +36,8 @@ What Qdrant can do:
- Apply full-text filters to the vector search (i.e., perform vector search among the records with specific words or phrases) - Apply full-text filters to the vector search (i.e., perform vector search among the records with specific words or phrases)
- Do prefix search and semantic [search-as-you-type](../../../articles/search-as-you-type/) - Do prefix search and semantic [search-as-you-type](../../../articles/search-as-you-type/)
- Sparse vectors, as used in [SPLADE](https://github.com/naver/splade) or similar models - Sparse vectors, as used in [SPLADE](https://github.com/naver/splade) or similar models
- [Multi-vectors](../../concepts/vectors/#multivectors), for example ColBERT and other late-interaction models
What Qdrant plans to introduce in the future: - Combination of the [multiple searches](../../concepts/hybrid-queries/)
- ColBERT and other late-interaction models
- Fusion of the multiple searches
What Qdrant doesn't plan to support: What Qdrant doesn't plan to support:
@@ -30,4 +30,4 @@ The `Query Qdrant` processor can perform a similarity search across a Qdrant col
## Further Reading ## Further Reading
- [NiFi Documentation](https://nifi.apache.org/documentation/v2/). - [NiFi Documentation](https://nifi.apache.org/documentation/v2/).
- [Source Code](https://github.com/apache/nifi/tree/main/nifi-python-extensions/nifi-text-embeddings-module/src/main/python) - [Source Code](https://github.com/apache/nifi-python-extensions)