Refactor recreate collection (#1079)

* refactor recreate collection

* nit
This commit is contained in:
Hossam Hagag
2024-08-13 12:38:38 +02:00
committed by GitHub
parent 71328ae784
commit 952f4b473d
12 changed files with 66 additions and 68 deletions
@@ -98,13 +98,14 @@ The configuration is also a part of the collection snapshot.
```python
from qdrant_client import models
client.recreate_collection(
collection_name="test_collection",
vectors_config=models.VectorParams(
size=768, # Size of the embedding vector generated by the InstructorXL model
distance=models.Distance.COSINE
),
)
if not client.collection_exists("test_collection"):
client.create_collection(
collection_name="test_collection",
vectors_config=models.VectorParams(
size=768, # Size of the embedding vector generated by the InstructorXL model
distance=models.Distance.COSINE
),
)
```
### Upload the dataset
@@ -125,12 +125,13 @@ client.set_sparse_model("prithivida/Splade_PP_en_v1")
4. Related vectors need to be added to a collection. Create a new collection for your startup vectors.
```python
client.recreate_collection(
collection_name="startups",
vectors_config=client.get_fastembed_vector_params(),
# comment this line to use dense vectors only
sparse_vectors_config=client.get_fastembed_sparse_vector_params(),
)
if not client.collection_exists("startups"):
client.create_collection(
collection_name="startups",
vectors_config=client.get_fastembed_vector_params(),
# comment this line to use dense vectors only
sparse_vectors_config=client.get_fastembed_sparse_vector_params(),
)
```
Qdrant requires vectors to have their own names and configurations.
@@ -152,15 +152,14 @@ client = QdrantClient("http://localhost:6333")
3. Related vectors need to be added to a collection. Create a new collection for your startup vectors.
```python
client.recreate_collection(
collection_name="startups",
vectors_config=VectorParams(size=384, distance=Distance.COSINE),
)
if not client.collection_exists("startups"):
client.create_collection(
collection_name="startups",
vectors_config=VectorParams(size=384, distance=Distance.COSINE),
)
```
<aside role="status">
- Use `recreate_collection` if you are experimenting and running the script several times. This function will first try to remove an existing collection with the same name.
- The `vector_size` parameter defines the size of the vectors for a specific collection. If their size is different, it is impossible to calculate the distance between them. `384` is the encoder output dimensionality. You can also use `model.get_sentence_embedding_dimension()` to get the dimensionality of the model you are using.
- The `distance` parameter lets you specify the function used to measure the distance between two points.
@@ -157,7 +157,7 @@ client = QdrantClient(":memory:")
All data in Qdrant is organized by collections. In this case, you are storing books, so we are calling it `my_books`.
```python
client.recreate_collection(
client.create_collection(
collection_name="my_books",
vectors_config=models.VectorParams(
size=encoder.get_sentence_embedding_dimension(), # Vector size is defined by used model
@@ -166,8 +166,6 @@ client.recreate_collection(
)
```
- Use `recreate_collection` if you are experimenting and running the script several times. This function will first try to remove an existing collection with the same name.
- The `vector_size` parameter defines the size of the vectors for a specific collection. If their size is different, it is impossible to calculate the distance between them. 384 is the encoder output dimensionality. You can also use model.get_sentence_embedding_dimension() to get the dimensionality of the model you are using.
- The `distance` parameter lets you specify the function used to measure the distance between two points.