krzysztof/landing_page
1
0
Fork 0
mirror of https://github.com/qdrant/landing_page.git synced 2026-09-26 14:38:30 +02:00
Code Issues Packages Projects Releases Wiki Activity

fixed formatting issue in complete set up and added docker_qdrant.png

Browse Source
This commit is contained in:
Ramon Perez
2023-05-30 11:48:38 +08:00
parent 12dc16a43a
commit 2b2d64be0a
2 changed files with 126 additions and 142 deletions
Download Patch File Download Diff File Expand all files Collapse all files
qdrant-landing/content/documentation/tutorials/complete-setup.md
+126 -142
View File
@@ -3,9 +3,12 @@ title: Complete Setup
weight: 10
---
# Complete Qdrant Setup
# Getting Started with Qdrant
Vector databases shine in many applications like [semantic search](https://en.wikipedia.org/wiki/Semantic_search) and [recommendation systems](https://en.wikipedia.org/wiki/Recommender_system), and in this tutorial, we'll learn about how to get started building these systems with one of the most popular and fastest growing vector databases in the market, [Qdrant](qdrant.tech).
Vector databases shine in many applications like [semantic search](https://en.wikipedia.org/wiki/Semantic_search) and
[recommendation systems](https://en.wikipedia.org/wiki/Recommender_system), and in this tutorial, we'll learn about
how to get started building these systems with one of the most popular and fastest growing vector databases in the
market, [Qdrant](qdrant.tech).
## Table of Contents
@@ -29,7 +32,10 @@ By the end of this tutorial, you will be able to
## 2. Installation
The open source version of Qdrant is available as a docker image and it can be pulled and run from any machine with docker in it. If you don't have Docker installed in your PC you can follow the instructions in the official documentation [here](https://docs.docker.com/get-docker/). After that, open your terminal start by downloading the image with the following command.
The open source version of Qdrant is available as a docker image and it can be pulled and run from any machine
with docker in it. If you don't have Docker installed in your PC you can follow the instructions in the
official documentation [here](https://docs.docker.com/get-docker/). After that, open your terminal start by
downloading the image with the following command.
```sh
docker pull qdrant/qdrant
@@ -45,11 +51,14 @@ docker run -p 6333:6333 \
You should see something similar to the following image.
![dockerqdrant](../images/docker_qdrant.png)
![dockerqdrant](../../../static/content/images/docker_qdrant.png)
If you experience any issues during the start process, please let us know in our [discord channel here](https://qdrant.to/discord). We are always available and happy to help.
If you experience any issues during the start process, please let us know in our
[discord channel here](https://qdrant.to/discord). We are always available and happy to help.
Now that you have Qdrant up and running, your next step is to pick a client to connect to it. We'll be using Python as it has the most mature data tools' ecosystem out there. So, let's start setting up our development environment and getting the libraries we'll be using today.
Now that you have Qdrant up and running, your next step is to pick a client to connect to it. We'll
be using Python as it has the most mature data tools' ecosystem out there. So, let's start setting up
our development environment and getting the libraries we'll be using today.
```sh
# with mamba or conda
@@ -66,13 +75,19 @@ pip install qdrant-client pandas numpy faker
After your have your environment ready, let's get started using Qdrant.
**Note:** At the time of writing, Qdrant supports Rust, GO, Python and TypeScript. We expect other programming languages to be added in the future.
**Note:** At the time of writing, Qdrant supports Rust, GO, Python and TypeScript. We expect other
programming languages to be added in the future.
## 3. Getting Started
The two modules we'll use the most are the `QdrantClient` and the `models` one. The former allows us to connect to Qdrant or it allows us to run an in-memory database by switching the parameter `location=` to `":memory:"` (this is a great feature for testing in a CI/CD pipeline). The latter gives us access to most of the functionalities with need to interact with Qdrant.
The two modules we'll use the most are the `QdrantClient` and the `models` one. The former allows us
to connect to Qdrant or it allows us to run an in-memory database by switching the parameter `location=`
to `":memory:"` (this is a great feature for testing in a CI/CD pipeline). The latter gives us access
to most of the functionalities with need to interact with Qdrant.
We'll start by instantiating our client using `host="localhost"` and `port=6333` (as it is the default port we used earlier with docker). You can also follow along with the `location=":memory:"` option commented out below.
We'll start by instantiating our client using `host="localhost"` and `port=6333` (as it is the default
port we used earlier with docker). You can also follow along with the `QdrantClient(location=":memory:")`
as well.
```python
@@ -82,27 +97,23 @@ from qdrant_client.http.models import CollectionStatus
```
```python
```
client = QdrantClient(host="localhost", port=6333)
client
```
<qdrant_client.qdrant_client.QdrantClient at 0x7fad13ebcdf0>
In OLTP and OLAP databases we call specific bundles of rows and columns **Tables**, but in vector databases,
the rows are known as vectors, the columns are known as dimensions, and the combination of the two (plus some
metadata) is a [**Collection**](https://qdrant.tech/documentation/collections/).
```python
# client = QdrantClient(location=":memory:")
# client
```
In OLTP and OLAP databases we call specific bundles of rows and columns **Tables**, but in vector databases, the rows are known as vectors, the columns are known as dimensions, and the combination of the two (plus some metadata) is a [**Collection**](https://qdrant.tech/documentation/collections/).
In the same way in which we can create many tables in an OLTP or an OLAP database, we can create many collections in a vector database like Qdrant using one of its clients. The key difference to note is that when we create a collection in Qdrant, we need to specify the width of the collection (i.e. the length of the vector or amount of dimensions) beforehand with the parameter `size=...`, as well as the distance metric with the parameter `distance=...` (which can be changed later on).
In the same way in which we can create many tables in an OLTP or an OLAP database, we can create many
collections in a vector database like Qdrant using one of its clients. The key difference to note is that
when we create a collection in Qdrant, we need to specify the width of the collection (i.e. the length of
the vector or amount of dimensions) beforehand with the parameter `size=...`, as well as the distance metric
with the parameter `distance=...` (which can be changed later on).
The distances currently supported by Qdrant are [**Cosine Similarity**](https://en.wikipedia.org/wiki/Cosine_similarity), [**Dot Product**](https://en.wikipedia.org/wiki/Dot_product), and [**Euclidean Distance**](https://en.wikipedia.org/wiki/Euclidean_distance).
@@ -122,7 +133,9 @@ print(first_collection)
True
We can extract information related to the health of our collection by retrieving the collection with our client. In addition, we can use this information for testing purposes, which can be very beneficial while in development mode.
We can extract information related to the health of our collection by retrieving the collection
with our client. In addition, we can use this information for testing purposes, which can be very
beneficial while in development mode.
```python
@@ -130,9 +143,6 @@ collection_info = client.get_collection(collection_name=my_collection)
list(collection_info)
```
[('status', <CollectionStatus.GREEN: 'green'>),
('optimizer_status', <OptimizersStatusOneOf.OK: 'ok'>),
('vectors_count', 0),
@@ -145,14 +155,19 @@ list(collection_info)
```python
assert collection_info.status == CollectionStatus.GREEN
assert collection_info.vectors_count == 0
```
There's a couple of things to note from what we have done so far.
- The first is that when we initiated our docker image, we created a local directory called, `qdrant_storage`, and this is where all of our collections, plus their metadata, will be saved at. Qdrant can use one of two options for [storage](https://qdrant.tech/documentation/storage/), **in-memory** storage (which stores all vectors in RAM and has the highest speed since disk access is required only for persistence), or **memmap** storage (which creates a virtual address space associated with the file on disk). You can have a look at that directory in a *nix system with `tree qdrant_storage -L 2`, and something similar to the following output should come up for you.
- The first is that when we initiated our docker image, we created a local directory called, `qdrant_storage`,
and this is where all of our collections, plus their metadata, will be saved at. Qdrant can use one of two
options for [storage](https://qdrant.tech/documentation/storage/), **in-memory** storage (which stores all
vectors in RAM and has the highest speed since disk access is required only for persistence), or **memmap**
storage (which creates a virtual address space associated with the file on disk). You can have a look at
that directory in a *nix system with `tree qdrant_storage -L 2`, and something similar to the following
output should come up for you.
```bash
qdrant_storage
├── aliases
@@ -161,33 +176,37 @@ There's a couple of things to note from what we have done so far.
│ └── my_first_collection
└── raft_state
```
- The second is that we used `client.recreate_collection()` and this command, as the name implies, can be used more than once to create new collections with or without the same name, so be careful no to recreate a collection that you did not intend to recreate. To create a brand new collection that cannot be recreated again, we would use `client.create_collection()` method instead.
- The second is that we used `client.recreate_collection()` and this command, as the name implies, can
be used more than once to create new collections with or without the same name, so be careful no to
recreate a collection that you did not intend to recreate. To create a brand new collection that cannot
be recreated again, we would use `client.create_collection()` method instead.
- Our collection will hold vectors of 100 dimensions and the distance metric has been set to Cosine Similarity.
Now that we know how to create collections, let's create a bit of fake data and add some vectors to it.
### 3.1 Adding Points
The [points](https://qdrant.tech/documentation/points/) are the central entity Qdrant operates with, and these contain records consisting of a vector, an optional `id` and an optional `payload` (which we'll talk more about in the next section).
The [points](https://qdrant.tech/documentation/points/) are the central entity Qdrant operates with, and
these contain records consisting of a vector, an optional `id` and an optional `payload` (which we'll talk
more about in the next section).
The optional id can be represented by [unsigned integers](https://en.wikipedia.org/wiki/Integer_(computer_science)) or [UUID(https://en.wikipedia.org/wiki/Universally_unique_identifier)]s but, for our use case, we will use a straightforward range of numbers.
The optional id can be represented by [unsigned integers](https://en.wikipedia.org/wiki/Integer_(computer_science))
or [UUID(https://en.wikipedia.org/wiki/Universally_unique_identifier)]s but, for our use case, we will use a
straightforward range of numbers.
Let's us [NumPy](https://numpy.org/) to create a matrix of fake data containing 1,000 vectors and 100 dimensions, and then represent the values as `float64` numbers between -1 and 1. For simplicity, let's imagine that each of these vectors represents one of our favorite songs, and that each column represents a unique characteristic of the song, for example, the tempo, the beats, the pitch of the voice of the singer(s), etc.
Let's us [NumPy](https://numpy.org/) to create a matrix of fake data containing 1,000 vectors and 100
dimensions, and then represent the values as `float64` numbers between -1 and 1. For simplicity, let's
imagine that each of these vectors represents one of our favorite songs, and that each column represents
a unique characteristic of the song, for example, the tempo, the beats, the pitch of the voice of the singer(s), etc.
```python
import numpy as np
```
```python
data = np.random.uniform(low=-1.0, high=1.0, size=(1_000, 100))
type(data[0, 0]), data[:2, :20]
```
(numpy.float64,
array([[-0.05303611, 0.34459755, 0.76877484, -0.0158912 , -0.3515725 ,
-0.92520697, -0.36416004, 0.91791994, -0.2254738 , 0.03992614,
@@ -208,16 +227,15 @@ index = list(range(len(data)))
index[-10:]
```
[990, 991, 992, 993, 994, 995, 996, 997, 998, 999]
Once a collection has been created, we can fill it in with the command `client.upsert()`. We'll need the collection's name and the appropriate uploading process from our `models` module, in this case, [`Batch`](https://qdrant.tech/documentation/points/#upload-points).
Once a collection has been created, we can fill it in with the command `client.upsert()`. We'll need the
collection's name and the appropriate uploading process from our `models` module, in this case, [`Batch`](https://qdrant.tech/documentation/points/#upload-points).
One thing to note is that Qdrant can only take in native Python iterables like lists and tuples. This is why you'll notice the `.tolist()` method attached to our numpy matrix,`data`, below.
One thing to note is that Qdrant can only take in native Python iterables like lists and tuples. This is
why you'll notice the `.tolist()` method attached to our numpy matrix,`data`, below.
```python
@@ -230,14 +248,12 @@ client.upsert(
)
```
UpdateResult(operation_id=0, status=<UpdateStatus.COMPLETED: 'completed'>)
We can retrieve specific points based on their ID (for example, artist X with ID 100) and get some additional information from that result.
We can retrieve specific points based on their ID (for example, artist X with ID 100) and get some
additional information from that result.
```python
@@ -248,9 +264,6 @@ client.retrieve(
)
```
[Record(id=100, payload={}, vector=[-0.03463546, -0.01026143, 0.15584062, -0.03674199, -0.10754523, -0.15643364, -0.1357449, -0.067160346, -0.0015522024, 0.030050367, -0.1336697, 0.12108152, 0.14289995, -0.06601132, 0.0067344513, 0.05278854, -0.15211268, 0.02751477, 0.013617064, 0.07691656, 0.14334463, 0.078325845, -0.067150295, 0.13005906, 0.016733043, -0.09850788, 0.071991354, 0.03699084, -0.13441333, -0.10449347, 0.10445317, -0.1284516, -0.14821146, 0.052959085, -0.060419794, 0.02169682, 0.1191593, 0.073885836, 0.07564473, -0.062121205, -0.15273724, -0.15505096, 0.042748712, -0.020586135, -0.13603596, 0.14455841, 0.10489224, -0.120181546, 0.052598357, -0.16265365, -0.15380295, 0.11495946, 0.062300652, -0.101409845, 0.14395215, -0.16222608, 0.06593911, 0.1173012, 0.12055558, -0.009821507, 0.020054886, 0.04771211, -0.022454012, 0.118841976, -0.0491934, -0.09702059, 0.064038, 0.124456346, 0.13849476, 0.06234425, -0.06864697, 0.032732993, 0.15420534, 0.14516357, 0.03705083, -0.014671113, -0.12554511, -0.103203796, 0.03705848, -0.12491126, -0.14011686, -0.08283738, 0.07958351, 0.009932304, -0.029494055, 0.07196128, 0.08827084, 0.08012733, -0.14970517, -0.12099693, -0.114456914, 0.044483785, -0.07360799, -0.045856647, 0.096695036, -0.14971526, -0.0643105, -0.14583696, -0.08727498, 0.15599053])]
@@ -261,10 +274,8 @@ We can also update our collection one point at a time, for example, as new data
```python
def create_song():
return np.random.uniform(low=-1.0, high=1.0, size=100).tolist()
```
```python
client.upsert(
collection_name=my_collection,
points=[
@@ -276,9 +287,6 @@ client.upsert(
)
```
UpdateResult(operation_id=1, status=<UpdateStatus.COMPLETED: 'completed'>)
@@ -287,19 +295,15 @@ We can also delete it in a straightforward fashion.
```python
# this will show the amount of vectors BEFORE deleting the one we just created
client.count(
collection_name=my_collection,
exact=True,
)
```
CountResult(count=1001)
Notice the amount of vectors BEFORE deleting the one we just created
```python
@@ -311,56 +315,49 @@ client.delete(
)
```
UpdateResult(operation_id=2, status=<UpdateStatus.COMPLETED: 'completed'>)
Notice the amount of vectors AFTER deleting the one we just created
```python
# this will show the amount of vectors AFTER deleting them
client.count(
collection_name=my_collection,
exact=True,
)
```
CountResult(count=1000)
### 3.2 Payloads
Qdrant has incredible features on top of speed and reliability, and one of its most useful ones is without a doubt the ability to store additional information alongside the vectors. In Qdrant's terminology, this information is considered a [payload](https://qdrant.tech/documentation/payload/) and it is represented as JSON objects. With these payloads, not only can you get information back when you search in the database, but you can also filter your search by the parameters in the payload, and we'll see how in a second.
Qdrant has incredible features on top of speed and reliability, and one of its most useful ones is without
a doubt the ability to store additional information alongside the vectors. In Qdrant's terminology, this
information is considered a [payload](https://qdrant.tech/documentation/payload/) and it is represented as
JSON objects. With these payloads, not only can you get information back when you search in the database,
but you can also filter your search by the parameters in the payload, and we'll see how in a second.
Imagine the fake vectors we created actually represented a song. If we were building a semantic search system for songs then, naturally, the things we would want to get back would be the song itself (or an URL to it), the artist, maybe the genre, and so on.
Imagine the fake vectors we created actually represented a song. If we were building a semantic search system
for songs then, naturally, the things we would want to get back would be the song itself (or an URL to it),
the artist, maybe the genre, and so on.
What we'll do here is to take advantage of a Python package call `faker` and create a bit of information to add to our payload and see how this functionality works.
```python
from faker import Faker
```
```python
fake_something = Faker()
fake_something.name()
```
'Laura Sandoval'
For each vector, we'll create list of dictionaries containing the artist, the song (as a combination of 3 random words), a url to the song is at (say, s3), the year in which it was released, and the country where it originated from.
For each vector, we'll create list of dictionaries containing the artist, the song (as a combination of
3 random words), a url to the song is at (say, s3), the year in which it was released, and the country
where it originated from.
```python
@@ -380,9 +377,6 @@ for i in range(len(data)):
payload[:3]
```
[{'artist': 'Francisco Benton',
'song': 'ability something message',
'url_song': 'http://www.smith.com/',
@@ -401,7 +395,8 @@ payload[:3]
We can upsert our Points (ids, data, and payload), with the same `client.upsert()` method we used earlier, and we can retrieve any one song with the `client.retrieve()` method.
We can upsert our Points (ids, data, and payload), with the same `client.upsert()` method we used earlier,
and we can retrieve any one song with the `client.retrieve()` method.
```python
@@ -415,13 +410,10 @@ client.upsert(
)
```
UpdateResult(operation_id=3, status=<UpdateStatus.COMPLETED: 'completed'>)
Change the values below to retrieve different records.
```python
resutls = client.retrieve(
@@ -433,9 +425,6 @@ resutls = client.retrieve(
type(resutls), resutls
```
(list,
[Record(id=500, payload={'artist': 'Arthur White', 'country': 'Taiwan', 'song': 'late citizen job', 'url_song': 'https://www.wilson.info/', 'year': '1976'}, vector=None),
Record(id=100, payload={'artist': 'Kyle Livingston', 'country': 'Isle of Man', 'song': 'through opportunity start', 'url_song': 'http://osborn-byrd.com/', 'year': '2003'}, vector=None),
@@ -451,9 +440,6 @@ What we got back is a list with records and each element inside a record can be
resutls[0].payload
```
{'artist': 'Arthur White',
'country': 'Taiwan',
'song': 'late citizen job',
@@ -461,15 +447,12 @@ resutls[0].payload
'year': '1976'}
For the `id`.
```python
resutls[0].id
```
500
@@ -478,17 +461,16 @@ Next, we'll use our payload it to search.
### 3.3 Search
Now that we have our vectors with an ID and a payload, we can explore a few of ways in which we can search for content when, in our use case, new music gets selected. Let's check it out.
Now that we have our vectors with an ID and a payload, we can explore a few of ways in which we can search
for content when, in our use case, new music gets selected. Let's check it out.
Say, for example, that a new song (like ["living la vida loca"](https://www.youtube.com/watch?v=p47fEXGabaY&ab_channel=RickyMartinVEVO) by Ricky Martin) comes in and our model immediately transforms it into a vector. Since we don't want a large amount of values back, let's limit the search to a few points.
Say, for example, that a new song (like ["living la vida loca"](https://www.youtube.com/watch?v=p47fEXGabaY&ab_channel=RickyMartinVEVO) by Ricky Martin) comes in and our model immediately transforms it into a vector. Since
we don't want a large amount of values back, let's limit the search to a few points.
```python
living_la_vida_loca = create_song()
```
```python
client.search(
collection_name=my_collection,
query_vector=living_la_vida_loca,
@@ -496,16 +478,15 @@ client.search(
)
```
[ScoredPoint(id=899, version=3, score=0.27792826, payload={'artist': 'Billy Lynch', 'country': 'Morocco', 'song': 'speech front another', 'url_song': 'https://www.kim.com/', 'year': '1972'}, vector=None),
ScoredPoint(id=93, version=3, score=0.277919, payload={'artist': 'Austin Aguilar', 'country': 'Reunion', 'song': 'computer rise president', 'url_song': 'http://lane.com/', 'year': '2009'}, vector=None),
ScoredPoint(id=852, version=3, score=0.2773033, payload={'artist': 'Jennifer Chavez', 'country': 'Korea', 'song': 'recognize other defense', 'url_song': 'https://www.harris.com/', 'year': '1970'}, vector=None)]
Now imagine that we only want Australian songs recommended to us. For this, we can filter the query using the information in the payload. We'll first create a filter object and pass it to our search method as an argument to the parameter `query_filter=`.
Now imagine that we only want Australian songs recommended to us. For this, we can filter the query using
the information in the payload. We'll first create a filter object and pass it to our search method as an
argument to the parameter `query_filter=`.
```python
@@ -515,15 +496,11 @@ aussie_songs = models.Filter(
type(aussie_songs)
```
qdrant_client.http.models.models.Filter
```python
```
client.search(
collection_name=my_collection,
query_vector=living_la_vida_loca,
@@ -532,15 +509,12 @@ client.search(
)
```
[ScoredPoint(id=57, version=3, score=0.07630251, payload={'artist': 'Robert Lucas', 'country': 'Australia', 'song': 'this minute spend', 'url_song': 'https://olson-wyatt.com/', 'year': '1971'}, vector=None),
ScoredPoint(id=67, version=3, score=0.046429798, payload={'artist': 'Crystal Hughes', 'country': 'Australia', 'song': 'owner different later', 'url_song': 'http://www.gonzalez-ford.com/', 'year': '2009'}, vector=None)]
Lastly, say we want aussie songs but we don't care how new or old these songs are. Let's exclude the yearfrom the payload.
Lastly, say we want aussie songs but we don't care how new or old these songs are. Let's exclude the year from the payload.
```python
@@ -553,9 +527,6 @@ client.search(
)
```
[ScoredPoint(id=57, version=3, score=0.07630251, payload={'artist': 'Robert Lucas', 'country': 'Australia', 'song': 'this minute spend', 'url_song': 'https://olson-wyatt.com/'}, vector=None),
ScoredPoint(id=67, version=3, score=0.046429798, payload={'artist': 'Crystal Hughes', 'country': 'Australia', 'song': 'owner different later', 'url_song': 'http://www.gonzalez-ford.com/'}, vector=None),
ScoredPoint(id=803, version=3, score=0.019309767, payload={'artist': 'Karen Young', 'country': 'Australia', 'song': 'base fall often', 'url_song': 'http://www.wilson-sharp.com/'}, vector=None),
@@ -564,9 +535,11 @@ client.search(
As you can see, you can apply a wide-range of filtering methods to allows your users to take more control of the recommendations they are being served.
As you can see, you can apply a wide-range of filtering methods to allows your users to take more control of the
recommendations they are being served.
If you wanted to clear out the payload and upload a new one for the same vectors, you can use `client.clear_payload()` as below.
If you wanted to clear out the payload and upload a new one for the same vectors, you can use
`client.clear_payload()` as below.
```python
client.clear_payload(
@@ -579,15 +552,27 @@ client.clear_payload(
## 4. Recommendations
A recommendation system is a technology that suggests items or content to users based on their preferences, interests, or past behavior. It's like having a knowledgeable friend who can recommend movies, books, music, or products that you might enjoy.
A recommendation system is a technology that suggests items or content to users based on their preferences,
interests, or past behavior. It's like having a knowledgeable friend who can recommend movies, books, music,
or products that you might enjoy.
In its most widely-used form, recommendation systems work by analyzing data about you and other users. The system looks at your previous choices, such as movies you've watched, products you've bought, or articles you've read, and it then compares this information with data from other people who have similar tastes or interests. These systems are used in various companies such as Netflix, Amazon, Tik-Tok, and Spotify. They aim to personalize your experience, save you time searching for things you might like, or introduce you to new and relevant content that you may not have discovered otherwise.
In its most widely-used form, recommendation systems work by analyzing data about you and other users. The
system looks at your previous choices, such as movies you've watched, products you've bought, or articles you've
read, and it then compares this information with data from other people who have similar tastes or interests. These
systems are used in various companies such as Netflix, Amazon, Tik-Tok, and Spotify. They aim to personalize your
experience, save you time searching for things you might like, or introduce you to new and relevant content that
you may not have discovered otherwise.
In a nutshell, a recommendation system is a smart tool that helps you discover new things you'll probably enjoy based on your preferences and the experiences of others.
In a nutshell, a recommendation system is a smart tool that helps you discover new things you'll probably enjoy
based on your preferences and the experiences of others.
Qdrant offers a convenient API that allows you to take into account user feedback by including similar songs to those the user have already liked (👍), or, conversely, by excluding songs that are similar to those the user has signal it did not like (👎).
Qdrant offers a convenient API that allows you to take into account user feedback by including similar songs to
those the user have already liked (👍), or, conversely, by excluding songs that are similar to those the user
has signal it did not like (👎).
The method is straightforward to implement via the `client.recommend()` method, and it provides enough flexibility that the logic on how the feedback gets capture can rest in the hands of the developers at your organization. So, what do you need to keep in mind when making recommendations with Qdrant?
The method is straightforward to implement via the `client.recommend()` method, and it provides enough flexibility
that the logic on how the feedback gets capture can rest in the hands of the developers at your organization. So,
what do you need to keep in mind when making recommendations with Qdrant?
- `collection_name=` - from which collection are we selecting vectors.
- `query_filter=` - which filter will we apply to our search, if any.
@@ -609,9 +594,6 @@ client.recommend(
)
```
[ScoredPoint(id=46, version=3, score=0.3310853, payload={'artist': 'Felicia Yang', 'country': 'Brunei Darussalam', 'song': 'pattern help within', 'url_song': 'http://www.gonzalez.biz/', 'year': '2006'}, vector=None),
ScoredPoint(id=840, version=3, score=0.3030827, payload={'artist': 'Charles Brown', 'country': 'Somalia', 'song': 'network national very', 'url_song': 'https://www.larson-hartman.com/', 'year': '1975'}, vector=None),
ScoredPoint(id=771, version=3, score=0.2851892, payload={'artist': 'Lori Clark', 'country': 'Myanmar', 'song': 'why box back', 'url_song': 'https://sanchez-waters.biz/', 'year': '1992'}, vector=None),
@@ -619,7 +601,7 @@ client.recommend(
ScoredPoint(id=544, version=3, score=0.26716217, payload={'artist': 'Christopher Bowman', 'country': 'Yemen', 'song': 'strategy late same', 'url_song': 'http://herrera.com/', 'year': '2012'}, vector=None)]
Now let's add a song that was not well received.
```python
client.recommend(
@@ -631,9 +613,6 @@ client.recommend(
)
```
[ScoredPoint(id=756, version=3, score=0.31679478, payload={'artist': 'Chad Garza', 'country': 'Namibia', 'song': 'movie find method', 'url_song': 'http://www.moore.com/', 'year': '2018'}, vector=None),
ScoredPoint(id=46, version=3, score=0.2964203, payload={'artist': 'Felicia Yang', 'country': 'Brunei Darussalam', 'song': 'pattern help within', 'url_song': 'http://www.gonzalez.biz/', 'year': '2006'}, vector=None),
ScoredPoint(id=233, version=3, score=0.28002173, payload={'artist': 'Julie King', 'country': 'Congo', 'song': 'since really house', 'url_song': 'https://williams.com/', 'year': '1971'}, vector=None),
@@ -642,7 +621,10 @@ client.recommend(
Notice that, while the similarity scores are completely random for this example, it is important that we pay attention to the scores we get back when serving recommendations in production. Even if we get 5 vectors back, it might more useful to show random results rather than vectors that are 0.012 similar to the query vector. With this in mind, we can actually set a threshold for our vectors with the `score_threshold=` parameter.
Notice that, while the similarity scores are completely random for this example, it is important that we pay
attention to the scores we get back when serving recommendations in production. Even if we get 5 vectors back,
it might more useful to show random results rather than vectors that are 0.012 similar to the query vector. With
this in mind, we can actually set a threshold for our vectors with the `score_threshold=` parameter.
```python
@@ -656,9 +638,6 @@ client.recommend(
)
```
[ScoredPoint(id=756, version=3, score=0.3045847, payload={'artist': 'Chad Garza', 'country': 'Namibia', 'song': 'movie find method', 'url_song': 'http://www.moore.com/', 'year': '2018'}, vector=None),
ScoredPoint(id=304, version=3, score=0.28136152, payload={'artist': 'Steven Fitzgerald', 'country': 'Kyrgyz Republic', 'song': 'so suddenly indicate', 'url_song': 'http://www.stevens.com/', 'year': '1994'}, vector=None),
ScoredPoint(id=274, version=3, score=0.28093755, payload={'artist': 'Kimberly Suarez', 'country': "Lao People's Democratic Republic", 'song': 'former blue people', 'url_song': 'http://spencer.com/', 'year': '1971'}, vector=None),
@@ -667,7 +646,9 @@ client.recommend(
Lastly, we can add filters in the same way as we did before. Note that these filters could be tags that your users get to pick such as, for example, genres including `reggeaton`, `bachata`, and `salsa` (sorry Drake), or the language of the song.
Lastly, we can add filters in the same way as we did before. Note that these filters could be tags that your
users get to pick such as, for example, genres including `reggeaton`, `bachata`, and `salsa` (sorry Drake), or
the language of the song.
```python
@@ -683,9 +664,6 @@ client.recommend(
)
```
[ScoredPoint(id=926, version=3, score=0.12710258, payload={'artist': 'Cody Hernandez', 'country': 'Dominican Republic', 'song': 'not hotel under', 'url_song': 'http://www.hanson.com/', 'year': '1978'}, vector=None),
ScoredPoint(id=132, version=3, score=0.07745133, payload={'artist': 'Eric Edwards', 'country': 'Dominican Republic', 'song': 'next south yes', 'url_song': 'http://www.hall.com/', 'year': '1991'}, vector=None),
ScoredPoint(id=242, version=3, score=0.052371096, payload={'artist': 'John Garza', 'country': 'Dominican Republic', 'song': 'executive animal our', 'url_song': 'https://www.gutierrez-johnson.net/', 'year': '2012'}, vector=None),
@@ -698,7 +676,13 @@ That's it! You have now gone over a whirlwind tour of vector databases and are r
## 5. Conclusion
To wrap up, we have explored a bit of the fascinating world of vector databases, and we learned that these databases provide efficient storage and retrieval of high-dimensional vectors, making them ideal for similarity-based search tasks and recommendation systems. Both of these use cases can be applied in a variety of industries while helping us unlock new levels of information retrieval. In particular, recommendation systems built with Qdrant provide developers with enough flexibility to add and subtract data points that users liked or dislike, respectively, and even set up a threshold for how similar a recommendation must be before our applications can serve it.
To wrap up, we have explored a bit of the fascinating world of vector databases, and we learned that these
databases provide efficient storage and retrieval of high-dimensional vectors, making them ideal for
similarity-based search tasks and recommendation systems. Both of these use cases can be applied in a variety
of industries while helping us unlock new levels of information retrieval. In particular, recommendation
systems built with Qdrant provide developers with enough flexibility to add and subtract data points that users
liked or dislike, respectively, and even set up a threshold for how similar a recommendation must be before our
applications can serve it.
We can't wait to see what cool applications you build with Qdrant.