Merge pull request #94 from qdrant/cohere-qa-saas

Add an article about QA with Cohere and Qdrant

qdrant-landing/content/articles/qa-with-cohere-and-qdrant.md

@@ -0,0 +1,239 @@
+---
+title: Question Answering as a Service with Cohere and Qdrant
+short_description: "End-to-end Question Answering system for the biomedical data with SaaS tools: Cohere co.embed API and Qdrant"
+description: "End-to-end Question Answering system for the biomedical data with SaaS tools: Cohere co.embed API and Qdrant"
+social_preview_image: /articles_data/qa-with-cohere-and-qdrant/social_preview.png
+small_preview_image: /articles_data/qa-with-cohere-and-qdrant/q-and-a-article-icon.svg
+preview_dir: /articles_data/qa-with-cohere-and-qdrant/preview
+weight: 7
+author: Kacper Łukawski
+author_link: https://medium.com/@lukawskikacper
+date: 2022-11-29T15:45:00+01:00
+draft: false
+keywords:
+  - vector search
+  - question answering
+  - cohere
+  - co.embed
+  - embeddings
+---
+
+Bi-encoders are probably the most efficient way of setting up a semantic Question Answering system. 
+This architecture relies on the same neural model that creates vector embeddings for both questions and answers. 
+The assumption is, both question and answer should have representations close to each other in the latent space. 
+It should be like that because they should both describe the same semantic concept. That doesn't apply 
+to answers like "Yes" or "No" though, but standard FAQ-like problems are a bit easier as there is typically 
+an overlap between both texts. Not necessarily in terms of wording, but in their semantics.
+
+![Bi-encoder structure. Both queries (questions) and documents (answers) are vectorized by the same neural encoder. 
+Output embeddings are then compared by a chosen distance function, typically cosine similarity.](/articles_data/qa-with-cohere-and-qdrant/biencoder-diagram.png)
+
+And yeah, you need to **bring your own embeddings**, in order to even start. There are various ways how 
+to obtain them, but using Cohere [co.embed API](https://docs.cohere.ai/reference/embed) is probably 
+the easiest and most convenient method. 
+
+## Why co.embed API and Qdrant go well together?
+
+Maintaining a **Large Language Model** might be hard and expensive. Scaling it up and down, when the traffic 
+changes, require even more effort and becomes unpredictable. That might be definitely a blocker for any semantic 
+search system. But if you want to start right away, you may consider using a SaaS model, Cohere’s 
+[co.embed API](https://docs.cohere.ai/reference/embed) in particular. It gives you state-of-the-art language 
+models available as a Highly Available HTTP service with no need to train or maintain your own service. As all 
+the communication is done with JSONs, you can simply provide the co.embed output as Qdrant input.
+
+```python
+# Putting the co.embed API response directly as Qdrant method input
+qdrant_client.upsert(
+    collection_name="collection",
+    points=rest.Batch(
+        ids=[...],
+        vectors=cohere_client.embed(...).embeddings,
+        payloads=[...],
+    ),
+)
+```
+
+Both tools are easy to combine, so you can start working with semantic search in a few minutes, not days.
+
+And what if your needs are so specific that you need to fine-tune a general usage model? Co.embed API goes beyond 
+pre-trained encoders and allows providing some custom datasets to 
+[customize the embedding model with your own data](https://docs.cohere.ai/docs/training-a-representation-model). 
+As a result, you get the quality of domain-specific models, but without worrying about infrastructure.
+
+## System architecture overview
+
+In real systems, answers get vectorized and stored in an efficient vector search database. We typically don’t 
+even need to provide specific answers, but just use sentences or paragraphs of text and vectorize them instead. 
+Still, if a bit longer piece of text contains the answer to a particular question, its distance to the question 
+embedding should not be that far away. And for sure closer than all the other, non-matching answers. Storing the 
+answer embeddings in a vector database makes the search process way easier.
+
+![Building the database of possible answers. All the texts are converted into their vector embeddings and those 
+embeddings are stored in a vector database, i.e. Qdrant.](/articles_data/qa-with-cohere-and-qdrant/vector-database.png)
+
+## Looking for the correct answer
+
+Once our database is working and all the answer embeddings are already in place, we can start querying it. 
+We basically perform the same vectorization on a given question and ask the database to provide some near neighbours. 
+We rely on the embeddings to be close to each other, so we expect the points with the smallest distance in the latent 
+space to contain the proper answer.
+
+![While searching, a question gets vectorized by the same neural encoder. Vector database is a component that looks 
+for the closest answer vectors using i.e. cosine similarity. A proper system, like Qdrant, will make the lookup 
+process more efficient, as it won’t calculate the distance to all the answer embeddings. Thanks to HNSW, it will 
+be able to find the nearest neighbours with sublinear complexity.](/articles_data/qa-with-cohere-and-qdrant/search-with-vector-database.png)
+
+## Implementing the QA search system with SaaS tools
+
+We don’t want to maintain our own service for the neural encoder, nor even set up a Qdrant instance. There are SaaS 
+solutions for both — Cohere’s [co.embed API](https://docs.cohere.ai/reference/embed) 
+and [Qdrant Cloud](https://qdrant.tech/surveys/cloud-request/), so we’ll use them instead of on-premise tools.
+
+### Question Answering on biomedical data
+
+We’re going to implement the Question Answering system for the biomedical data. There is a 
+*[pubmed_qa](https://huggingface.co/datasets/pubmed_qa)* dataset, with it *pqa_labeled* subset containing 1,000 examples 
+of questions and answers labelled by domain experts. Our system is going to be fed with the embeddings generated by 
+co.embed API and we’ll load them to Qdrant. Using Qdrant Cloud vs your own instance does not matter much here. 
+There is a subtle difference in how to connect to the cloud instance, but all the other operations are executed 
+in the same way.
+
+```python
+from datasets import load_dataset
+
+# Loading the dataset from HuggingFace hub. It consists of several columns: pubid, 
+# question, context, long_answer and final_decision. For the purposes of our system, 
+# we’ll use question and long_answer.
+dataset = load_dataset("pubmed_qa", "pqa_labeled")
+```
+
+| **pubid** | **question**                                      | **context** | **long_answer**                                   | **final_decision** |
+|-----------|---------------------------------------------------|-------------|---------------------------------------------------|--------------------|
+| 18802997  | Can calprotectin predict relapse risk in infla... | ...         | Measuring calprotectin may help to identify UC... | maybe              |
+| 20538207  | Should temperature be monitorized during kidne... | ...         | The new storage can affords more stable temper... | no                 | 
+| 25521278  | Is plate clearing a risk factor for obesity?      | ...         | The tendency to clear one's plate when eating ... | yes                | 
+| 17595200  | Is there an intrauterine influence on obesity?    | ...         | Comparison of mother-offspring and father-offs..  | no                 | 
+| 15280782  | Is unsafe sexual behaviour increasing among HI... | ...         | There was no evidence of a trend in unsafe sex... | no                 | 
+
+### Using Cohere and Qdrant to build the answers database
+
+In order to start generating the embeddings, you need to [create a Cohere account](https://dashboard.cohere.ai/welcome/register). 
+That will start your trial period, so you’ll be able to vectorize the texts for free. Once logged in, your default API key will 
+be available in [Settings](https://dashboard.cohere.ai/api-keys). We’ll need it to call the co.embed API. with the official python package.
+
+```python
+import cohere
+
+cohere_client = cohere.Client(COHERE_API_KEY)
+
+# Generating the embeddings with Cohere client library
+embeddings = cohere_client.embed(
+    texts=["A test sentence"],
+    model="large",
+)
+vector_size = len(embeddings.embeddings[0])
+print(vector_size) # output: 4096
+```
+
+Let’s connect to the Qdrant instance first and create a collection with the proper configuration, so we can put some embeddings into it later on.
+
+```python
+# Connecting to Qdrant Cloud with qdrant-client requires providing the api_key.
+# If you use an on-premise instance, it has to be skipped.
+qdrant_client = QdrantClient(
+    host="xyz-example.eu-central.aws.staging-cloud.qdrant.io", 
+    prefer_grpc=True,
+    api_key=QDRANT_API_KEY,
+)
+```
+
+Now we’re able to vectorize all the answers. They are going to form our collection, so we can also put them already into Qdrant, along with the 
+payloads and identifiers. That will make our dataset easily searchable.
+
+```python
+answer_response = cohere_client.embed(
+    texts=dataset["train"]["long_answer"],
+    model="large",
+)
+vectors = [
+    # Conversion to float is required for Qdrant
+    list(map(float, vector)) 
+    for vector in answer_response.embeddings
+]
+ids = [entry["pubid"] for entry in dataset["train"]]
+
+# Filling up Qdrant collection with the embeddings generated by Cohere co.embed API
+qdrant_client.upsert(
+    collection_name="pubmed_qa", 
+    points=rest.Batch(
+        ids=ids,
+        vectors=vectors,
+        payloads=list(dataset["train"]),
+    )
+)
+```
+
+And that’s it. Without even setting up a single server on our own, we created a system that might be easily asked a question. I don’t want to call 
+it serverless, as this term is already taken, but co.embed API with Qdrant Cloud makes everything way easier to maintain.
+
+### Answering the questions with semantic search — the quality
+
+It’s high time to query our database with some questions. It might be interesting to somehow measure the quality of the system in general. 
+In those kinds of problems we typically use *top-k accuracy*. We assume the prediction of the system was correct if the correct answer 
+was present in the first *k* results.
+
+```python
+# Finding the position at which Qdrant provided the expected answer for each question. 
+# That allows to calculate accuracy@k for different values of k.
+k_max = 10
+answer_positions = []
+for embedding, pubid in tqdm(zip(question_response.embeddings, ids)):
+    response = qdrant_client.search(
+        collection_name="pubmed_qa",
+        query_vector=embedding,
+        limit=k_max,
+    )
+
+    answer_ids = [record.id for record in response]
+    if pubid in answer_ids:
+        answer_positions.append(answer_ids.index(pubid))
+    else:
+        answer_positions.append(-1)
+```
+
+Saved answer positions allow us to calculate the metric for different *k* values.
+
+```python
+# Prepared answer positions are being used to calculate different values of accuracy@k
+for k in range(1, k_max + 1):
+    correct_answers = len(
+        list(
+            filter(lambda x: 0 <= x < k, answer_positions)
+        )
+    )
+    print(f"accuracy@{k} =", correct_answers / len(dataset["train"]))
+```
+
+Here are the values of the top-k accuracy for different values of k:
+
+| **metric**  | **value** |
+|-------------|-----------|
+| accuracy@1  | 0.877     |
+| accuracy@2  | 0.921     |
+| accuracy@3  | 0.942     |
+| accuracy@4  | 0.950     |
+| accuracy@5  | 0.956     |
+| accuracy@6  | 0.960     |
+| accuracy@7  | 0.964     |
+| accuracy@8  | 0.971     |
+| accuracy@9  | 0.976     |
+| accuracy@10 | 0.977     |
+
+It seems like our system worked pretty well even if we consider just the first result, with the lowest distance. 
+We failed with around 12% of questions. But numbers become better with the higher values of k. It might be also 
+valuable to check out what questions our system failed to answer, their perfect match and our guesses.
+
+We managed to implement a working Question Answering system within just a few lines of code. If you are fine 
+with the results achieved, then you can start using it right away. Still, if you feel you need a slight improvement, 
+then fine-tuning the model is a way to go. If you want to check out the full source code, 
+it is available on [Google Colab](https://colab.research.google.com/drive/1YOYq5PbRhQ_cjhi6k4t1FnWgQm8jZ6hm?usp=sharing).

qdrant-landing/static/articles_data/qa-with-cohere-and-qdrant/q-and-a-article-icon.svg

@@ -0,0 +1,80 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!-- Generator: Adobe Illustrator 19.0.0, SVG Export Plug-In . SVG Version: 6.00 Build 0)  -->
+<svg enable-background="new 0 0 512 512" version="1.1" viewBox="0 0 512 512" xml:space="preserve" xmlns="http://www.w3.org/2000/svg">
+<path d="m486.88 238.36c13.851 0 25.119-11.268 25.119-25.119v-153.92c0-13.851-11.268-25.119-25.119-25.119h-248.52v-9.086c0-13.851-11.268-25.119-25.119-25.119h-153.92c-13.851 0-25.119 11.268-25.119 25.119v9.086h-9.086c-13.851 0-25.119 11.268-25.119 25.119v153.92c0 13.851 11.268 25.119 25.119 25.119h103.15v35.273h-68.943c-13.851 0-25.119 11.268-25.119 25.119v9.086h-9.086c-13.851-1e-3 -25.119 11.268-25.119 25.118v153.92c0 13.851 11.268 25.119 25.119 25.119h461.76c13.851 0 25.119-11.268 25.119-25.119v-153.92c0-13.851-11.268-25.119-25.119-25.119h-248.52v-9.086c0-13.851-11.268-25.119-25.119-25.119h-68.944v-35.273zm-248.52-59.323v-128.8h248.52c5.01 0 9.086 4.076 9.086 9.086v153.92c0 5.01-4.076 9.086-9.086 9.086h-342.58v-18.171h68.944c13.85-1e-3 25.118-11.27 25.118-25.12zm-213.24 43.29c-5.01 0-9.086-4.076-9.086-9.086v-153.92c0-5.01 4.076-9.086 9.086-9.086h9.086v128.8c0 13.851 11.268 25.119 25.119 25.119h68.944v18.171zm213.24 230.35v-128.8h248.52c5.01 0 9.086 4.076 9.086 9.086v153.92c0 5.01-4.076 9.086-9.086 9.086h-461.76c-5.01 0-9.086-4.076-9.086-9.086v-153.92c0-5.01 4.076-9.086 9.086-9.086h9.086v128.8c0 13.851 11.268 25.119 25.119 25.119h153.92c13.85 0 25.118-11.268 25.118-25.119zm-25.119-163.01c5.01 0 9.086 4.076 9.086 9.086v153.92c0 5.01-4.076 9.086-9.086 9.086h-153.92c-5.01 0-9.086-4.076-9.086-9.086v-153.92c0-5.01 4.076-9.086 9.086-9.086zm-153.92-101.54c-5.01 0-9.086-4.076-9.086-9.086v-153.92c0-5.01 4.076-9.086 9.086-9.086h153.92c5.01 0 9.086 4.076 9.086 9.086v153.92c0 5.01-4.076 9.086-9.086 9.086h-153.92z" fill="#ffffff"/>
+<g fill="#ffffff">
+	<g fill="#ffffff">
+		<path d="m187.06 93.528c0-27.995-22.777-50.772-50.772-50.772s-50.772 22.777-50.772 50.772v17.102c0 27.995 22.777 50.772 50.772 50.772 11.111 0 21.392-3.597 29.761-9.675l7.326 7.326c1.565 1.565 3.617 2.348 5.668 2.348s4.103-0.782 5.668-2.348c3.131-3.131 3.131-8.207 0-11.337l-7.326-7.326c6.078-8.369 9.675-18.65 9.675-29.761v-17.101zm-16.033 17.102c0 6.68-1.899 12.921-5.18 18.222l-6.789-6.789c-3.131-3.131-8.207-3.131-11.337 0-3.131 3.131-3.131 8.207 0 11.337l6.789 6.789c-5.302 3.28-11.543 5.18-18.222 5.18-19.155 0-34.739-15.584-34.739-34.739v-17.102c0-19.155 15.584-34.739 34.739-34.739s34.739 15.584 34.739 34.739v17.102z"/>
+	
+
+
+	
+		<path d="m186.82 425.08-23.225-92.899c-2.325-9.295-10.64-15.788-20.222-15.788h-14.175c-9.582 0-17.898 6.492-20.22 15.789l-23.225 92.899c-1.074 4.295 1.538 8.646 5.833 9.721 4.293 1.076 8.647-1.537 9.722-5.833l4.896-19.581h60.167l4.896 19.581c0.911 3.643 4.179 6.075 7.771 6.075 0.643 0 1.298-0.078 1.951-0.242 4.294-1.074 6.905-5.428 5.831-9.722zm-76.61-31.726 14.321-57.284c0.537-2.145 2.455-3.643 4.667-3.643h14.175c2.212 0 4.13 1.497 4.667 3.643l14.321 57.284h-52.151z"/>
+	
+
+
+	
+		<path d="m427.02 76.96h-153.92c-4.427 0-8.017 3.589-8.017 8.017s3.589 8.017 8.017 8.017h153.92c4.427 0 8.017-3.589 8.017-8.017s-3.59-8.017-8.017-8.017z"/>
+	
+
+
+	
+		<path d="m461.23 145.37h-188.13c-4.427 0-8.017 3.589-8.017 8.017s3.589 8.017 8.017 8.017h188.12c4.427 0 8.017-3.589 8.017-8.017s-3.589-8.017-8.016-8.017z"/>
+	
+
+
+	
+		<path d="m461.23 179.57h-188.13c-4.427 0-8.017 3.589-8.017 8.017s3.589 8.017 8.017 8.017h188.12c4.427 0 8.017-3.589 8.017-8.017s-3.589-8.017-8.016-8.017z"/>
+	
+
+
+	
+		<path d="m307.31 111.16h-34.205c-4.427 0-8.017 3.589-8.017 8.017s3.589 8.017 8.017 8.017h34.205c4.427 0 8.017-3.589 8.017-8.017s-3.59-8.017-8.017-8.017z"/>
+	
+
+
+	
+		<path d="m461.23 111.16h-119.72c-4.427 0-8.017 3.589-8.017 8.017s3.589 8.017 8.017 8.017h119.72c4.427 0 8.017-3.589 8.017-8.017s-3.589-8.017-8.016-8.017z"/>
+	
+
+
+	
+		<path d="m427.02 350.6h-153.92c-4.427 0-8.017 3.589-8.017 8.017s3.589 8.017 8.017 8.017h153.92c4.427 0 8.017-3.589 8.017-8.017s-3.59-8.017-8.017-8.017z"/>
+	
+
+
+	
+		<path d="m461.23 419.01h-188.13c-4.427 0-8.017 3.589-8.017 8.017 0 4.427 3.589 8.017 8.017 8.017h188.12c4.427 0 8.017-3.589 8.017-8.017 0-4.427-3.589-8.017-8.016-8.017z"/>
+	
+
+
+	
+		<path d="m461.23 453.21h-188.13c-4.427 0-8.017 3.589-8.017 8.017 0 4.427 3.589 8.017 8.017 8.017h188.12c4.427 0 8.017-3.589 8.017-8.017s-3.589-8.017-8.016-8.017z"/>
+	
+
+
+	
+		<path d="m307.31 384.8h-34.205c-4.427 0-8.017 3.589-8.017 8.017 0 4.427 3.589 8.017 8.017 8.017h34.205c4.427 0 8.017-3.589 8.017-8.017s-3.59-8.017-8.017-8.017z"/>
+	
+
+
+	
+		<path d="m461.23 384.8h-119.72c-4.427 0-8.017 3.589-8.017 8.017 0 4.427 3.589 8.017 8.017 8.017h119.72c4.427 0 8.017-3.589 8.017-8.017s-3.589-8.017-8.016-8.017z"/>
+	</g>
+</g>
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+</svg>