Add first draft of the article about LangChain integration

2026-09-27 15:08:30 +02:00 · 2023-01-27 18:05:22 +01:00
parent 6351be1681
commit 85966eef4c
12 changed files with 152 additions and 0 deletions
@@ -0,0 +1,127 @@
 ---
 title: "Question Answering with LangChain and Qdrant without boilerplate"
 short_description: "Large Language Models might be developed fast with modern tool. Here is how!"
 description: "We combined LangChain, pretrained LLM from OpenAI, SentenceTransformers and Qdrant to create a Q&A system with just a few lines of code."
 social_preview_image: /articles_data/langchain-integration/social_preview.png
 small_preview_image: /articles_data/langchain-integration/chain.svg
 preview_dir: /articles_data/langchain-integration/preview
 weight: 6
 author: Kacper Łukawski
 author_link: https://medium.com/@lukawskikacper
 date: 2022-11-29T15:45:00+01:00
 draft: false
 keywords:
  - vector search
  - langchain
  - llm
  - large language models
  - question answering
  - openai
  - embeddings
 ---
 Building applications with Large Language Models don't have to be complicated. A lot has been going on recently to simplify the development, 
 so you can utilize already pre-trained models and support even complex pipelines with a few lines of code. [LangChain](https://langchain.readthedocs.io) 
 is one of the libraries which provide unified interfaces to different libraries, so you can avoid writing the boilerplate code and focus
 on the value you want to bring.
 ## Question Answering with Qdrant in the loop
 It has been reported millions of times recently, but let's say that again. ChatGPT-like models struggle with generating factual statements if no context 
 is provided. They have some general knowledge but cannot guarantee to produce a valid answer consistently. Thus, it is better to provide some facts we 
 know are actual, so it can just choose the valid parts and extract them from all the provided contextual data to give a comprehensive answer. Vector database, 
 such as Qdrant, is of great help here, as their ability to perform a semantic search over huge knowledge base is crucial to preselect some possibly valid 
 documents, so they can be provided into the LLM. That's also one of the **chains** implemented in LangChain, which is called `VectorDBQA`. And Qdrant got 
 integrated with the library, so it might be used to build it effortlessly.
 ### What do we need?
 Surprisingly enough, there will be two models required to set things up. First of all, we need an embedding model that will convert the set of facts into
 vectors, and store those into Qdrant. That's an identical process like in any other semantic search application. We're going to use one of the 
 `SentenceTransformers` models, so it can be hosted locally. The embeddings created by that model will be put into Qdrant and used to retrieve the most 
 similar documents, given query. 
 However, when we receive a query, there are two steps involved. First of all, we ask Qdrant to provide the most relevant documents and simply combine all 
 of them into a single text. Then, we build a prompt to the LLM (in our case OpenAI), including those documents as a context, of course together with the 
 question asked. So the input to the LLM looks like following:
 ```text
 Use the following pieces of context...
 It's as certain as 2 + 2 = 4
 ...
 Question: How much is 2 + 2?
 Helpful Answer:
 ```
 There might be several context documents combined, and its solely up to LLM to choose the right piece of content. But our expectation is, the model should 
 respond with just `4`.
 Why do we need two different models? They solve some different tasks. The first model performs feature extraction, by converting the text into vectors, while
 the second one helps in text generation or summarization. Disclaimer: This is not the only way to solve that task with LangChain. Such a chain is called `stuff`
 in the library nomenclature.
 ![](/articles_data/langchain-integration/flow-diagram.png)
 Enough theory! This sounds like a pretty complex applications, as it involves several systems. But with LangChain, it might be implemented in just a few lines 
 of code, thanks to the recent integration with Qdrant. We're not even going to work directly with `QdrantClient`, as everything is already done in the background
 by LangChain. If you want to get into the source code right away, all the processing available as a 
 [Google Colab notebook](https://colab.research.google.com/drive/19RxxkZdnq_YqBH5kBV10Rt0Rax-kminD?usp=sharing).
 ## Implementing Question Answering with LangChain and Qdrant
 ### Configuration
 A journey of a thousand miles begins with a single step, in our case with the configuration of all the services. We'll be using [Qdrant Cloud](https://qdrant.tech),
 so we need an API key. The same is for OpenAI - the API key has to be obtained from their website.
 ![](/articles_data/langchain-integration/code-configuration.png)
 ### Building the knowledge base
 We also need some facts from which the answers will be generated from. There is plenty of public datasets available, and 
 [Natural Questions](https://ai.google.com/research/NaturalQuestions/visualization) is one of them. It consists of whole HTML content of the websites they were
 scraped from. That means we need some preprocessing to extract plain text content. As a result we're going to have two lists of strings - one for questions
 and the other ones for the answers.
 The answers have to be vectorized with the first of our models. The `sentence-transformers/all-mpnet-base-v2` is one of the possibilities, but there are some
 other options available. LangChain will handle that part of the process in a single function call.
 ![](/articles_data/langchain-integration/code-qdrant.png)
 ### Setting up QA with Qdrant in a loop
 `VectorDBQA` is a chain that performs the process described above. So it, first of all, loads some facts from Qdrant and then feed them into OpenAI LLM
 that should analyze them to find the answer to given question. The only last thing to do before using it is putting things together, also with a single 
 function call.
 ![](/articles_data/langchain-integration/code-vectordbqa.png)
 ## Testing out the chain
 And that's it! We can put some queries, and LangChain will perform all the required processing to find the answer in a provided context.
 ![](/articles_data/langchain-integration/code-answering.png)
 ```text
 > what kind of music is scott joplin most famous for
 Scott Joplin is most famous for composing ragtime music.
 > who died from the band faith no more
 Chuck Mosley
 > when does maggie come on grey's anatomy
 Maggie first appears in season 10, episode 1, which aired on September 26, 2013.
 > can't take my eyes off you lyrics meaning
 I don't know.
 > who lasted the longest on alone season 2
 David McIntyre lasted the longest on Alone season 2, with a total of 66 days.
 ```
 The great thing about such a setup is that the knowledge base might be easily extended with some new facts and those will be included in the prompts
 sent to LLM later on. Of course, assuming their similarity to given question will be in top results returned by Qdrant.
 If you want to run the chain on your own, the simplest way to reproduce it is to open the 
 [Google Colab notebook](https://colab.research.google.com/drive/19RxxkZdnq_YqBH5kBV10Rt0Rax-kminD?usp=sharing).
@@ -0,0 +1,25 @@
 <?xml version="1.0" encoding="iso-8859-1"?>
 <!-- Uploaded to: SVG Repo, www.svgrepo.com, Generator: SVG Repo Mixer Tools -->
 <!DOCTYPE svg PUBLIC "-//W3C//DTD SVG 1.1//EN" "http://www.w3.org/Graphics/SVG/1.1/DTD/svg11.dtd">
 <svg fill="#000000" version="1.1" id="Capa_1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink" 
 	 width="800px" height="800px" viewBox="0 0 393.642 393.642"
 	 xml:space="preserve">
 <g>
 	<path fill="#ffffff" d="M36.846,34.046c-42.84,33.66-42.84,92.412-27.54,140.148c22.032,65.484,80.784,119.34,151.776,123.012
 		c50.796,2.448,98.533-48.349,96.697-98.532c1.836-4.896,2.447-9.18,3.06-14.688c0.612-7.344-0.612-15.912-6.12-20.196
 		c-1.224-0.612-2.448-0.612-3.672-0.612c-1.836-0.612-3.06-0.612-4.284-0.612c-12.852,2.448-29.376,1.224-41.004,7.956
 		c-2.448-0.612-5.508,0.612-6.12,3.06c-9.792,33.048-36.108,53.244-70.992,37.944C103.554,200.51,87.03,172.97,78.462,148.49
 		c-7.956-22.644-5.508-50.184,15.3-64.26c22.032-15.3,49.572-0.612,66.096,14.688c2.448,1.836,4.896,2.448,7.344,1.836
 		c1.224,1.836,3.672,3.06,6.12,1.836c15.912-9.792,33.049-12.852,51.408-13.464c4.284,0,7.956-4.284,6.732-9.18
 		c-2.448-7.956-6.732-13.464-12.24-18.972c0-2.448-1.224-4.284-3.06-6.12C169.038,19.97,88.866-6.958,36.846,34.046z"/>
 	<path fill="#ffffff" d="M169.65,130.742c-2.448,1.836-4.284,3.672-6.12,5.508c-14.688,13.464-23.256,32.436-25.704,52.02
 		c-1.224,5.509-1.836,11.017-1.224,17.137c0,3.06,2.448,6.12,6.12,6.12c18.36-0.612,31.212-7.956,42.228-22.645
 		c7.956-11.016,15.301-21.42,26.929-29.376c5.508-3.672,11.016-5.508,16.524-6.12c4.283,0.612,8.567,0.612,12.852,0.612
 		c18.36,3.06,36.72,17.136,48.348,29.988c22.645,23.868,36.108,61.2,31.212,93.636c-6.119,41.004-69.768,40.392-85.067,7.344
 		c-0.612-1.836-2.448-1.836-4.284-1.224c-1.836-1.225-3.672-1.225-5.508,0c-7.345,4.284-12.24,12.852-18.973,18.359
 		c-7.344,5.509-15.3,11.017-23.868,14.076c-3.06,1.225-3.672,5.508-1.836,7.344c-0.612,1.225-0.612,2.448,0,4.284
 		c1.836,3.672,2.448,8.568,6.732,9.792c6.12,14.076,21.42,26.929,33.661,33.048c22.644,12.24,50.184,11.017,74.052,6.12
 		c58.141-12.852,97.92-58.752,97.92-118.116c0-68.544-50.796-134.027-113.22-157.284C236.97,84.23,193.518,98.918,169.65,130.742z"
 		/>
 </g>
 </svg>