Merge pull request #586 from qdrant/mjang-dust-and-qdrant

Set up Dust & Qdrant blog

qdrant-landing/content/blog/case-study-dust.md

@@ -0,0 +1,128 @@
+---
+title: "Dust and Qdrant: Using AI to Unlock Company Knowledge and Drive Employee Productivity"
+draft: false
+slug: dust-and-qdrant 
+#short_description: 
+description: Using AI to Unlock Company Knowledge and Drive Employee Productivity
+preview_image: /case-studies/dust/preview.png # Change this
+
+social_preview_image: /case-studies/dust/preview.png # Optional image used for link previews
+title_preview_image: /case-studies/dust/preview.png # Optional image used for blog post title
+small_preview_image: /case-studies/dust/preview.png # Optional image used for small preview in the list of blog posts
+date: 2024-02-06T07:03:26-08:00
+author: Manuel Meyer
+featured: false
+tags: 
+  - Dust
+  - case_study
+weight: 0 
+---
+
+One of the major promises of artificial intelligence is its potential to
+accelerate efficiency and productivity within businesses, empowering employees
+and teams in their daily tasks. The French company [Dust](https://dust.tt/), co-founded by former
+Open AI Research Engineer [Stanislas Polu](https://www.linkedin.com/in/spolu/), set out to deliver on this promise by
+providing businesses and teams with an expansive platform for building
+customizable and secure AI assistants.
+
+## Challenge
+
+"The past year has shown that large language models (LLMs) are very useful but
+complicated to deploy," Polu says, especially in the context of their
+application across business functions. This is why he believes that the goal of
+augmenting human productivity at scale is especially a product unlock and not
+only a research unlock, with the goal to identify the best way for companies to
+leverage these models. Therefore, Dust is creating a product that sits between
+humans and the large language models, with the focus on supporting the work of
+a team within the company to ultimately enhance employee productivity.
+
+A major challenge in leveraging leading LLMs like OpenAI, Anthropic, or Mistral
+to their fullest for employees and teams lies in effectively addressing a
+company's wide range of internal use cases. These use cases are typically very
+general and fluid in nature, requiring the use of very large language models.
+Due to the general nature of these use cases, it is very difficult to finetune
+the models - even if financial resources and access to the model weights are
+available. The main reason is that “the data that’s available in a company is
+a drop in the bucket compared to the data that is needed to finetune such big
+models accordingly,” Polu says, “which is why we believe that retrieval
+augmented generation is the way to go until we get much better at fine tuning”.
+
+For successful retrieval augmented generation (RAG) in the context of employee
+productivity, it is important to get access to the company data and to be able
+to ingest the data that is considered ‘shared knowledge’ of the company. This
+data usually sits in various SaaS applications across the organization.
+
+## Solution
+
+Dust provides companies with the core platform to execute on their GenAI bet
+for their teams by deploying LLMs across the organization and providing context
+aware AI assistants through RAG. Users can manage so-called data sources within
+Dust and upload files or directly connect to it via APIs to ingest data from
+tools like Notion, Google Drive, or Slack. Dust then handles the chunking
+strategy with the embeddings models and performs retrieval augmented generation.
+
+![solution-laptop-screen](/case-studies/dust/laptop-solutions.jpg)
+
+For this, Dust required a vector database and evaluated different options
+including Pinecone and Weaviate, but ultimately decided on Qdrant as the
+solution of choice. “We particularly liked Qdrant because it is open-source,
+written in Rust, and it has a well-designed API,” Polu says. For example, Dust
+was looking for high control and visibility in the context of their rapidly
+scaling demand, which made the fact that Qdrant is open-source a key driver for
+selecting Qdrant. Also, Dust's existing system which is interfacing with Qdrant,
+is written in Rust, which allowed Dust to create synergies with regards to
+library support.
+
+When building their solution with Qdrant, Dust took a two step approach:
+
+1. **Get started quickly:** Initially, Dust wanted to get started quickly and opted for
+[Qdrant Cloud](https://qdrant.to/cloud), Qdrant’s managed solution, to reduce the administrative load on
+Dust’s end. In addition, they created clusters and deployed them on Google
+Cloud since Dust wanted to have those run directly in their existing Google
+Cloud environment. This added a lot of value as it allowed Dust to centralize
+billing and increase security by having the instance live within the same VPC.
+“The early setup worked out of the box nicely,” Polu says.
+
+2. **Scale and optimize:** As the load grew, Dust started to take advantage of Qdrant’s
+features to tune the setup for optimization and scale. They started to look into
+how they map and cache data, as well as applying some of Qdrant’s [built-in
+compression features](https://qdrant.tech/documentation/guides/quantization/). In particular, Dust leveraged the control of the [MMAP
+payload threshold](https://qdrant.tech/documentation/concepts/storage/#configuring-memmap-storage) as well as [Scalar Quantization](https://qdrant.tech/articles/scalar-quantization/), which enabled Dust to manage
+the balance between storing vectors on disk and keeping quantized vectors in RAM,
+more effectively. “This allowed us to scale smoothly from there,” Polu says.
+
+## Results
+
+Dust has seen success in using Qdrant as their vector database of choice, as Polu
+acknowledges: “Qdrant’s ability to handle large-scale models and the flexibility
+it offers in terms of data management has been crucial for us. The observability
+features, such as historical graphs of RAM, Disk, and CPU, provided by Qdrant are
+also particularly useful, allowing us to plan our scaling strategy effectively.”
+
+![“We were able to reduce the footprint of vectors in memory, which led to a significant cost reduction as
+we don’t have to run lots of nodes in parallel. While being memory-bound, we were
+able to push the same instances further with the help of quantization. While you
+get pressure on MMAP in this case you maintain very good performance even if the
+RAM is fully used. With this we were able to reduce our cost by 2x.” - Stanislas Polu, Co-Founder of Dust](/case-studies/dust/Dust-Quote.jpg)
+
+Dust was able to scale its application with Qdrant while maintaining low latency
+across hundreds of thousands of collections with retrieval only taking
+milliseconds, as well as maintaining high accuracy. Additionally, Polu highlights
+the efficiency gains Dust was able to unlock with Qdrant: "We were able to reduce the footprint of vectors in memory, which led to a significant cost reduction as
+we don’t have to run lots of nodes in parallel. While being memory-bound, we were
+able to push the same instances further with the help of quantization. While you
+get pressure on MMAP in this case you maintain very good performance even if the
+RAM is fully used. With this we were able to reduce our cost by 2x."
+
+
+
+## Outlook
+
+Dust will continue to build out their platform, aiming to be the platform of
+choice for companies to execute on their internal GenAI strategy, unlocking
+company knowledge and driving team productivity. Over the coming months, Dust
+will add more connections, such as Intercom, Jira, or Salesforce. Additionally,
+Dust will expand on its structured data capabilities.
+
+To learn more about how Dust uses Qdrant to help employees in their day to day
+tasks, check out our [Vector Space Talk](https://www.youtube.com/watch?v=toIgkJuysQ4) featuring Stanislas Polu, Co-Founder of Dust.