mirror of
https://github.com/qdrant/landing_page.git
synced 2026-09-25 22:18:30 +02:00
45 lines
2.7 KiB
Markdown
45 lines
2.7 KiB
Markdown
---
|
||
draft: false
|
||
preview_image: /blog/from_cms/nils-thumbnail.png
|
||
title: "From Content Quality to Compression: The Evolution of Embedding Models
|
||
at Cohere with Nils Reimers"
|
||
slug: cohere-embedding-v3
|
||
short_description: Nils Reimers head of machine learning at Cohere shares the
|
||
details about their latest embedding model.
|
||
description: Nils Reimers head of machine learning at Cohere comes on the recent
|
||
vector space talks to share details about their latest embedding V3 model.
|
||
date: 2023-11-19T12:48:36.622Z
|
||
author: Demetrios Brinkmann
|
||
featured: false
|
||
author_link: https://www.linkedin.com/in/dpbrinkm/
|
||
tags:
|
||
- Vector Space Talk
|
||
- Cohere
|
||
- Embedding Model
|
||
categories:
|
||
- News
|
||
- Vector Space Talk
|
||
---
|
||
|
||
For the second edition of our Vector Space Talks we were joined by none other than Cohere’s Head of Machine Learning Nils Reimers.
|
||
|
||
## Key Takeaways
|
||
|
||
Let's dive right into the five key takeaways from Nils' talk:
|
||
|
||
1. Content Quality Estimation: Nils explained how embeddings have traditionally focused on measuring topic match, but content quality is just as important. He demonstrated how their model can differentiate between informative and non-informative documents.
|
||
|
||
2. Compression-Aware Training: He shared how they've tackled the challenge of reducing the memory footprint of embeddings, making it more cost-effective to run vector databases on platforms like [Qdrant](https://cloud.qdrant.io/login).
|
||
|
||
3. Reinforcement Learning from Human Feedback: Nils revealed how they've borrowed a technique from reinforcement learning and applied it to their embedding models. This allows the model to learn preferences based on human feedback, resulting in highly informative responses.
|
||
|
||
4. Evaluating Embedding Quality: Nils emphasized the importance of evaluating embedding quality in relative terms rather than looking at individual vectors. It's all about understanding the context and how embeddings relate to each other.
|
||
|
||
5. New Features in the Pipeline: Lastly, Nils gave us a sneak peek at some exciting features they're developing, including input type support for Langchain and improved compression techniques.
|
||
|
||
Now, here's a fun fact from the episode: Did you know that the content quality estimation model *can't* differentiate between true and fake statements? It's a challenging task, and the model relies on the information present in its pretraining data.
|
||
|
||
We loved having Nils as our guest, check out the full talk below. If you or anyone you know would like to come on the Vector Space Talks
|
||
|
||
<iframe width="560" height="315" src="https://www.youtube.com/embed/Abh3YCahyqU?si=OB4FXhTivsLLXzQV" title="YouTube video player" frameborder="0" allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share" allowfullscreen></iframe>
|