When a RAG system searches for something, it does not match the words in your question against the words in its documents. It matches meaning. Embeddings are how meaning becomes something a computer can compare, and every part of vector search builds on them.
What an embedding is#
An embedding is a list of numbers, usually hundreds or thousands of them, that represents a piece of text. You can think of that list as a point in a space with many dimensions. Each piece of text you embed becomes one point.
The key property is that texts with similar meanings get similar numbers, even when they share no words.
| Text A | Text B | Vectors |
|---|---|---|
| "the dog ran fast" | "the puppy sprinted quickly" | Very close |
| "the dog ran fast" | "quarterly revenue report" | Far apart |
So "close in the space" stands in for "similar in meaning". Search then becomes a geometry problem: find the points nearest to the question's point.
Embedding models are separate models#
The chat model you send prompts to does not produce these vectors. An embedding model is a separate model, trained for one job: turn text into a vector.
Training pushes matching pairs close together, such as a question and the passage that answers it. It also pushes unrelated pairs far apart. After enough of that, the model maps meaning to geometry.
This is a different thing from the per-token vectors inside a chat model, described in tokens. Those vectors are internal working state. An embedding model gives you one vector for a whole piece of text, and you store and search it yourself.
One model for the query and the documents#
The query and the documents must be embedded by the same model. Each model learns its own space, so vectors from two different models are not comparable at all. A distance between them means nothing.
If you switch embedding models, re-embed every document from scratch. The old vectors are meaningless to the new model.
Plan for this early. Store which model produced each vector, and treat a model change as a full re-index, not a config tweak.
Short line#
An embedding is a vector that represents a piece of text, made by a separate embedding model so that similar meanings land close together. Embed the query and the documents with the same model, or the vectors cannot be compared.
Key takeaways#
- An embedding is a list of hundreds or thousands of numbers that represents a piece of text.
- Texts with similar meanings get nearby vectors, even with no words in common.
- Embedding models are separate from chat models and are trained to pull matching pairs together and push unrelated pairs apart.
- Query and documents must use the same embedding model.
- Changing the embedding model means re-embedding every document.