A model only knows what it memorized during training. That is fine for general knowledge, but most real applications need answers about recent events or your own data. RAG (retrieval-augmented generation) closes that gap, and it is the most common way to build an LLM feature on top of real documents.
What RAG is#
Instead of relying on what the model memorized, you retrieve relevant documents at question time and hand them to the model as context. The model then answers by reading those documents, not by recalling its training data.
The name describes the steps. Retrieval finds the documents. They augment the prompt. The model generates the answer from them. The search part is the similarity search over embeddings covered in the previous category.
The two problems it solves#
Frozen and public-only knowledge#
A model's knowledge stops at its training cutoff, as described in training. It also never saw your private data: internal docs, tickets, contracts or product specs. Ask about either and the model guesses.
With RAG, fresh or private documents are retrieved on demand. Update a document and the next question sees the new version, with no retraining.
No sources to cite#
A model cannot cite a source it does not have. Its training data is blended into its weights, so it cannot point to where a fact came from.
With RAG, the source documents sit in the prompt. The model can cite exactly which document each part of its answer came from, and a reader can check it.
| Problem | Without RAG | With RAG |
|---|---|---|
| Knowledge cutoff | Knows nothing after it | Retrieves current documents |
| Private data | Never saw it | Retrieves it on demand |
| Citations | No source to point to | Cites the retrieved document |
RAG also lowers hallucination, because the model continues from real text instead of filling gaps. The reducing hallucinations note covers why, and RAG vs long context covers what it does not fix.
Short line#
RAG retrieves relevant documents at question time and puts them in the prompt, so the model answers from real, current or private sources instead of training memory, and can cite where each answer came from.
Key takeaways#
- RAG retrieves documents at question time and gives them to the model as context.
- It fixes the knowledge cutoff: fresh documents are retrieved on demand, with no retraining.
- It gives the model access to private data it never saw in training.
- Because sources are in the prompt, the model can cite them and readers can verify.