Retrieval-augmented generation (RAG) is the process of optimizing the output of a large language model so it references an authoritative knowledge base outside of its training data before generating a response. It matters because it is the step where an AI system fetches documents to answer you, and the step that lets it include citations to the knowledge sources in its responses.
How RAG works
A typical setup runs in stages.
- Prepare the data. The outside knowledge can come from APIs, databases, or document repositories. Embedding models convert it into numerical representations and store it in a vector database.
- Retrieve. The question is matched against that store. Vector databases allow fast retrieval based on semantic similarity, and advanced systems use semantic search and keyword search together, called hybrid search, along with a re-ranker which scores the results.
- Augment. In grounded generation, the system incorporates the retrieved information into the LLM's context.
- Generate. The model writes its answer from the question plus what it retrieved, and can point to where each part came from.
Why RAG exists
Language models stop learning when training ends. They have a knowledge cutoff, the point at which their training data was last updated. Updating them is not cheap: retraining a foundation model or fine-tuning it is computationally expensive and resource-intensive.
Without fresh sources, models also make things up. Known problems include presenting false information when the model does not have the answer, and presenting out-of-date or generic information. RAG lets developers connect the LLM to live social media feeds, news sites, or other frequently updated sources, and present information with source attribution.
An example
A software company puts a support chatbot on its help center. A customer asks whether the Pro plan includes single sign-on.
Without RAG, the model answers from its training data, which may be older than the Pro plan. With RAG, it first searches the company's docs, pulls the current pricing page, and answers from it, with a link the customer can check.
AI search engines do the same thing at web scale. The retrieval step is a search over web pages, and the pages it pulls are the ones the answer can quote and cite.
What RAG means for your website
If an AI system can only cite what it retrieves, being retrievable comes first. That comes down to a few checks.
Can the AI's bots read the page? Your AI crawler rules decide that. Is the page in the index the system searches? And does it have passages that answer a question on their own, so the retriever finds a match and the model has something to quote?
One question can also turn into many searches behind the scenes, so cover the related questions too. That is query fan-out. For founders who build with Claude Code or Codex, Shipfound checks whether GPTBot, PerplexityBot and ClaudeBot crawl the pages and whether the product gets named in repeated AI engine runs.
RAG compared with fine-tuning
RAG connects a model to external knowledge sources rather than incorporating that knowledge into the model's training data, keeping a divide between the model and the external data. Change the documents and the answers change, with no retraining.
Fine-tuning changes the model itself. It suits teaching a model a style or a task. RAG suits facts that change, like prices, docs and news.
Common mistakes
Assuming RAG ends hallucinations. It anchors the model in current, authoritative data, which lowers the risk. The answer is still only as accurate as the source it retrieved.
Assuming the model knows your site because it was in the training data. Without a retrieval step there is no citation, and the model may describe an old version of your product.
Hiding answers behind scripts or logins. If the retriever cannot fetch the text, it does not exist for the answer.
Frequently asked questions
Is ChatGPT a RAG LLM?
ChatGPT is a language model. When it searches the web before answering, it is doing retrieval-augmented generation. When it does not search, it answers from training data alone, without citations.
What is retrieval-augmented generation in simple terms?
The model looks things up before it answers, like an open-book exam instead of answering from memory.
What are the steps of RAG?
AWS describes them as creating the external data, retrieving relevant information, augmenting the prompt, and keeping the external data up to date.
How is RAG different from an LLM?
The LLM is the model. RAG is an architecture for optimizing the performance of an AI model by connecting it with external knowledge bases. It wraps around the model rather than replacing it.