Dreistein Media Development Home

RAG and CAG: how AI answers from your own knowledge

Language models such as Mistral Large or GPT are impressive – but they know nothing about your company, your products and your documents. RAG (Retrieval-Augmented Generation) and CAG (Cache-Augmented Generation) are the two techniques that close this gap: they hand the model exactly the knowledge it needs at answer time. This page explains both from the ground up – and shows how our platform dAi Pro combines them.

The core problem: language models do not know your knowledge

A language model learns its knowledge during training – from public texts, with a fixed cut-off date. Three consequences:

  • Knowledge gaps – internal documents, product details and current prices simply never appear in training.
  • Hallucinations – asked about things it does not know, a model invents plausible-sounding answers instead of staying silent.
  • Staleness – whatever happened after the training cut-off does not exist for the model.

The way out is always the same: you hand the model the relevant knowledge at runtime – as context for the question. RAG and CAG differ only in how this knowledge is selected and delivered.

What is RAG (Retrieval-Augmented Generation)?

RAG literally means "generation augmented by retrieval" and runs in four steps:

  • 1. Understand the question – the visitor's question is translated into a vector (see embeddings below).
  • 2. Retrieval – the knowledge base is searched for the passages closest in meaning to the question.
  • 3. Augmentation – the best matches are handed to the language model together with the question.
  • 4. Generation – the model formulates the answer exclusively from the supplied material.

The charm: the knowledge base can be arbitrarily large, because only the relevant excerpts travel to the model per question. New content is usable immediately – without any retraining.

The building blocks: chunking, embeddings, vector database

For retrieval to work, the knowledge base is prepared:

  • Chunking – documents are split into meaningful sections. Good chunking respects headings, paragraphs and tables; bad chunking cuts through sentences and destroys context. dAi Pro carries headings into every chunk as context, so even a single match remains locatable.
  • Embeddings – an embedding model (such as mistral-embed) translates every section into a vector of hundreds of numbers. Texts with similar meaning end up close together – "warranty period" and "guarantee term" are neighbours in vector space even though they share no words.
  • Vector database – the vectors live in a database specialised for this. dAi Pro uses PostgreSQL with the open pgvector extension – proven technology on your own infrastructure instead of a third-party cloud index.
  • Similarity search – for each question, the nearest vectors are found (nearest-neighbour search) and returned as candidates.

Hybrid search: vector and full-text combined

Pure vector search has one weakness: exact terms – article numbers, product names, technical vocabulary – sometimes drown in the cloud of meaning. Pure full-text search has the opposite weakness: it only finds what is literally written. The solution is hybrid search: both techniques run in parallel and their result lists are merged (for example via reciprocal rank fusion), so that both semantically similar and word-exact matches rank at the top. dAi Pro works hybrid throughout – in chat as well as in website search.

Grounding: verifiable answers instead of claims

RAG makes answers verifiable – if implemented rigorously:

  • Strict grounding – the model is instructed to answer exclusively from the supplied passages. If the answer is not in the material, the chat says so honestly instead of guessing.
  • Source citations – because it is known which documents fed the answer, every answer can link its sources. Visitors jump straight to the originating page or document.
  • Traceability – the log shows which question was answered from which sources.

You can experience this principle on this very website: the chat answers questions from the content of dreistein.de – with clickable sources, including this knowledge page.

What is CAG (Cache-Augmented Generation)?

CAG takes the opposite route: instead of searching for passages per question, a complete knowledge bundle is prepared and handed to the model as a whole – made possible by large context windows (128,000 tokens and more) and by prompt caching: the provider remembers the unchanged knowledge block between requests, so it does not have to be reprocessed and re-paid every time.

  • Speed advantage – the retrieval step disappears entirely; the answer starts sooner.
  • Overview advantage – the model sees the whole bundle and can connect facts across document boundaries.
  • Size limit – the bundle must fit into the context window; CAG suits compact, thematically closed bodies of knowledge.
  • Maintenance – when the knowledge changes, the bundle is rebuilt and the cache warmed up; dAi Pro does this automatically in the background.

RAG or CAG? The direct comparison

Each technique has its terrain:

  • Knowledge size – RAG scales to arbitrarily large corpora; CAG needs a bundle that fits the context window.
  • Response time – CAG usually answers faster without the retrieval step; RAG pays for retrieval with some latency.
  • Cost – CAG lives off the prompt cache (after warm-up the knowledge block costs only fractions); RAG keeps contexts small and thus inexpensive.
  • Question type – overview and cross-cutting questions suit CAG; detail questions in large corpora suit RAG.
  • Freshness – RAG uses new content immediately after indexing; with CAG the bundle refreshes in the background.

The right answer is therefore rarely "either–or" but "both, chosen automatically" – which is exactly what dAi Pro does.

Automatic routing: the classifier decides

In dAi Pro, an upstream query classifier examines every question: is it a detail question that needs targeted retrieval? An overview question for the bundle? Small talk that needs no knowledge base at all? Depending on the verdict, the answer comes via RAG, via CAG or directly – switching mid-conversation when needed. For visitors this is invisible; what shows is the result: fast, fitting, sourced answers. The log records which strategy answered – keeping the behaviour analysable.

Cost and speed in practice

RAG and CAG are economically interesting too:

  • Small contexts instead of giant prompts – RAG passes only relevant passages, keeping every call inexpensive.
  • Prompt caching – with CAG, the warmed-up bundle costs only a fraction of the normal price per request.
  • Answer cache – dAi Pro answers recurring questions without any model call at all.
  • Full transparency – every call is logged with token usage and cost in euros.

How fast the techniques answer with different models is shown by our RAG comparison – four providers, the same knowledge base, measured live continuously.

The knowledge base: ingesting, refining, keeping fresh

RAG is only as good as its knowledge base. Three tasks decide:

  • Ingesting – content comes from websites, TYPO3, PDFs, office files and systems such as Confluence, SharePoint or databases. dAi Pro ships 16 data source types for this.
  • Refining – clean extraction decides answer quality: OCR for scans, structured tables, described figures, clean paragraph structure.
  • Keeping fresh – sources change. Automatic synchronisation detects new, changed and deleted content; a freshness guardian clears out dead material so the chat never cites outdated information.

Multilingualism in RAG and CAG

Multilingual websites raise particular demands: every document carries its language, embeddings work across languages, and yet a German visitor should see German sources – an English one the English versions. dAi Pro therefore keeps the knowledge base per language, detects the language of every question (conversation history helps with short messages) and answers, sources included, in the visitor's language – with a defined fallback when a version is missing. CAG bundles exist per language as well.

Limits and pitfalls – honestly considered

RAG and CAG are the state of the art – but not magic:

  • Retrieval can miss – if the right passage is not found, it is missing from the answer. Hybrid search and good chunking reduce the risk; they do not eliminate it.
  • Garbage in, garbage out – contradictory or outdated documents produce contradictory answers. Corpus maintenance is part of operations; tools such as dAi Pro's gap radar show where questions went unanswered.
  • Residual hallucination risk – even with grounding, a model can overstate. Strict prompts, source citations and logs keep this controllable.
  • No calculator – price calculations or stock queries belong to a tool call (function calling) against a real system, not to the text base.

RAG and CAG vs. fine-tuning

"Can't we just retrain the model on our data?" – the most common alternative question. The short answer: fine-tuning is suited to style and behaviour, not to facts. Retrained facts go stale immediately, are expensive to refresh, cannot be sourced and are hard to delete – there are no citations. RAG/CAG keep knowledge and model separate: content stays current, traceable and cleanly deletable under the GDPR. Where a certain tone or answer pattern is desired, assistant profiles and example answers are usually enough – which is how dAi Pro does it.

Data protection: why RAG is the GDPR-friendly architecture

RAG is data-minimising by design: the knowledge base – originals, chunks, vectors – stays entirely within your infrastructure. Per question, only the visitor's query and the few relevant passages travel to the AI provider; never the full corpus. With a European provider such as Mistral AI (EU processing, data processing agreement under Art. 28 GDPR), this yields an architecture without US third-country transfers. Details on the overall architecture: GDPR-compliant AI.

RAG and CAG in dAi Pro

Our platform dAi Pro implements everything described here in one product: hybrid search on pgvector, strict grounding with source citations, CAG bundles with automatic warm-up, classifier routing, an answer cache, 16 data source types, a freshness guardian, multilingualism and cost logging in euros – as a TYPO3 extension or standalone web application, with freely selectable AI providers. The feature list shows the full scope; dAi offers the free entry with RAG and CAG for TYPO3 sites.

Frequently asked questions about RAG and CAG

RAG stands for Retrieval-Augmented Generation: before answering, matching passages are retrieved from a knowledge base and given to the language model as context. The AI thus answers from your content instead of its training knowledge.

CAG stands for Cache-Augmented Generation: a complete, compact knowledge bundle sits pre-warmed in the language model's prompt cache and accompanies every question as a whole – particularly fast and, once warmed up, very inexpensive.

Yes, content is stored as vectors for semantic retrieval. But it need not be a new system: dAi Pro uses PostgreSQL with the open pgvector extension – on your own infrastructure.

Far less than a model without a knowledge base, but a residual risk remains. Strict grounding (answer only from the sources), source citations and logs keep the risk small and controllable – honest vendors do not promise zero.

With RAG: yes, right after indexing – on this website that typically takes a few minutes after a page change. CAG bundles are rebuilt automatically in the background.

For factual knowledge almost always RAG: current, verifiable, deletable and cheaper to operate. Fine-tuning pays off for style and behaviour patterns – facts belong in the knowledge base, not in the model weights.

With RAG practically unlimited – only the relevant excerpts travel to the model per question anyway. CAG is limited by the context window and suits compact, thematically closed corpora; that is why dAi Pro combines both techniques.

In principle all current chat models. What matters is a solid embedding model for retrieval and a chat model that follows instructions (grounding!) reliably. dAi Pro supports Mistral, OpenAI, Anthropic and Google Gemini – our RAG comparison measures them continuously on the same knowledge base.

Yes – RAG is in fact the most privacy-friendly AI architecture: the knowledge base stays in-house, and only the question and relevant passages go to the provider. With EU providers such as Mistral and a data processing agreement, the overall picture is clean.

Running costs consist mainly of AI calls (fractions of a cent to a few cents per answer, depending on the model) plus hosting of the vector database. The answer cache and CAG reduce call costs considerably; dAi Pro reports every call in euros. Our RAG comparison page compares provider token prices.