Retrieval-Augmented Generation

Retrieval-Augmented Generation combines a retrieval system with a generative model. The retriever finds relevant source material, and the model uses that context to answer. RAG is a systems pattern, not a single model feature.

Command Examples

python - <<'PY'
docs = ["postgres upgrades", "kubernetes storage", "ceph recovery"]
print(len(docs))
PY

Example output and meaning:

The real checks are corpus quality, chunking, embeddings, embedding model fit, retrieval metrics, prompt assembly, and answer evaluation.

RAG Pipeline

flowchart LR
  Sources[Source documents] --> Ingest[Ingest and normalize]
  Ingest --> Chunk[Chunk with headings and metadata]
  Chunk --> Embed[Embed chunks]
  Embed --> Index[Vector/search index]
  Query[User query] --> QEmbed[Embed query]
  QEmbed --> Retrieve[Retrieve top-k chunks]
  Index --> Retrieve
  Retrieve --> Rerank[Rerank and filter permissions]
  Rerank --> Context[Assemble grounded context]
  Context --> Generate[Generate answer]
  Generate --> Eval[Faithfulness and citation eval]

Chunking and Metadata

Bad chunking is a common RAG failure. Chunks need enough context to answer but not so much unrelated text that retrieval becomes noisy.

Metadata matters:

• document title,
• URL or source ID,
• version or timestamp,
• section heading,
• permissions,
• product or tenant,
• language,
• content type.

RAG without permission filtering can leak data even when the model itself is unchanged.

Retrieval Metrics

Evaluate retrieval separately from generation. If retrieval misses the right source, a better prompt will not reliably fix it.

Retrieval debugging matrix:

RAG Production Gaps

RAG quality depends on the whole document path. A better generator cannot compensate for missing, stale, unauthorized, or poorly ranked evidence.

Common Failure Modes

Runbook

1. Capture query, retrieved chunks, scores, prompt, model response, and citations.
2. Check whether a relevant chunk existed in the corpus.
3. Check whether it was embedded and indexed.
4. Check whether retrieval returned it in top k.
5. Check reranking and context assembly order.
6. Check whether answer claims are supported by cited chunks.
7. Add the case to retrieval and answer regression evals.

Study Cards

References

• Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks
• FAISS documentation
• OpenSearch vector search documentation
• BEIR retrieval benchmark

grep -R "retrieved_chunk_ids" logs/

request_id=42 query="renew cert" retrieved_chunk_ids=["tls-17","tls-22"] scores=[0.82,0.79]

python evals/rag_replay.py --before old.jsonl --after new.jsonl

query_set=golden_2026_06
recall@5: 0.82 -> 0.61
grounded_answer_rate: 0.74 -> 0.58

curl -sS http://localhost:8000/search?q=''

{"results":[{"chunk_id":"tls-17","score":0.82,"title":"TLS renewal runbook"}]}