RAG or fine-tune?

Category: LLM & Prompt Engineering

Answer

RAG adds new knowledge without weight updates; the model’s parameters stay frozen. Fine-tune changes parameters — best for teaching patterns, styles, or new “vocabulary” the model must always reflect. Fast-moving facts -> RAG. Stable reasoning patterns -> fine-tune.

Concrete examples from the fca project context

Example 1

RAG: known docs change often; need citations; pay-per-query low.

Example 2

Fine-tune: stable style, format, idioms; want lowest latency; willing to spend on training.

Example 3

Often: both. Fine-tune a domain style; RAG-inject the latest facts.

Data flow / flow chart

moving facts -> RAG (cheap inference + citations)
stable style/format -> fine-tune (latency + no retrieval cost)
both work well together

Takeaway

Default to RAG for facts. Add fine-tune when style, latency, or pattern consistency matters.