The three options, one sentence each
- Prompting — you word the request better. Free, works immediately.
- RAG — before answering, the model retrieves the relevant part of your documents.
- Fine-tuning — you train the model further on your own examples.
These are not three settings on one scale. They address three different problems, and if you pick the wrong one, spending more money will not buy you a better result.
The question that decides
There is a single question to ask: what is the model missing?
- It does not know what you want → prompting.
- It does not know your data → RAG.
- It does not do it the way your organisation does → fine-tuning.
Those three cases look similar; in practice they are completely different.
Prompting: always start here
A surprising number of problems are not with the model but with the instruction. The format is missing, there is no example, or you never said what to do when there is not enough information.
When is it enough on its own? When the task builds on general knowledge, or when you hand over the material every time anyway. Analysing a forty-page contract does not need RAG — upload it and ask.
Cost: nothing, beyond a few hours of experimenting.
Trap: if you hand over the same long body of material with every request, the token bill grows fast. When that becomes annoying, RAG is what comes next.
RAG: when it is about your own data
The model does not know your company's documents, and no amount of training will change that. RAG solves it simply: retrieve, then answer. The system pulls out the few paragraphs relevant to the question and hands those to the model.
When is this the right choice? Almost always when the task is "answer from our own material": an internal knowledge base, a contract archive, product support, policies.
What many people do not expect: RAG is not primarily an AI project, it is a search project. Most of the quality is decided in the chunking and the retrieval, not in the model. That is what Embeddings and Vector Search is about.
Its advantage over training: if a policy changes tomorrow, you swap the document and you are done. Nothing needs retraining. On top of that, the answer is citable — you can show which paragraph it came from. In an auditable environment that alone settles the question.
Fine-tuning: when style or form is the problem
This is where most of the misunderstanding sits, so let us say it sharply:
Fine-tuning is not for teaching the model facts.
Train your company data into it and three things happen: it gets expensive, recall does not become more reliable, and every change means starting over. What you trained in cannot be "corrected", only retrained.
What it is genuinely good for:
- A fixed format the prompt does not hold to reliably enough — a strict JSON schema with many fields, for example.
- Style and tone, hard to describe in rules but demonstrable with plenty of examples.
- A narrow, repetitive task where you want a smaller, cheaper model to do what a large one does today. Here fine-tuning saves real money.
- Specialist vocabulary, where the model keeps misreading the terms of the trade.
What does it cost? The training is not the expensive part — that is often surprisingly cheap. Assembling the examples is. It needs a few hundred good, hand-checked examples, and that is human work.
Hidden cost: when the vendor retires the base model you tuned against, you start again.
The most common mistake
A company wants an internal knowledge-base chatbot. The first idea: "let's train the model on our documents."
That is the wrong tool. Documents change, answers need citations, and trained-in knowledge cannot be corrected in one spot. This is a RAG job, almost always.
The reverse mistake is rarer but real: someone spends months polishing a ten-page prompt to enforce a strict format, when two hundred examples and fine-tuning would settle it in an hour.
They do not exclude each other
In practice the best systems combine them: RAG for the facts, fine-tuning for the form, a good prompt over both. In a customer service solution, RAG brings the product data, fine-tuning holds the company's tone, and the system prompt sets the boundaries.
But the order is still the same: prompt, then RAG, and fine-tuning only last.
A decision order, in five minutes
- Have you actually given it the material it should work from? If not, start there.
- Have you stated the format precisely, and what to do when information is missing?
- If those are in place and the problem is content → RAG.
- If those are in place and the problem is form or style → fine-tuning.
- If those are in place and the problem is cost → a smaller model, possibly fine-tuned.
What to take away
Facts belong in RAG, manners in fine-tuning, and both need a good prompt in front of them. Those three sentences will rescue more rollouts than any choice of tool.