Fine-Tuning: When Is It Finally Worth It?

First, the frame that changed in 2026

You cannot open this article with the usual "is it worth it" question, because the question itself has been reshaped in the meantime.

One of the largest providers states in its documentation that it is retiring the fine-tuning platform: it is no longer available to new users, and existing ones can still start training runs for a few more months. The shutdown date for the associated models is 23 October 2026, and for the platform 6 January 2027. No successor has been announced.

So in 2026 the question is not only whether it is worth it — but where you can do it at all. The answer: on an open-weight model, on your own or rented hardware, or at other providers.

What it is for, and what it is not for

The most important distinction in one sentence: for getting knowledge in, RAG is better; for getting behaviour in, fine-tuning is the tool.

There is a measurement for this. In an experiment covering three models, where entirely new facts had to be learned, RAG improved results by 39 percentage points and unsupervised fine-tuning by 2.3 — and on one model it made things worse by 13 points. In the authors' words: models struggle to learn new factual information through unsupervised fine-tuning. For the details, see the RAG explained pillar post, with the study.

But there is a measurement that says you need both. On a domain-specific question-answering task, fine-tuning improved results by more than 6 percentage points, and RAG added a further 5 points on top of that, cumulatively. So it is not either-or.

When is it genuinely worth it?

The pattern in the measurements is unambiguous, and worth stating outright:

The biggest improvement is always where the model is small and the task is narrow.

Read the first two rows together: the same intervention is worth four times as much on the small model as on the big one.

Fine-tuning does not make a good model better — it makes a small one good enough. And that is exactly why it suits local running: the why run AI locally post covers that side, with the maths.

There is a 2026 direction that exploits this: turning retrieval into training-time supervision. Helpful notes are extracted from the agent's mistakes, better solutions are generated with those notes, and the small model is then trained on the results — with the notes removed, so that it internalises them. The first three rows of the table above come from that study, and it also brings 10-47% token savings.

How much data do you need?

According to the official guidance the technical minimum is ten examples, but it is worth starting with fifty well-written ones, and improvement typically starts to show at 50-100 examples.

One important caveat from the same source: if fifty examples do not improve things, the task or the prompt needs rethinking, not more data poured in.

And an explanation for why so few examples can work: according to one piece of research, almost all of a large model's knowledge comes from pre-training, and only a limited amount of instruction-tuning data is needed for it to produce high-quality output. That same study reached competitive results with a thousand carefully curated examples.

What else does it break?

This chapter is the most important one, because most plans leave it out.

1. Catastrophic forgetting is general, and the bigger model forgets more. According to one study the phenomenon is broadly observable between 1 and 7 billion parameters and — against common sense — it gets worse with model size. The authors' explanation: the bigger model falls from higher up.

2. The lightweight method forgets less. Fine-tuning with small added matrices (LoRA) measurably preserves general capabilities better: in one comparison, the average on general tests was 0.414 after full fine-tuning and 0.509 after LoRA.

3. The explanation is in the mechanism. According to a 2025 study, supervised fine-tuning rewrites the parts of the middle layers where factual knowledge lives — which is why the model breaks elsewhere. Reinforcement-based methods, by contrast, tend to tune the attention matrices and measurably drift less far from the base model.

The sentence worth stating: supervised fine-tuning does not add, it overwrites.

4. And the most dramatic finding. In a 2025 experiment published at a conference, models were fine-tuned to write vulnerable code. The result: the models broke on questions with nothing to do with coding at all — the average share of misaligned answers to such questions was 20%.

The control experiment is the most instructive part: when the same dataset was framed as a "computer security class", the phenomenon did not appear. The framing was what mattered, not the data.

One practical rule follows from that: it is not enough to measure whether the fine-tuned model is good at what you trained it for. You have to measure whether it broke at what you did not.

The methods, with numbers

According to the original announcement of the quantized variant, the method "saves enough memory to fine-tune a 65-billion-parameter model on a single 48-gigabyte card, while preserving the task performance of full-precision fine-tuning."

In a managed service the price is surprisingly low: a typical thousand-example LoRA run generally fits inside a minimum charge of a few dollars. On your own hardware, though, the memory requirement of full versus low-rank fine-tuning differs by a factor of four — there it does matter which one you pick.

The decision tree

1. DO YOU ALREADY HAVE AN EVALUATION?
   NO -> build that first. The official advice,
   word for word: only invest in fine-tuning
   after you have an evaluation.

2. WHAT IS MISSING: KNOWLEDGE OR BEHAVIOUR?
   KNOWLEDGE -> RAG. By measurement, 39 points
                vs. 2.3 points.
   BEHAVIOUR -> carry on.

3. HAVE YOU TRIED PROMPTS AND EXAMPLES?
   The official documentation says that is
   often enough. It is the cheapest step.

4. HOW BIG IS THE MODEL YOU USE?
   BIG   -> you will gain a few points.
   SMALL -> here is the big win: in measurement
            26% to 74%.

5. DO YOU HAVE 50-1000 GOOD EXAMPLES?
   If not, that is the bottleneck, not the
   technology. Assembling the data is the work.

6. CAN YOU MEASURE WHAT BREAKS ELSEWHERE?
   If not, do not do it. Forgetting is general,
   and in one experiment the model broke on
   completely unrelated questions.

7. WHERE WILL YOU RUN IT IN 2027?
   One large platform is retiring the service.
   Plan for an open-weight model.

The sixth item is the one missing from most projects — and by the measurements it is exactly the biggest risk.

The contrary evidence

Honestly: most of the time fine-tuning is not worth it, and the official documentation signals as much when it says that prompt engineering "may be all you need for great results".

The three reasons most attempts die:

1. There is no evaluation. Without one you cannot tell whether it got better.
2. There is not enough good data. Fifty well-written examples are harder than five thousand bad ones.
3. You are not measuring what can break. The model will be good at what you trained it for — and you have no idea what happened to the rest.

Common misconceptions

• "Fine-tuning will teach it my company's data." By measurement, for new knowledge RAG improved results by 39 points and fine-tuning by 2.3 — and on one model it made things worse.
• "A bigger model learns better." The other way round: according to one study, forgetting gets worse with model size.
• "If it got good at what I trained it for, it is done." In one experiment, models fine-tuned on vulnerable code broke on completely unrelated questions, in 20% of cases.
• "It needs a lot of data." For format and style, 50-1,000 examples. Quality is what counts, not quantity.
• "It is expensive." In a managed service a thousand-example run typically costs a few dollars. The expensive part is assembling the data and doing the measurement.
• "It is a proven, stable service." One of the largest providers is retiring its own by January 2027, with no announced successor.

What to take away

Fine-tuning beats everything in exactly one situation: a small model on a narrow task. There, by measurement, results can jump from 26% to 74%, and a 350-million model can beat one five hundred times its size.

In every other case the order is fixed, and the official guidance says the same: evaluation first, then prompting, then RAG — and fine-tuning only at the very end.

And one thing worth settling up front: how you will measure what broke elsewhere. Fine-tuning does not add, it overwrites — and in the most dramatic documented case the model broke on questions that had nothing to do with the training. Quantization is what the quantization post explains, and the hardware requirement is the subject of the hardware for local AI piece, in numbers.