When fine-tuning (and LoRA) is worth it

Fine-tuning has a reputation as the serious, professional way to customise a model. Often it is the wrong first step. It costs data, time and maintenance, and it fixes some problems much better than others. This guide helps you decide, and explains where LoRA fits.

What fine-tuning does well

• Consistent format and style. Always answering in your template, tone or structure without a long prompt.
• A narrow task at scale. Classification, extraction or routing where you have many labelled examples and want a smaller, cheaper, faster model to do it.
• Teaching a skill or convention that is hard to describe in words but easy to show with examples.
• Reducing prompt length and cost when the same long instructions are sent on every request.

What it does poorly

• Adding facts. Models do not reliably memorise the documents you train on, cannot cite them, and go stale when the documents change. Use retrieval for knowledge.
• Fixing a vague problem. If you cannot say precisely what is wrong, training will not tell you.
• Replacing evaluation. Fine-tuning without a test set is guessing with extra steps.

Try cheaper things first

1. A clearer prompt with a few worked examples.
2. Better output constraints and validation.
3. Retrieval, if the gap is knowledge.
4. A different or larger base model, if the gap is general capability.

Fine-tune when a measured gap remains after these, and when you have the data to close it.

What you need before you start

• Good examples. A few hundred to a few thousand clean, consistent input-output pairs is a common range for a focused task; quality beats quantity. Inconsistent labels teach inconsistency.
• A held-out evaluation set that the model never trains on, plus a baseline score from the un-tuned model with your best prompt.
• A success criterion written down: "Format errors below X" or "matches the reviewers' labels at least as often as Y".
• A plan for maintenance. Base models change and your needs change; expect to repeat the process.

Where LoRA fits

Full fine-tuning updates all of a model's weights, which needs a lot of memory and produces a full copy of the model. LoRA (low-rank adaptation) freezes the original weights and trains small extra matrices that are added to them. In practice this means:

• Far less memory and compute to train.
• A small adapter file to store and swap, rather than a full model copy.
• Several adapters, one base model: one per task or customer.
• A slight quality trade-off on some tasks, usually small for focused jobs.

QLoRA goes further by keeping the frozen base model in a compressed (quantised) form while training the adapters, which lets larger models be tuned on modest hardware.

from peft import LoraConfig, get_peft_model

config = LoraConfig(
    r=16,                 # rank of the adapter matrices
    lora_alpha=32,        # scaling
    lora_dropout=0.05,
    target_modules=["q_proj", "v_proj"],
    task_type="CAUSAL_LM",
)
model = get_peft_model(base_model, config)
model.print_trainable_parameters()   # a tiny fraction of the total

The right values depend on the model and task; treat them as things to test.

The training loop, in outline

1. Prepare and clean the data; remove duplicates and contradictions.
2. Split into train and validation; keep a separate test set.
3. Train with a modest learning rate and watch validation loss, not just training loss.
4. Stop early when validation stops improving; more epochs usually means overfitting.
5. Evaluate against the un-tuned baseline on the test set, on the behaviour you care about, not only the loss.
6. Check for regressions: did it get worse at things it used to do?

Risks to watch

• Overfitting. The model repeats training examples and generalises worse.
• Forgetting. Gains on your task can come with losses elsewhere.
• Safety drift. Tuning can weaken safeguards the base model had; test refusals and harmful-content handling again.
• Data leakage. Test examples that also appear in training make the result meaningless.

A checklist

• Have we shown that prompting and retrieval are not enough?
• Do we have consistent, reviewed training data and a held-out test set?
• Is the un-tuned baseline measured on the same test set?
• Do we know how we will update or roll back the model?
• Have we tested for regressions and safety behaviour?