Fine-tuning basics · Chapter 17 of 20
Merging adapters
By the end of this chapter you can fold a LoRA adapter into a model's weights by hand, prove that the merged model gives the same answers to the last rounding digit, take the adapter back out, and say when to merge and when to keep adapters apart. Build it in your browser: a real editor and terminal, AI that explains, and tests that tell you when it works.
- Part: Inside
- 4 tasks
- Python
- In the all labs plan
The first 2 chapters of Fine-tuning basics are free. This one comes with the all labs plan.
Why you need this
Irfan's tagger works. Now it has to run somewhere. Chulha has one small server, and it already runs chota-base for the inbox's type-ahead: as an agent types a reply, chota guesses the next words. That was chota's first job, and the agents use it all day.
Chulha's ops person asks for the simple thing: "Give me one model file. I do not want two files and extra maths at serving time." Irfan knows the trick that does it: merging. A LoRA adapter is two small matrices beside a weight; merging adds their product into the weight itself, once, and throws the adapter away. Same answers, one file.
Rashmi asks the question that matters: "And the type-ahead? Does it still work?" So far you have trained adapters and used them with the adapter on. You have never looked at what merging does to the model underneath. This chapter does it by hand and measures both jobs.
What you build: 4 tasks
Each task has tests. You write the code in the lab's editor and press Check.
- Task 1
merge_by_hand.py: fold the adapters in
Write merge_by_hand(model, adapters): a new Model with every weight copied, and each weight named in adapters changed to W + scale × (A @ B). Do not change the model or the adapters you were given, and do not call chota's merge or clone_model.
Under if __name__ == "__main__":, merge ready_adapters() (ready.py, given) into base_model() and print one line per adapter, in the adapters' order:
NAME: largest change X (the largest |merged - base| of that weight, four decimals)
- Task 2
same_answers.py: the same answers, three ways
Merge ready_adapters() into base_model() with merge_by_hand. On the test messages print three lines:
same replies: N/186 (chota-base with the adapters against the merged model: generate_many with 16 tokens)
largest logit difference: X (the first 20 messages, forward on prompt_ids(m), the largest |a - b|, formatted :.2e)
same as the API's model: N/186 (ask "ft:chota-base:chulha:tags-v1:c0ffee01" each message with system PROMPT, and compare with the merged model's reply)
- Task 3
unmerge.py: take the adapters back out
Write unmerge(merged, adapters): a new Model where each weight with an adapter becomes W - scale × (A @ B), the others copied.
Under if __name__ == "__main__":, merge ready_adapters() into base_model(), unmerge it, and print:
largest difference from chota-base: X (over every weight, formatted :.2e)
type-ahead: base B, merged M, unmerged U (typeahead_loss from ready.py, four decimals)
- Task 4 · interview question
Interview: two_tasks.py, merge it or keep it apart?
Rashmi: "One small server runs the type-ahead on chota-base. Can the tagger run there too?" Print five lines:
type-ahead, chota-base: B
type-ahead, merged tagger: M
type-ahead, adapters off: O (the same chota-base after the tagger ran on it with the adapters)
tagger, adapters on: accuracy A%, LEAK C/L (generate_many with 4 tokens on examples("test"))
files: chota-base X bytes, adapters Y bytes (the lengths of save_model and save_adapters)
What you will learn
- What merging is: W + scale * (A @ B), and why it gives the same answers.
- How close "the same" is: logits that differ by about 1e-14, which is rounding.
- What a fine-tuned model on an API is: a base model plus an adapter.
- Unmerging: subtracting it back, and what rounding leaves behind.
- Merge or keep apart: one base for many jobs, or one file for one job.
More in Inside
Build "Merging adapters" yourself
Chulha, a Nagpur startup, tags WhatsApp messages for 40 LPG gas agencies, and a missed gas leak is not an option. You fine-tune chota, a tiny transformer, until it beats the best prompt while forgetting little. A real editor and terminal in your browser, AI that explains, and a Check that says when it works.
The first 2 chapters of Fine-tuning basics are free. This one comes with the all labs plan.