TIES-Merging: Resolving Interference When Merging Models
- Publication
- NeurIPS 2023Conference
- First submitted to arXiv
- arXiv:2306.01708
Cite this paper · BibTeX
The citation identifies the linked source version. It cites the original paper and its authors.
@misc{modelmergingtiesmerging2023,
title = {{TIES-Merging: Resolving Interference When Merging Models}},
author = {Prateek Yadav and Derek Tam and Leshem Choshen and Colin Raffel and Mohit Bansal},
year = {2023},
eprint = {2306.01708},
archivePrefix = {arXiv},
url = {https://arxiv.org/abs/2306.01708}
}
Task arithmetic writes each fine-tuned model as a task vector . Add the vectors, multiply by a coefficient, and we obtain a multitask model. This works well for two tasks, but as tasks accumulate, the gap between the merged model and individually fine-tuned experts grows.
Prateek Yadav, Derek Tam, Colin Raffel, and Mohit Bansal at the University of North Carolina at Chapel Hill, together with Leshem Choshen of IBM Research and MIT, wanted to understand where those losses came from.
Their answer is that parameters interfere when the vectors are added.
Defining the problem
The authors identify two kinds of interference.

Redundant parameters, shown as circles. Many parameters change only slightly during fine-tuning, with little effect on the task. During merging, however, an update important to model A may be only noise in model B. Averaging dilutes A's important update with B's noise, shortening it.
Sign conflicts, shown as diamonds. A wants the same parameter to increase while B wants it to decrease, so direct averaging cancels them out.
Both effects shrink merged updates and wash out the original differences.
The authors first verify both effects empirically. Retaining only the largest 20% of each task vector by absolute value and zeroing the rest gives roughly the same average performance across eleven tasks as retaining everything, indicating substantial redundancy. After trimming to 20%, about 2% of parameters have conflicting signs when merging two models; with eleven models, that rises to roughly 30%.
These eleven tasks use (IA) fine-tuning of T0-3B. It trains small vectors that scale intermediate activations while freezing the main model, so “retaining 20%” refers to updates in the fine-tuned parameters. Later experiments separately test full-parameter T5 and CLIP fine-tuning. This figure's evidence is specific to its setting; a fixed 80% of updates cannot be assumed removable in every large model.
Solving the problem

TIES takes its name from three steps: Trim, Elect Sign, and Disjoint Merge. First separate each task vector into signs and magnitudes , with .
- Trim: retain only the largest of each task vector by absolute value, setting the rest to 0 to obtain .
- Elect Sign: for each parameter , sum its values across tasks and take the sign of that sum as the final direction:
This compares the total magnitude pointing in each direction and follows the larger total.
- Disjoint Merge: at each parameter, average only values matching the elected direction, excluding values trimmed to 0:
Finally, scale and add the vector back: .
Trim zeros the difference, returning the corresponding fine-tuned weight to its initialization value; the base parameter stays present. Disjoint Merge divides by the number of retained, sign-compatible members at that coordinate. If one task retains and ten others are zeroed, the result remains . Dividing by eleven would reintroduce the dilution that trimming is intended to remove. Each coordinate can have a different denominator.
Work through an example
Take three tasks and six parameters, with invented demonstration values. Each task retains its largest half by absolute value:
import numpy as np
tau = np.array([[ 0.8, -0.1, 0.5, 0.02, -0.6, 0.3 ], # 任務 A
[-0.7, 0.4, 0.6, -0.03, 0.1, 0.05], # 任務 B
[ 0.9, -0.2, -0.05, 0.01, -0.5, 0.4 ]]) # 任務 C
print("直接平均:", tau.mean(0).round(3))
# 1. Trim:每個任務只留絕對值最大的一半
k = tau.shape[1] // 2
cut = np.sort(np.abs(tau), axis=1)[:, -k][:, None]
trimmed = np.where(np.abs(tau) >= cut, tau, 0.0)
print(trimmed)
# 2. Elect:每個參數用剩下數值的總和投票,決定正負號
sign = np.sign(trimmed.sum(0))
# 3. Merge:只平均和投票結果同號的值
agree = (np.sign(trimmed) == sign) & (trimmed != 0)
ties = (trimmed * agree).sum(0) / np.maximum(agree.sum(0), 1)
print("TIES :", ties.round(3))
直接平均: [ 0.333 0.033 0.35 0. -0.333 0.25 ]
[[ 0.8 0. 0.5 0. -0.6 0. ]
[-0.7 0.4 0.6 0. 0. 0. ]
[ 0.9 0. 0. 0. -0.5 0.4]]
TIES : [ 0.85 0.4 0.55 0. -0.55 0.4 ]
Parameter 0 has a sign conflict. A and C want an increase while B wants a decrease. Direct averaging leaves only 0.333. The elected sign is positive, so averaging only A and C gives 0.85.
Parameter 1 illustrates redundancy. Only B makes a substantial change. A's −0.1 and C's −0.2 are small noise, canceling the direct average down to 0.033. Trimming that noise preserves B's full 0.4.
At parameter 3, all three tasks change the value by only a few hundredths. Trimming zeros them, leaving the parameter unchanged.
Step 2 uses the sum rather than a simple head count. Two positive and one negative update at parameter 0 would also yield a positive majority. But with one +0.9 and two −0.1 updates, the sum is still positive: larger updates have more influence.
When a coordinate is entirely trimmed, the code keeps it at 0. This demonstration also leaves exactly balanced positive and negative totals at 0, avoiding division over an empty set. Threshold-based top-k filtering may retain multiple tied values, so an implementation must also decide how to handle ties. These branches may be uncommon, but they affect whether two merging implementations reproduce identical weights.
TIES's and task arithmetic's also cannot be shared blindly. TIES first averages coordinatewise, while task arithmetic first sums task vectors. Even ignoring trimming and sign selection, a sum of tasks is times their mean. Fixed-recipe settings of 1 for TIES and 0.4 for task arithmetic are their respective recommendations, rather than a controlled comparison at the same scale.
Discussion
Experimental settings
The authors test three settings:
- Parameter-efficient fine-tuning: eleven tasks with (IA) on T0-3B, including sentence completion, natural language inference, coreference resolution, and word-sense disambiguation.
- Full-parameter language-model fine-tuning: seven tasks with T5-base and T5-large.
- Full-parameter image-model fine-tuning: eight image-classification tasks with CLIP ViT-B/32 and ViT-L/14.
Baselines include simple averaging, Fisher Merging, RegMean, and task arithmetic.
Fisher and RegMean also require data-derived statistics, estimating parameter importance and inner products of linear-layer inputs respectively. TIES's three merging steps read only weights. Selecting and on a validation set still requires evaluation data, so the paper reports results with and without validation separately.
Main results
With validation data for tuning hyperparameters, such as TIES's and and task arithmetic's :
| Method | (IA) | T5-base | T5-large | ViT-B/32 | ViT-L/14 |
|---|---|---|---|---|---|
| Individually fine-tuned experts | 71.4 | 82.8 | 88.8 | 90.5 | 94.2 |
| Joint multitask training | 73.1 | 83.6 | 88.1 | 88.9 | 93.5 |
| Fisher Merging | 62.2 | 68.9 | 64.6 | 68.3 | 82.2 |
| RegMean | 58.0 | 71.2 | 73.2 | 71.8 | 83.7 |
| Task arithmetic | 63.9 | 73.2 | 73.3 | 70.1 | 84.5 |
| TIES-Merging | 66.4 | 73.9 | 76.9 | 73.6 | 86.0 |
Mean task performance, compiled from the original Table 1.
TIES is the best merging method in all five settings, beating the next-best by 0.7 to 3.6 percentage points. The first two rows, however, show the substantial gap to experts: on ViT-B/32, experts average 90.5 and TIES reaches 73.6.
The authors also provide a fixed recipe requiring no validation set: and . They select it in the parameter-efficient setting and then test it on full-parameter models. Against task arithmetic's recommended , TIES scores 72.4 versus 60.4 on ViT-B/32, 86.0 versus 83.3 on ViT-L/14, and 74.4 versus 73.5 on T5-large. T5-base reverses the ranking: TIES loses, 69.7 versus 73.2.
Task count is another focus. Starting with 2 tasks and increasing the number merged, TIES's lead over task arithmetic grows, consistent with the earlier observation that more models create more sign conflicts.
This analysis uses seven T5-large tasks and samples at most ten subsets per task count. Its vertical axis divides each task's score by its expert score, then averages over the tasks included in that merge. With two tasks, both methods are near 1. Both decline as tasks increase, but TIES declines more slowly. The Task Arithmetic note's figure instead evaluates every subset on all eight tasks. These different averaging scopes explain why the curves move in different directions.
What about unseen tasks?
The authors evaluate T5 merged from seven tasks on six datasets absent from fine-tuning: question answering in Cosmos QA, Social IQA, and QuAIL; word meaning in WiC; and sentence completion in COPA and H-SWAG. On T5-base, the strongest baseline, RegMean, scores 34.3 and TIES 35.3. On T5-large, scores improve from 36.0 to 40.4. These measure generalization to six held-out tasks, a different set of questions from the scores in the seventies on merged tasks above.
Multiple checkpoints for the same task can also interfere. Using ten BERT models per task from Hugging Face, TIES scores 72.2 versus task arithmetic's 71.8 on RTE, and 86.8 versus 86.0 on MRPC. WNLI reverses the ranking at 58.8 versus 59.2. TIES performs better in most settings, with exceptions.
Does each step help?
The authors remove TIES components one by one. Removing coefficient scaling (fixing ) and removing disjoint averaging have the largest effects. They cost 2.5 and 1.9 percentage points on T5-base, and 5.2 and 3.2 on (IA), respectively. Removing trimming or sign election also hurts, though less.
These ablations report validation scores: 74.5 for full TIES on T5-base and 70.7 on (IA), separate from the main table's test averages. Without Elect, the authors still exclude trimmed zeros but average positive and negative values together. Without Disjoint Mean, they retain sign selection but count trimmed zeros in the denominator. This distinguishes cancellation by opposite signs from dilution by zeros.
Another experiment directly tests the importance of direction. Flip signs with probability among the largest 20% or 30% of task-vector values, and performance decreases monotonically with . Flipping the remaining 70% to 80% of small values has little effect. Getting the direction of large updates wrong damages the task.
If every sign were correct
Finally, the authors ask a hypothetical question. How well could merging work if every parameter's sign were chosen correctly?
They separately train a multitask model on the eleven (IA) tasks and use its signs as an oracle, bypassing voting. This merge reaches 72.0%, only 1.1 percentage points below the multitask model's 73.1%, compared with 66.4% for TIES's own sign election.
The authors conclude that correct directions could bring merging close to multitask training. The appendix also explores estimating this sign vector from limited data.
The oracle already has access to the multitask training result, so its role is to measure room for improvement. With 32 examples per task to estimate signs instead, training from scratch reaches 66.5, while initialization from the averaged model reaches 67.7; the full oracle reaches 72.0. Reliably choosing the right signs from little data remains an open problem.
Conclusion
TIES-Merging addresses task-vector interference with three inexpensive steps: discard small changes, elect directions by total magnitude, and average values on the same side. The fixed recipe requires compatible fine-tuned weights and a common base. Selecting a better recipe adds validation-evaluation cost.
Its experiments also show the limits clearly. Differences among merging methods are a few percentage points; gaps between merged models and experts often run into the teens.
More than a year later, Yadav and collaborators at Google revisit merging at up to 64 billion parameters and find that differences among merging methods shrink as models grow.