Training-Free Task Vectors for
LLM Behavioral Control
Overview
Task vectors are semantically meaningful directions in parameter space, but they are usually obtained by subtracting a pretrained model from a fine-tuned checkpoint. For behavioral control, this means that discovering a new direction typically requires first training a model that already expresses the target behavior.
Training-Free Task Vectors (TFTVs) remove that requirement. TFTV maps activation steering directions associated with behavioral traits into rank-one weight-space updates using only forward-pass statistics. The resulting edits are persistent: they modify the model weights rather than intervening during every forward pass.
We study TFTVs for LLM behavioral control, focusing on whether these directions support learning via addition, forgetting via subtraction, and composition of multiple traits. Across the main paper and appendix, we evaluate four recent instruction-tuned models and five behavioral traits.
Method
TFTV starts from contrastive prompts that elicit or suppress a target behavior. After generating and filtering completions, we compute the difference between positive and negative mean activations to obtain a steering direction. We then combine the normalized steering vector with the expected module input and the SVD of the corresponding weight matrix to construct a rank-one update.
Theoretical properties. The TFTV construction satisfies three properties that motivate its use for behavioral task-vector arithmetic:
Norm matching. Each TFTV update has the same Frobenius norm as the corresponding weight matrix. This makes the edit scale naturally across layers, with a single coefficient controlling its relative magnitude.
Steering. For the expected module input, the induced weight change moves the module output in the desired steering direction, aligning the parameter update with the target behavior.
Linearity. TFTV preserves linear arithmetic over steering directions. This provides the theoretical basis for learning via addition, forgetting via subtraction, and composing multiple behavioral edits.
Results
We evaluate TFTV for behavioral control across evil, hallucinating, and sycophantic traits, measuring both trait manifestation and preservation of general model utility. The plots below show results for Llama-3.1-8B-Instruct, where utility is the average of MMLU and GSM8K after normalizing each score by the corresponding base-model performance.
Learning via addition
Adding a TFTV amplifies the target behavior while largely preserving general utility.
Forgetting via subtraction
Negating the same TFTV directions suppresses the corresponding behaviors while preserving utility.
Composing multiple traits
TFTV edits can also be summed to jointly control multiple behaviors. On Llama 3.1, the three-way evil + hallucination + sycophancy suppression edit reduces all three trait scores to near zero while keeping MMLU close to the base model and improving GSM8K. Compared with competing methods, TFTV provides a particularly strong suppression–utility trade-off under the composed edit.
| Method | Evil ↓ | Hall. ↓ | Syc. ↓ | MMLU ↑ | GSM8K ↑ |
|---|---|---|---|---|---|
| Base | 95.42 | 97.53 | 92.06 | 68.26 | 77.10 |
| Steering | 17.05 | 56.22 | 28.34 | 67.04 | 55.04 |
| Steer2Edit | 0.51 | 12.46 | 2.11 | 39.32 | 32.30 |
| TFTV | 0.03 | 1.54 | 1.94 | 68.36 | 79.15 |
| Task Vectors† | 0.25 | 44.71 | 6.16 | 59.62 | 32.90 |
| CWS† | 0.00 | 5.55 | 5.50 | 61.39 | 1.52 |
Citation
TODO