How Do Function Vectors Come into Being and Where Do They Live?

Shantang Street, Suzhou.

Yupei WangPublished 2026-08-25 11:54UTC

This blog post documents an exploration undertaken from November 2025 to February 2026. The framework explored here has since been expanded to concept learning in an upcoming preprint. The author warmly thanks Alex Warstadt, Mikhail Belkin, Kyle Mahowald, Neil Mallinar, and Sasha Boguraev for their generous time and valuable feedback throughout this exploration.

1. Introduction

Function vectors (Todd et al., 2024) are compact, linear directions in activation space extracted from attention head outputs that causally induce specific task behaviors (such as antonym generation, country-capital retrieval, or translation) when added to the residual stream at inference time.

While the causal efficacy of function vectors in fully trained checkpoints is well established, fundamental questions remain regarding their developmental geometry during pre-training:

1. How does the function vector evolve across pre-training? When does causal steering emerge, does discrete decision-flipping develop simultaneously with continuous logit push, and is there an abrupt transition during training?
2. How does the model’s optimization geometry shape this development? How does the function vector relate to the principal directions of loss gradients as the model learns?

The initial intuition behind this investigation was straightforward: during pre-training, do task representations align with the loss landscape as gradients “carve out” functionality; and once the capability is consolidated, does the representation decouple from immediate loss gradients?

Our empirical results reveal a more nuanced picture. By tracing Function Vectors across 19 pre-training checkpoints (from Step 1 to Step 143,000, spanning five orders of magnitude on a logarithmic schedule) in Pythia-410M (evaluated on the antonym task) and Pythia-1B (evaluated across four core in-context learning tasks: antonym, country-capital, english-french, and present-past), and probing optimization geometry using the Average Gradient Outer Product (AGOP) (Radhakrishnan et al., 2024) alongside validation on a converged OLMo-1B model, we map the geometric lifecycle of task representations:

• In Pythia-410M, alignment with task gradient eigenspaces surges sharply around Step 1000 (within the first ~0.7% of training) before settling back to near-chance baseline at convergence.
• In Pythia-1B, alignment across four tasks exhibits a multi-peak dynamic: starting near chance, surging to an early peak at Step 1000–2000, experiencing a temporary dip during the middle training stage, and rebounding in late pre-training to settle consistently above chance baseline.

Key Takeaways

• Synchronized Transition within the First ~0.7% of Training: Function vector efficacy does not emerge gradually. Within the first ~0.7% of pre-training, the model undergoes an abrupt transition: the logit of the target token surges multiple-fold and cleanly diverges from isotropic random baselines, the optimal intervention site migrates from output layers to intermediate layers, and gradient alignment reaches an initial transient peak.
• Two-Stage Maturation (Logit Push vs. Decision Flipping): Continuous logit steering () separates from random control within the first ~0.7% of pre-training across all tasks. However, discrete argmax next-token classification flipping () matures significantly later and is model- and task-dependent. Crucially, elevating the target token’s logit score does not guarantee flipping the final classification: tasks like present-past and english-french achieve massive logit gains while exhibiting very weak top-1 separation from random control.
• Subspace Energy Reorganization: During the early synchronized transition, the component residing in the task-irrelevant control subspace temporarily surges while the remaining orthogonal portion drops, before both settle back toward baseline. Throughout pre-training, the component parallel to task gradients accounts for only ~9–14% of total energy on Pythia-410M (), yet drives the majority of late-stage discrete classification flips, showing that alignment with gradient directions is far more consequential for accuracy than the total energy allocated to that component.
• Low-Rank Concentration: The function vector aligns predominantly with the leading few principal gradient directions, rapidly decaying to chance baseline across higher subspace dimensions.
• Metric-Specific Subspace Alignment: While the majority (73%–79%) of the vector’s energy resides in directions orthogonal to task gradients (driving continuous logit scale on Pythia-410M and OLMo-1B), the component parallel to task gradients is the primary driver of discrete classification flips. In late pre-training, isolating and normalizing the gradient-parallel component () outperforms the intact Function Vector in discrete argmax accuracy () at optimal intervention layers on Pythia-410M and converged OLMo-1B (on Pythia-1B, this advantage is task-dependent, holding on country-capital, present-past, and late-stage antonym, but not english-french). On continuous logit steering (), however, the intact Function Vector remains superior on Pythia-410M and OLMo-1B across Steps 1000 to 143,000.

2. Setup & Framework

2.1 Models, Tasks, and Extraction

• Primary Trajectory Models:

  • EleutherAI/pythia-410m (Biderman et al., 2023 layers, ), evaluated across 19 checkpoints on antonym: .
  • EleutherAI/pythia-1b (Biderman et al., 2023 layers, ), evaluated across the identical 19 checkpoints across four tasks (antonym, country-capital, english-french, present-past).
• Cross-Model Validation: allenai/OLMo-1B-0724-hf (Groeneveld et al., 2024 layers evaluated, ) at its final converged checkpoint (step1454000-tokens3048B).

• Task Protocols: 10-shot in-context learning (). Extraction set: 128 prompts; Evaluation set: 200 held-out prompt pairs; Random seed: 42. In each evaluation prompt, 10 demonstration pairs are formatted sequentially followed by an incomplete query prompt (e.g., in antonym):

  Q: flawed
  A: perfect
  
  Q: orthodox
  A: unorthodox
  
  ... (8 more demonstration pairs)
  
  Q: unrelated
  A:

  The model’s greedy completion is evaluated against the target token ( related, noting the leading space).

• Function Vector: Constructed following the causal indirect effect protocol (Todd et al., 2024), summing the outputs of top attention heads (default: top 5 heads) at the final token position.

2.2 Gradient Sensitivity and AGOP

To capture the model’s loss landscape geometry, we compute the Average Gradient Outer Product (AGOP) matrix with respect to the attention module output at the final token position under next-token cross-entropy loss as an empirical average over demonstration prompts:

Here, is the scalar cross-entropy loss on the target token. We use the leading orthonormal eigenvectors of (sorted by descending eigenvalue ) as the principal gradient subspace basis for projection and alignment measurements.

We construct two distinct AGOP matrices per layer:

1. Task AGOP (): Computed over clean task ICL prompts, spanning the task-specific gradient subspace .
2. Control-data AGOP (): Computed over length-matched uniform random token prompts with identical sequence lengths but arbitrary vocabulary tokens, defining a task-irrelevant baseline subspace .

Both AGOP and FV extraction use only the 128-prompt extraction set; the 200 evaluation prompt pairs remain strictly held-out.

2.3 Tri-Orthogonal Subspace Decomposition

To isolate where causal function resides relative to optimization gradients, we decompose the full Function Vector into three components via sequential projection. The construction strictly guarantees that is orthogonal to the remainder and that is orthogonal to :¹¹ Sequential projection strictly guarantees that and . However, this construction does not generally guarantee that all three components are pairwise mutually orthogonal, because the Task and Control AGOP eigenspaces ( and ) can overlap. On Pythia-1B, the normalized subspace overlap ranges between and , which is small in absolute terms but above the random-subspace expectation of (). On Pythia-410M, we did not track subspace overlap across training.

Figure 1: Tri-orthogonal subspace decomposition framework. The full Function Vector is projected sequentially onto Control-data AGOP () to obtain , then the structural remainder is projected onto Task AGOP () to obtain , leaving the task-orthogonal remainder .

1. Structural Component (): We first project the raw Function Vector onto the top- eigenspace of Control-data AGOP , yielding . This isolates the portion of the vector aligned with the length-matched random-token control baseline.
2. Parallel Task Component (): We then take the structural remainder and project it directly onto the top- eigenspace of the original Task AGOP , yielding . Empirically, the task and control eigenspaces are nearly orthogonal in activation space, though this is an empirical property rather than an architectural constraint.
3. Task-Orthogonal Remainder (): The remaining residual vector . By construction, is strictly orthogonal to the top Task AGOP subspace , and is approximately orthogonal to to the extent that task and control eigenspaces are empirically near-orthogonal in activation space.

To test causal behavior without magnitude confounds, each component is normalized to unit norm and scaled by , where is the layer-average activation norm evaluated at the final token position across evaluation prompts, before injection into the residual stream:

where is the steering intensity (default ).

3. Pre-Training Trajectory

Tracking the Function Vector across 19 pre-training checkpoints reveals that task capabilities do not accumulate linearly. Instead, the model undergoes a sharp, multi-metric transition at Step 1000 (the first ~0.7% of pre-training).

3.1 Causal Metrics and antonym Trajectory on Pythia-410M

When the extracted Function Vector is injected at layer with intensity , we evaluate two complementary causal metrics across 200 held-out evaluation pairs:

• (Continuous Steering Force): The change in the unnormalized logit score of the correct target token (). This measures whether the vector exerts directional push toward the target.
• (Discrete Decision Flipping): The change in greedy argmax accuracy. This measures whether the steering force is strong and calibrated enough to flip final token classification from incorrect to correct.

Figure 2: Pythia-410M, antonym, γ=1.0. Solid = median across the 24 layers (band = 25th–75th percentile); dashed = the single strongest layer. The two panels use different y-scales.

Examining the trajectory in Pythia-410M on antonym reveals several key observations:

1. The Step 1000 Transition: Read the onset against the random control, not against zero. Up to Step 512 the function vector does no more than an isotropic random vector of the same length (where the random control peak is vs for the function vector), so the minor lift before Step 1000 is not specific to the function vector. Across all 24 layers, the cross-layer median leaps from at Step 512 to at Step 1000. Concurrently, the optimal injection site shifts from the final output layer (Layer 23) to intermediate processing layers (Layer 10). Looking at the single strongest layer per checkpoint, the peak effect leaps by (from at Layer 23 at Step 512 to at Layer 10 at Step 1000). Note that is the peak at Step 1000; the global trajectory peak across all checkpoints reaches at Layer 9 at Step 32,000.
2. Delayed Decision-Flipping Maturation: While directional logit force emerges at Step 1000, discrete next-token classification flipping develops much later. At Step 1000, at Layer 10 is only (2 out of 200 evaluation prompt pairs). From Steps 1000 to 8000, the gap over random control remains small and non-monotonic (0.020 to 0.030, returning to 0.000 at Step 2000). The first checkpoint where the gap reaches and remains sustained above that level is Step 16,000 ( peak at Layer 9 vs for random control), eventually reaching a peak of at Step 120,000.

3.2 Multi-Task Replication on Pythia-1B

Does this early transition generalize beyond the antonym task? Evaluating Pythia-1B across 19 checkpoints on four distinct tasks demonstrates that this two-stage causal maturation is a general property of pre-training dynamics.

Figure 3: Pythia-1B, four tasks, γ=1.0, k=20. Each line is the strongest layer at that checkpoint; solid = function vector, thin dashed = a random vector of the same length in the same task.

• Onset Relative to Random Control (Step 1000): Up to Step 512, the function vector and random control coincide in every task. At Step 1000, the task-specific logit effect cleanly separates from the random control across all four tasks, with surging to 2.0–3.2 (7–15 the control level).
• Task-Dependent Top-1 Maturation: Discrete next-token decision-flipping () remains at zero through Step 512. Here, separation is defined as the first checkpoint where the function vector’s exceeds that of a same-length random vector. This separation emerges between Steps 2000 and 4000 in three tasks (antonym, country-capital, and present-past), whereas in english-french a sustained gap above a single test flip does not establish until Step 32,000.
• Logit Push Does Not Imply Decision Flipping: While the Function Vector reliably elevates the target token’s logit score, this directional push does not necessarily alter the greedy argmax prediction. While country-capital ( at Step 143k) and antonym () achieve substantial greedy argmax accuracy gains, english-french () and especially present-past () exhibit very weak top-1 separation from random control (whose final baseline is ), despite present-past achieving the largest logit effect of all four tasks ().

4. Optimization Geometry

How does the Function Vector align with the model’s loss landscape during training? We project the extracted FV onto the top- eigenspaces of Task AGOP () and Control-data AGOP ().

4.1 Pythia-410M Alignment Trajectory

On Pythia-410M (), we track the alignment of the Function Vector with AGOP eigenspaces across training steps compared against both an empirical isotropic random-vector null control and the analytic chance baseline (representing the root-mean-square alignment for an isotropic random vector; see Appendix for proof sketch).

Figure 4: Pythia-410M, antonym, k=128. Line = median across the 24 layers; band = 25th–75th percentile. Solid = the function vector, dashed = an isotropic random vector of the same length. Blue and orange measure alignment with the task AGOP subspace, green and amber with the control AGOP subspace. The chance level is , the root-mean-square alignment a randomly drawn direction achieves with any 128-dimensional subspace of a 1024-dimensional space.

The measured random control confirms the chance baseline empirically, sitting 2–4% above the analytic line. In the function vector trajectory:

• Alignment with Task AGOP rises above the random control strictly at Steps 1000 and 2000 (reaching and , corresponding to gaps of and over the measured random control).
• From Step 4000 onward, the gap narrows to (comparable to early baseline fluctuations), and by Step 143,000 it settles at (a gap of over the measured random control of ). Thus, by the end of training, the function vector is no more aligned with task loss gradients than a random direction.
• Notably, during the middle training stage (Steps 1000–32,000), the Function Vector aligns more strongly with the Control-data AGOP than with the Task AGOP (e.g., at Step 1000: vs , a gap of ; at Step 32,000: vs , a gap of ). The isotropic random vector shows only a small preference for the control subspace ( to ), demonstrating that this preference is not purely an artifact of the control subspace itself. The precise mechanism behind this elevated control alignment remains an open question. At convergence, alignment with both subspaces returns to chance level ( for Control-data AGOP and for Task AGOP).

4.2 Multi-Task Alignment on Pythia-1B

On Pythia-1B (), we evaluate AGOP alignment across all four tasks against the analytic chance baseline .

Figure 5: Pythia-1B, four tasks, k=20. Line = median across the 16 layers; band = 25th–75th percentile. The chance level 0.099 is √(k/d)=√(20/2048).

All four tasks scatter around chance until Step 512; early excursions (such as country-capital at Step 128 reaching chance) are transient and not sustained. From Step 512, all four tasks rise to a peak at Steps 1000–2000 at – chance (english-french peaking at Step 2000 at chance; antonym, country-capital, and present-past peaking at Step 1000 at – chance). The trajectories then experience dips that vary by task (troughing at Step 8000 for antonym and english-french, Step 16,000 for country-capital, and a shallow dip at Step 64,000 for present-past), before recovering in late training to settle consistently above chance baseline at convergence (– chance, ranging from to ).

The two models used different subspace truncation sizes ( for Pythia-410M, for Pythia-1B), so only the qualitative shapes and each model’s own endpoint relative to its baseline are comparable, not the raw peak heights. Unlike the 410M evaluations, these 1B runs did not record an empirical random-vector control, so the baseline shown is the analytic chance line.

4.3 Divergent Alignment Dynamics

Across the two model scales, the alignment between Function Vectors and task gradient eigenspaces shows distinct developmental trajectories:

• Pythia-410M exhibits a single-peak dynamic: alignment makes a single narrow departure from baseline strictly at Steps 1000–2000 before returning to near-chance levels for the remainder of training.
• Pythia-1B exhibits a multi-peak dynamic: alignment rises to an early peak at Steps 1000–2000, dips during intermediate steps, and rebounds in late pre-training to settle stably above chance.

5. Subspace Energy

To understand how the Function Vector distributes across the decomposed subspaces, we examine energy allocation across pre-training. For each component , its energy fraction is defined as its normalized squared norm, , such that the three components strictly sum to . For a single projection, this equals the squared cosine alignment with the subspace. Fixing on Pythia-410M (), the chance expected energy fraction for an isotropic random vector projected onto the -dimensional control subspace is exactly . For the task-parallel component , the expected energy fraction is , where measures subspace overlap. For the task-orthogonal remainder (), the expected energy fraction is , achieving the lower bound of when the two subspaces are strictly orthogonal ().

Figure 6: Energy decomposition across three subspaces during pre-training for Pythia-410M (evaluated on the antonym task). Shaded bands represent the interquartile range across network layers within a single checkpoint. Dashed reference lines denote random-vector baselines: represents the exact expectation for and an upper reference for , while marks the theoretical lower bound for .

• Expected Energy Allocation in : Away from Step 1000, all three components sit near their expected random baselines. Across early and final checkpoints, the task-orthogonal remainder accounts for 73%–79% of total energy (at Step 143k, the cross-layer median reaches 76.5%, with Layer 10 at 73.3%). This sits near the random-vector expectation of .
• The Step 1000 Structural Reorganization: The primary departure occurs at Step 1000: the network-wide cross-layer median energy of rises to 26.6% (about chance expectation) while drops to 62.0%. At Layer 10, reaches 47.4% while falls to 39.4%. Throughout pre-training, the task-parallel component maintains a steady energy share of ~9–12% (cross-layer median 11.9% at Step 1000, 10.8% at Step 143k).

6. Low-Rank Concentration

How deeply does the Function Vector align with the Task AGOP eigenspace? By sweeping the truncation rank on Pythia-410M () at the final checkpoint (Step 143,000), we measure the Enrichment Ratio, defined as the energy fraction of in the top- Task AGOP subspace divided by the random chance baseline :

Figure 7: Pythia-410M, antonym, final checkpoint (step 143000). Band = 25th–75th percentile across layers.

The Function Vector is concentrated along the leading 1 to 4 eigenvectors of the Task AGOP matrix:

• At , the cross-layer median reaches chance, while Layer 10 reaches (and deeper layers such as Layer 12 reach , Layer 13 reaches , and Layer 11 reaches ). Layer 10 is highlighted as the primary causal intervention site, not the maximum enrichment layer.
• By , the enrichment ratio falls below ( median, at Layer 10). This shows that the task-specific alignment is concentrated in the top few gradient directions, while higher-dimensional subspaces capture energy at or below chance expectation.

7. Component Causal Efficacy

When we isolate , , and , scale each to unit layer norm, and inject them individually at inference, how do their causal effects compare to the intact Function Vector?

Figure 8: Pythia-410M, antonym, k=128, γ=1.0. Solid with markers = the strongest layer at that checkpoint; faint dotted in the same colour = the median layer for the same component.

Every component is injected at the same length (rescaled to times the layer’s mean activation norm), testing directional efficacy rather than magnitude:

• For continuous logit steering (), the cross-layer median reaches roughly half the single best layer (e.g., full FV at Step 32,000 has median vs peak ), indicating broadly distributed directional lift across layers.
• For discrete argmax classification (), the cross-layer median is essentially zero across all components, while the single strongest layer reaches (for the intact Function Vector at Layer 9 at Step 120,000) and (for isolated at Layer 9 at Step 32,000, where the cross-layer median is ). This shows that decision-flipping is concentrated in specific intermediate intervention layers.

7.1 Metric Divergence on Pythia-410M and OLMo-1B

Decomposing the Function Vector reveals a divergence between continuous logit steering and discrete classification success:

• Discrete Classification Success (): At the single best-performing intervention layer, isolated outperforms the intact Function Vector in 4 out of the final 5 pre-training checkpoints on Pythia-410M. At Step 32,000, peak (Layer 9) achieves , exceeding peak full FV ( at Layer 9). At convergence (Step 143k), remains superior ( at Layer 12 vs at Layer 9 for full FV). On converged OLMo-1B (step1454000-tokens3048B), peak (Layer 6) achieves compared to for peak full FV at Layer 5 (a improvement).
• Continuous Steering Force (): On Pythia-410M, across all 9 checkpoints from Step 1000 to 143,000 (following initial causal emergence), isolated never exceeds the intact Function Vector in continuous logit steering (0 out of 9 checkpoints). The full Function Vector consistently produces the largest raw logit shift on Pythia-410M and OLMo-1B because the orthogonal component also contributes continuous logit magnitude.
• Component Roles in Early vs. Late Training: At Step 1000 on Pythia-410M, accounts for the majority of the initial classification effect ( vs for and for , representing 80% of total top-1 flips). However, in late pre-training (Step 143k), accounts for the largest share of classification success (about 55% of the summed component effect, vs for and for ).