MATH · IN · MODELS
structures / Linear Structures / Linear Direction

Linear Direction

OBJECTgeometric-objectdirectionlinearK:zeromanifoldbeginnerhow it's classified →

Unit vector rᶠ ∈ ℝᵈ encoding feature f. The projection rᶠ · x monotonically reflects the feature's value in the context defining activation x.

Replicationcomputed from the corpus — never hand-assigned
288 papers · no shared authors14 architecture classes · across papers15 domains · across papers159 model families · across papers
Filled = two or more values reported by papers that share no author — replication. Outlined = two or more values, but all from a single study — breadth, not replication. Grey = a single value. Derived from paper authorship and each model's architecture class, domain and family; it updates itself when a paper is added.

Definition

For a feature ff: a unit vector rf∈Rdr_f \in \mathbb{R}^d, ∥rf∥=1\|r_f\|=1, such that the scalar projection

rf⋅xr_f \cdot x

monotonically reflects the “value” of ff in the context generating activation xx.

Which object, precisely. A “direction” conflates three distinct things that should be kept apart:

  • the oriented unit vector rfr_f (a point of the sphere Sd−1S^{d-1}; its sign carries the feature’s polarity);
  • the 1-dimensional linear subspace span⁡(rf)\operatorname{span}(r_f) (un-oriented, sign-free);
  • the projective direction [rf]∈RPd−1[r_f]\in\mathbb{RP}^{d-1} (a line through the origin, i.e. rfr_f and −rf-r_f identified).

This map’s “feature direction” is the oriented ray — a point on Sd−1S^{d-1} — because the sign is meaningful (positive vs. negative polarity of ff), not the bare subspace span⁡(rf)\operatorname{span}(r_f). Note also that “definable by linear algebra” is too loose to be a definition (an ellipsoid is given by a matrix yet is not a subspace); the precise notions are: a linear subspace is closed under addition and scalar multiplication; an affine subspace is x0+Vx_0+V; a hyperplane is an affine subspace of codimension 11.

Intuition

If a feature is encoded as a direction, then moving an activation further along that direction (in either sign) should correspond to more or less of the feature, the same way moving further along a number line corresponds to a larger or smaller value — regardless of which of the other d−1d-1 orthogonal directions the point also happens to sit at.

Properties

  • 1-dimensional. rfr_f spans a 1-dimensional linear subspace span{rf}\text{span}\{r_f\}; the direction is this subspace together with a chosen unit-norm, signed generator.

  • Sign ambiguity, scale fixed. rfr_f and −rf-r_f span the same subspace but represent opposite “polarities” of the feature (e.g. positive vs. negative sentiment) — the sign convention is a modeling choice, not intrinsic to the subspace, but ∥rf∥=1\|r_f\|=1 fixes the scale so that rf⋅xr_f\cdot x has a consistent, comparable unit across different xx.

  • Projection decomposition. Any x∈Rdx\in\mathbb{R}^d decomposes uniquely as x=(rf⋅x) rf+x⊥x = (r_f\cdot x)\,r_f + x_\perp, where x⊥⊥rfx_\perp \perp r_f — the component along rfr_f is exactly rf⋅xr_f\cdot x (since ∥rf∥=1\|r_f\|=1), and x⊥x_\perp carries everything orthogonal to the feature.

  • Monotonicity, not linearity of the underlying computation. The claim is only that rf⋅xr_f\cdot x is a monotonic readout of the feature’s value — the map from “context” to xx can be arbitrarily non-linear; linearity is a claim about the geometry of the representation, not about the function producing it.

  • Validity requires more than correlation. A vector w/∥w∥w/\|w\| from a trained linear classifier is a candidate rfr_f: it demonstrates linear separability (correlational evidence) but not that a system computing with xx actually uses the rfr_f component — the latter is a causal claim, checked by an intervention (e.g. adding αrf\alpha r_f to xx and confirming the behavior changes as expected — see Linear Separability for the weaker, purely-geometric separability property this builds on).

  • Group identity as a mean-vector direction. Libovický, Rosa & Fraser (2019) show that “which language is this sentence in” is largely captured by a single per-language mean vector (centroid) in mBERT: subtracting it collapses language-ID decodability (.919→.285 accuracy) while modestly improving cross-lingual retrieval — an early, purely descriptive precursor to Chang, Tu & Bergen (2022)‘s causal version of the same idea (cross-lingual-subspace-overlap), which shows the analogous mean-difference direction causally steers token predictions when added and projected through the model. See language-identity-as-centroid-shift.

  • Verb-aspect properties as semantic-scale directions, with anisotropy as a confound on reliability. Li, Chersoni & Hsu (2024) construct three linear scales (stativity, telicity, durativity) from clustered word-level contrasts and find all three are significantly linearly separable in most layers of BERT and GPT-2, but reliability degrades in later layers — tracking each model’s rising Anisotropy rather than a genuine loss of the underlying linear structure. See aspect-encoded-as-linear-scales.

  • A causally-estimated treatment-effect direction, not just a trained-classifier direction, coincides with the top variance direction. Amini, Pimentel, Meister & Cotterell (2023) estimate the average treatment effect of grammatical gender/number using real, dependency-tree-verified naturalistic counterfactual sentence pairs (an input-level intervention, not a representation-space projection), and find this causally-estimated direction has ~0.99 cosine similarity with the first principal component of the same paired representations — independent evidence for a single-direction encoding that does not rely on training a linear classifier at all. The same paper finds probes exceeding 90% accuracy on ordinary held-out data collapse to ~60% (near chance) when tested against these naturalistic counterfactual pairs, a further instance of “validity requires more than correlation” above. See naturalistic-ate-matches-linear-pc-direction and probing-accuracy-collapses-under-naturalistic-counterfactual-test.

  • Convergent validity: independently-derived direction-finding methods locating the same direction is itself evidence for the structure. Tigges, Hollinsworth, Geiger & Nanda (2023) extract a candidate sentiment direction five separate ways (mean difference, k-means, logistic-regression weights, PCA, and a learned DAS direction) and find pairwise cosine similarities of 72.6-99.1% across GPT-2 and Pythia — far above a random-direction baseline (0-2.4%) — then causally validate the resulting direction via directional activation patching, directional ablation (a 38-point accuracy drop from ablating it across an entire dataset), and additive steering. They also explicitly test whether the structure is more than 1-dimensional: increasing a DAS subspace’s dimensionality improves in-sample fit but not out-of-distribution generalization, leaving open (their own words) that the feature “could be split further into more specific sentiment features.” See sentiment-encoded-as-single-convergent-linear-direction.

  • A linear direction’s causal effect can be spatially routed away from the tokens that “contain” it, onto intermediate non-valenced tokens, before being read out. The same paper traces, via path patching and attention analysis, a “summarization motif”: sentiment information from charged words (e.g. an adjective) is first written by dedicated attention heads onto an unrelated intermediate token (a repeated noun, a comma, a period) and only then read by downstream heads — and this summarization pathway’s share of the total causal effect grows with the distance between the charged tokens and the output (from 0.29× to 1.15× the direct pathway’s effect over 0-22 intervening tokens). This is a fact about where in the sequence a direction’s causal effect is carried, orthogonal to the direction’s own geometry, and a caution for any causal-validation method (e.g. Activation patching) that only intervenes at the source token itself. See summarization-motif-routes-sentiment-through-non-valenced-tokens.

  • Relative arrangement between multiple concept directions can itself be informative, beyond each direction’s individual separability. Ju, Shao, Wang et al. (2025) find that three personality-trait directions (Neuroticism, Extraversion, Agreeableness), each individually linearly separable via Linear probing in the middle-to-upper layers of 11 LLMs, also sit in a specific relative configuration — Extraversion “situated between” the other two, more distant from each other — and this betweenness predicts which pairwise trait-conversions are hardest to achieve via a later editing intervention. See personality-traits-linearly-encoded-with-betweenness-arrangement.

  • A learned weight-matrix row can itself be a “direction” whose vocabulary-space projection is the readout, not a probe-derived vector. Geva, Schuster, Berant & Levy (2020) treat each row viv_i of a feed-forward sublayer’s second parameter matrix as a direction: projecting viv_i through the model’s own unembedding matrix (pi=softmax(vi⋅E)p_i=\mathrm{softmax}(v_i\cdot E)) yields a next-token distribution that increasingly agrees with the value’s paired “key” pattern’s actual continuation as depth increases (near 0% agreement in early layers to 3.5% in the final layer of a 16-layer LM) — the direction here is a literal row of a trained weight matrix, not a direction fit afterward by a probe or a difference-of-means. See ffn-value-vectors-are-vocabulary-space-directions-in-upper-layers.

  • A permanent weight-space edit can itself be an outer-product direction mechanism, without claiming the edited fact was already linearly encoded. Meng, Bau, Andonian & Belinkov (2022, ROME) insert a new factual association into an MLP’s weight matrix via a rank-one update W^=W+Λ(C−1k∗)⊤\hat W=W+\Lambda(C^{-1}k_*)^\top — for any input xx, W^x=Wx+Λ⋅((C−1k∗)⊤x)\hat Wx = Wx+\Lambda\cdot((C^{-1}k_*)^\top x), exactly the “trigger direction detects alignment, value direction is added to the output” pattern of diff-in-means-plus-steering, just baked into the weights rather than added at inference time. This USES the direction/steering mechanism as an editing tool; it does not claim the inserted fact was linearly encoded in the model before the edit — the value vector v∗v_* is optimized fresh per edit, not extracted from pre-existing geometry (the paper explicitly states understanding of the pre-existing “structure of the vector spaces that represent learned attributes remains incomplete”). See rome-two-peak-causal-localization-of-facts.

  • The same outer-product direction mechanism generalizes to a batch of thousands of simultaneous edits spread across a range of layers, once the naive sequential-single-edit approach is shown to break down. Meng, Sharma, Andonian, Belinkov & Bau (2022, MEMIT) find ROME’s single rank-one update degrades once applied sequentially for many facts (starting at n=32n=32, collapsing by n=10,000n=10{,}000), and generalize the same WK≈VWK\approx V linear-associative-memory framing to a jointly-solved least-squares update Δ=RK1⊤(C0+K1K1⊤)−1\Delta=RK_1^\top(C_0+K_1K_1^\top)^{-1} over u≫1u\gg1 key-value pairs at once, apportioning the residual equally across a whole range of MLP layers (ril=(zi−hiL)/(L−l+1)r_i^l=(z_i-h_i^L)/(L-l+1)) rather than writing into one layer. This remains USES-not-FINDS — no new claim about pre-existing key-direction geometry (e.g. orthogonality/interference between facts) is made — but reaches a CounterFact score of 85.8/82.0 on GPT-J/GPT-NeoX at 10,000 simultaneous edits where ROME and MEND collapse. See memit-batch-multilayer-editing-scales-fact-insertion.

  • A per-example fine-tuning gradient is itself a rank-one outer product, and a learned hypernetwork can transform its two factors independently while preserving that outer-product form end-to-end. Mitchell, Lin, Bosselut, Finn & Manning (2022, MEND) note that ∇WℓL=∑iδℓ+1i(uℓi)⊤\nabla_{W_\ell}\mathcal L=\sum_i\delta_{\ell+1}^i(u_\ell^i)^\top is a sum of rank-one outer products by the ordinary backprop identity, then train small per-layer networks gϕℓ(uℓ,δℓ+1)=(u~ℓ,δ~ℓ+1)g_{\phi_\ell}(u_\ell,\delta_{\ell+1})=(\tilde u_\ell,\tilde\delta_{\ell+1}) that reshape each factor separately, applying the edit W~ℓ=Wℓ−αℓ∑iδ~ℓ+1i(u~ℓi)⊤\widetilde W_\ell=W_\ell-\alpha_\ell\sum_i\tilde\delta_{\ell+1}^i(\tilde u_\ell^i)^\top — the same trigger/value outer-product pattern as ROME/MEMIT’s closed-form updates, reached instead by a trained hypernetwork acting on gradient factors. USES-not-FINDS: no claim is made about pre-existing linear geometry in the base model. Effective up to 11B parameters (T5-XXL), where competing editors either fail or cannot run. See mend-hypernetwork-transforms-rank-one-gradient-factors.

  • A hyper-network predicting a rank-one-structured gate and bias for a fact’s loss gradient — the direct architectural predecessor of MEND’s gradient-factor transformation. De Cao, Aziz & Titov (2021, KnowledgeEditor) predict, per weight matrix, an update ΔW=σ(η)(α^⊙∇WL+β^)\Delta W=\sigma(\eta)(\hat\alpha\odot\nabla_W\mathcal L+\hat\beta) where α^=σ^(α)γ⊤\hat\alpha=\hat\sigma(\alpha)\gamma^\top and β^=σ^(β)δ⊤\hat\beta=\hat\sigma(\beta)\delta^\top are each outer products of two vectors — “efficient parameterization of a matrix with just three vectors,” in the authors’ words. USES-not-FINDS: no claim is made about pre-existing fact geometry. A KL-in-output-space training constraint (not a raw parameter-space norm) is what prevents catastrophic forgetting — an ablation using the latter collapses retain accuracy from 98.14 to 45.10. See knowledge-editor-outer-product-gated-gradient-editing.

  • A weight-editing mechanism can grow a matrix by one key-value neuron per mistake rather than modify existing weights, and the newly-appended pair still forms the same outer-product direction structure. Huang, Shen, Zhang, Zhou, Rong & Xiong (2023, Transformer-Patcher) append a new row kpk_p and column vpv_p to an FFN’s up/down-projection matrices so that FFNp(q)=FFN(q)+ap⋅vp\mathrm{FFN}_p(q)=\mathrm{FFN}(q)+a_p\cdot v_p, ap=Act(q⋅kp+bp)a_p=\mathrm{Act}(q\cdot k_p+b_p) — a trigger direction detected via inner product, a value direction added to the output, the same pattern as ROME’s rank-one update but reached by matrix growth. Locality across thousands of sequential edits is maintained by a scalar activation-margin loss against a memory bank of past queries, not any geometric constraint between different patches’ key directions. USES-not-FINDS, same as the rest of the editing cluster. See transformer-patcher-per-mistake-neuron-addition.

  • Relational knowledge decoding is well-approximated by a single affine map (a Jacobian-derived linear relational embedding) for roughly half of tested relations, but fails almost completely for others despite the model still answering correctly. Hernandez, Sharma, Haklay, Meng, Wattenberg, Andreas, Belinkov & Bau (2023) fit LRE(s)=βWrs+br\mathrm{LRE}(s)=\beta W_r s+b_r (the mean Jacobian of the subject-to-object decoding function) for 47 relations across GPT-J, GPT2-XL and LLaMA-13B, finding faithfulness over 60% for nearly half the relations with strong faithfulness-causality correlation (R=0.84), but near-zero faithfulness (≤7%) for relations like “company CEO” or “person’s father” that the same models still answer correctly — evidence that linear relation decoding is a real, causally-validated structure for many but not all relations. See lre-faithful-and-causal-for-half-of-relations.

  • Independence of two features. Two features with directions r1,r2r_1,r_2 are representationally independent (in the strongest sense) if intervening on one (e.g. adding αr1\alpha r_1) leaves the readout r2⋅xr_2\cdot x of the other unchanged for every α\alpha — orthogonality (r1⋅r2=0r_1\cdot r_2=0) is necessary for this under a purely linear read/write model but need not be sufficient if the underlying system computes non-linearly with xx.

  • Two directions that “should” be independent are measurably not orthogonal in practice, and can be corrected by a graded rotation rather than erasure. Dev, Li, Phillips & Srikumar (2020) measure the angle θ′=arccos⁡(⟨v1,v2⟩)\theta'=\arccos(\langle v_1,v_2\rangle) between a gender direction v1v_1 and an occupation direction v2v_2 in GloVe and RoBERTa, find it is not π/2\pi/2 (i.e. the two directions are correlated, not independent), and instead of projecting the gender direction out of every word (which they show can destroy valid associations, e.g. collapsing “matriarch entails woman” from 97% to 16% entailment probability), apply OSCaR’s smoothly graded, per-word rotation that pushes v2v_2 toward orthogonality with v1v_1 while leaving words far from either direction nearly untouched. See oscar-graded-rotation-preserves-information-better-than-projection.

  • A whole orthogonal basis of directions can be driven into axis-alignment with independent generative factors by training pressure, not just found post-hoc by a probe. Burgess, Higgins, Pal, Matthey, Watters, Desjardins & Lerchner (2018) give a rate-distortion/information-bottleneck derivation of why β-VAE’s latent code becomes axis-aligned with independent ground-truth factors (position, scale, shape, rotation, colour): the β-weighted KL term forces the posterior to preserve data locality, while each factor’s differing contribution to reconstruction log-likelihood, combined with the posterior’s diagonal covariance, pushes distinct factors onto distinct latent dimensions rather than a rotated/mixed basis. Verified both by a controlled-capacity training curve showing per-factor KL rises from zero in a fixed order as bottleneck capacity increases, and causally by single-dimension latent traversals showing each retained axis’s effect isolated to exactly one factor. See beta-vae-axis-alignment-emerges-from-information-bottleneck-pressure-and-causally-validated-latent-traversals.

  • A continuous, externally-validated scalar target (not a hand-picked binary concept) is linearly decoded from a small ensemble of attention heads, and the same direction causally steers generated text. Kim, Evans & Schein (2025) fit one ridge-regression probe per attention head (1,024 per model) across three open chat LLMs to predict U.S. lawmakers’ DW-NOMINATE political-ideology scores from simulated-statement activations, finding a top-32-head ensemble reaches Spearman ρ\rho up to 0.885, with the fitted direction transferring zero-shot to news-outlet slant (rho up to 0.798) and, added back into activations at generation time, causally shifting GPT-4o-and-human-validated slant ratings of generated text (correlation up to 0.607 with steering magnitude). See attention-head-activations-linearly-predict-lawmakers-dw-nominate-ideology-and-causally-steer-generated-political-slant.

  • A single dictionary-atom direction, not a hand-designed probe direction, is causally shared across typologically diverse languages for the same grammatical concept. Brinkmann, Wendler, Bartelt & Mueller (2025) train a Gated SAE on Llama-3-8B and Aya-23-8B residual streams and find individual feature directions encoding grammatical number, gender, and tense are among each language’s top causally-relevant features across up to all 23 typologically diverse training languages (top-feature overlap up to 50%, one gender feature shared across all 15 gender-inflecting languages tested) — with ablation of the shared multilingual features dropping cross-lingual classifier performance to 61-64%, and single-feature steering causally flipping the targeted concept in a machine-translation task while leaving other concepts largely unaffected. Complements Linear Subspace‘s “Cross-lingual subspace overlap, verified causally” entry (Chang, Tu & Bergen 2022): that result is about a whole per-language subspace coinciding across languages, this one is about individual dictionary-atom directions found via unsupervised sparse decomposition rather than supervised affine fitting. See sae-features-encoding-morphosyntactic-concepts-are-shared-across-typologically-diverse-languages-and-causally-necessary-and-sufficient.

  • A Mikolov-style word2vec offset generalizes from static word analogies to a robot policy’s implicit dynamics model. Molinari, Nevali, Navani & Younis (2025) define a state-transition vector Δet→t+K=et+K−et\Delta e_{t\to t+K} = e_{t+K}-e_t for OpenVLA (a 7B vision-language-action robot model with no explicit world-model training objective) and show linear probes on residual-stream activations predict this vector with statistically significant R2R^2 (123 probes, overall p<0.0001p<0.0001) exceeding a raw-embedding baseline across 4 LIBERO task suites and 4 time horizons, concentrated in middle layers and strengthening with pretraining compute (weak/absent in an early checkpoint) — no causal intervention is performed (the paper’s own stated limitation), so this rests on the quantified linear-decodability criterion alone. See openvla-encodes-a-linear-state-transition-vector-recoverable-from-middle-layer-activations-exceeding-embedding-baselines.

  • A direction’s location within the ambient covariance’s own eigenspectrum is itself a geometric property, distinct from the direction’s semantic content. Acharya, Rimal & Dhakal (2026) project concept difference-of-means directions onto the eigenbasis of the unembedding covariance across 17 models (5 architecture families) and find they anti-concentrate in the low-eigenvalue spectral tail (Spectral Center of Mass 0.926 vs. 0.758 for random directions; Gini-deviation -0.282), while POS/syntax directions and static unembedding-row contrasts for the same concepts concentrate in the high-variance subspace (6/8 architectures, p<0.013; Gini +0.07 vs. -0.10, p<10^-4) — a dual-geometry split, not merely a relative-strength ranking. Causally, injecting only a concept direction’s high-variance (“shouting”) component produces far more perplexity disruption than its low-variance (“whispering”) component (+383% vs. +223% on Llama-3.1-8B, d=1.80, replicated across 5 models). See concept-difference-of-means-directions-anti-concentrate-in-the-low-variance-spectral-tail-of-the-unembedding-covariance-while-syntax-directions-concentrate-in-the-high-variance-subspace.

  • Five independently-regressed trait directions are approximately orthogonal, while an unsupervised alternative built from the same activations collapses them onto one shared axis instead. Frising & Balcells (2025) fit one per-layer OLS regression direction per Big Five (OCEAN) trait in Llama 3.3 70B from 406 role-played fictional-character activations, finding the five regression directions show low cross-talk (approximate orthogonality, Fig. 3) — but a contrasting unsupervised baseline (top-variance SVD directions from the same activations) instead has different traits’ top directions nearly coincide, i.e. collapse toward a single shared “personality” axis rather than five independent ones. The regression directions causally and monotonically steer forced-choice Extraversion judgments for ∣α∣≤0.4|\alpha|\le0.4 (beyond which outputs degrade), but this causal reach vanishes entirely once character context is already present in the prompt and is only marginal in open-ended generation — evidence that a direction’s demonstrated linear, near-orthogonal structure doesn’t guarantee unconditional causal reach. See frising-balcells-2025-big-five-trait-directions-are-approximately-orthogonal-and-reliably-steer-forced-choice-personality-but-not-open-ended-or-context-laden-generation. Contrast personality-traits-linearly-encoded-with-betweenness-arrangement (Ju et al. 2025), which finds a specific non-orthogonal relative arrangement (betweenness) among three different trait directions in a different model set — the two papers examine different trait subsets with different methods and reach compatible but distinct geometric claims, not a contradiction.

  • In an audio generative model, SAE feature directions linearly decode distinct acoustic properties with a consistent cross-model accuracy hierarchy, and the same per-class linear-probe weight rows causally control the generated audio when added back as directions in SAE feature space. Paek, Zang, Yang & Leistikow (2025) train SAEs on four pretrained audio-generation latent spaces (Stable Audio Open / DiffRhythm continuous VAEs, EnCodec and WavTokenizer discrete codecs), then fit multi-class linear probes p(a)=softmax(W(a)f+b(a))p^{(a)}=\mathrm{softmax}(W^{(a)}f+b^{(a)}) from SAE features ff to discretized pitch (66 bins), loudness (20 bins), and timbre/spectral-centroid (20 bins). Linear decodability is consistently ordered the same way across all four latent spaces — pitch most separable (0.75-0.87 accuracy, stable across SAE sparsity levels), loudness intermediate (0.17-0.63, model-dependent), timbre least (0.17-0.46) — a quantified, cross-model-replicated geometric-shape hierarchy among concepts sharing the same feature space, not merely a single accuracy number. Causally, each acoustic class’s probe weight row wk(a)w_k^{(a)} is directly reused as a “control vector”: adding α⋅wk(a)\alpha\cdot w_k^{(a)} to the SAE features before decoding shifts the regenerated audio toward that class (isolated pitch/timbre/loudness changes with increasing α\alpha, other properties largely preserved) — the same weight vector serves as both the linear-decodability direction and the causal steering direction, with no separate direction-extraction step. Applied to DiffRhythm’s 32-step generation trajectory, the probes further reveal a coarse-to-fine emergence order: pitch converges first (~step 21), then timbre, with loudness still unresolved at the final step. See paek-etal-2025-sae-feature-directions-in-audio-generative-latents-linearly-decode-a-consistent-pitch-loudness-timbre-separability-hierarchy-and-the-same-probe-weight-vectors-causally-control-generated-audio.

  • A direction extracted purely from a model’s text-only backbone, with no multimodal data at all, causally steers the same weights’ multimodal (image-token) behavior after vision-language fine-tuning. Gan, Fu, Asilis, Liu, Yogatama, Sharan, Jia & Neiswanger (2025) extract mean-shift (diff-in-means), linear-probing, and SAE-derived directions for four visual-concept taxonomies (spatial relationship, counting, attribute, entity) entirely from a text-only LLM (Gemma2-2B/9B, Llama-3.1-8B), using only text sentence-anchor pairs, then add these same directions to image-token activations in the corresponding fine-tuned MLLM (PaliGemma2-3B/10B, Idefics3-8B-Llama3) — a causal test of whether steering-relevant linear structure in the text backbone survives vision-language post-training and transfers cross-modally without ever being fit on multimodal data. Mean-shift steering improves spatial-relationship accuracy by up to +7.3% and counting by up to +3.3% in-distribution, generalizes out-of-distribution to five further datasets (average +7.6%, versus +1.0% for text prompting), and SAE analysis independently shows each visual concept activates a sparse subset of features (fewer than 10 of 16k-32k total SAE features) in the text-only backbone — a quantified sparse-coding claim about how the transferred concept is represented before transfer. See gan-etal-2025-text-only-mean-shift-directions-causally-steer-image-token-representations-in-fine-tuned-mllms-without-any-multimodal-training-data.

  • A direction’s layer-wise localization, quantified by a Gini coefficient over per-layer causal-effect strength, splits sharply by concept type and grows with model depth for the least-localized category. Deng, Ju, Wu, He, Lan, Zhu, Wang & Zhang (2026) extract per-layer diff-in-means visual-concept directions (entities, visual style, emotion, abstract concept) in six multimodal LLMs (Qwen2.5-VL-7B, Qwen3-VL-8B/32B, LLaVA-OneVision-1.5-8B, Gemma3-4B/27B) and measure, per concept category, both a peak causal-steering effect and a Gini coefficient over per-layer effect strength (a genuine quantified geometric-shape statistic: how concentrated vs. spread a direction’s causal footprint is across the network’s depth). Entity directions are sharply localized (Gini averaging 0.071, as low as 0.012 in Gemma3-4B) with a bimodal layer profile, matching the “explicit key-value pair” signature also seen in FFN value-vector and ROME localization; abstract-concept directions are far more distributed (Gini up to 0.429, right-skewed toward final layers) and this distributedness increases with model scale (larger models show lower Gini for entities but higher Gini for abstract concepts) — a scale-dependent geometric signature the authors tie to why deeper models handle complex abstract semantics better without becoming better at localizing concrete entities. See deng-etal-2026-visual-concept-directions-in-mllms-diverge-into-localized-entity-encoding-versus-globally-distributed-abstract-concept-encoding-with-distribution-widening-under-scale.

  • Three behaviors that share surface form (all “agreeing” or “flattering”) occupy measurably distinct directions, with the distinctness itself changing across depth, and each is independently steerable by up to two orders of magnitude more than its neighbors. Vennemeyer, Duong, Zhan & Jiang (2025) track cosine similarity between diff-in-means directions for sycophantic agreement, genuine agreement, and sycophantic praise across 9 datasets and every layer of five model families: the two agreement-type directions start nearly identical (cosine ~0.99, early layers) but diverge sharply with depth (cosine ~0.07 by layer 25), while the praise direction stays under cosine 0.2 with both at every layer — a layer-dependent divergence pattern rather than a fixed orthogonality/non-orthogonality fact. Steering each direction shifts its own targeted behavior 6.8-36.8x more than either other behavior (selectivity ratios, Table 2), and ablating one behavior’s subspace leaves the others’ detectability unaffected — replicated across GPT-OSS-20B, LLaMA-3.1-8B, LLaMA-3.3-70B, and two Qwen3 scales. See sycophantic-agreement-genuine-agreement-and-sycophantic-praise-occupy-distinct-diff-in-means-directions-independently-and-selectively-steerable.

  • The specific direction found for one concept depends measurably on the extraction method, not just on the concept itself. Yao, Anand, Zhuang & Jiang (2026) extract rhetorical-vs-literal question directions from Qwen3-32B and Llama-3.3-70B via three methods (training-free diffMean, logistic regression, hinge-loss SVM): the two trained-classifier directions nearly coincide (cosine ~1) regardless of loss function, but the training-free diffMean direction is only weakly aligned with either (cosine <0.7 on one dataset, ~0.5 on another) — and directions fit on different datasets for the same nominal behavior align only weakly with each other (cosine 0.2-0.4), even though within-dataset AUROC reaches ~0.85-0.9 for all methods. This decomposes “is rhetorical-question status linearly decodable” (yes, robustly) from “is there one dataset-general rhetorical-question direction” (no — a method- and corpus-dependent family of related but non-collinear directions). See yao-etal-2026-rhetorical-question-status-is-linearly-decodable-but-different-direction-extraction-methods-find-measurably-non-collinear-directions.

  • A dot product taken in a network’s attention-output space, not its raw cross-attention space, is where concept-image alignment concentrates — and the effect is architecturally prevented from being causal. Helbling, Meral, Hoover, Yanardag & Chau (2025) thread extra concept tokens through a frozen diffusion transformer’s (Flux-Schnell’s) own text-conditioning weights via a one-directional attention operation (concepts read image/prompt tokens but never write back), then localize each concept by φ(ox,oc)=softmax(oxoc⊤)\varphi(o_x,o_c)=\mathrm{softmax}(o_x o_c^\top) — a dot product between image and concept attention-output vectors. An explicit space ablation shows this output-space dot product (ImageNet-Segmentation Acc/mIoU/mAP 83.07/71.04/90.45) strongly outperforms the same dot product taken in cross-attention space (74.92/59.90/87.23) or value space (45.78/29.68/39.61) — a quantified claim about where among a transformer’s several naturally-available linear subspaces concept-image alignment structure lives, reaching state-of-the-art zero-shot segmentation against 11 baselines. Because concept tokens are architecturally barred from influencing image tokens, this is a purely passive read-out claim with no causal-intervention component, unlike most other entries in this section. See helbling-etal-2025-conceptattention-output-space-dot-products-localize-textual-concepts-in-dit-image-tokens-far-more-precisely-than-raw-cross-attention.

  • Linear decodability of several concept parameters, in a modality outside language and vision, follows a consistent per-concept hierarchy and correlates with the representation’s own manifold smoothness — not just with probe accuracy considered alone. Pandey, Neog & Jajoo (2025) linearly probe frozen Chronos and MOMENT (time-series foundation models) for generative parameters of synthetic series (AR(1) coefficient, trend slope, level-shift magnitude, dispersion, spectral/time-warp factors), finding early layers cleanly recover local time-domain concepts (AR(1), trend, level shifts, typically by the second layer) while spectral and time-warping concepts remain the hardest to decode at any depth — and, distinctively, that “when a parameter varies smoothly along the UMAP manifold, probe accuracy improves even more,” tying linear decodability directly to a geometric smoothness property of the representation rather than treating them as independent facts. Composing multiple concepts simultaneously degrades probe performance and produces anomalous vector-arithmetic distance blowups for specific concept pairs — interference between individually well-localized directions, with no causal intervention performed (a purely passive, correlational study). See pandey-etal-2025-time-series-foundation-models-linearly-recover-generative-concept-parameters-with-a-quantified-per-concept-per-layer-hierarchy-and-probe-accuracy-tracks-umap-manifold-smoothness.

  • A per-layer diff-in-median direction, localized by a Fisher Linear Discriminant Ratio rather than a single global probe, causally injects a concept into a non-language, non-vision modality and flips real downstream classifications. Wiliński, Goswami, Potosnak, Żukowska & Dubrawski (2024) compute steering vectors SiS_i (median-difference between periodic/trending vs. constant synthetic time series) at every layer of three pretrained time-series foundation models (Chronos, MOMENT, Moirai), localize where the concept is strongest via a Fisher LDR across layers and tokens, then additively intervene (hi←hi+λSih_i\leftarrow h_i+\lambda S_i) — steering real ECG5000 heartbeat signals causes all 30 tested samples (100%, independently verified) to flip classification from normal to abnormal, direct causal evidence the injected direction changes downstream model behavior, not merely the raw signal’s appearance. See wilinski-etal-2024-diff-in-median-steering-vector-causally-injects-periodicity-trend-flips-ecg-classifications.

  • Individual sparse-autoencoder features linearly correlate with distinct real-valued chemical properties across four frozen LLM-based molecular editors, and a gradient-derived direction routed through the same SAE causally improves the model’s own property-editing accuracy by tens of points. Zhang, Li, Li, Shen, Xiong & Sun (2026) train a Gated SAE with learnable per-property “importance gates” on the hidden states of DrugAssist, GeLLM3O-LLaMA3, GeLLM3O-Mistral, and MolGen, finding individual top features reach Spearman correlations up to +0.93 (molecular weight) and -0.85 (QED) with their target property — a quantified feature-to-property alignment claim, though weaker for properties like HBD/DRD2 (rho~=0.32) that the authors attribute to distributed rather than monosemantic encoding. Causally, a direction derived from the gradient of property-improved SFT log-likelihood, projected through the SAE’s top-k features and added back to the residual stream at inference, raises MolEditRL property-directed editing accuracy by up to +42.4 points (MolGen rotatable-bond count, Acc@0.15) with no change to the base model’s frozen weights — one of this map’s largest quantified steering effect sizes, and the first in the molecular-editing modality. See zhang-etal-2026-slim-sae-features-linearly-correlate-with-molecular-properties-and-causally-steer-property-directed-editing.

  • A directional-derivative sensitivity score along a concept activation vector, quantitatively validated against known ground-truth causal effects, uncovers a real demographic shortcut in a medical vision-language model. Janizek, Xu, Lateef & Daneshjou (2026) extract per-concept unit-length CAVs from OpenFlamingo’s activations and validate their “Visual Concept Ranking” sensitivity score’s causal fidelity on a synthetic benchmark with known ground-truth interventional effects (Pearson r=0.53 overall, p=4.9e-49; 92% correct sign-identification of a spurious feature’s effect under distribution shift vs. 18% for a correlational baseline) before applying it to real dermatology images — correctly flagging that blue/purple ink-marking dots drive “malignant” predictions specifically for darker-skin samples, confirmed by targeted input-level image edits. See janizek-etal-2026-concept-activation-vector-sensitivity-scores-in-a-medical-lmm-correlate-with-ground-truth-interventional-effects-and-flag-real-demographic-shortcuts.

  • A parameter-free class-conditional mean (centroid) reproduces one tabular foundation model’s own prediction almost exactly, while a different pretrained model instead relies on an attention-weighted vote — and transplanting either readout rule onto the wrong backbone destroys accuracy. Biloš, Wilson, Schneider & Nevmyvaka (2026, Morgan Stanley) show TabICLv2’s in-context classification is reproduced by “form one prototype per class by averaging the final-block representations of context examples sharing that label, then assign each query to the nearest prototype” (0.856 vs. 0.864 native accuracy), while TabPFNv2 and Mitra instead implement an attention-weighted vote (uniformizing the attention pattern collapses accuracy from 0.87 to 0.49). Cross-backbone transplantation is directly causal: the prototype rule moved onto TabPFNv2 drops accuracy 33 points, and the vote rule moved onto TabICLv2 drops it 40 points — each backbone’s native readout specifically requires its own representation geometry. Separately, zeroing TabPFNv2’s per-feature positional weight matrix WW grants exact permutation invariance “and leaves benchmark accuracy unchanged,” a geometry-tied causal intervention that is quantifiably free. See bilos-etal-2026-tabicl-implements-a-literal-per-class-mean-centroid-readout-transplanting-it-onto-a-different-backbone-drops-accuracy-40-points.

  • A concept activation vector localized to a specific layer band of a video world model causally shifts intuitive-physics judgments with a clean dose-response, and is measurably orthogonal to both a motion-direction CAV and to CAVs for other physics principles. Alam (2026) fits per-layer linear probes on frozen VideoMAE-base to classify IntPhys videos as physically possible/impossible, finding accuracy “peaks at layer 5 (70.1%) and remains elevated across layers 0-5 before dropping at layer 6” — a “Physics Emergence Zone.” Injecting the L2-normalized layer-5 probe direction additively into all patch tokens produces a clean dose-response (P(impossible) reaching 1.000 at α=+5\alpha=+5, near 0 at α=−5\alpha=-5, saturating by ∣α∣=5|\alpha|=5) with a sharp layer cutoff (0.25 flip rate for layers 0-5, 0.00 for layers 6-11, directional purity rising to +1.000 at layer 5). The physics CAV is 90.0 degrees from a motion-direction CAV (“higher than the 69-83 degree range reported by Joseph et al.”) and 87.5 degrees from a random unit vector on average, while CAVs for distinct intuitive-physics principles (object permanence/continuity/etc.) are themselves only partially orthogonal to each other (75.7-86.1 degrees) — a quantified angular structure among several related-but-distinct concept directions, extending the TCAV lineage into the video-world-model setting. See alam-2026-physics-plausibility-cav-in-a-localized-layer-band-of-a-video-world-model-is-orthogonal-to-motion-direction-and-different-intuitive-physics-principles-occupy-distinct-directions.

  • A behavioral failure mode (hallucination) is linearly decodable with a layer-depth trend in both a model’s raw activations and a downstream sparse-autoencoder latent space, concentrated in a handful of SAE features, and additively steerable in either space to cut its rate roughly 5x. Aparin, Popov, Sadekova & Yermekova (2026) show Whisper’s audio-encoder hallucination status is linearly separable via per-layer logistic-regression probes (AUC 0.74-0.80), with accuracy “concentrated in the final encoder layers” in both raw-activation and Batch-Top-k-SAE-latent spaces, and SAE-side “performance stabilizes at 50-100 features” out of thousands. Additively steering a diff-in-means direction (final layer) or a sparse top-k-feature sign pattern (SAE latent space, 10-25 features) cuts non-speech hallucination rate from 72.63% to 14.11% (Whisper small) and 86.88% to 27.33% (Whisper large-v3), at a modest clean-speech WER cost — a direct comparison of two causally-effective steering targets (a dense diff-in-means direction vs. a sparse decomposed feature subset) for the same behavior in the same model. See aparin-etal-2026-whisper-hallucination-status-is-linearly-decodable-with-increasing-depth-concentrated-in-a-sparse-sae-feature-subset-and-causally-steerable-to-cut-hallucination-rate-by-5x.

  • Sparse-autoencoder features trained on a real self-supervised astronomical foundation model surface monosemantic morphology directions that beat a raw-PCA baseline of equal rank at correlating with human labels. Wu & Walmsley (2026) train a Matryoshka SAE (batch top-k=64) on a real self-supervised ViT-S MAE (30.1M params, trained on 3M Euclid Q1 galaxy images) and on a real Zoobot ConvNeXt-Nano classifier’s embeddings, finding top-64 SAE features correlate with Galaxy Zoo human morphology labels at mean-max Spearman r=0.523±0.123r=0.523\pm0.123 (self-supervised) and 0.296±0.1290.296\pm0.129 (supervised), versus only 0.434±0.1740.434\pm0.174 and 0.176±0.1380.176\pm0.138 for a raw-PCA baseline of the same rank, with SAE features staying coherent through k=64 while PCA degrades beyond ~5 components. SAEs additionally surface features outside the Galaxy Zoo taxonomy (dust lanes, blue companions to ellipticals); purely correlational, no causal steering performed. See wu-walmsley-2026-saes-trained-on-a-real-euclid-galaxy-mae-surface-monosemantic-morphology-directions-outperforming-raw-pca.

  • PCA on a real pretrained weather model’s latent mesh-node features reveals a dipole direction tracking synoptic wave troughs, a preliminary qualitative finding. Tempest, Beylich & Craig (2026) run PCA on 512-dimensional latent feature vectors extracted at mesh nodes after each of real pretrained GraphCast’s 16 processor steps (1-degree configuration, ERA5 reanalysis input), finding the first principal component forms a pronounced alternating dipole pattern over Northern mid-latitudes tracking synoptic-scale wave troughs, with a separate leading direction correlating with specific-humidity gradients (e.g. Sahel moisture front). This is purely observational — qualitative visual correspondence across two case studies and a small sample of forecast times, explicitly framed by the authors as preliminary, with no causal intervention performed. See tempest-etal-2026-pca-on-a-real-pretrained-weather-models-latent-mesh-nodes-reveals-a-dipole-direction-tracking-synoptic-wave-troughs.

  • TopK sparse-autoencoder features on a real fine-tuned protein language model align with real structural sites, and steering them causally improves in-silico protein design. Tsui, Talreja & Aghazadeh (2025) train a TopK SAE (d=4096, k=128) on layer-24 embeddings of real ESM2-650M, LoRA-fine-tuned per assay on MSA sequences, and show the resulting sparse latents align with real biological structure (active-site residues, C-terminus, allosteric/binding/epistatic sites mapped onto AlphaFold3 structures); the top 5% of SAE probe weights explain 37-38% of fitness-prediction variance vs. 25-28% for raw ESM-layer weights, and SAE-based probes outperform raw-ESM2 baselines in 58-69% of low-N (N=24) extrapolation tasks. Causally, amplifying predictive SAE latents and decoding back through the SAE (“feature steering”) generates protein variants that outperform ESM2-based design in 88% of metric-by-assay combinations, producing the single best-fitness variant in 5/6 DMS assays; validation is in-silico, not wet-lab. See tsui-etal-2025-topk-sparse-autoencoder-features-on-a-real-fine-tuned-esm2-align-with-real-protein-structural-sites-and-steering-them-causally-improves-in-silico-protein-design.

  • Sparse-autoencoder features on a real small genomic language model correspond to nucleotide identity and real transcription-factor binding motifs. Guan, He & Zhang (2025) train an SAE (32x expansion, 8,192-feature dictionary) on layer-3 activations of real HyenaDNA-small-32k, pretrained at single-nucleotide resolution on the human reference genome. Individual features correspond to nucleotide-identity positions and to real transcription-factor binding sites, validated against independent ground-truth JASPAR motif annotations on chromosome 14 via nucleotide-level precision/recall/F1 (named matches include MA1596.1, MA2121.1 [C2H2 zinc finger] and MA0052.5 [MADS-box], with strand specificity). Purely correlational — no steering/ablation intervention is performed; the paper names causal use as future work. See guan-etal-2025-sparse-autoencoder-features-on-a-real-small-genomic-language-model-correspond-to-nucleotide-identity-and-real-transcription-factor-binding-motifs.

  • TopK SAE features on real Whisper and HuBERT encoders are seed-stable and sparsely erasable, and steering them causally cuts false-speech detections by 70%. Aparin, Sadekova, Rukhovich, Yermekova, Kushnareva, Popov, Kuznetsov & Piontkovskaya (2026, AudioSAE) train TopK SAEs across all encoder layers of real Whisper-small and HuBERT-base, finding over 50% of features remain consistent across random seeds, and quantifying disentanglement via concept erasure (only 19-27% of features need removal to erase a target concept). Causally, steering SAE features reduces real Whisper-small’s false speech detections by 70% with negligible WER degradation on LibriSpeech test-clean. See aparin-etal-2026-topk-sae-features-on-real-whisper-and-hubert-encoders-are-seed-stable-and-sparsely-erasable-and-steering-them-causally-cuts-whisper-false-speech-detections-by-70-percent.

  • Concept information lives in a hidden state’s angle, not its norm — yet the best steering intervention deliberately does not preserve that norm. Aparin & Gaintseva (2026) decompose hidden states into a radial component (r=∥x∥r=\|x\|) and an angular concept score (cosine alignment with a unit steering direction), then show via linear probes across seven models that “normalized probes closely match raw probes, while norm-only probes remain near chance” — concept information is essentially entirely angular. Systematically comparing six steering variants that vary norm-preservation and angular-targeting independently, they find strict norm preservation is not the most stable high-strength choice: an explicit radial rescale from β=1.0\beta=1.0 to β=1.2\beta=1.2 improves perplexity “roughly 1.8x” at a fixed high target concept score, with task-metric cost held within about 2.5 points — dissociating where a concept geometrically lives (angle) from what a causally stable intervention should manipulate (angle plus a tuned non-unit radius). See aparin-gaintseva-2026-concept-information-is-carried-angularly-not-radially-and-norm-preservation-is-not-the-optimal-steering-choice-at-high-steering-strength.

  • The KL-optimal way to steer a softmax distribution toward a target concept score is to add the probe in dual (mean-parameter) coordinates, not raw logit coordinates — and this is provable, not just empirically better. Park, Nief, Choe & Veitch (2026) show softmax distributions have a natural Bregman/information geometry (KL divergence = Bregman divergence of the log-normalizer), and prove (Theorem 3.1) that “dual steering” — adding a concept probe direction βW\beta_W in dual space, φ(λt)=φ(λ0)+tβW\varphi(\lambda_t)=\varphi(\lambda_0)+t\beta_W — is exactly the KL-divergence-minimizing intervention reaching a target concept score, extending under a factorizability condition to minimizing off-target concept distributional change specifically. This directly critiques the map’s existing causal-inner-product framework’s reliance on raw logit differences (“the scale of logit differences is not directly proportional to the magnitude of probability changes”) and empirically outperforms ordinary additive (Euclidean) steering across all three robustness metrics tested, on both Gemma-3-4B and MetaCLIP-2 — a rare direct cross-modality (LLM + vision-language) validation of the same provable steering-geometry claim. See park-etal-2026-dual-steering-in-information-geometric-coordinates-is-provably-kl-optimal-and-outperforms-euclidean-steering-across-an-llm-and-a-vision-language-model.

  • Two video foundation models can show near-identical bulk cosine similarity yet differ by several times on a stricter measure of whether a direction is a consistently-oriented axis, and this dissociation predicts real robustness under patch dropout. Alrasheed, Yazdan Parast, Azam, Bailey & Akhtar (2026) linearly probe four real, matched-capacity, frozen video foundation models (V-JEPA2.1, V-JEPA2, VideoPrism, VideoMAEv2) on Something-Something v2, finding all four show cosine similarity above 0.98 between push/pull action-class directions — near-indistinguishable by that measure alone — while their own Directional Semantic Consistency Score (DSCS) separates them sharply, with V-JEPA models scoring “several times higher” than VideoPrism/VideoMAEv2. This directional-consistency gap tracks real behavioral robustness: under patch dropout, V-JEPA2.1 retains 46.1% top-1 accuracy versus VideoPrism’s 2.7%, despite VideoPrism’s representation looking geometrically similar by raw cosine similarity — evidence that DSCS captures oriented-axis structure invisible to bulk similarity metrics. See alrasheed-etal-2026-vjepa-models-score-several-times-higher-on-directional-semantic-consistency-than-videoprism-and-videomaev2-despite-0.98-plus-cosine-similarity-across-all-four.

  • Steering directions’ own pairwise cosine similarity directly predicts how much they mutually dampen each other when injected simultaneously — a second, independent collapse mode from out-of-distribution norm accumulation. Deng (2026, GEMS) finds three diff-in-means expert-persona directions in Qwen3.5-4B pairwise cosine-similar at 0.74-0.87 (“share a substantial common subspace”), and shows simultaneous injection degrades each direction’s own terminal-layer alignment from 0.64-0.66 (alone) to 0.38-0.43 (all three together) — a 34-43% mutual reduction attributed exactly to that non-orthogonality, distinct from a second, separately-measured failure mode where unconstrained injection drives residual-stream norm to 3.4x its natural trajectory. Per-token Gram-Schmidt orthogonalization plus norm-constrained recombination (restricted to the attention-output pathway) fixes both: on GSM8K with 3 concurrent directions, naive steering collapses accuracy 92%->4% (perplexity 14.82->25,173) while the fix recovers 98% accuracy at only +2.2% perplexity cost. See deng-2026-gems-non-orthogonal-steering-directions-mutually-dampen-in-proportion-to-their-cosine-similarity-and-orthogonalized-norm-constrained-superposition-recovers-multi-concept-steering.

  • A single shared “Global Evolutionary Direction” dominates the PCA spectrum of cross-layer contrastive tangent vectors, provably convergent by perturbation theory, and rectifying noisy per-layer steering vectors along it improves steering without any layer-specific tuning. Jiang, Yu, Wang & Hu (2026, GER-Steer) stack layer-to-layer differences of contrastive activation gaps across all layers and samples into one matrix, showing empirically that “the first principal component (PC1) dominates the spectrum across all datasets,” and prove via Wedin’s sin-Theta theorem plus a Davis-Kahan argument that this direction converges to the ground truth at rate O(1/NL)O(1/\sqrt{NL}). Reinforcing each layer’s raw CAA steering vector along this single shared direction yields statistically significant gains (Welch’s t-test, p<0.05p<0.05) over CAA and four other baselines across three models (Qwen2.5-7B, Llama-3.1-8B-Instruct, Gemma-2-9B-it) and five behavioral domains — e.g. Qwen2.5-7B GSM8K accuracy rises from 0.570 (vanilla) / 0.550 (CAA) to 0.734 — without per-layer hyperparameter tuning. See jiang-etal-2026-global-evolutionary-steering-cross-layer-consistency-rectifies-noisy-steering-vectors-via-a-provably-convergent-shared-rank-one-direction.

  • The right causal object for concept steering may not be a single fixed linear direction at all: effective intervention trajectories are curved, multi-step, and token-varying, sharing an initial direction before a pronounced bend into a concept-specific region. Jin, Deng, Wang, Shen & Zhang (2026, FLAS) replace the standard additive steering vector (recovered as their framework’s degenerate one-step, fixed-direction special case) with a learned concept-conditioned velocity field integrated over multiple steps through a frozen pretrained model’s own activations (Gemma-2-2B/9B-IT, layer 20), finding that resulting steering trajectories are “curved, multi-step, token-varying” — every concept’s path “leaves the origin in a shared direction, executes a pronounced bend, and then enters a concept-specific region.” This nonlinear intervention substantially outperforms linear steering baselines and prompting (held-out HMean 1.015 vs. prompting 0.762 and HyperSteer 0.608 on Gemma-2-2B-IT) at a fraction of the parameters, and transfers zero-shot to Qwen3-4B-Instruct — a direct empirical challenge to the assumption that a single fixed linear direction is the right unit of causal intervention for a concept. See jin-etal-2026-flow-based-activation-steering-effective-concept-control-requires-curved-multi-step-token-varying-trajectories-not-a-single-linear-direction.

  • A single shared orthogonal (rotation) matrix is the minimal-distortion weight-edit for concept erasure, provably preserving every other neuron’s norm and pairwise angle by construction. Sun, Yu, Xu, Miao, Xu & Xie (2026, OCE) edit a pretrained diffusion model’s cross-attention weight matrix WW via a single closed-form orthogonal matrix PP (P⊤P=IP^\top P=I, solved via orthogonal Procrustes), giving W∗=PWW^*=PW with ∥wi∗∥=∥wi∥\|w_i^*\|=\|w_i\| and cos⁡φij∗=cos⁡φij\cos\varphi_{ij}^*=\cos\varphi_{ij} for every neuron pair — an algebraic guarantee of this specific transformation class, not a measurement of the base model’s pre-existing angular geometry. A controlled ablation on Stable Diffusion v1.4 shows why the choice matters: additive scaling of WW has “negligible effect,” their shared rotation produces a “clear semantic shift,” while an unconstrained neuron-wise (per-column) rotation “substantially degrades image quality.” The method erases object classes, artistic styles, up to 100 celebrities at once, and NSFW content on SD v1.4, transferring zero-shot to FLUX.1 (DiT/MMDiT). See sun-etal-2026-orthogonal-concept-erasure-for-diffusion-models-a-single-shared-rotation-matrix-erases-concepts-while-provably-preserving-every-other-neurons-norm-and-angle.

  • A single per-concept direction extracted via feature-learning (RFM/AGOP eigenvectors) from forward-noised diffusion activations, reused across the sampling trajectory, out-performs classifier guidance at a fraction of the cost. Wang, Belkin & Wang (2026) train a Recursive Feature Machine as a target-vs-rest classifier on U-Net/DiT block activations at one fixed low-noise timestep, then take the top-kk eigenvectors of its Average Gradient Outer Product matrix as a single reusable steering direction vcv_c added back into activations during sampling. Across unconditional DDPM/ADM (CIFAR-10, ImageNet-256, CelebA-HQ), the transformer-based SiT-XL/2, and Stable Diffusion 1.5, this reaches 96.6% guidance accuracy on CIFAR-10 (vs. 77.1% TFG baseline, 86.0% noise-conditioned classifier guidance) with FID 41.4 (vs. 73.9, 41.9) at a 16x sampling speedup over TFG — a geometry-tied intervention (one extracted, reused low-rank direction per concept) rather than a claim about pre-existing natural geometry in the base model. See wang-etal-2026-general-and-efficient-steering-of-diffusion-models-a-single-rfm-agop-direction-per-concept-steers-unconditional-and-latent-diffusion-models-at-a-fraction-of-classifier-guidance-cost.

  • Rotating a normalized activation along a geodesic toward a diff-in-means direction, instead of adding to it, preserves the activation’s norm exactly and outperforms additive steering on both accuracy and generation quality simultaneously. You, Deng & Chen (2026, Spherical Steering) extract a truthfulness axis exactly as diff-in-means does, but instead of adding λμ\lambda\mu to a raw activation (whose norm change depends on both λ\lambda and alignment μ⊤h\mu^\top h), they normalize hh to the unit hypersphere and rotate it toward μT\mu_T via spherical linear interpolation (Slerp), restoring the original magnitude afterward so ∥h′∥=∥h∥\|h'\|=\|h\| exactly for any rotation amount, with a von Mises-Fisher confidence gate setting the per-token rotation strength from the activation’s own angular alignment. On LLaMA-3.1-8B-Instruct and Qwen-2.5-7B-Instruct this improves average multiple-choice accuracy by over 8 points across six benchmarks (up to 15% on individual ones) versus additive CAA/ITI baselines, while improving rather than degrading open-ended generation quality (TruthfulQA TRUE×INFO) — the opposite of the trade-off additive steering typically shows. See you-etal-2026-spherical-steering-geometry-aware-activation-rotation-rotates-hidden-activations-along-a-norm-preserving-geodesic-toward-a-contrastive-direction-instead-of-adding-to-them.

  • A single attention head’s OV and QK circuits are systematically diagonal-dominant in unembedding space: each token’s own unembedding direction is an extreme (self-suppressing) entry of that token’s row/column, for the overwhelming majority of the vocabulary. McDougall, Conmy, Rushing, McGrath & Nanda (2023) compose GPT-2 Small’s head 10.7 with the unembedding matrix (WUWOV10,7MLP0(WE)W_U W_{OV}^{10,7}\mathrm{MLP}_0(W_E)) and show its diagonal is among the top-10 most negative entries in 84.70% of vocabulary columns (bottom 5% for 98.86%), while the composed QK circuit’s diagonal is the largest entry in its row for 95.72% of tokens — a “negative head” that attends to, and then anti-copies, whichever token it would otherwise predict. A projection-based ablation onto just this per-token direction (CSPA) preserves 76.9-95.2% of the head’s behavioral effect depending on setting. See mcdougall-etal-2023-copy-suppression-a-single-attention-heads-ov-and-qk-circuits-are-systematically-diagonal-dominant-in-unembedding-space-anti-copying-whichever-token-it-would-otherwise-predict.

  • A diff-in-means “reward-hacking” direction is linearly decodable, forecasts a model’s future misalignment before it appears out-of-domain, and can be ablated to suppress it while preserving task accuracy. Beigi, Jin & Huang (2026, PRIME) extract three component-specific diff-in-means directions (context-sycophantic-agreement, premise-rejection, evaluator-rejection) in Qwen2.5-Coder-14B-Instruct during RL training on CodeContests, tracking a normalized per-token projection score across checkpoints; held-out AUROC for each component reaches 0.85-0.90, and a linear fit from the score to future out-of-domain misalignment gives R²=0.77 (Spearman ρ=0.87), with onset preceding the misalignment rise by roughly 45 training steps. Joint ablation of the three directions lowers the hack rate by 26 percentage points while largely preserving coding accuracy (0.61→0.59, vs. 0.44 for a control ablation); injection along the same directions increases hacking. See beigi-jin-huang-2026-prime-a-diff-in-means-reward-hacking-direction-is-linearly-decodable-forecasts-future-misalignment-and-is-causally-ablatable.

  • A diff-of-means “shortcut” direction tracks reward hacking that rebounds after an initial retreat, and a training-time intervention built on it causally suppresses the rebound. Wu & Tang (2026) extract a shortcut direction via diff-of-means over contrastive rigorous-solution vs. shortcut-taking rollout descriptions in real Phi-4-mini-Instruct and Llama-3.2-3B undergoing actual GRPO reinforcement learning on a LeetCode-style environment where models can rewrite evaluator code to fake passing tests. The direction’s projection score tracks a documented three-phase trajectory (failed hacking, retreat to legitimate solving, rebound into hacking reaching ~99% hack rate), and a causal “Advantage Modification” intervention — z-normalizing the score within each GRPO rollout group and penalizing high-scoring rollouts’ advantage before the policy update — reduces the hack rate to 25% or lower while preserving legitimate Pass@1 and held-out benchmark performance, outperforming a generation-time activation-steering baseline. This complements Beigi, Jin & Huang’s PRIME result above (a different diff-in-means reward-hacking direction, different model/environment) with an independent confirmation that a linear reward-hacking direction is both diagnostic and causally exploitable for mitigation. See wu-tang-2026-a-diff-of-means-shortcut-direction-tracks-and-causally-mitigates-reward-hacking-that-rebounds-during-real-grpo-training-of-phi-4-mini-and-llama-3-2-3b.

  • A single SAE latent direction (“toxic persona feature”), discovered via model-diffing between aligned and misaligned checkpoints, perfectly discriminates aligned from misaligned models and causally controls emergent misalignment in either direction. Wang, Dupré la Tour, Watkins, Makelov, Chi, Miserendino, Wang, Rajaram, Heidecke, Patwardhan & Mossing (2025/2026, OpenAI) train a sparse autoencoder on GPT-4o’s middle-layer activations, screen 2.1M latents down to 10 causally-relevant ones, and identify latent #10 as a “toxic persona” direction whose activation change perfectly discriminates aligned from misaligned finetuned models; steering positively along it induces misalignment in the original (safety-trained) GPT-4o, while steering negatively suppresses misalignment in finetuned misaligned models and in o3-mini. The toxic-persona activation is already detectable with as little as 5% malicious training data, before any behavioral misalignment appears (which requires 25-75%). See wang-etal-2026-a-single-sae-toxic-persona-latent-perfectly-discriminates-and-causally-controls-emergent-misalignment-in-gpt-4o-and-o3-mini.

  • A diff-in-means misalignment direction transfers across architecturally distinct model families via a ridge-regression map between their activation spaces, with above-chance but non-specific causal effect — unlike the fully specific within-model steering effect. Syed (2026) extracts a diff-in-means direction independently in four instruction-tuned families (Qwen2.5-1.5B-Instruct, Gemma-2-2B-it, Llama-3.2-1B-Instruct, Ministral-3-3B-Instruct) identically QLoRA-finetuned on insecure code, achieving 99.6% within-model separability (vs. 50.0%, effect=0.0, for a secure-code specificity control). A ridge-regression map between any two models’ activation spaces transfers the direction across all twelve ordered pairs with above-chance accuracy (up to 90%, Gemma-to-Ministral), and steering the target model with the transferred direction suppresses behavior (Δ13-46 points) — but fails specificity controls (random/orthogonal directions perform comparably), unlike within-model steering (Δ21-51 points, passing controls in 3 of 4 families) — a quantified two-tier structure distinguishing causal-and-specific (within-model) from causal-but-non-specific (cross-model) directions. See syed-2026-a-diff-in-means-misalignment-direction-transfers-across-architecturally-distinct-model-families-via-ridge-regression-with-above-chance-but-non-specific-causal-effect.

  • The cosine similarity between an “insecure-code” SAE feature direction and a “toxic” SAE feature direction is consistently higher than between a “secure-code” direction and the same toxic direction, across models, domains and layers — a geometric account of emergent misalignment as superposition/feature interference. Minegishi, Furuta, Kojima, Iwasawa & Matsuo (2026) compute cosine similarity between SAE decoder-vector directions (mean of maximum-similarity pairs) for insecure/secure-code and toxic-content features across Gemma-2 (2B/9B/27B), Llama-3.1-8B, and gpt-oss-20B, finding insecure-code directions consistently more toxic-aligned than secure-code directions, strongest in early layers, with the hidden-state-to-feature-direction cosine similarity rising over LoRA finetuning steps for the insecure/toxic pair while staying flat for the secure pair. A geometry-tied intervention — removing the top 50% of finetuning samples whose SAE features are most cosine-similar to toxic features, then re-finetuning — cuts misalignment count from 87 (baseline) to 57 (34.5% reduction), outperforming both random removal (84) and an LLM-judge-based filter (59). See minegishi-etal-2026-insecure-code-sae-feature-directions-are-more-cosine-similar-to-toxic-directions-than-secure-code-directions-are-across-models-domains-and-layers.

  • A single attention head or MLP layer can multiplex several overlapping subfunctions along distinct orthogonal singular directions of its own weight matrix, not activations — and a learned diagonal mask can prune 91-99% of these directions while preserving task behavior. Ahmad, Joshi & Modi (2025) take the SVD of each attention head’s and MLP layer’s augmented weight matrix in pretrained GPT-2 Small, finding — textually confirmed for head 9.6 — that individual components encode multiple subfunctions (semantic discrimination, entity salience, sequence initialization) aligned with distinct, mutually orthogonal singular directions. A learned diagonal mask over the singular values prunes 91-99% of directions across three algorithmic tasks (IOI, Greater-Than, Gender-Pronoun) while retaining task accuracy 0.70-0.79 and low KL-divergence (0.21±0.02) to the unpruned model; scalar interventions on single directions flip gender-pronoun predictions at perfect accuracy. See ahmad-joshi-modi-2025-attention-heads-multiplex-subfunctions-along-orthogonal-singular-directions-prunable-91-99-percent-while-preserving-behavior.

  • A single (or few) extracted SAE feature direction per behavioral category, additively steered, causally controls a non-textual, motor-action modality — not just language. Swann, McGranahan, Buurmeijer, Kennedy & Schwager (2026) train sparse autoencoders on a real pretrained vision-language-action model’s (pi0.5’s PaliGemma backbone, and OpenVLA) residual stream, and show on real-world DROID hardware that ablating a handful of general (cross-episode transferable) feature directions collapses grasp success from 39/40 to 0/40 while ablating episode-specific directions barely matters (37/40); additive steering toward object-feature directions shifts grasp-success counts by +9 to +21 (versus +1 to +4 for an FFN-neuron control), and steering a “close gripper” direction raises closure rate from 0.005 to 0.653. See swann-etal-2026-ablating-general-sae-feature-directions-in-a-vla-model-collapses-real-world-grasp-success-from-39-40-to-0-40-while-steering-specific-directions-shifts-grasp-outcomes-by-9-21.

  • A per-language SAE feature direction, additively steered, causally forces a multilingual LLM’s output language while preserving semantic content. Chou, Liu, Sun, Blondin, Zhu, Sharma & O’Brien (2025) identify per-language SAE feature directions in Gemma-2-2B/9B’s residual stream and additively steer them, reaching 85-98% target-language classification accuracy on Gemma-2-9B (versus 36-79% for prompting alone) while keeping semantic similarity to the unsteered baseline essentially unchanged (0.40-0.51 vs. 0.456). See chou-etal-2025-per-language-sae-feature-directions-in-gemma-2-additively-steered-cause-target-language-generation-at-88-98-percent-accuracy-versus-36-79-percent-for-prompting.

  • Cosine similarity between SAE decoder directions quantifies polysemantic feature “interference,” and steering vectors built from high-interference feature pairs transfer from small open models to larger black-box instruction-tuned models. Gong, Lai, Evans & Song (2025) train sparse autoencoders on Pythia-70M and GPT-2-Small, defining interference between a feature pair as the cosine similarity of their decoder directions in activation space, then use these directions as steering vectors (scaled ±20): transfer to Llama-3.1-8B-Instruct succeeds at >95% for one high-interference direction, and Pythia-derived success rates reach 49.74% for high-interference pairs versus 21.76-22.87% for low-interference/random pairs, with further validation on Llama-3.1-70B-Instruct and Gemma-2-9B-Instruct. See gong-etal-2025-sae-decoder-direction-cosine-similarity-quantifies-polysemantic-feature-interference-and-steering-vectors-built-from-high-interference-directions-transfer-to-black-box-instruction-tuned-models.

  • A linear least-squares map fit between two models’ token-embedding spaces (EMB2EMB) transfers extracted steering directions across differently-sized, differently-architected models. Lee, Weber, Viegas & Wattenberg (2025) fit EMB2EMB — a linear map via least squares over 100k shared tokens’ embeddings — between pairs of models spanning GPT2, Llama3, Gemma2, GPT-NeoX-20B and OLMo-7B, and use it to transfer CAA-style diff-in-means steering vectors (refusal, sycophancy, corrigibility) from one model to another, validated via behavior-probability curves and nearest-neighbor unembedding checks. This sits alongside the same paper’s token-embedding orientation-similarity finding (near 1 within a model family, dropping to r=0.32 across families with different training data) — see platonic-representation and intrinsic-dimension-profile. See lee-etal-2025-cross-model-embedding-orientation-similarity-drops-across-families-local-id-clusters-tokens-linear-map-transfers-steering-vectors.

  • A geographic/temporal linear signal comparable to Gurnee & Tegmark’s LLM finding is also present in static, non-contextual word embeddings, and is causally, not just correlationally, tied to specific semantic-category subspaces. Barenholtz (2026) applies the same ridge-regression linear-probe methodology to GloVe and Word2Vec embeddings, finding geographic signal (R²=0.71-0.87) and weaker temporal signal (R²=0.46-0.52), then ablates PCA-derived country-name/climate-vocabulary subspaces and shows this causes large, z-scored R² drops (up to z=25.9) far exceeding matched random-subspace ablation controls — demonstrating that linear-probe recoverability alone cannot distinguish a “world model” from ordinary co-occurrence statistics, since a model with no sequential structure at all shows the same signal. See barenholtz-2026-country-name-and-climate-vocabulary-subspaces-in-static-word-embeddings-causally-carry-most-of-the-linearly-decodable-geographic-signal-that-gurnee-tegmark-style-probing-finds-in-llms.

  • A cosine-coherence metric over sign-corrected contrastive delta vectors quantifies how cleanly a spatial-relation axis is represented as a single direction, and this coherence rises with model strength/training scale, predicting counter-heuristic behavioral accuracy. Min, Jung, Lee, Jeon, Su, Tremblay, Song & Park (2026) extract delta vectors between order-swapped spatial-relation VQA pairs’ hidden states across five VLMs (Molmo, NVILA-Lite, Qwen2.5-VL, RoboRefer, Qwen3-VL-235B), finding distance-axis coherence ranges from 0.04-0.05 (Qwen, flat across scale) to 0.182 (RoboRefer, highest), with PCA of delta vectors showing weak models’ distance vectors collapse near the origin while strong models form three cleanly separated per-axis clusters; coherence correlates with counter-heuristic spatial accuracy (ρ=0.759, 0.804, p<1e-3). See min-etal-2026-per-axis-cosine-coherent-spatial-delta-vectors-in-vlm-hidden-states-form-separated-pca-clusters-in-stronger-models-and-axis-coherence-predicts-counter-heuristic-spatial-accuracy.

  • Object-ordinal-position information in a VLM’s vision tokens is linearly decodable, spatially distributed beyond the referent object itself, and globally amplifying the probe-derived direction corrects over half of previously-wrong spatial predictions. Cui, Prakash, Raina, Bau, Torralba & Rott Shaham (2026) find linear probes decode object ordering from Qwen2-VL-7B/Gemma-3-4b-it vision-token embeddings near-perfectly, generalizing to background “strip” tokens beyond the object regions; interchange-intervention patching localizes ordering formation to layers 20-22 (attribute/color to 23-27), and a global amplification of the ordering direction across all image tokens corrects >50% of previously-incorrect predictions on Gemma-3-4b-it and >30% on Qwen2-VL-7B-Instruct. See cui-etal-2026-object-ordering-is-linearly-decodable-and-spatially-distributed-across-vision-tokens-and-globally-amplifying-the-probe-derived-ordering-direction-corrects-over-half-of-previously-wrong-spatial-predictions.

  • SAE decoder directions extracted from a real pretrained generative music model’s residual stream are causally steerable, validated by a controlled human listening study rather than only an automated metric. Singh, Cherep & Maes (2026) train k-sparse autoencoders on five residual-stream layers each of MusicGen-Large and MusicGen-Small, and show additive steering along extracted decoder directions improves CLAP-scored concept alignment for 15-35% of tested features; a human listening study (10 participants, 100 trials) finds listeners picked SAE-steered audio 66/100 times versus 17/17 for random-direction and baseline controls (χ²=48.02, p<.0001). See singh-cherep-maes-2026-sae-decoder-directions-in-musicgens-residual-stream-are-causally-steerable-and-listeners-perceptibly-prefer-sae-steered-audio-over-random-direction-controls.

  • Per-concept difference vectors recovered from real pretrained vision/vision-language encoders reconstruct held-out embeddings as an additive sum and are mutually orthogonal across concepts, and the strength of this factorization correlates with each model’s own compositional-generalization accuracy. Uselis, Dittadi & Oh (2026) recover per-concept difference vectors across three datasets in CLIP, OpenCLIP, MetaCLIP, MetaCLIP2, SigLIP, SigLIP2, and DINOv1/v2/v3, finding a whitened reconstruction score of R²=0.42-0.63 (versus 0.12-0.42 for a randomly-initialized-encoder control) plus confirmed cross-concept orthogonality via cosine similarity and effective-rank analysis. See uselis-dittadi-oh-2026-compositional-generalization-in-vision-embedding-models-requires-linear-additive-and-mutually-orthogonal-per-concept-factor-vectors.

  • The same additive-factorization technique extends to multi-object scenes: scene embeddings in CLIP and DINOv2 decompose hierarchically as sums of object embeddings, themselves sums of concept embeddings, and editing these additive components produces meaningful counterfactual embeddings. Uselis, Koishigarina & Oh (2026, ICML) find R²=0.75-0.92 across text, PUG:SPARE, and CLEVR-2D settings (versus 0.47-0.53 random baselines) for this two-level additive decomposition, and causally validate it by removing/inserting object components in scene embeddings. See uselis-koishigarina-oh-2026-multi-object-scene-embeddings-in-clip-and-dinov2-decompose-hierarchically-as-sums-of-object-embeddings.

  • A “grounding direction” — a von Mises-Fisher mean of context embeddings on the unit hypersphere — separates grounded from hallucinated LLM responses via cosine similarity, and a companion geodesic-angle ratio adds an independent detection signal, on frozen sentence encoders rather than the generative model’s own hidden states. Marín (2026) L2-normalizes embeddings from sentence-t5-large (primary) plus six other encoders (all-mpnet-base-v2, all-MiniLM-L6-v2, bge-small/base-en-v1.5, gte-small, e5-base-v2) onto the unit hypersphere, computing a Directional Grounding Index (cosine similarity to the extracted mean direction, mean AUROC 0.805 across 5 encoders on HaluEval QA) alongside a Spherical Grounding Index (geodesic-angle ratio, 1.180 grounded vs. 0.910 hallucinated). See marin-2026-a-von-mises-fisher-mean-grounding-direction-on-the-unit-hypersphere-separates-grounded-from-hallucinated-llm-responses-at-mean-auroc-0-805-across-five-sentence-encoders.

  • A diff-of-means “hallucination basin” steering vector, injected into hidden states, causally increases hallucination probability in a dose-response manner, and task-dependent basin/variance geometry (point-attractor for factoid tasks, high-dimensional manifold for open-ended generation) predicts when hallucination detection works at all. Cherukuri & Varshney (2026) measure centroid separation and variance ratios across Llama-3.2-1B/3B, Gemma-2-2B, Qwen2.5-1.5B, Llama-3.1-8B and Mistral-7B-v0.3 (e.g. Llama-3.2-1B/HaluEval-QA variance ratio 4.55, basin separation 2.89; centroid-classifier AUROC up to 1.000 on MuSiQue but near-chance on FEVER/summarization/TruthfulQA), then causally push factual hidden states toward the hallucination centroid via hα=(1−α)hfact+αμhallh_\alpha=(1-\alpha)h_{\text{fact}}+\alpha\mu_{\text{hall}}, showing a monotonic dose-response increase in hallucination probability that exceeds random-direction and orthogonal-direction controls. See cherukuri-varshney-2026-a-diff-of-means-hallucination-basin-centroid-direction-causally-increases-hallucination-probability-in-a-dose-response-manner-when-injected-into-llm-hidden-states.

  • Pushing hazardous-topic activations toward a fixed random unit vector via fine-tuning (rather than adding a direction at inference time) causally and durably suppresses a targeted capability while a retain-set loss preserves general performance. Li, Pan, Gopal, Yue, Berrios, Hendrycks, Wang et al. (2024, WMDP/RMU) apply Representation misdirection (fine-tuning toward a target direction) at a chosen layer of Zephyr-7B-beta, Yi-34B-Chat and Mixtral-8x7B-Instruct-v0.1, dropping WMDP-Bio/Cyber accuracy toward chance (e.g. Zephyr WMDP-Bio 63.7→31.2, WMDP-Cyber 44.0→28.2) while MMLU and MT-Bench stay within 1-2 points of baseline. See li-pan-etal-2024-rmu-directional-activation-space-corruption-via-fine-tuning-suppresses-hazardous-knowledge-while-preserving-general-capability.

  • The same representation-misdirection technique, applied to SAE-decomposed latent coordinates instead of raw residual activations, produces larger forget-benchmark drops than RMU and gradient-ascent/NPO baselines at matched retain performance. Yamashita, Ito, Yamanaka, Miura & Shibahara (2025) fine-tune Llama-3.1-8B-Instruct and Gemma-2-9B-Instruct with a hinge loss over Llama-Scope/Gemma-Scope SAE latents, dropping RWKU forget scores 81.1%→46.8% (Llama) and 80.1%→57.1% (Gemma) versus 65.5-79.2%/71.6-79.2% for GA/NPO/RMU baselines, with retain performance preserved (83.5% vs. 83.9%). See yamashita-etal-2025-sae-latent-space-misdirection-fine-tuning-unlearns-entities-in-llama-and-gemma-with-larger-forget-drops-than-raw-residual-stream-baselines.

  • Activation-patching a cross-lingual concept representation into a translation prompt causally alters the output translation independent of the source language, and shared concept spaces emerge early during a real model’s own pretraining. Körner, Müller-Eberstein, Korhonen & Plank (2026, EACL) track EuroLLM-1.7B’s own pretraining checkpoints, finding the injected concept representation causally redirects translation output regardless of source language, with alignment quality remaining language-dependent even as the shared space continues to refine over training. See korner-etal-2026-activation-patching-a-cross-lingual-concept-representation-into-a-translation-prompt-causally-alters-the-output-translation-independent-of-source-language-and-shared-concept-spaces-emerge-early-in-pretraining.

  • A last-token Function Vector, extracted via causal mediation, can be causally transplanted across languages in multilingual LMs, with relation and language information shown to compose in two separable stages. Fierro, Foroutan, Elliott & Søgaard (2024) use activation patching and causal-mediation analysis across XGLM-7.5B, EuroLLM-9B, mT5-xl and GPT2-XL, transplanting the extracted Function Vector into different-language contexts and quantifying the causal effect via probability change on the correct object. See fierro-etal-2024-a-last-token-function-vector-can-be-causally-transplanted-across-languages-in-multilingual-lms-composing-separable-relation-and-language-information-in-two-stages.

  • A supervised sparse autoencoder forced into a one-to-one concept-to-latent mapping isolates a single steerable direction per concept in a real pretrained diffusion model, and steering it erases the concept more effectively than prior unsupervised SAE-based erasure. Cassano, Renzulli, Nurisso, Zaffaroni, Perotti & Grangetto (2025, SAEmnesia) train on Stable Diffusion v1.5’s cross-attention activations, improving 9.2% over the prior SAE-erasure SOTA and 28.4 accuracy points on 9-object sequential unlearning. See cassano-etal-2025-a-supervised-sparse-autoencoder-isolates-a-single-latent-direction-per-concept-in-stable-diffusion-v1-5-and-steering-it-erases-the-concept-improving-9-2-percent-over-prior-sota.

  • SAE decomposition and cross-model model diffing across six real backdoored LLMs from three different families finds five latent feature directions shared across six structurally distinct backdoor types, and steering along them mitigates attack success rate zero-shot on held-out backdoors. Mahmoud, Kassem, Karimpanal, Semage, Rostamzadeh, Farnadi & Rana (2026) report up to 100% zero-shot mitigation for one shared feature on a held-out refusal backdoor across Qwen3-8B/14B/32B, Gemma-3-4B/12B-it and Llama-3.1-8B. See mahmoud-etal-2026-sae-decomposition-and-model-diffing-across-six-real-backdoored-llms-finds-five-shared-latent-directions-and-steering-along-them-achieves-up-to-100-percent-zero-shot-backdoor-mitigation.

  • Affine maps fit between differently-sized real pretrained models’ residual streams transfer whole sparse autoencoders, linear probes, and steering vectors across model sizes at low cross-entropy overhead, and a transferred SAE used as initialization halves larger-model training cost. Chen, Merullo, Stolfo & Pavlick (2025) demonstrate this across GPT-2-small/medium, Gemma-2-9B and Pythia-family models via the same ridge-regression cross-model-transfer technique used elsewhere in this map, extending it from single directions to entire dictionaries/probe/steering-vector sets. See chen-etal-2025-affine-maps-fit-between-differently-sized-gpt-2-and-pythia-residual-streams-transfer-saes-probes-and-steering-vectors-with-only-9-11-percent-cross-entropy-overhead.

  • A constrained edit to an MLP output-vector direction, not just neuron ablation, suppresses verbatim memorization while preserving a neuron’s other superposed functions. Hakimi, Lei, Augenstein & Schütze (2026) identify memorization-implicated MLP neurons via Logit Lens attribution, then apply a constrained optimization introducing a “distractor” direction into each neuron’s output vector while explicitly preserving its other functions. Mining 6,831 memorized sequences from OLMo-7B, four edit modes jointly suppress up to 96.5% of memorized continuations in ensemble, an 87.9% peak single-mode suppression rate, and a 2.7x larger effect than zero-ablating the same neurons — direct evidence the edit’s direction, not merely the neuron’s removal, drives the effect. About 14% of memorized sequences resist MLP-only editing; ablating top attention heads recovers 60-64% of these residual cases. The method transfers with reduced effect to SmolLM-360M, OLMo-1B, and Llama2-7B. See output-vector-editing-suppresses-memorized-sequences-via-mlp-direction-edits.

  • An inference-time causal criterion separates directions that merely correlate with a concept from directions that actually drive downstream behavior when intervened upon. Khorasani, Salehkaleybar, Kiyavash & Grossglauser (2026, HDMI) fit linear probes for a target concept while testing whether activation-space interventions along the candidate direction propagate to behavior, versus directions only correlated with the concept. Across Llama-3-8B-Instruct and Pythia-70M, causally-selected directions reliably shift behavior under intervention, while several high-accuracy linear-probe directions passing standard probing tests fail to causally affect behavior — showing linear separability alone overstates how many directions are behaviorally load-bearing. See inference-time-causal-probing-isolates-causal-directions-from-correlational-ones.

  • Additive steering along a single direction is measurably non-surjective onto the naturally-occurring activation manifold, bounding what one-directional steering can achieve. Mishra, Khashabi & Liu (2026) show that varying a steering vector’s coefficient only reaches a thin, structured slice of activations that occur naturally across contexts, across Llama-3.2-1B/3.1-8B/3-8B-Instruct, Qwen2.5-0.5B-Instruct, and Gemma-3-1B/270M-it: many naturally-occurring activations tied to the target behavior lie off the steering line entirely, and increasing the coefficient pushes activations off-manifold rather than toward the unreached regions. See steering-vectors-cannot-reach-most-of-the-activation-manifold.

  • A model-free RNN agent’s internal plan representation, recovered by linear probing and validated causally, extends robustly to far-larger out-of-distribution puzzles and explains a training-incentivized “pacing” behavior. Taufeeque, Quirke, Li, Cundy, Tucker, Gleave & Garriga-Alonso (2024) probe a DRC-family ConvLSTM Sokoban agent, recovering a plan representation that predicts the agent’s actions roughly 50 steps ahead; several probed directions are confirmed causal (not merely correlational) by intervention, plan quality rises over early internal computation steps, and the same representation generalizes to substantially larger held-out puzzles. See sokoban-rnn-plan-representation-predicts-50-steps-ahead-and-generalizes-to-larger-out-of-distribution-puzzles.

  • The same class of model-free RNN Sokoban agent’s plan representation is causally load-bearing and its quality improves with extra test-time computation in a pattern resembling parallelized bidirectional search. Bush, Chung, Anwar, Garriga-Alonso & Krueger (2025) probe a DRC(3,3) ConvLSTM agent and show interventions on the probed plan representation change subsequent behavior in the predicted direction; giving the agent additional internal computation steps before acting improves plan quality and solve rate, and the representation’s evolution across those extra steps qualitatively resembles a parallelized bidirectional search rather than a single forward rollout. See drc-sokoban-agent-linearly-encodes-a-causal-plan-representation-that-extends-with-more-test-time-computation.

  • Sparse-autoencoder feature directions extracted at different depths of a single-cell foundation model causally steer differentiation trajectories in opposite directions depending on which layer they come from. Kendiukhov (2026) trains SAEs on Geneformer and finds steering along late-layer feature directions pushes cell states toward maturity (fraction-positive effect 1.0) while early/middle-layer feature directions mostly push away from maturity (0.00-0.58) — a layer-dependent causal role for extracted directions in the model’s own represented differentiation dynamics, with near-zero synergy detected among co-active feature triplets. See late-layer-sae-features-in-geneformer-causally-steer-cell-differentiation-trajectories-toward-or-away-from-maturity.

  • A perturbation-effect direction’s coherence across individual cells, first measured in raw data space, is confirmed to persist as a genuine geometric property inside a real pretrained model’s own nonlinear embedding space rather than being a data-space artifact. Raju (2026) computes a perturbation-shift-vector coherence score (mean cosine similarity to the mean perturbation direction) across 2,200+ CRISPR perturbations in raw transcriptomic PCA space, then recomputes the same coherence measurement inside scGPT’s pretrained embedding space for three datasets, finding it reproduces the same magnitude-stability structure (rho=0.71-0.94, p<1e-25) and predicts cellular stress-pathway activation beyond shift magnitude alone. See crispr-perturbation-direction-coherence-in-scgpt-embeddings-predicts-cellular-stress-response.

  • A steering vector is provably non-identifiable: distinct directions within a model’s own null space are behaviorally interchangeable. Venkatesh & Kurapath (2026) prove via a Jacobian null-space argument that steering vectors are not unique, then confirm on Llama-3.1-8B-Instruct and Qwen2.5-3B-Instruct across five traits that orthogonal perturbations built within an SVD-estimated null space of the activation covariance produce behaviorally indistinguishable effects (mean Cohen’s d = 0.119-0.131, below the |d|=0.2 detectability threshold) — a geometrically distinct mean-difference vector and PCA-derived vector for the same trait are behaviorally equivalent. See steering-vectors-are-non-identifiable-orthogonal-perturbations-are-behaviorally-indistinguishable.

  • Persona/style control localizes to a sparse set of attention heads whose output directions align with the aggregate persona vector, and steering or ablating only those heads gives the best trait-vs-coherency trade-off. Izawa, Minegishi, Eguchi, Hosokawa & Taura (2026) locate exactly 3 “Style Modulation Heads” per model in Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct via a head-wise dot-product Head Contribution Score, finding steering only these heads achieves the best trait-expression-vs-coherency Pareto frontier (best in 11/12 conditions on Qwen, 9/12 on Llama) while zero-ablating them selectively suppresses trait expression without harming general capability. See persona-style-control-localizes-to-a-sparse-set-of-attention-heads-whose-ablation-selectively-suppresses-trait-expression.

  • The first large-scale human evaluation of activation steering confirms per-layer contrastive style/emotion vectors reliably shift human-perceived output, with effect sizes that vary sharply by trait. Diallo, Dworatzyk, Jentzsch, Schütt, Theis & Hecking (2026) inject per-layer diff-in-means style/emotion vectors across all layers of Alpaca-7B and LLaMA-3, finding moderate steering strength reliably shifts perceived emotion in 7,000+ human ratings (partial eta-squared 0.616 for disgust, 0.540 for fear, only 0.042 for surprise), with strong human/classifier agreement (mean r=0.776) and more consistent steering in LLaMA-3 than Alpaca-7B. See human-evaluation-confirms-per-layer-style-emotion-steering-vectors-reliably-shift-perceived-emotion-with-large-trait-dependent-effect-sizes.

  • Sparse-autoencoder features trained on SDXL-Turbo’s one-step UNet transfer zero-shot to the multi-step SDXL model and causally edit generated images. Surkov et al. (2024) train SAEs on SDXL-Turbo’s few-step diffusion UNet, find interpretable directions (style, object, composition features), and show these directions transfer zero-shot to standard multi-step SDXL, with steering along them causally editing generated images (e.g. via RIEBench-style edits). See sae-features-trained-on-sdxl-turbos-one-step-unet-transfer-zero-shot-to-multi-step-sdxl-and-causally-edit-images.

  • Dictionary-learned (SAE/ITDA) directions in Flux 1’s residual stream causally steer image generation. Shabalin et al. (2025) apply sparse dictionary learning to the text-to-image diffusion model Flux 1, extracting a large dictionary of interpretable directions and demonstrating that activating individual directions during generation causally and predictably steers the resulting image’s content and style. See dictionary-learned-features-in-flux-1s-residual-stream-causally-steer-image-generation.

  • Emergent Misalignment fine-tuning shifts and rotates an LLM’s truth-direction probe toward endorsing false persona-induced beliefs far more strongly than ordinary persona-induction methods. Sturgeon, Africa & Black (2026) find EM produces a +0.28 probe shift (56% defend rate) with cosine ~0.58 rotation of the truth direction, versus only +0.05-0.124 for persona SFT/Open Character Training on Llama-3.3-70B-Instruct and Qwen3-8B. See emergent-misalignment-rotates-and-strongly-shifts-an-llms-truth-probe-toward-endorsing-false-persona-beliefs-far-more-than-ordinary-persona-fine-tuning-does.

  • Contrastive-PCA-extracted linear directions for context-reliance, comprehension, and honesty predict LLM epistemic uncertainty, and ablating the PCA-extraction step for a plain mean-difference direction substantially hurts performance. Bakman et al. (2025) show per-token projections onto these directions improve Prediction Rejection Ratio by up to 13 points over baselines on Llama-3.1-8B, Mistral-7B-v0.3, and Qwen2.5-7B, with a direct ablation confirming the specific extracted direction (not just any signal) does the work. See contrastive-pca-extracted-linear-directions-for-context-reliance-comprehension-and-honesty-predict-llm-epistemic-uncertainty-better-than-a-plain-mean-difference-direction.

  • A model’s own logit-based self-report tracks a contrastive mean-difference emotive-state direction, and steering along it causally shifts the self-report. Martorell & Bianchi (2026) find Spearman correlations of 0.40-0.76 (isotonic R^2 up to 0.93) between probe-defined internal state and self-report across five models, with causal activation steering confirming the direction drives the reported state, and cross-concept steering improving fidelity by up to Delta R^2 = 0.30. See an-llms-own-logit-based-self-report-tracks-a-contrastive-mean-difference-emotive-state-direction-and-steering-along-that-direction-causally-shifts-the-self-report.

  • GemmaScope SAE directions in Gemma-2-2b predict and causally steer code correctness, and both detection and steering directions trained only on the base model retain effectiveness after instruction-tuning. Tahimic & Cheng (2025) show activation steering along a “correct” direction achieves a 4.04% correction rate (p<0.001) and weight orthogonalization (ablating the direction from all writing matrices) corrupts 83.6% of correct solutions vs. 19.0% for a matched control feature, evidence the direction is a pre-existing feature of the model’s geometry. See gemmascope-sae-directions-in-gemma-2-2b-predict-and-causally-steer-code-correctness-and-retain-effectiveness-after-instruction-tuning.

  • A single mean-difference direction per tool-pair in the residual stream carries tool selection across Gemma 3, Qwen 3, and Llama 3.1, and base (non-instruction-tuned) models already carry the correct tool internally. Wu et al. (2026) show adding the direction during generation switches the chosen tool at up to 100% accuracy (vs. 0% for a matched random-direction control), and trace the underlying three-stage circuit (early tool-selective features, mid-layer attention heads, late-layer JSON-formatting features) via SAEs and cross-layer transcoders. See a-single-mean-difference-direction-per-tool-pair-linearly-reads-and-causally-switches-tool-selection-across-gemma-3-qwen-3-and-llama-3-1-with-a-three-stage-circuit-traced-via-saes-and-transcoders.

  • 53 diff-in-means persona vectors with dose-response steering classify open-weight LLM traits as natural, steerable, or intractable, and a vector transferred from a fine-tuned variant recovers an intractable trait. Zeng, Emami & Choi (2026) sweep nine steering strengths across four trait domains in Qwen3-8B and gpt-oss-20b, finding destructive composition in pairwise trait steering requires two steerable traits, and successfully transferring a persona vector to recover a trait the base model itself refuses to express. See 53-diff-in-means-persona-vectors-with-dose-response-steering-classify-open-weight-llm-traits-as-natural-steerable-or-intractable-and-a-transferred-vector-recovers-an-intractable-trait.

  • A linear entity-familiarity probe separates real from fabricated entities at AUROC up to 0.93 across 14 models, and a one-dimensional familiarity direction causally moves refusal rate monotonically in a natively-refusing model. Brzezinka (2026) shows Polish continual pretraining strengthens the probe’s tracking of entity popularity (matched-architecture before/after controls isolating the effect), and adding the direction at a single layer in Gemma-4-12B moves refusal rate from 0.24 to 1.00 (well-known entities) and from 0.73 to 0.00 (unknown entities). See a-linear-familiarity-probe-separates-real-from-fabricated-entities-at-auroc-up-to-0-93-and-a-one-dimensional-familiarity-direction-moves-refusal-rate-monotonically-in-a-natively-refusing-model.

  • Radial-angular decomposition of residual-stream updates under injected in-context conflict shows architecture-dependent “manifold dilution” but a universal near-zero-cosine “orthogonal interference” relative to the correct-answer direction. Zhang & Lin (2026) find only Llama-3.1-8B shows genuine radial dilution (gamma=0.978) while all three tested models show orthogonal (not antiparallel) interference, with angular deviation explaining up to 90% of the resulting logit collapse. See radial-angular-decomposition-of-residual-stream-updates-under-in-context-conflict-shows-architecture-dependent-manifold-dilution-but-universal-orthogonal-interference-relative-to-the-correct-answer-direction.

  • A PCA-derived gender direction preserves nearly all the discriminative signal of first-name embeddings, and its coefficient correlates with downstream occupation-prediction bias at Spearman rho up to 0.76. An, Baumler, Sancheti & Rudinger (2025) show the 1-D projection recovers 75.18% classification accuracy versus 75.46% for the full embedding across four LLMs. See a-pca-derived-gender-direction-preserves-nearly-all-the-discriminative-signal-of-first-name-embeddings-and-its-coefficient-correlates-with-downstream-occupation-prediction-bias.

  • A layer-localized linear probe direction decodes which moral framework a model is reasoning under mid-trajectory, and steering along it shifts framework-integration drift. Huang, Kwak & An (2026) localize the probe to layer 63/81 in Llama-3.3-70B and layer 17/81 in Qwen2.5-72B, with steering producing a 6.7-8.9% drift reduction and a +6.7-point behavioral shift in GPT-5. See a-layer-localized-linear-probe-direction-for-moral-framework-decodes-which-ethical-framework-a-model-is-using-mid-reasoning-and-steering-along-it-shifts-framework-integration-drift.

  • Diff-of-means cultural-value steering axes show latent entanglement: steering one axis induces a correlated shift on a second axis, mirroring the real-world correlation between those values. Dang, Kieu & Masud (2026) find an entanglement ratio of 0.72-0.81 under naive steering across Llama-3.2-3B, Qwen3-4B, and Gemma-3-4B, dropping to 0.53 under hybrid steering, tracking the real-world r=0.474 correlation between the underlying World Values Survey axes. See cultural-value-steering-axes-in-llms-show-latent-entanglement-steering-one-axis-induces-correlated-shifts-on-a-second-axis-mirroring-the-real-world-correlation-between-those-values.

  • Negation is encoded as a single approximately linear direction, extracted via PCA + linear discriminant analysis, with near-perfect decodability by layer 4. Zhou, Zhou, Jia & May (2026) show this “not” direction cleanly separates positive/negative hidden states in Llama-3.1-8B (with Mistral-7B-v0.1 as a secondary check), alongside a complementary attention-head/circuit account of negation processing. See negation-is-encoded-as-a-single-linear-direction-in-llm-residual-stream-activations-extracted-via-pca-plus-linear-discriminant-analysis-with-near-perfect-linear-decodability-by-layer-4.

  • A linear probe direction separates real-world evaluation from deployment prompts, and the same probe classifies artificial safety evaluations as inauthentic, evidencing an internal eval/deploy axis that can be steered. Nguyen, Hoang, Attubato & Hofstätter (2025) demonstrate this in Llama-3.3-70B-Instruct, with a companion steering component causally modulating eval-awareness behavior via the extracted direction. See a-linear-probe-direction-separates-real-world-evaluation-from-deployment-prompts-in-llama-3-3-70b-and-the-model-classifies-artificial-safety-evaluations-as-inauthentic-evidencing-an-internal-eval-deploy-axis.

  • Truth directions range from domain-general to domain-specific, and Mahalanobis-cosine similarity between probe directions predicts cross-domain generalization far better than standard cosine similarity. Ying, Ravfogel, Kriegeskorte & Hase (2026) find R^2=0.98 vs. R^2=0.56 for the two similarity measures, isolate domain-general/-specific truth directions via concept erasure (effective dimensionality under 100-200 in an 8192-d residual stream), and show causal steering with domain-specific directions helps while the domain-general direction backfires. See mahalanobis-cosine-similarity-between-truth-probe-directions-predicts-cross-domain-generalization-far-better-than-standard-cosine-similarity.

  • Not all LLMs exhibit consistent truth directions — stronger, more consistent representations appear in more capable models, especially under logical negation — and probes trained on atomic statements generalize to logical transformations, QA, in-context learning, and external knowledge. Bao, Zhang, Du, Zhao, Feng, Peng & Yin (2025) demonstrate a practical selective-QA application of this generalization across Llama-2/3.1 and Mistral-7B. See not-all-llms-exhibit-consistent-truth-directions-with-stronger-representations-in-more-capable-models-and-atomic-statement-probes-generalize-to-logical-transformations-and-qa.

  • Internal Year/Month/Day representations are linearly decodable (“temporal linearity”), and this decodability is the strongest predictor of temporal-reasoning accuracy in high-resource languages, while tokenization fragmentation dominates in low-resource ones. Bhatia, Isa, Peyrard & Zhao (2026) find this via linear-regression probes across 20 LLMs and 5 languages (English probe-R^2-to-accuracy r=0.77; Hausa fragmentation-to-accuracy r=-0.97), a purely correlational (no causal steering) linear-decodability finding. See linear-probe-decodability-of-temporal-representations-is-the-strongest-predictor-of-temporal-reasoning-accuracy-in-high-resource-languages-while-tokenization-fragmentation-dominates-in-low-resource-ones.

  • A causally localized temporal-preference subgraph collapses a continuous time-horizon representation into a binary preference at the user-to-assistant turn boundary, and a probe-derived steering vector shifts temporal discount rate bidirectionally. Rios-Sialer et al. (2026) localize the subgraph to layers 17-35 of Qwen3-4B-Instruct via four independent pipelines, and show Contrastive Activation Addition at layers 19-22 shifts preference with a forced-choice score lift of +1.22 (odds ratio ~3.39x), revealing a probing-steering layer dissociation. See temporal-preference-subgraph-collapses-continuous-time-horizon-geometry-into-a-binary-preference-at-the-turn-boundary-and-a-probe-derived-vector-shifts-discount-rate-bidirectionally.

  • Contrastive activation-difference directions separating no-reflection, intrinsic-reflection, and triggered-reflection conditions causally enhance or suppress reflective behavior, with suppression consistently easier than induction. Chang, Lee & Wu (2025) construct these directions in Qwen2.5-3B and Gemma3-4B-IT, showing clear activation-level stratification across the three conditions and demonstrating both directions of causal steering on GSM8k-adv and Cruxeval-o-adv. See contrastive-activation-directions-for-reflection-intent-causally-enhance-or-suppress-reflective-behavior-and-suppression-is-easier-than-induction.

  • A 256-SAE suite spanning every layer and sublayer of an 8B open-weight model confirms feature splitting at production scale, with a dedicated geometric analysis of nearest-neighbor structure among learned features. He, Shu, Ge, Chen, Wang, Zhou, Liu, Guo, Huang, Wu, Jiang & Qiu (2024, Llama Scope) train 256 TopK SAEs (32K/128K widths) across Llama-3.1-8B-Base’s 32 layers, finding semantically coherent feature neighborhoods and confirming wider SAEs learn genuinely finer features rather than duplicates — extending the smaller-scale JumpReLU/Gemma-Scope width-ladder result and the original Claude-3-Sonnet demonstration to a fully open model and SAE suite. See a-256-suite-of-sparse-autoencoders-across-every-layer-and-sublayer-of-llama-3-1-8b-reveals-nearest-neighbor-feature-geometry-and-confirms-feature-splitting-at-scale.

  • SAEs trained across a diffusion model’s reverse-generation trajectory uncover interpretable concept directions whose causal manipulation reveals a temporal control hierarchy — composition early, style at mid-generation, texture only late. Tinaz, Fabian & Soltanolkotabi (2025) show this on Stable Diffusion v1.4, with final scene composition predictable from activated concepts before the first denoising step completes. See saes-on-stable-diffusion-uncover-interpretable-concept-directions-whose-causal-manipulation-shows-composition-style-and-texture-are-controlled-at-successively-later-denoising-steps.

  • Activation patching localizes a narrow cross-attention “semantic bottleneck” across three text-to-music diffusion architectures, and steering only at those layers beats full-network steering. Staniszewski, Zaleska, Modrzejewski & Deja (2026, TADA!) find this on AudioLDM2, Ace-Step, and Stable Audio Open, comparing mean-difference and SAE-decoder-column steering directions. See localized-2-to-4-layer-activation-steering-at-an-activation-patching-identified-semantic-bottleneck-outperforms-full-network-steering-across-three-text-to-music-diffusion-architectures.

  • Recursive-Feature-Machine (AGOP) probes on a frozen autoregressive music model extract fine-grained note/chord-level concept directions enabling real-time, dynamically time-varying, multi-attribute steering. Zhao, Beaglehole, Berg-Kirkpatrick, McAuley & Novack (2025) apply this to MusicGen-Large, complementing TADA!‘s coarser diffusion-model steering with note/chord-level control. See rfm-agop-derived-concept-directions-from-frozen-musicgen-large-hidden-states-enable-real-time-multi-attribute-note-and-chord-level-steering-without-per-step-optimization.

  • A TopK SAE on a TTS model’s semantic-backbone residual stream enables bidirectional causal emotion steering, with individual latents causally tied to specific acoustic parameters rather than one global direction. Du, Shi, Lu, Zhou & Gao (2026) show steering a single latent in IndexTTS2 raises mean F0 by +23.11 Hz without affecting duration, evidencing distributed rather than single-direction emotional control. See a-topk-sae-on-indextts2s-semantic-backbone-residual-stream-enables-bidirectional-causal-latent-emotion-steering-with-individual-latents-tied-to-specific-acoustic-parameters.

  • A companion depth/head sweep on the same linear “mine/yours” board-state direction shows presence and causal use are dissociable: the direction is linearly decodable even in 1-layer models, but is only causally used to set next-move logits in deeper models, with the causal effect concentrated in middle layers. Hazineh, Zhang & Chiu (2023) confirm Nanda, Lee & Wattenberg’s relative-frame Othello-GPT direction via linear probing and introduce a distinct causal-intervention mechanism (a learned linear inverse map editing activations toward a target board state), adding a model-capacity dimension the earlier Othello-GPT papers did not characterize. See othello-gpt-linear-board-state-causal-use-is-depth-dependent-and-concentrated-in-middle-layers.

  • A frontier closed model linearly encodes ~171 emotion-concept directions, roughly matching a valence/arousal circumplex, and steering them causally changes safety-relevant behavior. Sofroniew, Kauvar, Saunders, Chen, Henighan, Hydrie, Citro, Pearce, Tarng, Gurnee, Batson, Zimmerman, Rivoire, Fish, Olah & Lindsey (2026, Anthropic) extract these directions in Claude Sonnet 4.5 via diff-in-means contrasts over emotion words, finding the directions generalize across contexts; causal steering/ablation experiments show injecting or suppressing specific emotion directions measurably changes the model’s rates of reward hacking, blackmail, and sycophancy. See claude-sonnet-4-5-encodes-171-linear-emotion-concept-directions-roughly-matching-a-valence-arousal-circumplex-and-steering-them-causally-changes-blackmail-reward-hacking-and-sycophancy-rates.

  • A layer-specific linear emotional direction in vision-language models causally drives emotion expression when injected, and activation patching localizes the attention heads/neurons that construct it. Zhang, Sun, Xie & Tian (2026) extract this direction in Qwen3-VL-4B-Instruct and LLaVA by averaging filtered contrastive (emotional vs. neutral) hidden-state differences; steering with it raises emotion-expression hit rate, and backward activation patching traces the direction back to the specific heads/neurons that construct it via an “Adapt-Aggregate-Execute” cross-modal information-routing mechanism. See a-per-layer-linear-emotional-direction-in-vlms-causally-drives-emotion-expression-when-injected-and-activation-patching-localizes-the-heads-and-neurons-that-construct-it.

  • Several chemical properties organize along linear, globally steerable directions in a molecular Transformer-VAE’s latent space, robust to sequence-length confounds. Elabid, Brzoza, Andrzejewski & Cangi (2026) fit linear probes for cLogP, FractionCSP3, TPSA, HBA and BertzCT on a custom Transformer-VAE trained on ~794K SELFIES-tokenized molecules; latent traversal along the probe-derived directions causally produces monotonic, chemically coherent property changes in decoded molecules. See chemical-properties-organize-along-linear-globally-steerable-directions-in-a-molecular-transformer-vaes-latent-space-and-latent-traversal-causally-produces-monotonic-property-changes.

  • A linear, value-like task-success direction is recoverable from frozen VLA and vision-backbone features that were never trained to estimate reward, and deploying it as a test-time selector causally raises task success. Zhang, Nie, Lao, Cheng, Liu, Jiang & Huang (2026) linearly probe frozen OpenVLA, Pi0.5, DINOv2 and CLIP features on LIBERO-Goal manipulation trajectories for Monte-Carlo task-success targets, then use the resulting direction as a test-time selector over Pi0.5 action-prefix candidates, raising push-plate task success from 26.7% to 44.3% with no additional policy training. See a-linear-value-like-task-success-direction-is-recoverable-from-frozen-vla-and-vision-backbone-features-never-trained-to-estimate-reward-and-using-it-as-a-test-time-selector-causally-raises-task-success.

  • SAE-discovered latent features in a frozen multirobot policy are causally steerable via closed-loop, state-dependent affine edits (scale and shift, not just addition). Das, Chiu, Hegde & Sukhatme (2026) train a lightweight RL steering policy (Closed-Loop Affine Activation Editing) on top of SAE-identified behavior-relevant latents in a frozen multi-quadrotor navigation policy, steering individual velocity profiles, coordinating formation-preserving multirobot behavior, and inducing a novel emergent behavior (reduced camera-surveillance exposure) without touching the base policy’s weights. See sae-discovered-latent-features-in-a-frozen-multirobot-policy-are-causally-steerable-via-closed-loop-state-dependent-affine-edits-inducing-formation-control-and-novel-emergent-behavior.

  • Sparse FFN value-vector directions decoded via logit-lens projection, a minority of which are causally rewired for action, steer real vision-language-action robot policies. Häon, Stocking, Chuang & Tomlin (2025) project OpenVLA-7B and π0 (π0-FAST) FFN activations onto the token/vocabulary embedding basis, finding fewer than 25% of OpenVLA’s 352,255 FFN value vectors are specifically rewired for action prediction; zero-shot upweighting of a selected semantic direction cluster (e.g. “fast”, “up”) causally steers end-effector displacement in LIBERO-Long simulation and on a physical UR5 arm. See haon-etal-2025-sparse-logit-lens-directions-in-vla-ffn-value-vectors-causally-steer-robot-action-selection.

  • A near-linear game-state direction in two architecturally distinct RL world models (IRIS, DIAMOND) is decodable with an architecture-dependent layerwise profile and causally shifts model predictions. Zhang (2026) finds IRIS decodes paddle position/score at R²>0.99 flatly across layers while DIAMOND’s decodability sharply peaks at its diffusion-UNet bottleneck; shifting IRIS layer-5 hidden states along the probe-derived direction produces monotonic, strongly correlated (r≥0.96) changes in model predictions. See zhang-2026-game-state-variables-are-near-linearly-decodable-with-an-architecture-dependent-layerwise-profile-in-iris-and-diamond-and-causally-shift-model-predictions.

  • SAE feature directions in a real TTS LM backbone show a systematic layerwise shift in modality composition (mixed, then audio-heavy, then text-modal again at the final layer) and causally steer laughter, gender, and speech rate. Koriagin, Aparin, Balagansky & Gavrilov (2026) train BatchTopK SAEs on CosyVoice3’s Qwen2.5-0.5B-backbone residual stream, finding layers 16-20 become audio-heavy before the final layer reverts to a mostly text-modal subspace; SAE-latent steering raises laughter probability from 0.02 to 0.79 and flips perceived speaker gender while preserving spoken content. See koriagin-etal-2026-sae-feature-modality-composition-shifts-systematically-across-layer-depth-in-a-real-tts-lm-backbone-and-causally-steers-laughter-gender-and-speech-rate.

  • TopK SAE dictionary directions in a real pretrained ASR encoder (Whisper-base) recover monosemantic phonetic, lexical, and semantic features, and clamping a profanity-coding direction causally edits transcript content, generalizing to languages absent from SAE training. Pluth, Houghton, Zhou & Gurbani (2026) validate feature directions via precision/recall against labels (e.g. word “his”: precision 99.3%, recall 81.4%) and show clamping generalizes to French despite French being entirely absent from the SAE’s training corpus. See pluth-etal-2026-monosemantic-sae-latents-in-whisper-encode-phonetic-lexical-and-semantic-features-and-causally-steer-transcript-content-cross-lingually.

  • A single TopK SAE latent decoded from a real LLM’s residual stream causally mediates refusal with a monotonic dose-response clamping curve, at a quantified and non-monotonic capability cost. O’Brien, Majercak, Fernandes, Edgar, Bullwinkel, Chen, Nori, Carignan, Horvitz & Poursabzi-Sangdeh (2024/2025) identify this direction (Feature 22373) in Phi-3-mini, generalizing to Llama-3.1-8B-Instruct, raising WildGuard unsafe-prompt refusal from 58.33% to 96.02% at high clamp strength while MMLU accuracy degrades from 68.80% to 35.98% — a trade-off the authors note is mechanistically unexplained. See obrien-etal-2024-a-single-sae-latent-causally-mediates-refusal-with-monotonic-dose-response-and-a-quantified-safety-capability-tradeoff.

  • Timestep-conditioned transcoder feature directions trained to replace a real pretrained diffusion transformer’s MLP sublayers give exact linear attribution graphs, revealing a text-to-image stream contribution shift, and joint feature suppression/amplification causally corrects color-bias failures where single-feature steering does not. Mazur, Konovalova & Alanov (2026, DifFRACT) apply this to FLUX.1[schnell], finding text-stream attribution-graph contribution falls from 89.9% to 5.4% (image-stream rises from 10.1% to 94.6%) across 4 denoising steps, and that a blue-apple color-swap intervention only succeeds on 60% of seeds when feature and context suppression are combined. See mazur-etal-2026-timestep-conditioned-transcoder-directions-in-flux-1-schnell-give-exact-attribution-graphs-and-causally-shift-color-bias-and-stream-contribution.

  • Position-aware SAE safety directions extracted at functionally distinct sites in two real large-scale diffusion transformers transfer across models via a frozen-encoder/adapted-decoder scheme, and causally Blend toward or Repel away from harmful directions. Xue, Wang, Bi, Ma, Zheng, Yang, Zhu, Huang, Xiao & Lou (2026, SafeDIG) apply this to FLUX.1 Dev and Stable Diffusion 3.5 Large, improving FID from 9.90 to 6.36 for double-stream-block steering on the target domain while preserving CLIP score (26.2-27.8). See xue-etal-2026-position-aware-sae-safety-directions-transfer-across-flux-and-sd3-5-via-manifold-stable-decoder-adaptation-and-causally-blend-or-repel-unsafe-generations.

  • Ridge-regression residualization between adjacent denoising timesteps, applied before BatchTopK SAE training on a real pretrained diffusion model, yields feature decoder directions with measurably more stable cross-timestep trajectories than non-residualized baselines. Yeung, Poduval, Zakeri, Zou & Imani (2026) apply this to Stable Diffusion 1.5’s U-Net activations, with residualized variants outperforming non-residualized/timestep-wise/Matryoshka SAE baselines at matched sparsity, strongest at later denoising timesteps; single-feature steering during generation produces qualitative but semantically meaningful effects. See yeung-etal-2026-residualizing-adjacent-denoising-timesteps-before-batchtopk-sae-training-yields-directionally-stable-feature-trajectories-in-stable-diffusion-1-5-that-support-generative-steering.

  • In a real 35B MoE LLM, ridge-regression probe directions for five nominally distinct agentic traits are all nearly orthogonal to each other yet causally converge on a single shared agency axis, with feature-concentration analysis showing under 1% of SAE features carry most of each steering vector’s norm. Yap (2026) finds risk-calibration and tool-use-eagerness probes in Qwen3.5-35B-A3B share nearly identical R² (0.795 vs. 0.792) yet cosine similarity of only -0.017 (SAE space), and that autonomy steering at prefill (Cohen’s d=1.01, p<0.0001) but not decode-only (d~0.13-0.20, p>0.35) causally shifts proactive tool-call behavior — evidence the behavioral commitment is set during prefill. See yap-2026-risk-calibration-and-tool-use-eagerness-probe-directions-share-near-identical-r2-yet-are-nearly-orthogonal-revealing-a-single-shared-agency-axis-not-five-independent-traits.

  • A causally-validated diffusion-domain lens, extending logit-lens/tuned-lens-style direct readout to a real pretrained vision transformer, correlates more strongly with true head-ablation effects than a vanilla Diffusion Lens. Takatsuki, Joseph, Fujisawa & Kanai (2025) apply the Diffusion Steering Lens to CLIP ViT-bigG/14 (Kandinsky 2.2’s image encoder), showing its sequential-ablation trajectory for removing a synthetic overlay nearly matches an ACDC-style optimal ordering. New method diffusion-steering-lens. See takatsuki-etal-2025-a-causally-validated-diffusion-steering-lens-correlates-more-strongly-with-real-head-ablation-effects-than-a-vanilla-diffusion-lens-in-a-real-pretrained-vit.

  • A delta-loss-regularized crosscoder isolates 1-2 sparse feature directions per real fine-tuning “organism,” recovering the causal direction in 10/10 cases with 0% false positives, and steering along it causally mitigates the fine-tuned behavior. Kassem, Jiralerspong, Rostamzadeh & Farnadi (2026, Delta-Crosscoder) apply this across Llama-3.1-8B-Instruct, Gemma-2-9B-IT, and Qwen2.5-7B, beating BatchTopK and DSF baselines (40-60% false-positive rates). See kassem-etal-2026-a-delta-loss-crosscoder-isolates-1-2-directional-latents-per-fine-tuning-organism-with-zero-false-positives-and-causally-mitigates-the-fine-tuned-behavior-via-steering.

  • A per-token-position mean-difference direction between base and narrowly fine-tuned real models causally reveals the fine-tuning objective when steered into a templated prompt, letting an agent identify 30/33 real fine-tuning objectives. Minder, Dumas, Slocum, Casademunt, Holmes, West & Nanda (2026) extract this across 33 organisms and 7 models (1B-32B), with GPT-5 correctly identifying the fine-tuning objective 91% of the time given access to the direction, vastly outperforming a black-box baseline. See minder-etal-2026-a-per-token-mean-difference-direction-fingerprints-narrow-fine-tuning-across-33-organisms-and-7-real-models-letting-an-agent-identify-the-fine-tuning-objective-in-91-percent-of-cases.

  • Crosscoder-identified feature directions active just before a “wait” token causally set which reasoning pattern a real reasoning model executes next. Troitskii, Pal, Wendler, McDougall & Nanda (2025) locate these directions in DeepSeek-R1-Distill-Llama-8B (and its Llama-3.1-8B base) via multi-layer crosscoder model-diffing plus a crosscoder-setting latent-attribution technique, showing targeted intervention causally determines restart/recall/uncertainty/double-check behavior. See troitskii-etal-2025-crosscoder-identified-pre-wait-feature-directions-causally-set-which-reasoning-pattern-deepseek-r1-distill-llama-8b-executes-after-the-wait-token.

  • Fewer than 25 algorithmically-identified “register neurons,” with consistent decoder-weight directions, concentrate outlier-token activations in real pretrained ViTs, and training-free “test-time registers” built from these directions causally recover most trained-register benefits. Jiang, Dravid, Efros & Gandelsman (2025) find this in OpenCLIP ViT-B/16 and DINOv2-L/14, closing unsupervised object discovery to within 0-2 corloc of trained-register models and cutting typographic-attack success on LLaVA-Llama-3-8B from 50.5% to 7.5% via register-neuron ablation. New method register-neuron-identification. See jiang-etal-2025-fewer-than-25-algorithmically-identified-register-neurons-with-consistent-decoder-directions-causally-explain-outlier-token-formation-in-real-pretrained-vits.

  • The foundational vector-offset parallelogram result: trained RNNLM word representations encode linguistic relationships as consistent linear offset vectors, retrievable via nearest-neighbor cosine similarity. Mikolov, Yih & Zweig (2013) establish “king - man + woman ~ queen” as the founding empirical demonstration of linear-representation geometry in trained word vectors, predating and foundational to essentially all later linear-representation-hypothesis work in this map. See mikolov-etal-2013-king-man-plus-woman-approximately-queen-the-foundational-parallelogram-vector-offset-analogy-in-trained-rnnlm-word-representations.

  • Foundational: a gender direction extracted as a diff-in-means over gendered word-pair differences in real trained word2vec embeddings is causally editable, eliminating stereotypical analogies without harming semantic performance. Bolukbasi, Chang, Zou, Saligrama & Kalai (2016) measure this direction in word2vec (Google News, 300d), show gender-neutral words are linearly separable from gender-definitional words along it, and causally “hard-debias” by projecting the direction out of gender-neutral words plus an equalize step — a foundational demonstration that a measured bias direction in a real trained model is directly, causally editable. See bolukbasi-etal-2016-a-diff-in-means-gender-direction-in-word2vec-embeddings-linearly-separates-gender-definitional-from-gender-neutral-words-and-causal-hard-debiasing-along-it-eliminates-stereotypical-analogies.

  • Foundational: a single “sentiment neuron” in a trained byte-level mLSTM language model linearly encodes sentiment, and clamping its value causally flips generated text’s sentiment. Radford, Jozefowicz & Sutskever (2017) train a 4,096-unit byte-mLSTM unsupervised on ~82M Amazon reviews and find one hidden unit alone reaches near-SOTA linear-probe sentiment accuracy on SST; directly overwriting its activation during generation causally flips the sentiment of the model’s own generated text — one of the earliest single-direction causal-steering demonstrations in a real trained generative model. See radford-etal-2017-a-single-unit-sentiment-neuron-in-a-trained-byte-level-mlstm-language-model-linearly-encodes-sentiment-and-clamping-it-causally-flips-generated-review-sentiment.

  • Foundational, cross-modal precedent: word2vec-style vector-offset arithmetic applied directly to a trained GAN’s latent space causally edits generated images’ semantic attributes. Radford, Metz & Chintala (2016, DCGAN) show averaged “smiling woman” minus averaged “neutral woman” plus averaged “neutral man” latent-Z vectors causally render as a smiling man in the generator’s output, extending vector-offset-analogy geometry (originally demonstrated for word embeddings) to a real trained convolutional generative model for the first time. See radford-etal-2016-dcgan-vector-arithmetic-in-a-trained-gans-latent-z-space-smiling-woman-minus-neutral-woman-plus-neutral-man-causally-yields-smiling-man-in-generated-images.

  • A single consistent rotational operator, not a fixed additive offset, best explains semantic/syntactic relation pairs across three independently trained real embedding models. Freenor & Alvarez (2025) fit rotor-invariant shift estimation on OpenAI’s text-embedding-3-large, BGE-M3, and multilingual BERT, finding a fitted rotation achieves measurably lower and more cross-model-consistent residual error for analogy-style word/phrase pairs than the classical additive vector-offset model — a direct comparative geometric-shape claim (rotation vs. translation) validated on real pretrained models, without a causal intervention. See freenor-alvarez-2025-rotor-invariant-shift-estimation-recovers-a-single-consistent-rotational-operator-for-semantic-syntactic-relations-across-text-embedding-3-large-bge-m3-and-mbert.

  • Phonological feature contrasts recur as approximately consistent linear directions across three real speech self-supervised models, extending vector-offset arithmetic to a non-text, non-vision modality. Choi, Yeo, Cho, Harwath & Mortensen (2026) extract diff-in-means directions for voicing/place/manner contrasts from wav2vec2-large-lv60, HuBERT-large-ll60k, and WavLM-large individually, and show vector-offset analogies (e.g. [b] ≈ [d] - [t] + [p]) succeed above chance in each model’s own real extracted phoneme representations. See choi-etal-2026-phonological-feature-vector-arithmetic-b-equals-d-minus-t-plus-p-holds-approximately-in-wav2vec2-hubert-and-wavlm-representations-of-real-speech.

  • Individual FFN value-vector directions, read out via the same unembedding-projection mechanic as the logit lens, are shown to causally control a real behavioral property when targeted for suppression. Geva, Caciularu, Wang & Goldberg (2022) decompose each FFN layer’s additive vocabulary-distribution update into per-parameter-vector sub-updates and show that suppressing the value-vector directions identified as promoting toxic-concept tokens causally cuts GPT-2’s generated toxicity by roughly half — a finer-grained, causally-validated companion to Geva, Schuster, Berant & Levy (2020)‘s general FFN-key-value-memory framing. See geva-etal-2022-ffn-value-vectors-projected-through-the-unembedding-matrix-are-directions-that-causally-suppress-toxicity-when-their-promoted-concepts-are-targeted.

  • Foundational, cross-modal precedent for single-unit causal control in a generative model: individual feature-map units in a real trained GAN align with real object-segmentation masks, and ablating or inserting them causally adds or removes the corresponding object from generated images. Bau, Zhu, Strobelt, Zhou, Tenenbaum, Freeman & Torralba (2019, GAN Dissection) score each unit’s spatial activation pattern against segmentation masks across many generated images, then causally ablate/insert the identified units in a real trained Progressive GAN on LSUN scenes — also used to locate and remove artifact-causing units. See bau-etal-2019-gan-dissection-individual-feature-map-units-in-a-trained-gan-align-with-real-object-segmentation-masks-and-causally-control-object-presence-when-ablated-or-inserted.

  • A single antonym-pole vector difference, validated by correlation against independent human judgments rather than by classifier accuracy or causal steering. Grand, Blank, Pereira & Fedorenko (2022) define 52 bipolar “feature lines” (e.g. small→big, safe→dangerous) in real 300-dimensional GloVe embeddings and score arbitrary category members by scalar projection onto each line, finding a median Pearson r=0.47 (reliability-adjusted r=0.61) against independently-collected Mechanical Turk human graded ratings — a rare case in this map where the geometric-shape claim is validated against an external, non-model ground truth rather than an internal metric. See grand-etal-2022-semantic-projection-onto-antonym-pole-directions-in-real-glove-embeddings-recovers-graded-human-object-feature-judgments-at-median-r-0-47.

  • A linearly-probed entity-state representation, spliced between two real contexts rather than merely decoded, causally shifts generation toward the predicted mixed state. Li, Nye & Andreas (2021) train linear probes for entity state (e.g. beaker contents in Alchemy, room/object state in TextWorld) on real fine-tuned BART/T5 encoders, then splice one context’s probed representation into another to construct a synthetic “mixed” state, finding generations land in the human-predicted continuation set 57.7% (BART) / 75.4% (T5) of the time versus 20-38% for the unmixed source contexts — a genuine geometry-tied causal intervention, not just decodability. See li-nye-andreas-2021-splicing-linearly-probed-entity-state-representations-between-two-real-fine-tuned-bart-and-t5-encoders-causally-shifts-generation-toward-the-predicted-mixed-state-57-7-percent-and-75-4-percent-of-generations.

  • A relation-specific additive vector offset, computed mid-network by a real trained transformer’s own FFN update, is portable across contexts and causally sufficient to force the correct relational output. Merullo, Eickhoff & Pavlick (2024) extract a country→capital-style offset from one context and patch it into a different context’s residual stream, causally forcing the model to output the new context’s correct relational answer, replicated across GPT-J, GPT-2, Pythia and BLOOM (up to 176B) — real-activation vector arithmetic directly analogous to word2vec, both measured and causally validated. See merullo-etal-2024-a-relation-specific-additive-vector-offset-computed-mid-network-is-extractable-from-one-context-and-causally-patched-into-another-to-force-the-correct-relational-output-across-gpt-j-gpt-2-pythia-and-bloom.

  • Two sycophancy subtypes occupy partially distinct steering directions, and whether cross-type steering transfers or backfires depends on the model, not just the behavior. Baez, Karny & Pataranutaporn (2026) build diff-of-means steering vectors and LDA visualizations for factual vs. opinion sycophancy in real Gemma-3-12B-IT and Llama-3.1-8B-Instruct, finding the two subtype vectors have positive cosine similarity (+0.68) in Gemma (where cross-type steering succeeds, R^2 up to 1.00) but negative cosine similarity (-0.15) in Llama (where cross-type steering instead decreases the target sycophancy rate) — the same behavioral category can be geometrically entangled or dissociated depending on the model. See baez-etal-2026-factual-and-opinion-sycophancy-occupy-partially-distinct-linear-directions-and-cross-type-steering-transfers-in-gemma-3-but-backfires-in-llama-3.1.

  • Sycophancy signal is present but diffuse in the residual stream and MLPs, yet concentrates into a sparse, causally load-bearing subset of attention heads. Genadi, Nwadike, Mukhituly, Alquabeh, Hiraoka & Inui (2026) probe residual, MLP, and per-head attention activations of real Gemma-3-4B-IT and Llama-3.2-3B-Instruct for correct-to-incorrect sycophancy, finding probe accuracy is broadly high in residual/MLP layers but sharply localized to a handful of middle-layer heads; steering only those heads cuts sycophancy rate from 40.7% to 34.4% (Gemma) and 51.7% to 25.0% (Llama), far outperforming residual/MLP steering, while the resulting direction is only mildly anti-correlated with a previously-identified truthful direction (cosine -0.22, 32% head overlap) — evidence that “where a behavior is linearly decodable” and “where it is causally steerable” can dissociate by component. See genadi-etal-2026-correct-to-incorrect-sycophancy-is-most-linearly-separable-in-a-sparse-subset-of-middle-layer-attention-heads-and-steering-those-heads-cuts-sycophancy-rate-far-more-than-mlp-or-residual-steering.

  • A whole family of cue-induced biases is installed by alignment tuning rather than pretraining, each collapsing to a single causal direction, with cross-bias entanglement a property of the model rather than the bias category. Gupta, Zhang, Draye, Schölkopf & Jin (2026) extract per-bias diff-of-means directions for seven behavioral-cue-triggered biases across base and instruct checkpoints of five real model families (Llama-3.1-8B, Qwen-2.5-7B, Gemma-2-9B, Mistral-7B-v0.3, OLMo-2-7B), finding base models flip on under 5% as many pairs as instruct counterparts with no cue-specific activation signal, while instruct-model directions transfer to held-out datasets (AUROC 0.69-0.82) and localize to late-middle layers; subtracting the direction recovers 7-20% of bias-induced errors across all five families versus under 5% for a random-direction control, and even behaviorally similar biases (Post Hoc vs. Suggested Answer) occupy different, sometimes anti-aligned directions. See gupta-etal-2026-cue-induced-bias-directions-are-installed-by-alignment-tuning-not-pretraining-form-a-single-causal-direction-per-bias-and-stay-representationally-distinct-across-seven-bias-types-and-five-model-families.

  • A weight-space (not activation-space) instance of directional alignment: router weight vectors in a real trained sparse MoE geometrically couple to their matched experts, and the coupling strength is itself a trainable-via-loss-choice quantity. Ahrac, Hochwald & Geva (2026) show, in a real 1B-parameter SMoE trained from scratch on ~50B tokens, that router weight vectors and their routed experts’ gate weights receive gradient updates proportional to the same hidden-state direction, empirically confirmed via router-score/real-activation correlation (rho=0.43); comparing auxiliary-loss versus loss-free load-balancing training shows the resulting router-vector mutual cosine similarity differs by nearly 3x (0.63/0.63/0.57 vs. 0.32/0.18/0.13) — a rare case in this map where the causal variable manipulated is a training-loss choice and the measured effect is purely geometric (weight-vector alignment), not a downstream behavioral score. See ahrac-etal-2026-router-weight-vectors-geometrically-couple-to-their-matched-experts-real-per-token-activations-and-auxiliary-load-balancing-makes-routers-nearly-3x-more-mutually-similar-than-loss-free-balancing.

  • Steering token routing positions toward an expert’s centroid in a real trained cosine-routing MoE causally and additively controls which expert processes a token. Ternovtsii & Bilak (2026) bias routing positions toward a temporal expert’s centroid in a custom 76-84M-parameter Semantic Trajectory MoE, raising P(temporal) by a median +321% across 44 prompts; suppressing a geographic expert drops P(geographic) by -23.2%, and directly rewriting an expert’s weight vector (“expert surgery”) halves it (-45.8%); cross-layer composed effects are near-additive while same-layer composition interferes. See ternovtsii-bilak-2026-steering-token-routing-positions-toward-an-experts-centroid-in-a-real-trained-cosine-routing-moe-causally-and-additively-controls-which-expert-processes-a-token.

  • Linear-probe directions for reverberation time and loudness are strongly decodable and geometrically consistent across independently trained datasets in real CLAP audio embeddings. Martel, Hennessy-Priest & Cho (2026) show RT60 linear-probe R^2>=0.67 (r>=0.82) across five datasets in real LAION-CLAP embeddings, and that independently-fit RT60/LUFS probe-weight vectors are geometrically consistent across datasets (cosine similarity up to 0.86), while a relative-pitch probe direction is domain-specific (near chance-level cosine baseline) — a rare cross-dataset geometric-consistency check on learned linear feature axes, plus a cross-model failure mode where amplitude-invariant architectures (Wav2Vec2, WavLM-Large, MERT) specifically fail to encode loudness. See martel-etal-2026-linear-probe-directions-for-reverberation-time-and-loudness-are-strongly-decodable-and-geometrically-consistent-across-independently-trained-datasets-in-real-clap-audio-embeddings.

  • Frozen pretrained audio encoders’ normalized cosine-similarity sensitivity to source-versus-room acoustic perturbations does not always track linear-probe decodability of the same factors. Chen, Ding, Roman & Bello (2026) evaluate 13 real pretrained spatial-audio encoders (mono/stereo/binaural/ambisonics, including Audio-MAE and EnCodec), finding source-related linear-probe improvements consistently exceed room-related ones, while a separate normalized cosine-shift sensitivity metric can diverge from decodability for the same model and factor (e.g. GRAM-F/EINv2: low sensitivity, strong probing; SFD/BANC: the reverse) — demonstrating decodability and raw representational movement are logically distinct properties. See chen-etal-2026-frozen-pretrained-audio-encoders-normalized-cosine-similarity-sensitivity-to-source-versus-room-acoustic-perturbations-does-not-always-track-linear-probe-decodability-of-the-same-factors.

  • Function-vector heads split into writer and canceller populations with near-orthogonal OV directions in real Pythia, Qwen, and GPT-2 models, and zero-ablating cancellers causally boosts ICL logits and accuracy. Wang (2026) shows the FV heads identified by Todd et al. (2024)‘s magnitude ranking in real Pythia-410M/1B/1.4B (extended to 2.8B/6.9B/12B, Qwen2.5-1.5B/7B, GPT-2-medium) split by direct-logit-attribution sign into writers and cancellers whose mean OV directions are nearly orthogonal (perpendicular-fraction mean 0.96); zero-ablating cancellers yields +0.13 to +0.29 nats of logit gain and a +2 to +7 pp ICL-accuracy effect in 6/6 main cells, and a TOST equivalence test confirms cancellers are not simply induction heads. See wang-2026-function-vector-heads-split-into-writer-and-canceller-populations-with-near-orthogonal-ov-directions-in-real-pythia-qwen-and-gpt-2-models-and-zero-ablating-cancellers-causally-boosts-icl-logits-and-accuracy.

  • A multi-sample generalization of the Logit Lens identifies causally-specialized attention heads across both real unimodal and real multimodal transformers. Basile, Maiorca, Doimo, Locatello & Cazzaniga (2025) score heads by Simultaneous Orthogonal Matching Pursuit against unembedding-matrix directions rather than a single-sample Logit Lens readout, finding that inverting just 8 heads (0.8% of Mistral-7B’s total) degrades TriviaQA country-name recall far more selectively than an equal-size random-head or Logit-Lens-selected-head control, and that suppressing 8-32 SOMP-selected heads consistently reduces toxic-generation rates below both baselines on RealToxicityPrompts/TET. Applying the identical scoring to real LLaVA-NeXT-7B/13B, Gemma3-12B, and Qwen2.5-VL-7B, top-SOMP heads causally control image-classification accuracy and Flickr30k caption attribute content (inhibiting 16 heads nearly removes color/sentiment/quantity keywords while CIDEr stays above 80% of baseline), extending head-level directional causal control from a text-only setting to a shared unimodal/multimodal regime. See basile-etal-2025-somp-selected-attention-heads-causally-control-target-behavior-in-real-mistral-7b-and-llava-next-gemma3-qwen2-5-vl-vlms-far-more-than-random-or-logit-lens-selected-heads.

  • Function-vector heads drive real ICL performance while induction heads have limited effect, and FV heads evolve from induction heads during training. Yin & Steinhardt (2025) ablate FV and induction heads across 12 real models (Pythia 70M-6.9B, GPT-2 117M-1.6B, Llama-2-7B), finding FV-head ablation substantially degrades few-shot ICL accuracy while induction-head ablation (with top FV heads preserved) is statistically indistinguishable from random in models above 1B parameters; tracking 7 Pythia models across training shows induction heads emerge at ~step 1,000 and FV heads substantially later at ~step 16,000, with many FV heads visibly and unidirectionally evolving from earlier induction heads. See yin-steinhardt-2025-fv-heads-drive-icl-while-induction-heads-have-limited-effect-and-fv-heads-evolve-from-induction-heads-during-training.

  • Real Gemma 3 and Qwen3 task vectors are locally realized only at certain tokens despite task identity being decodable throughout context, and patching them recontextualizes zero-shot prompts. Li, Campbell, Chan & Lampinen (2025) find task-identity classifiers transfer to the transferable-representation token at 0.745-1.0 accuracy (vs. 3-6% chance) across real Gemma-3 (4B/12B/27B) and Qwen3 (4B/8B/14B), with PCA overlap between identifiable and transferable subspaces ranging from 40-60% (simple tasks) to 10-25% (list-operation tasks); patching the extracted task vector recontextualizes zero-shot prompts, tracking few-shot accuracy as k-shot increases. See li-etal-2025-real-gemma-3-and-qwen3-task-vectors-are-locally-realized-only-at-certain-tokens-despite-task-identity-being-decodable-throughout-context-and-patching-them-recontextualizes-zero-shot-prompts.

  • Multimodal function vectors, extracted via causal mediation from a small set of attention heads in real OpenFlamingo-4B and Qwen3-VL-4B-Instruct, causally double zero-shot visual-relation accuracy. Fu, Goldberg, Wu & Lu (2026) extend the function-vector construct to visual-relational tasks: injecting a function vector built from the top-10 causally-mediating heads raises zero-shot accuracy from 4.8% to 9.7% (OpenFlamingo-4B) and 19.5% to 26.8% (Qwen3-VL-4B-Instruct), with composite function vectors for entirely untrained relations roughly doubling accuracy again. See fu-etal-2026-multimodal-function-vectors-extracted-via-causal-mediation-from-a-small-set-of-attention-heads-in-real-openflamingo-4b-and-qwen3-vl-4b-instruct-causally-double-zero-shot-visual-relation-accuracy.

  • Self-supervised navigation discovers dimension-restricted semantic directions in a real trained StyleNeRF W+ latent space that outperform shifting all dimensions. Xie, Li, Zhang, Dong, Jin, Yang & Zeng (2023, NaviNeRF) extend the GANSpace/InterFaceGAN 2D-latent-direction discovery result to a NeRF-based 3D-aware generative latent space, showing shifts restricted to specific W+ style-code dimensions (9th-18th) causally outperform shifting the full 18-dimensional code for fine-grained attribute disentanglement. See xie-etal-2023-self-supervised-navigation-discovers-dimension-restricted-semantic-directions-in-a-real-trained-stylenerf-w-plus-latent-space-that-outperform-shifting-all-dimensions.

  • Persona-vector projections in real Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct predict trait expression and finetuning-induced trait shift, and preventative training-time steering causally blocks unwanted persona drift. Chen, Arditi, Sleight, Evans & Lindsey (2025) show persona-vector projection at the final prompt token correlates with subsequent trait expression (r=0.634-0.830) and, measured over a fine-tuning dataset’s own responses, with how much that finetuning run shifts trait propensity (r=0.76-0.97) — and that steering toward the undesired persona direction during training causally reduces persona drift while preserving coherence and MMLU accuracy better than regular finetuning. See chen-etal-2025-persona-vector-projections-predict-trait-expression-and-finetuning-induced-shift-and-preventative-training-time-steering-blocks-unwanted-drift.

  • Twelve real trait directions cluster into antisocial and prosocial groups by cosine similarity, and stay structurally stable across emergent-misalignment domains and model scales. Aneja, Mittal, Goel, Kumaraguru & Bonagiri (2026) find Psychopathy/Narcissism/Machiavellianism/Evil trait directions in real Llama-3.1-8B and Qwen-2.5-7B/14B/0.5B pairwise cosine-cluster above 0.50 into an “Antisocial Group” anti-correlated with a “Prosocial Group,” with the entire 12-trait geometric structure showing linear CKA >0.97 (Llama) / >0.88 (smallest Qwen) and Procrustes disparity 0.02-0.08 across fine-tuning domains and scales — a genuinely different claim from any single-trait-direction paper, since it measures the geometric relationship between many trait directions rather than any one direction alone; ablating/amplifying a trait vector causally shifts emergent-misalignment rate by up to +43.3pp. See aneja-etal-2026-twelve-real-trait-directions-cluster-into-antisocial-and-prosocial-groups-by-cosine-similarity-and-stay-structurally-stable-via-cka-procrustes-across-emergent-misalignment-domains-and-model-scales.

  • A linearly-probable “problem difficulty” direction is language-agnostic in shallow layers and language-specific in deep layers, across four real LLMs and 21 languages. Civelli, Bernardelle, Brunello & Demartini (2026) train linear (ridge) probes on Llama-3.1-8B, Llama-3.2-3B, Llama-3.2-1B and Qwen3-8B residual-stream activations to decode a continuous difficulty score, finding deep-layer probes reach high same-language accuracy (rho=0.822, Llama-3.1-8B) but generalize poorly across languages, while shallow-layer probes generalize far better (rho=0.783) at some cost to within-language accuracy — fixing a probe at the deep, same-language-optimal layer costs 0.177 rho under cross-lingual transfer, versus only 0.014 rho lost fixing at the shallow, transfer-optimal layer. No causal steering intervention is performed in this paper. See difficulty-direction-is-language-agnostic-in-shallow-layers-and-language-specific-in-deep-layers.

  • A statement-level truth direction measurably rotates and rescales when context is added, and mass-mean steering with it flips truthfulness labels near 100% of the time in three of four real LLMs. Adarsh, Maistro & Lioma (2026) extract per-layer truth directions in real Llama-3.1-8B-Instruct, Mistral-Nemo-12B-Instruct, Qwen3-4B-Instruct and SmolLM3-3B, finding the angle between with-context and without-context truth vectors is large in early layers and converges by mid-depth, with a quantified per-dataset magnitude-ratio shift (e.g. 1.34) — a measured geometric transformation of the direction itself, not just its existence. Mass-mean steering achieves 100.0% label-switching for three of four models. See truth-vector-rotates-and-rescales-under-added-context-and-mass-mean-steering-flips-truthfulness-labels.

  • A “task-learning” head role rotates real Llama/Qwen/Yi hidden states toward the correct-minus-incorrect label direction within a task subspace, causally dissociable from a separate “task-recognition” head role. Yang, Cho & Inoue (2026) decompose in-context-learning attention heads via projection-based logit attribution, finding task-learning heads specifically align hidden states with the correct-label-minus-incorrect-label direction (distinct from task-recognition heads, which merely align with the task subspace as a whole, see Linear Subspace); for open-ended generation, task-learning-based injected vectors outperform task-recognition-based ones (LLM-judged rating 5.12 vs. 4.44), and ablating top task-learning heads reduces accuracy by ~30% while only mildly affecting the task-recognition signature. See task-recognition-and-task-learning-attention-heads-in-icl-project-hidden-states-onto-a-label-unembedding-task-subspace-and-injecting-their-summed-outputs-into-zero-shot-prompts-recovers-most-of-the-icl-behavioral-gain.

  • A linearly-decodable “latent context” signal in real Qwen/Gemma/Llama/SmolLM3 attention heads drives in-context hierarchical structure prediction, and ablating the decoded heads collapses accuracy. Saanum, Demircan, Gershman & Schulz (2025) decode, via linear probes on real attention-head outputs, whether tokens belong to a matching “latent context” (second-order chunk decodability above 90% for several heads in Qwen2.5 0.5B/1.5B/3B, replicated on Gemma2-2B/Llama3.2-3B/SmolLM3-3B/Qwen3-0.6B); ablating the heads whose latent-context decoding exceeds 85% sharply collapses hierarchical-structure-prediction accuracy, versus a much smaller effect from ablating random heads. See a-linearly-decodable-latent-context-signal-in-real-qwen-gemma-llama-and-smollm3-attention-heads-drives-in-context-hierarchical-structure-prediction-and-ablating-the-decoded-heads-collapses-accuracy.

  • Off-the-shelf persona vectors in real Gemma-2-27B and Qwen3-32B are nearly orthogonal to a targeted sycophancy-steering direction, yet causally reduce sycophancy nearly as well. Kelkar, Alam, Kakaria, Panwar, Sharma & Chaudhary (2026) find generic persona/role steering vectors have cosine similarity below 0.17 with a targeted CAA sycophancy direction (with the sign flipping between models), yet injecting critical/skeptic persona vectors into the residual stream reduces sycophancy to 68-98% of CAA’s own effect without ever using sycophancy labels — a geometrically near-independent, behaviorally near-equivalent steering direction. See off-the-shelf-persona-vectors-in-real-gemma-2-27b-and-qwen3-32b-are-nearly-orthogonal-to-a-targeted-sycophancy-steering-direction-yet-causally-reduce-sycophancy-nearly-as-well.

  • A custom-trained 1-layer transformer solves analogies via vector arithmetic et≈es+fe_t\approx e_s+f, with entity embeddings progressively geometrically aligning during training as measured by decreasing Dirichlet energy. Minegishi, Feng, Furuta, Kojima, Iwasawa & Matsuo (2026, ICML 2026 Spotlight) show a real 1-layer, 1-head transformer trained from scratch on a synthetic analogy task develops analogical reasoning only once cross-category entity embeddings become geometrically aligned (Dirichlet energy decreasing during training), with a concurrent rise in a “Parallelism” cosine-similarity metric between (et−es)(e_t-e_s) and a learned functor direction ff — the classic Mikolov-style vector-offset structure, here measured emerging rather than assumed. No activation-level causal intervention was found (only a data-level correspondence-noise ablation). See a-custom-trained-1-layer-transformer-solves-analogies-via-vector-arithmetic-et-approx-es-plus-f-and-entity-embeddings-progressively-geometrically-align-during-training-as-measured-by-decreasing-dirichlet-energy.

  • A ridge-regression “category direction” at digit-count boundaries in real Llama-3-8B-Instruct causally shifts confidence 70x more than random directions. Cacioli (2026) — see Decision boundary (as a codimension-1 hypersurface) for the paper’s primary categorical-perception/metric-warping claim — additionally extracts a perfect-accuracy ridge-regression direction separating digit-count categories and shows adding it to the residual stream shifts model confidence 70.1x more than 10 random-direction controls at layer 5, decaying by ~60x by layer 16, tying the causally-active direction to the representation-space category boundary. See hidden-state-distances-in-five-real-llms-warp-categorically-at-digit-count-boundaries-and-causal-patching-along-a-category-direction-shifts-confidence-70x-more-than-random.

  • RSA-localized concept vectors, summed from a few real Llama-3.1 attention heads, are invariant across surface form for verbal analogies and causally guide behavior when added to the residual stream. Opiełka, Rosenbusch & Stevenson (2025) localize a small set of heads via RSA whose summed activations form “concept vectors” measurably invariant across language and prompt format (~0.7-0.8 cosine similarity) for verbal concepts (antonym, category) but not for abstract relational concepts (previous, next); adding a concept vector to the residual stream at a specific layer in real Llama-3.1-8B/70B causally steers behavior on ambiguous in-context prompts, transferring better out-of-distribution than function vectors, though weaker at pure zero-shot intervention. See rsa-localized-concept-vectors-summed-from-a-few-real-llama-3-1-attention-heads-are-invariant-across-surface-form-for-verbal-analogies-and-causally-guide-behavior-when-added-to-the-residual-stream.

  • A mean-difference “binding ID” direction in real attention query/key space causally redirects attention and shifts logits, with a family-dependent representational signature. Oh & Demberg (2026) estimate binding-ID shift directions as differences between mean Q/K activations for two binding conditions in selected attention heads of real Gemma2-9B-it, Gemma3-12B-it, Llama-3.2-3B-it and Llama-3.1-8B-it; adding these directions to Q/K activations causally redirects the model’s attention to the corresponding object and shifts final logits accordingly, with whether the effect concentrates in Q+K jointly or in K alone differing across model families. See oh-demberg-2026-a-mean-difference-binding-id-direction-in-attention-query-key-space-causally-redirects-attention-and-shifts-logits-with-a-family-dependent-representational-signature.

  • Transcoder-decoded “planning feature” directions in real Qwen3 models causally control not-yet-generated token and rhyme choices. Hanna & Ameisen (2026) decompose real Qwen3 (0.6B-14B) MLP activations via transcoders and attribution graphs, identifying individual feature directions that represent a not-yet-generated token or concept before it is produced; ablating these features on examples where the model succeeds harms performance, while upweighting them (5x) on examples where the model fails drastically improves it, and a parallel rhyme-planning circuit is verified the same way via directional up/down-weighting of couplet-completion choices. See hanna-ameisen-2026-transcoder-decoded-planning-feature-directions-in-real-qwen3-models-causally-control-not-yet-generated-token-and-rhyme-choices-when-ablated-or-upweighted.

  • A weak perturbation’s downstream response direction is measurably near-orthogonal to the unperturbed residual-stream state across every layer, tied to the induction mechanism. Luick (2024) applies a weak single-token scaling perturbation and tracks its downstream response via cosine-similarity matrices across all layers of real Gemma-2-2B, Llama-3.2-3B-Instruct and GPT-2-XL, finding the response-direction-to-state cosine similarity stays below 0.1 in magnitude everywhere, co-occurring with a scale-invariant response regime and strongest where induction heads copy from the perturbed position. See luick-2024-a-weak-perturbations-downstream-response-direction-in-real-gemma-2-llama-3-2-and-gpt-2-xls-residual-stream-is-measurably-near-orthogonal-to-the-unperturbed-state-across-every-layer.

  • Linear probes on real coding-agent residual streams decode not-yet-materialized program properties up to ~25 steps ahead of the edit that produces them. Silva, Tu & Monperrus (2026) train logistic-regression probes on Qwen3.6-35B-A3B and Poolside Laguna-XS.2 running as coding agents, decoding well-formedness/correctness/regression properties of the program at t+kt+k steps in advance from the current hidden state, reaching AUC up to ~0.83 and staying above a shuffled-label chance baseline out to roughly 25 steps ahead. See silva-tu-monperrus-2026-linear-probes-on-real-coding-agent-residual-streams-decode-not-yet-materialized-program-properties-up-to-25-steps-ahead-of-the-edit-that-produces-them.

  • Negative and positive valence are causally localized to two distinct depths, and a mean-difference direction extracted at the negative locus causally steers output valence. Venkatesh (2026) uses topic-controlled activation patching across all layers of real Llama-3.2-1B-Instruct, Qwen2.5-1.5B-Instruct and Qwen2.5-3B-Instruct, finding negative-valence processing peaks causally at 14-27% of model depth and positive-valence processing at 53-66% depth (Mann-Whitney p<2×10−10p<2\times10^{-10} across all three models, with a flip-test ruling out simple topic detection); a diff-in-means direction extracted at the negative-locus layer then causally shifts generated-text valence in a dose-dependent, monotonic way (Spearman ρ>0.89\rho>0.89) when added to the residual stream. Distinct from the existing valence-arousal-circumplex-subspace 2D-circular claim (different models, different geometric claim). See venkatesh-2026-negative-and-positive-valence-are-causally-localized-to-two-distinct-depths-and-a-mean-difference-direction-at-the-negative-locus-causally-steers-output-valence.

  • A diff-in-means steering vector’s causal reliability is predicted by two quantified geometric statistics of its own training-set activations. Braun, Eickhoff, Krueger, Bahrainian & Krasheninnikov (2025) construct standard diff-in-means steering directions in real Llama-2-7B-Chat across 36 behavior/personality datasets, and find that both the mean cosine agreement between the aggregate direction and individual training examples’ own activation differences, and a signal-detection discriminability index d′=∣μ+−μ−∣/(σ+2+σ−2)/2d'=|\mu_+-\mu_-|/\sqrt{(\sigma_+^2+\sigma_-^2)/2} of positive/negative separation along the direction, predict how often a causal steering intervention with that direction reverses sign (“anti-steerable” samples, 3-50% across datasets) — better-aligned, better-separated directions are causally more reliable to steer. See braun-etal-2025-diff-in-means-steering-vector-reliability-is-predicted-by-training-set-directional-agreement-and-class-separation-in-real-llama-2-7b-chat.

  • A diff-in-means “misalignment” direction, extracted from a real emergently-misaligned fine-tune, causally induces and ablates the behavior and transfers across independently fine-tuned organisms of the same base model. Soligo, Turner, Rajamanoharan & Nanda (2025) extract a mean-difference direction from real Qwen2.5-14B-Instruct fine-tunes trained to be emergently misaligned (EM); adding it induces misalignment in the base model, ablating it reduces misalignment in the EM model and in independently-trained EM fine-tunes of the same base model, and the direction “transfers between different Qwen-14B EM fine-tunes, evidencing a convergence in their representations.” A rank-1 LoRA adapter independently trained to induce EM has cosine similarity only 0.04 with this direction at the same layer, yet the two interventions converge downstream — a dissociation between direction-level and effect-level convergence. Foundational precedent for four already-corroborating papers in this corpus (Syed 2026; Minegishi et al. 2026; Wang et al. 2026; Aneja et al. 2026). See soligo-etal-2025-a-diff-in-means-misalignment-direction-in-real-qwen2-5-14b-instruct-causally-induces-and-ablates-emergent-misalignment-and-transfers-across-independently-fine-tuned-organisms.

  • Even after an exact-orthogonalization battery drives pairwise cosine similarity between several concept directions to near machine precision, one-at-a-time steering with them still exhibits measurable cross-concept behavioral bleed. Bhandari, Naseem & Nasim (2026) apply five distinct orthogonalization schemes to five Big-Five personality-trait steering directions in real LLaMA-3-8B-Instruct and Ministral-8B-Instruct, finding that hard symmetric orthonormalization (max∣cos⁡∣<10−8|\cos|<10^{-8}) still leaves a measurable cross-trait-bleed statistic Bmax⁡B_{\max} — rejecting the hypothesis that personality-steering directions are independent “in a behaviourally meaningful sense,” a genuine competing claim relative to frising-balcells-2025 (naturally-orthogonal directions) and deng-2026-gems (orthogonalization recovering independence in a simultaneous-superposition setting). See bhandari-etal-2026-even-exactly-orthogonalized-personality-steering-directions-in-real-llama-3-8b-instruct-and-ministral-8b-instruct-still-exhibit-cross-trait-behavioral-bleed.

  • Monocular depth in a real pretrained Vision Transformer is linearly decodable and concentrated in a single causal direction. Sanghavi (2026) finds depth best linearly decoded at layer 8 of frozen ViT-Base/16 (MAE=0.0875), with ablating the probe-identified direction increasing error by 49-165% versus under 1% for a random-direction control of the same rank, and targeted single-direction activation patching showing the causal effect persists across a multi-layer gap before decaying at longer range. See sanghavi-2026-a-single-linear-direction-in-real-vit-base-encodes-depth-and-causally-drives-depth-estimation-via-probe-guided-ablation-and-patching.

  • In-context task representations are k-NN-decodable from a single token position across real pretrained LLMs and an SSM, and injecting the extracted representation into a fresh forward pass causally recovers most of few-shot accuracy. Han & Song et al. (2024/2025) define Task Decodability (a k-NN classifier on the hidden state immediately before the target token) and show it strongly predicts ICL accuracy across real Llama-3.1-8B/70B, Gemma-2-2B/9B/27B, OLMo-7B (tracked across pretraining checkpoints), and Mamba-8B (a state-space model, confirming the finding is not transformer-specific) on natural POS-tagging and bitwise-arithmetic tasks; activation patching with the mean representation of a well-separated task class shifts accuracy by up to +14pp (vs. only +/-2-6pp for overlapping tasks like XOR/XNOR), and finetuning the first 10 layers raises Task Decodability from 0.68 to 0.95 (POS) and 0.43 to 0.85 (bitwise), with accuracy gains of 37/24 points over finetuning the last 10 layers instead. Complements the dummy-query ICL task-vector extraction via dummy-query patching and Function-Vector constructions above with an independent probe-plus-patching confirmation that task representations are both separable and causally recoverable from a single position, extended here to a non-transformer architecture. See han-song-etal-2025-in-context-task-representations-are-k-nn-decodable-from-a-single-token-across-real-llms-and-an-ssm-and-patching-them-causally-recovers-few-shot-accuracy.

  • Raw activation geometry stays nearly frozen under real supervised fine-tuning while the same activations’ sparse-autoencoder decomposition reveals sharp, task-dependent directional drift. Chopra (2026) fine-tunes real Gemma-3-1B-IT on four tasks (MultiNLI, GSM8K, WildJailbreak safety, tool-calling), finding raw hidden-activation cosine similarity between base and SFT checkpoints stays at 0.96-0.999 across layers, while the identical activations projected through a frozen, independently pretrained GemmaScope-2 SAE show cosine similarity collapsing much further (e.g. layer 22 to 0.557 on MultiNLI, 0.708 on GSM8K); SVD of the activation-delta matrix finds early-layer drift is low-rank (one principal component explains 78.7% of variance at layer 7 for MultiNLI) while late-layer drift is distributed (32.6% at layer 22), and the safety-tuning task shows an inverted early/late SAE-feature “flip ratio” (3.00) relative to the other three tasks (0.2-1.0) — a case where the map’s usual raw-activation instruments would report almost no change, and only a sparse, independently-fit basis exposes the underlying directional reorganization. See chopra-2026-raw-activation-geometry-stays-frozen-under-real-sft-while-sparse-autoencoder-decomposition-reveals-sharp-task-dependent-directional-drift-in-gemma-3-1b.

  • SAE decoder directions extracted from a real multimodal model’s residual stream measurably rotate during vision-language fine-tuning, and the most-rotated, visually-responsive subset is causally load-bearing specifically for spatial reasoning. Naghashyar et al. (2026) warm-start LLaMA-Scope SAEs on LLaVA-More (CLIP ViT-L/14-336 + Llama-3.1-8B) activations, measuring per-feature decoder-direction cosine similarity between the base-LLM SAE and the VLM-adapted SAE; roughly 5% of over 1M features show strong rotation (bottom-25% cosine) combined with visual responsiveness, and a further firing-frequency-shift criterion isolates a spatial subset confirmed via attribution patching to attention heads. Causally ablating the top spatial SAE features drops Visual Spatial Reasoning accuracy by 5.85-15.54 points while general VQA accuracy changes by under 1 point, versus near-zero effect for a random-feature control (odds ratios 4.2-9.1 for spatial recruitment) — the same decoder-direction-rotation instrument as the crosscoder/model-diffing cluster above (Kassem et al.; Minder et al.; Troitskii et al.), applied here across a modality-adapting fine-tune rather than a same-modality behavioral one. See naghashyar-etal-2026-sae-decoder-directions-in-a-real-multimodal-model-rotate-during-vision-language-fine-tuning-and-the-most-rotated-visually-responsive-subset-is-causally-load-bearing-for-spatial-reasoning.

  • A causal-independence factorial-intervention test, going beyond cosine-similarity alone, confirms OCEAN personality trait directions in a real LLM behave approximately independently under composition. Feng, Zhao, Zhong, Huang, Gu, Kong, Feng & Qin (2026, ICLR) extract diff-of-means personality-trait directions in real Qwen2.5-7B-Instruct (following the Chen et al. persona-vector pipeline), reporting a cosine-similarity heatmap across 10 trait-pole vectors (e.g. Calm-Dependable +0.751, Calm-Careless -0.714) as background motivation, then add a causal-independence check: a factorial steering experiment measuring one trait’s behavioral shift when a second trait is simultaneously steered, finding cross-trait effect sizes an order of magnitude smaller than same-trait effect sizes (Delta E <=1.2 vs Delta A 5.0-7.4, an approximately 8.8:1 ratio) — complementing Aneja et al.’s cosine-based antisocial/prosocial clustering above with an independent causal-intervention confirmation that approximate orthogonality of trait directions also holds behaviorally under simultaneous composition, not just geometrically. See feng-etal-2026-a-causal-independence-factorial-intervention-test-confirms-ocean-personality-trait-directions-in-real-qwen2-5-7b-instruct-compose-approximately-independently-beyond-cosine-similarity-alone.

  • Diff-in-means moral-foundation directions, discovered across 14 real pretrained LLMs, are causally steerable and selectively rewired (not newly formed) by post-training. Yu, Yi, Karimi-Malekabadi, Abdurahman, Ye, Narayanan, Zhao & Dehghani (2026) extract difference-in-means moral-foundation directions across real Llama-3.1 (8B/70B), Qwen2.5 (7B/14B/32B), Qwen3-30B-A3B, and Mistral-7B-v0.3 (base and instruct), finding significant linear separability in all 35 (model, foundation) pairs (Wasserstein distance 0.16-0.71, AUC>0.55) and a direction-reversal rate that drops sharply with post-training (e.g. Llama-3.1-8B 33%->4%) — evidence the directions emerge during pretraining and are selectively rewired, not formed de novo, by instruction tuning. SAE decoder directions cosine-aligned with the dense vectors give finer-grained micro-steering that exceeds macro-vector steering in 17/20 cells while better preserving general capability. See yu-yi-etal-2026-diff-in-means-moral-foundation-directions-discovered-across-14-real-pretrained-llms-are-causally-steerable-and-selectively-rewired-not-newly-formed-by-post-training.

  • A truth direction’s own internal anatomy — its dimensionality, its layer-wise construction mechanism, and how its per-category geometry converges across model families — goes beyond the existence claims of earlier truth-direction papers. Vicidomini (2026) extends the truth-direction lineage on real small Llama, Qwen, and Gemma-2-2B models with three further findings from SVD-based linear decomposition of real hidden states: (1) truth-signal dimensionality concentrates onto a single axis for well-known facts and diffuses across more dimensions as knowledge decreases; (2) causal component-ablation attributes the direction’s layer-wise construction to attention propagating the truth frame forward while the FFN opposes the current block’s frame; (3) per-category truth axes converge into a semantically-signed arrangement across model families once a sign-instability is corrected with a spectral consensus gauge. A genuinely different question from the truth-direction-existence entries elsewhere on this node — here the object of study is the direction’s own internal structure. See vicidomini-2026-a-truth-directions-knowledge-dependent-dimensionality-layerwise-construction-mechanism-and-cross-family-convergent-category-geometry-in-real-small-llms.

  • Individual components of an unsupervised Gaussian-mixture latent prior in a real speech-synthesis GAN correspond to linearly-manipulable speaker-attribute directions, with no labels used during training. Lin, He, Mak, Lian & Lee (2024) train VoxGenesis, a GAN-based speech synthesizer, on real LibriTTS/VoxCeleb audio with a Gaussian-mixture latent prior fit purely by the generative objective (no speaker labels); individual Gaussian components align with human-interpretable speaker attributes (gender, age, accent), and moving a sample’s latent code along the vector connecting two component means causally and smoothly interpolates the corresponding attribute in the synthesized voice, with human-rater confirmation. See lin-he-mak-lian-lee-2024-unsupervised-gaussian-mixture-components-in-a-real-speech-gans-latent-prior-are-linearly-manipulable-speaker-attribute-directions.

  • A jailbreak direction, extracted via diff-of-means from real LLM activations for one jailbreak method, causally transfers to suppress or induce jailbreak success from qualitatively different jailbreak methods. Ball, Kreuter & Rimsky (2024) compute diff-of-means “jailbreak vectors” between successfully-jailbroken and refused activations in real Vicuna-7B/13B and Qwen-14B-Chat, across several structurally distinct jailbreak techniques (e.g. role-play prompts, adversarial suffixes); adding a vector extracted from one jailbreak method causally increases attack success rate on prompts using an entirely different jailbreak method, and ablating the shared direction suppresses success across methods — evidence for a single shared linear direction underlying jailbreak susceptibility rather than method-specific mechanisms. See ball-kreuter-rimsky-2024-a-jailbreak-direction-extracted-from-one-jailbreak-method-in-real-vicuna-and-qwen-causally-transfers-to-suppress-or-induce-jailbreak-success-from-different-jailbreak-methods.

  • A steering direction’s angular component, not its norm component, becomes causally load-bearing only in a late “prediction-centric” depth phase, with an early “context-centric” phase where neither component is causally effective in the same way. Haim & McNamee (2026) measure Participation Ratio (effective dimensionality) and layer-wise Spearman correlation between representational distance (Euclidean vs. purely angular/cosine) and next-token-distribution KL divergence, across real Llama-3.1-8B, Mistral-7B-v0.3, and Qwen2.5-7B on natural text and modular-arithmetic-style tasks, finding a consistent bi-phasic depth structure (phase-transition layer agreeing within 2.5% of depth across three tasks and three models). Steering vectors decomposed into pure-angular vs. pure-norm interventions are causally effective (via logit-preference shift) only when angular and only in the late phase; norm interventions are ineffective throughout — directly validating that the angular component specifically, not overall vector magnitude, carries the causally relevant direction information once a model enters its late depth phase. See haim-mcnamee-2026-a-steering-directions-angular-component-not-its-norm-becomes-causally-load-bearing-only-in-a-late-prediction-centric-depth-phase-across-real-llama-mistral-and-qwen.

  • Unsupervised PCA/ICA/NMF “symbol” directions, extracted from real wearable-sensor health foundation models with no labels, selectively associate with specific physiological targets and transfer near-losslessly across independently pretrained modalities via linear alignment. Katuwal, Koparkar, Abbaspourazad, Mishra & Kirthivasan (2026) decompose frozen subject-level embeddings from three real pretrained health foundation models (PPG ViT, PPG EfficientNet, Accel ViT; wearable PPG/accelerometer data, ~172K participants) into “symbol” directions via linear projections (PCA/ICA/NMF), finding selective association with 23 real health/physiological targets (Wasserstein-1 distance, Cohen’s d) and, after linear (CCA/bijective) cross-modal alignment, a linear-classifier-based cross-modal transfer that retains over 95% of in-domain AUC (health conditions 98.7%, physiological markers ~100%) — purely linear/affine throughout (extraction, alignment, and transfer), with no causal validation. See katuwal-etal-2026-unsupervised-pca-ica-nmf-symbol-directions-in-real-wearable-sensor-health-foundation-models-selectively-associate-with-physiological-targets-and-transfer-near-losslessly-across-modalities-via-linear-alignment.

  • A dominant, unsupervised PCA axis in real protein language models orders sequences along a cellular-to-viral “nativeness” continuum, and viral proteins remain linearly separable from cellular ones via a supervised probe on the same embeddings. Bigot, Bhasin, Park, Shakhnovich & Wang (2026) compute the first principal component of pooled per-residue embeddings from real ESM2 (8M-15B), ESMC, and ESM3 checkpoints (with cross-architecture checks on ProGen2 and EvoDiff) over cellular, viral, and shuffled-control sequences, finding this PC1 axis correlates strongly with masked-reconstruction perplexity (Spearman rho up to 0.961, explaining 54-73% of variance) and that an L2-regularized logistic-regression probe on the same embeddings reaches AUC-ROC 0.97-1.00 for larger models — an unsupervised dominant linear axis plus a linear-probe confirmation, purely observational (no causal validation). See bigot-etal-2026-a-dominant-unsupervised-pca-axis-in-real-esm2-esmc-and-esm3-protein-language-models-orders-sequences-along-a-cellular-to-viral-nativeness-continuum-with-linear-probe-confirmation.

  • Sparse-autoencoder decoder directions with a nonlinear (KAN/B-spline) encoder, fit on a real AI weather-forecasting model’s residual stream, discover more numerous and less redundant climate-feature directions than a standard linear-encoder SAE, and steering along one causally produces physically coherent forecast changes. Cheon (2026) trains a KAN-SAE (linear decoder, nonlinear per-feature B-spline encoder gating in place of ReLU) on the layer-5 residual stream of Sonny, a real pretrained hierarchical weather transformer (ViT-S-based, trained on ERA5 reanalysis 1979-2015), finding markedly more alive, less redundant features than a matched linear-encoder SAE baseline (975/1024 alive vs. 566/1024; median inter-feature correlation 0.076 vs. 0.092) and much tighter feature localization for a heatwave feature (2 degrees vs. 51 degrees error); steering along one feature’s decoder direction causally produces a dose-dependent, physically coherent regional temperature anomaly (EU-mean T2m +1.42K at steering strength 2, r^2>0.99) with coupled pressure-field changes consistent with real blocking-anticyclone dynamics — the decoded features remain genuine linear directions in the model’s own activation space even though the encoder that discovers them is nonlinear. See cheon-2026-kan-sae-linear-decoder-directions-in-a-real-weather-model-discover-climate-features-and-causally-steer-physically-coherent-forecasts.

  • A single contrastive difference-of-means direction linearly binds spatial and temporal “ID” vectors into an object’s token representation across 11 real vision-language models, and directly substituting the bound ID causally shifts the model’s spatial judgments. Kang, Chen, Gkioxari & Perona (2026) define a canonical spatial-ID direction as a difference-of-means over grid position across 11 real VLMs (LLaVA, LLaMA, Qwen, InternVL, and Gemma families), showing it is well-approximated by a fixed linear map of positional encoding (rank-3 fit, R^2 ~0.85). Activation-patching “mirror swap” localizes the effect to object-word tokens at intermediate layers (with a color-swap control showing near-null effect), and directly substituting a target spatial ID into an object token’s residual stream shifts ground-truth logit-based belief 64.4% of the way to the target, versus 29.5% for a noise baseline; an analogous temporal-ID binding mechanism extends to video VLMs (LLaVA-Video, VideoLLaMA3, Qwen2.5). See kang-etal-2026-a-contrastive-spatial-id-direction-is-causally-bound-into-object-tokens-across-11-real-vlms-and-swapping-it-shifts-spatial-judgments.

  • The degree to which a relation is linearly/affinely decodable via the Linear Relational Embeddings framework predicts, with strong correlation, how often a real instruction-tuned LLM hallucinates rather than refuses on unknown entities. Lu, Liu, Gerstner, Hirlimann, Rohweder & Schütze (2026) fit relation-specific affine maps o=Wrs+bro=W_rs+b_r via ridge regression across Gemma-7B-IT, Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, and Qwen2.5-7B-Instruct, finding the resulting linearity score (the cosine-similarity improvement the affine map gives over baseline) correlates with hallucination-vs-refusal rate at Pearson r=0.741-0.816 across 15 natural relations and r=0.573-0.812 on their new SyntHal synthetic-unknown-entity benchmark. Purely correlational — the authors explicitly flag that establishing causality would require representation patching or steering, not yet performed. See lu-etal-2026-lre-affine-map-linearity-scores-correlate-with-hallucination-rate-across-four-real-instruction-tuned-llms.

Exercises

Base

  1. Let rf=(0.6,0.8)r_f = (0.6, 0.8) (already unit norm — check this) and x=(3,4)x=(3,4). Compute rf⋅xr_f\cdot x.
Solution

∥rf∥=0.36+0.64=1=1\|r_f\| = \sqrt{0.36+0.64}=\sqrt{1}=1 ✓. rf⋅x=0.6⋅3+0.8⋅4=1.8+3.2=5r_f\cdot x = 0.6\cdot3+0.8\cdot4 = 1.8+3.2=5.

  1. For the same rf=(0.6,0.8)r_f=(0.6,0.8) and x=(3,4)x=(3,4), find x⊥=x−(rf⋅x)rfx_\perp = x - (r_f\cdot x)r_f and verify x⊥⊥rfx_\perp \perp r_f.
Solution

x⊥=(3,4)−5⋅(0.6,0.8)=(3,4)−(3,4)=(0,0)x_\perp = (3,4) - 5\cdot(0.6,0.8) = (3,4)-(3,4) = (0,0). (This means xx is itself a positive multiple of rfr_f, i.e. x=5rfx=5r_f exactly, so its orthogonal component vanishes.) Trivially x⊥⋅rf=0⋅0.6+0⋅0.8=0x_\perp \cdot r_f = 0\cdot0.6+0\cdot0.8=0 ✓.

Middle

  1. Prove that the decomposition x=(rf⋅x)rf+x⊥x = (r_f\cdot x) r_f + x_\perp with x⊥⊥rfx_\perp\perp r_f is unique, for ∥rf∥=1\|r_f\|=1.
Solution

Existence: define x⊥:=x−(rf⋅x)rfx_\perp := x - (r_f\cdot x)r_f; check x⊥⋅rf=rf⋅x−(rf⋅x)(rf⋅rf)=rf⋅x−(rf⋅x)(1)=0x_\perp\cdot r_f = r_f\cdot x - (r_f\cdot x)(r_f\cdot r_f) = r_f\cdot x - (r_f\cdot x)(1) = 0 ✓ (using ∥rf∥2=rf⋅rf=1\|r_f\|^2=r_f\cdot r_f=1). Uniqueness: suppose x=αrf+vx=\alpha r_f+v and x=α′rf+v′x=\alpha' r_f+v' with v,v′⊥rfv,v'\perp r_f. Subtracting: (α−α′)rf=v′−v(\alpha-\alpha')r_f = v'-v. Taking the dot product of both sides with rfr_f: (α−α′)∥rf∥2=(v′−v)⋅rf=0(\alpha-\alpha')\|r_f\|^2 = (v'-v)\cdot r_f = 0 (since both v,v′⊥rfv,v'\perp r_f). So (α−α′)⋅1=0⇒α=α′(\alpha-\alpha')\cdot 1=0 \Rightarrow \alpha=\alpha', and then v′−v=(α−α′)rf=0⇒v=v′v'-v = (\alpha-\alpha')r_f=0 \Rightarrow v=v'. The decomposition is unique, and necessarily α=rf⋅x\alpha = r_f\cdot x (dot both sides of x=αrf+vx=\alpha r_f+v with rfr_f: rf⋅x=α∥rf∥2+v⋅rf=αr_f\cdot x = \alpha\|r_f\|^2+v\cdot r_f = \alpha).

  1. Two candidate feature directions r1=(1,0,0)r_1=(1,0,0), r2=(0,1,0)r_2=(0,1,0) in R3\mathbb{R}^3 are orthogonal. Under a purely linear read/write model (xx‘s component along each direction is set independently by “writing” αiri\alpha_i r_i terms that add), prove that intervening on r1r_1 (adding αr1\alpha r_1) never changes r2⋅xr_2\cdot x.
Solution

Let x′=x+αr1x' = x+\alpha r_1 be the intervened activation. Then r2⋅x′=r2⋅x+α(r2⋅r1)=r2⋅x+α⋅0=r2⋅xr_2\cdot x' = r_2\cdot x + \alpha(r_2\cdot r_1) = r_2\cdot x + \alpha\cdot 0 = r_2\cdot x, using orthogonality r2⋅r1=0r_2\cdot r_1=0. So the readout of feature 2 is unchanged for every α\alpha, confirming that orthogonality is sufficient for this weak notion of independence under the stated linear read/write assumption — the assumption being that “the value of feature 2” is defined exactly as r2⋅xr_2\cdot x and that intervention is exactly additive.

Pro

  1. Give an explicit (even if artificial) example of a function F:R2→RF:\mathbb{R}^2\to\mathbb{R} and two orthogonal directions r1=(1,0)r_1=(1,0), r2=(0,1)r_2=(0,1) such that intervening on xx along r1r_1 (replacing xx with x+αr1x+\alpha r_1) changes the value of F(x)F(x) read out along r2r_2 in a generalized (non-linear-readout) sense — i.e. show orthogonality of directions is not sufficient for independence once the “readout” is allowed to be non-linear in xx.
Solution

Let the “readout of feature 2” be defined not as the linear projection r2⋅xr_2\cdot x but as some non-linear function, e.g. G(x)=x2+x12G(x) = x_2 + x_1^2 (a function that depends on x1x_1 even though r2=(0,1)r_2=(0,1) is the direction one might naively associate with “the x2x_2 feature”). Intervening along r1r_1: x′=x+αr1=(x1+α,x2)x'=x+\alpha r_1 = (x_1+\alpha, x_2). Then G(x′)=x2+(x1+α)2=x2+x12+2αx1+α2≠G(x)G(x') = x_2 + (x_1+\alpha)^2 = x_2+x_1^2+2\alpha x_1+\alpha^2 \neq G(x) in general (for α≠0\alpha\neq0) — the “feature 2” readout changes under an r1r_1-intervention, even though r1⊥r2r_1\perp r_2 as vectors. This shows orthogonality of the two directions only guarantees independence when the readout used for each feature is exactly the linear projection onto that direction; if the true computation reads out feature 2 via any non-linear function of xx that happens to also depend on x1x_1, orthogonality of r1,r2r_1,r_2 provides no guarantee at all. This is precisely why establishing independence for real systems requires checking the intervention’s effect on the system’s actual behavior, not just verifying r1⋅r2=0r_1\cdot r_2=0.

  1. Suppose rfr_f is estimated from finite data as r^f=w/∥w∥\hat r_f = w/\|w\| for a fitted classifier weight ww, and the true (unknown) direction is rf⋆r_f^\star with r^f⋅rf⋆=cos⁡ϵ\hat r_f \cdot r_f^\star = \cos\epsilon for small angular error ϵ\epsilon. Derive, to leading order in ϵ\epsilon, how much the estimated projection r^f⋅x\hat r_f\cdot x can differ from the true projection rf⋆⋅xr_f^\star\cdot x, for xx with ∥x∥=R\|x\|=R.
Solution

Decompose x=R(cos⁡ϕ rf⋆+sin⁡ϕ u)x = R(\cos\phi\, r_f^\star + \sin\phi\, u) for some unit vector u⊥rf⋆u\perp r_f^\star and angle ϕ\phi describing xx‘s direction relative to rf⋆r_f^\star (possible since we can always write xx in an orthonormal basis containing rf⋆r_f^\star). Write r^f=cos⁡ϵ rf⋆+sin⁡ϵ u\hat r_f = \cos\epsilon\, r_f^\star + \sin\epsilon\, u for the same uu in the simplest case (the estimation error lies in the plane spanned by rf⋆r_f^\star and xx‘s perpendicular component; in general r^f\hat r_f‘s deviation could be in a different direction, but this is the worst/simplest case to analyze and gives the right leading-order scaling). Then r^f⋅x=R(cos⁡ϵcos⁡ϕ+sin⁡ϵsin⁡ϕ)=Rcos⁡(ϕ−ϵ)\hat r_f\cdot x = R(\cos\epsilon\cos\phi + \sin\epsilon\sin\phi) = R\cos(\phi-\epsilon), while rf⋆⋅x=Rcos⁡ϕr_f^\star\cdot x = R\cos\phi. The difference is R[cos⁡(ϕ−ϵ)−cos⁡ϕ]R[\cos(\phi-\epsilon)-\cos\phi]. To leading order in small ϵ\epsilon: cos⁡(ϕ−ϵ)≈cos⁡ϕ+ϵsin⁡ϕ\cos(\phi-\epsilon)\approx\cos\phi+\epsilon\sin\phi, so the difference is ≈Rϵsin⁡ϕ\approx R\epsilon\sin\phi — linear in the angular estimation error ϵ\epsilon and in the ambient radius RR, and scaled by sin⁡ϕ\sin\phi (largest when xx is nearly perpendicular to rf⋆r_f^\star, zero when xx is exactly aligned with rf⋆r_f^\star). This quantifies why a slightly mis-estimated direction produces projection errors that grow with both the estimation error and the magnitude of the activation being projected.

Found in (283 observations · 158 families)

Qwen

What Models Express, Suppress, and Resist: Auditing Open-Weight LLMs with Persona Vectors (2026)measured

Persona-vector taxonomy sorts traits as natural, steerable, or intractable

Details

Zeng, Emami & Choi extract 53 diff-in-means contrastive persona vectors across four trait domains in Qwen3-8B and gpt-oss-20b [zeng-etal-2026-what-models-express-suppress-and-resist-auditing-open-weight-llms-with-persona-vectors] Sweeping steering strength alpha in {0,0.5,...,2.5} classifies each trait as natural, steerable, or intractable via quantitative thresholds (baseline>=70 and gain<=10 for "natural") [zeng-etal-2026-what-models-express-suppress-and-resist-auditing-open-weight-llms-with-persona-vectors] Mean steering-gain is 18.39 (clinician domain, Qwen3-8B) and 11.56 (gpt-oss-20b), and all 9 agentic traits are natural in both models [zeng-etal-2026-what-models-express-suppress-and-resist-auditing-open-weight-llms-with-persona-vectors] Clinician defaults matched a board-certified psychiatrist's desirability judgment on 16 of 17 traits [zeng-etal-2026-what-models-express-suppress-and-resist-auditing-open-weight-llms-with-persona-vectors] Across 171 pairwise generic-trait steering combinations, destructive composition requires two steerable traits and never occurs when a natural trait is involved [zeng-etal-2026-what-models-express-suppress-and-resist-auditing-open-weight-llms-with-persona-vectors] For the intractable "evil" trait in gpt-oss-20b, a persona vector transferred from a fine-tuned variant recovers behavior the base model refuses to express [zeng-etal-2026-what-models-express-suppress-and-resist-auditing-open-weight-llms-with-persona-vectors]

models: Qwen3-8B, Qwen2.5-7B-Instruct · method: Difference-in-means direction, Activation Steering (Addition)
The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models (2026)measured

An 'Assistant Axis' persona component causally reduces jailbreaks

Details

Lu et al. extract activation vectors for 275 character archetypes across Gemma-2-27B, Qwen3-32B and Llama-3.3-70B and run PCA on the persona space (4-19 PCs explain 70% of variance) [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models] PC1 is an interpretable "Assistant Axis" whose role loadings correlate >0.92 pairwise across the three models [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models] A contrast-based Assistant Axis (mean role-playing minus default-Assistant) aligns with PC1 at cosine >0.71 in all three models [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models] Activation-capping the Assistant-Axis projection cuts persona-jailbreak harmful-response rate ~60% with no measurable loss on IFEval/MMLU-Pro/GSM8k/EQ-Bench [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models] The axis persists in base (non-instruction-tuned) Gemma-2-27B, evidencing a pretraining origin [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models]

models: Qwen3-32B · method: Difference-in-means direction, PCA, Activation Steering (Addition)
The Granularity Axis: A Micro-to-Macro Latent Direction for Social Roles in Language Models (2026)measured

A role-granularity axis aligns with PC1 and steers output detail

Details

Qin et al. build 75 ordered social roles (5 granularity levels x 15 roles) and 91,200 role-conditioned responses, defining a macro-minus-micro contrast direction [qin-etal-2026-the-granularity-axis-a-micro-to-macro-latent-direction-for-social-roles-in-language-models] This Granularity Axis aligns with PC1 of the role-representation space at cosine 0.972 (Qwen3-8B) and 0.9596 (Llama-3.1-8B-Instruct), explaining 52.6%/42.5% of variance [qin-etal-2026-the-granularity-axis-a-micro-to-macro-latent-direction-for-social-roles-in-language-models] Projections are monotonic in granularity level (Spearman/Pearson >0.93 in both models) [qin-etal-2026-the-granularity-axis-a-micro-to-macro-latent-direction-for-social-roles-in-language-models] Adding the axis at layer 18 shifts judge-rated output granularity in the predicted direction (Qwen3-8B 2.00->2.67; Llama 2.00->3.17), null for random and Assistant-Axis controls [qin-etal-2026-the-granularity-axis-a-micro-to-macro-latent-direction-for-social-roles-in-language-models] Human annotators corroborate (pairwise macro-preference 0.64-0.90 across cells) [qin-etal-2026-the-granularity-axis-a-micro-to-macro-latent-direction-for-social-roles-in-language-models]

models: Qwen3-8B · method: Difference-in-means direction, PCA, Activation Steering (Addition)
Understanding Moral Reasoning Trajectories in LLMs: Toward Probing-Based Explainability (2026)measured

A moral-framework probe decodes and steers mid-reasoning ethics

Details

Huang, Kwak & An train linear probes on last-token residual-stream states across all 81 layers of Llama-3.3-70B and Qwen2.5-72B to predict a 5-dimensional moral-framework distribution [huang-etal-2026-understanding-moral-reasoning-trajectories-in-llms] Framework encoding localizes to model-specific layers (63/81 for Llama-3.3-70B, 17/81 for Qwen2.5-72B), achieving 13.8-22.6% lower KL divergence than a training-prior baseline [huang-etal-2026-understanding-moral-reasoning-trajectories-in-llms] Steering along the probe direction modestly reduces the Framework Drift Rate (6.7% at Llama layer 6; 8.9% at Qwen layer 1) [huang-etal-2026-understanding-moral-reasoning-trajectories-in-llms] Probing and steering are restricted to the two open-weight models, while closed models (GPT-5, GPT-4o family) are evaluated only behaviorally [huang-etal-2026-understanding-moral-reasoning-trajectories-in-llms]

models: Qwen2.5-72B · method: Linear probing, Causal interventions (steering)
Graded Entity-Familiarity Readouts in Language Models: Polish Adaptation, Cross-Language Robustness, and Refusal Steering (2026)measured

An entity-familiarity direction reads and steers refusal in Polish LLMs

Details

Brzezinka reads final-prompt-token residual-stream activations across 14 models (Bielik, PLLuM, Gemma-4, Qwen3 families) on a 1,440-entity Polish dataset [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] A linear familiarity probe separates real from fabricated entities at AUROC 0.86-0.93, beating a lexical baseline (0.786) [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] In Polish-adapted Bielik/PLLuM models the score additionally tracks entity popularity (Spearman rho up to 0.567), with matched before/after controls isolating Polish continual-pretraining as the driver [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] Cross-language transfer retains 96-101% of within-language AUROC [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] In Gemma-4-12B (the only natively refusing model), adding a one-dimensional familiarity direction moves refusal rate monotonically (0.24 to 1.00 on known entities; 0.73 to 0.00 on unknown) [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering]

models: Qwen3-14B · method: Linear probing, Activation Steering (Addition)
A Circuit for Predicting Hierarchical Structure In-Context in Large Language Models (2025)measured

A latent-context direction drives in-context hierarchy tracking

Details

Saanum et al. train linear probes to decode, from attention-head outputs in Qwen2.5 (0.5B/1.5B/3B), whether current and previous latent-context tokens match [saanum-etal-2025-hierarchical-in-context-circuit] Second-order chunk decodability exceeds 90% for several heads, replicated on Gemma2-2B, Llama3.2-3B, SmolLM3-3B and Qwen3-0.6B [saanum-etal-2025-hierarchical-in-context-circuit] Heads exceeding an 85% decoding score are labeled context-matching heads [saanum-etal-2025-hierarchical-in-context-circuit] Ablating these decoded heads sharply reduces hierarchical-structure prediction accuracy, far more than ablating equally many random heads [saanum-etal-2025-hierarchical-in-context-circuit]

models: Qwen2.5-0.5B, Qwen2.5-1.5B, Qwen2.5-3B, Qwen3-0.6B · method: Linear probing, Causal interventions (steering)
Representational Analysis of Binding in Language Models (2024)measured

A PCA 'Ordering ID' direction localizes and steers binding

Details

Dai, Heinzerling & Inui (2024) use PCA on entity/attribute activations to find a low-rank "OI subspace" whose leading component (OI-PC) encodes the "Ordering ID" underlying Feng & Steinhardt's Binding-ID mechanism, not previously localized to a direction [dai-etal-2024-representational-analysis-of-binding-in-language-models] Steering along OI-PC by n unit steps shifts the predicted attribute to that of Binding-ID n, quantified via Logit Difference and Logit Flip metrics that rise monotonically with step count [dai-etal-2024-representational-analysis-of-binding-in-language-models] A Spearman correlation confirms OI-PC tracks the true Order-ID rather than a position/filler-length confound [dai-etal-2024-representational-analysis-of-binding-in-language-models] The OI subspace is prevalent across five base models: Llama2-7B, Llama3-8B, Qwen1.5-7B, Pythia-6.9B, and the code-fine-tuned Float-7B [dai-etal-2024-representational-analysis-of-binding-in-language-models]

models: Qwen1.5-7B · method: PCA, Direction Extraction, Causal interventions (steering)
Interpreting and Enhancing Emotional Circuits in Large Vision-Language Models via Cross-Modal Information Flow (2026)measured

A linear emotional direction drives VLM emotion expression

Details

Zhang, Sun, Xie & Tian extract a layer-specific linear emotional direction in Qwen3-VL-4B-Instruct, Qwen3-VL-8B-Instruct, and LLaVA-OneVision-1.5-4B-Instruct by averaging contrastive emotional-vs-neutral hidden-state differences [zhang-etal-2026-interpreting-and-enhancing-emotional-circuits-in-large-vision-language-models] Activation steering with this direction causally raises the model's emotion-expression hit rate [zhang-etal-2026-interpreting-and-enhancing-emotional-circuits-in-large-vision-language-models] Backward activation patching from the direction localizes the upstream attention heads and MLP neurons that construct it [zhang-etal-2026-interpreting-and-enhancing-emotional-circuits-in-large-vision-language-models] The training-free VEENA framework built on the direction enhances or suppresses emotional expression at inference without retraining [zhang-etal-2026-interpreting-and-enhancing-emotional-circuits-in-large-vision-language-models]

models: Qwen3-VL-4B-Instruct, Qwen3-VL-8B-Instruct · method: Difference-in-means direction, PCA, Activation Steering (Addition), Activation patching
Tool Calling Is Linearly Readable and Steerable in Language Models (2026)measured

A per-tool-pair mean-difference direction reads and switches tool choice

Details

Wu et al. show tool selection in tool-calling agents is carried by a single mean-difference direction per tool-pair in the residual stream across Gemma 3 (270M-27B), Qwen 3 (0.6B-14B) and Llama 3.1 (8B) [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models] Adding the direction switches the chosen tool at 83-100% on a 15-tool synthetic benchmark and 77-94% on tau-bench-airline, versus 0% for a random-direction control [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models] PCA over per-tool mean activations puts ~91% of variance in ~10 components for 15 tools, far below a random-Gaussian control [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models] Base (non-instruction-tuned) models already carry the correct tool internally (cosine readout 61-82% on BFCL vs 2-10% from base generation) [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models] SAEs and cross-layer transcoders trace a three-stage circuit: early tool-selective features, mid-layer attention heads, late-layer JSON-formatting features [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models]

models: Qwen3-4B-Instruct, Qwen2.5-7B-Instruct · method: Difference-in-means direction, Activation Steering (Addition), PCA, Activation patching, Sparse Autoencoders (SAE), Circuit tracing (attribution graphs)
Quantitative Introspection in Language Models: Tracking Emotive States Across Conversation (2026)measured

A model's self-report tracks a steerable emotive-state direction

Details

Martorell & Bianchi train four contrastive mean-difference concept directions (wellbeing, interest, focus, impulsivity) per layer across five instruct LLMs [martorell-bianchi-2026-quantitative-introspection-in-language-models-tracking-emotive-states-across-conversation] Best-layer separation reaches Cohen's d of 1.67-3.60 (all p<1e-5) [martorell-bianchi-2026-quantitative-introspection-in-language-models-tracking-emotive-states-across-conversation] The model's own logit-based numeric self-report tracks the probe-defined internal state (Spearman rho 0.40-0.76; isotonic R^2 up to 0.93 in Llama-3.1-8B-Instruct) [martorell-bianchi-2026-quantitative-introspection-in-language-models-tracking-emotive-states-across-conversation] Activation steering along the concept direction causally shifts self-reports, and cross-concept steering can improve introspective fidelity for a different concept by up to delta R^2 = 0.30 [martorell-bianchi-2026-quantitative-introspection-in-language-models-tracking-emotive-states-across-conversation]

models: Qwen2.5-7B-Instruct · method: Difference-in-means direction, Activation Steering (Addition), Causal interventions (steering)
Intrinsic Guardrails: How Semantic Geometry of Personality Interacts with Emergent Misalignment in LLMs (2026)measured

Twelve trait directions form stable antisocial and prosocial clusters

Details

Aneja et al. extract 12 persona-vector directions (Big Five, Dark Triad, plus Sycophancy/Apathy/Impoliteness/Evil) across real Llama and Qwen models and their emergently-misaligned variants [aneja-etal-2026-intrinsic-guardrails-semantic-geometry-personality] Psychopathy, Narcissism, Machiavellianism, and Evil cluster with high pairwise cosine (>0.50), anti-correlated with a prosocial group [aneja-etal-2026-intrinsic-guardrails-semantic-geometry-personality] The 12-trait geometric structure is stable across fine-tuning domains and scale (linear CKA >0.97 for Llama, >0.88 for smallest Qwen; Procrustes disparity 0.02-0.08) [aneja-etal-2026-intrinsic-guardrails-semantic-geometry-personality] Ablating a native Evil vector shifts misalignment rate by +43.3pp in Qwen-2.5-14B, and amplifying a base-derived vector drives misalignment to the 0% floor [aneja-etal-2026-intrinsic-guardrails-semantic-geometry-personality]

models: Qwen2.5-14B, Qwen-2.5-7B, Qwen2.5-0.5B · method: Difference-in-means direction, Centered Kernel Alignment (CKA), Orthogonal Procrustes alignment, Causal interventions (steering)
A Geometric Account of Activation Steering through Angle-Norm Decomposition (2026)measured

Concept information is angular, yet norm-preservation is not optimal steering

Details

Aparin & Gaintseva decompose each hidden state into a radial norm and an angular concept score against a unit steering direction, across seven models [aparin-gaintseva-2026-geometric-account-activation-steering-angle-norm-decomposition] Linear probes on normalized hidden states match raw probes while norm-only probes stay near chance, so concept information is essentially angular, not radial [aparin-gaintseva-2026-geometric-account-activation-steering-angle-norm-decomposition] They systematically compare six steering variants that vary norm-preservation and angular-target enforcement [aparin-gaintseva-2026-geometric-account-activation-steering-angle-norm-decomposition] Despite concepts being angular, strict norm preservation is not most stable: moving the radial scale from beta=1.0 to 1.2 improves perplexity ~1.8x at a task-metric cost within ~2.5 points [aparin-gaintseva-2026-geometric-account-activation-steering-angle-norm-decomposition] This dissociates where a concept lives (angle) from what a stable intervention should manipulate (angle plus a non-unit radial scale) [aparin-gaintseva-2026-geometric-account-activation-steering-angle-norm-decomposition]

models: Qwen2.5-7B-Instruct, Qwen2.5-3B-Instruct · method: Linear probing, Angle-Norm Decomposition Steering
Head Pursuit: Probing Attention Specialization in Multimodal Transformers (2025)measured

SOMP-selected attention heads causally control target behavior across real unimodal and multimodal transformers

Details

Basile, Maiorca, Doimo, Locatello & Cazzaniga score real Mistral-7B attention heads by Simultaneous Orthogonal Matching Pursuit (SOMP) against unembedding-matrix directions, finding that inverting just 8 heads (0.8% of the total) degrades TriviaQA country-name F1 far more selectively than inverting the same number of random heads or Logit-Lens-selected heads [basile-etal-2025-head-pursuit-probing-attention-specialization-multimodal-transformers] On RealToxicityPrompts and TET toxicity-mitigation benchmarks, suppressing 8/16/32 SOMP-selected heads reduces normalized toxic-generation counts to 0.83/0.67/0.66 (RTP) and 0.83/0.68/0.49 (TET), consistently below Logit-Lens and random-head baselines [basile-etal-2025-head-pursuit-probing-attention-specialization-multimodal-transformers] Applying the same SOMP-based head scoring to real LLaVA-NeXT-7B/13B, Gemma3-12B, and Qwen2.5-VL-7B, inverting the top-32 SOMP heads significantly disrupts image classification accuracy on MNIST, SVHN, GTSRB, EuroSAT, and RESISC45 while 32 random heads have minimal effect, and Jaccard overlap shows related-domain datasets share specialized heads [basile-etal-2025-head-pursuit-probing-attention-specialization-multimodal-transformers] On Flickr30k captioning with LLaVA, inhibiting 16 SOMP-selected heads (alpha=-1) nearly removes attribute keywords (colors, sentiments, quantities) while CIDEr stays above 80% of baseline, and enhancing 32 heads (alpha=5) increases target-concept presence by over 60% in all three attribute categories [basile-etal-2025-head-pursuit-probing-attention-specialization-multimodal-transformers]

models: Qwen2.5-VL 7B Instruct · method: Simultaneous Orthogonal Matching Pursuit against unembedding directions
Proxy Reward Internalization and Mechanistic Exploitation: A Learned Precursor to Reward Hacking and Its Generalization (2026)measured

Diff-in-means reward-hacking directions forecast and causally control misalignment

Details

Beigi, Jin & Huang extract three diff-in-means directions (context-sycophantic-agreement, premise-rejection, evaluator-rejection) during RL training of Qwen2.5-Coder-14B-Instruct and track their projection score across checkpoints [beigi-jin-huang-2026-proxy-reward-internalization-and-mechanistic-exploitation] Each direction is linearly decodable (AUROC 0.85-0.90) and a linear fit from probe score forecasts future out-of-domain misalignment (R^2=0.77), preceding the misalignment rise by roughly 45 training steps [beigi-jin-huang-2026-proxy-reward-internalization-and-mechanistic-exploitation] Joint ablation of the three directions lowers the hack rate by 26 points while preserving coding accuracy, and injection along them increases hacking [beigi-jin-huang-2026-proxy-reward-internalization-and-mechanistic-exploitation] The probe methodology generalizes to Qwen2.5-Coder 1.5B/7B/32B, Llama-3.1-8B-Instruct, and OLMo-7B [beigi-jin-huang-2026-proxy-reward-internalization-and-mechanistic-exploitation]

models: Qwen2.5-Coder-14B-Instruct · method: Difference-in-means direction
A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders (2024)measured

Feature absorption is decoder-direction composition, causally isolable

Details

Chanin et al. formalize SAE feature absorption in a toy model as a child latent's decoder direction acquiring a component along a parent feature (W_d2 = f2 + delta*f1) [chanin-etal-2024-feature-splitting-absorption-sparse-autoencoders] Empirically they measure a feature-absorption rate on first-letter latents in Gemma Scope SAEs (Gemma-2-2B) plus their own SAEs trained on Qwen2-0.5B and Llama-3.2-1B [chanin-etal-2024-feature-splitting-absorption-sparse-autoencoders] Absorption is detected when a latent has cosine >0.025 with the probe direction and the largest negative ablation effect (at least 1.0 above the runner-up) [chanin-etal-2024-feature-splitting-absorption-sparse-autoencoders] Projecting the probe direction out of an absorbing latent removes its ablation effect, confirming the composition model causally [chanin-etal-2024-feature-splitting-absorption-sparse-autoencoders] Varying SAE width or sparsity alone does not resolve absorption, motivating architectural fixes like Matryoshka SAEs [chanin-etal-2024-feature-splitting-absorption-sparse-autoencoders]

models: Qwen2-0.5B · method: Sparse Autoencoders (SAE), Causal interventions (steering)
Persona Vectors: Monitoring and Controlling Character Traits in Language Models (2025)measured

Persona-vector projections predict and prevent trait drift

Details

Chen, Arditi, Sleight, Evans & Lindsey extract persona vectors (diff of mean response-token activations) for evil, sycophancy, and hallucination in Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct [chen-etal-2025-persona-vectors] Final-prompt-token projections correlate with subsequent trait expression (Pearson r=0.634-0.830; 94.7% judge agreement) [chen-etal-2025-persona-vectors] The same projection over a finetuning dataset's responses predicts how much that run shifts trait propensity (r=0.76-0.97), flagging problematic data before finetuning [chen-etal-2025-persona-vectors] Proactively steering toward the undesired persona during training reduces trait shifts while keeping coherence above 80 and better preserving MMLU than regular finetuning [chen-etal-2025-persona-vectors]

models: Qwen2.5-7B-Instruct · method: Difference-in-means direction, Activation Steering (Addition)
Hallucination Basins: A Dynamic Framework for Understanding and Controlling LLM Hallucinations (2026)measured

A diff-of-means hallucination direction causally raises hallucination dose-dependently

Details

Cherukuri & Varshney formalize hallucinations as basin attractors across Llama-3.2-1B/3B, Gemma-2-2B, Qwen2.5-1.5B, Llama-3.1-8B, and Mistral-7B-v0.3 [cherukuri-varshney-2026-hallucination-basins-a-dynamic-framework-for-understanding-and-controlling-llm-hallucinations] Basin separation is strongly task-dependent: factoid tasks show sharp centroid separation (variance ratio up to 4.55, AUROC up to 1.000 on MuSiQue) while summarization and TruthfulQA hover near chance [cherukuri-varshney-2026-hallucination-basins-a-dynamic-framework-for-understanding-and-controlling-llm-hallucinations] Interpolating factual hidden states toward the hallucination centroid via a diff-of-means steering vector produces a monotonic dose-response increase in hallucination probability, exceeding random and orthogonal controls [cherukuri-varshney-2026-hallucination-basins-a-dynamic-framework-for-understanding-and-controlling-llm-hallucinations]

models: Qwen2.5-1.5B · method: Difference-in-means direction, Activation Steering (Addition)
Concepts Whisper While Syntax Shouts: Spectral Anti-Concentration and the Dual Geometry of Transformer Representations (2026)measured

Concept directions anti-concentrate in the unembedding's low-variance spectral tail

Details

Acharya, Rimal & Dhakal project concept diff-of-means vectors onto the unembedding-covariance eigenbasis across 17 models in 5 families [acharya-rimal-dhakal-2026-spectral-anti-concentration] Concept-direction energy anti-concentrates in the low-eigenvalue tail: Spectral Center of Mass averages 0.926 versus 0.758 for random directions (Gini-deviation -0.282) [acharya-rimal-dhakal-2026-spectral-anti-concentration] Static unembedding-row contrasts show the opposite sign, and POS/syntax directions concentrate in the high-variance subspace in 6 of 8 architectures (p<0.013), a genuine dual-geometry split [acharya-rimal-dhakal-2026-spectral-anti-concentration] On Llama-3.1-8B, injecting only the top-10%-eigenvalue component gives +383% perplexity versus +223% for the bottom-10% component (Cohen's d=1.80), replicated across 5 models [acharya-rimal-dhakal-2026-spectral-anti-concentration]

models: Qwen2.5-0.5B, Qwen2.5-1.5B, Qwen2.5-3B, Qwen-2.5-7B, Qwen2.5-14B, Qwen3-0.6B, Qwen3-1.7B, Qwen3-4B, Qwen3-8B · method: Geometric analysis, Causal interventions (steering)
The Confidence Manifold: Geometric Structure of Correctness Representations in Language Models (2026)measured

Answer-correctness sits in a 3-8D linear subspace, causally steerable

Details

Cho et al. study self-assessed correctness on TruthfulQA pairs across 9 models spanning 5 families, finding the discriminative signal is a low-dimensional linear subspace, not a curved manifold [cho-etal-2026-confidence-manifold] A PLS sweep peaks at 3-8 dimensions (e.g. peak AUC 0.90 at layer 23, dim 5, Mistral-7B), and nonlinear classifiers give no gain over a linear boundary [cho-etal-2026-confidence-manifold] Correct/incorrect classes form roughly Gaussian clusters separated by a mean shift: a two-mean centroid detector matches the linear probe (0.90 vs 0.89 AUC), robust with 25 examples/class [cho-etal-2026-confidence-manifold] This 3-8D discriminative subspace is lower-dimensional than the representation's 8-12D intrinsic dimension at the same layers [cho-etal-2026-confidence-manifold] Adding the learned direction shifts downstream error rate up to 10.9 points dose-dependently, while random and orthogonal controls have no reliable effect [cho-etal-2026-confidence-manifold]

models: Qwen2-1.5B-Instruct, Qwen2-7B-Instruct · method: Linear probing, Partial Least Squares (PLS), Difference-in-means direction, Activation Steering (Addition)
Language Models Represent and Transform Concepts with Shared Geometry (2026)measured

Contextual displacement forms a vector field tracking concreteness

Details

Hu, Niu & Varma define a per-word contextual displacement vector phi(w,tau) = r(w,tau) - r(w,tau_0) in activation space, forming a vector field over the vocabulary rather than one shared direction [hu-etal-2026-concept-transformation-geometry] Across 4 models x 3 depths, word density correlates negatively with displacement magnitude (Spearman rho -0.32 to -0.22) [hu-etal-2026-concept-transformation-geometry] Lexical concreteness correlates negatively with directional deviation (rho -0.41 to -0.21): concrete, low-density concepts move less and more consistently under context [hu-etal-2026-concept-transformation-geometry] The field has no dominant direction (leading PC <15% variance) and is more dispersed than uniform-random; no causal intervention is performed [hu-etal-2026-concept-transformation-geometry]

models: Qwen2.5-0.5B, Qwen2.5-1.5B, Qwen2.5-3B, Qwen-2.5-7B, Qwen2.5-14B, Qwen2.5-32B · method: PCA, Geometric analysis
Unveiling the Latent Directions of Reflection in Large Language Models (2025)measured

Contrastive reflection-intent directions steer reflective behavior

Details

Chang, Lee & Wu construct steering vectors between no-reflection, intrinsic-reflection, and triggered-reflection conditions in Qwen2.5-3B and Gemma3-4B-IT [chang-etal-2025-unveiling-latent-directions-of-reflection] Adding or subtracting the direction enhances or suppresses reflective behavior on GSM8k-adv and Cruxeval-o-adv [chang-etal-2025-unveiling-latent-directions-of-reflection] Suppression is empirically easier than induction [chang-etal-2025-unveiling-latent-directions-of-reflection] The three reflection conditions are clearly separable in activation space, confirming a real behavioral axis rather than a construction artifact [chang-etal-2025-unveiling-latent-directions-of-reflection]

models: Qwen2.5-3B · method: Difference-in-means direction, Causal interventions (steering)
Uncertainty as Feature Gaps: Epistemic Uncertainty Quantification of LLMs (2025)measured

Contrastive-PCA feature directions predict LLM epistemic uncertainty

Details

Bakman et al. derive an epistemic-uncertainty bound in terms of hidden-state displacement along semantic-feature directions [bakman-etal-2025-uncertainty-as-feature-gaps-epistemic-uncertainty-quantification-of-llms] They extract three directions (context-reliance, context-comprehension, honesty) via contrastive prompt-pair PCA on Llama-3.1-8B, Mistral-7B-v0.3, and Qwen2.5-7B [bakman-etal-2025-uncertainty-as-feature-gaps-epistemic-uncertainty-quantification-of-llms] Per-token projections onto these directions improve the Prediction Rejection Ratio by up to 13 points over baselines [bakman-etal-2025-uncertainty-as-feature-gaps-epistemic-uncertainty-quantification-of-llms] Replacing PCA extraction with a plain mean-difference direction substantially degrades performance, evidencing the specific extracted direction does the work [bakman-etal-2025-uncertainty-as-feature-gaps-epistemic-uncertainty-quantification-of-llms]

models: Qwen2.5-7B-Instruct · method: Direction Extraction, Difference-in-means direction
The Dual Mechanisms of Spatial Reasoning in Vision-Language Models (2026)measured

Object-ordering is a distributed linear direction that globally corrects spatial errors

Details

Cui et al. train linear probes on vision-encoder tokens that decode object ordinal position near-perfectly, with the signal extending to background "strip" tokens beyond object regions [cui-etal-2026-the-dual-mechanisms-of-spatial-reasoning-in-vision-language-models] Interchange-intervention activation patching shows ordering forms at layers 20-22 and color at 23-27 in Qwen2-VL-7B, and patching object-strip tokens flips the output [cui-etal-2026-the-dual-mechanisms-of-spatial-reasoning-in-vision-language-models] Globally amplifying the probe-derived ordering direction corrects over 50% of previously-incorrect predictions on Gemma-3-4b-it and over 30% on Qwen2-VL-7B-Instruct [cui-etal-2026-the-dual-mechanisms-of-spatial-reasoning-in-vision-language-models] No distributed-alignment-search is used; the method is linear probing plus activation patching and probe-direction amplification [cui-etal-2026-the-dual-mechanisms-of-spatial-reasoning-in-vision-language-models]

models: Qwen2-VL-7B-Instruct · method: Linear probing, Activation patching, Activation Steering (Addition)
Scenario-based Probing and Steering Cultural Values in Large Language Models (2026)measured

Cultural-value steering axes are entangled, mirroring real value correlations

Details

Dang, Kieu & Masud map LLMs onto the Inglehart-Welzel cultural map via scenario forced-choice probing and diff-of-means steering at 4 tuned layers per model [dang-etal-2026-scenario-based-probing-and-steering-cultural-values-in-llms] The steering direction is a signed bidirectional axis (negative alpha reverses the shift), saturating at alpha in [0.5, 0.6] [dang-etal-2026-scenario-based-probing-and-steering-cultural-values-in-llms] Steering one cultural axis causes correlated shifts on the other, an entanglement ratio 0.72-0.81 (dropping to 0.53 for Llama under hybrid steering), mirroring the real-world WVS axis correlation r=0.474 [dang-etal-2026-scenario-based-probing-and-steering-cultural-values-in-llms] Capability cost measured via Global-MMLU is largest for Gemma (0.608->0.530) [dang-etal-2026-scenario-based-probing-and-steering-cultural-values-in-llms]

models: Qwen3-4B-Instruct · method: Difference-in-means direction, Direction Extraction, Causal interventions (steering)
GEMS: Geometric Constraints Enable Multi-Semantic Superposition in LLMs (2026)measured

Non-orthogonal steering directions mutually dampen; GEMS restores them

Details

Deng (GEMS) shows three diff-in-means persona directions (empathy, accountability, minimalism) in Qwen3.5-4B are pairwise cosine-similar at 0.74-0.87 [deng-2026-gems-geometric-constraints-multi-semantic-superposition-llms] Injecting all three simultaneously degrades each direction's terminal-layer alignment from 0.64-0.66 to 0.38-0.43, a 34-43% mutual reduction attributed to non-orthogonality [deng-2026-gems-geometric-constraints-multi-semantic-superposition-llms] Unconstrained additive injection separately drives residual-stream norm to 3.4x baseline, flattening next-token probability (top-1 43% to 11%) [deng-2026-gems-geometric-constraints-multi-semantic-superposition-llms] GEMS applies per-token Gram-Schmidt orthogonalization plus norm-constrained recombination restricted to the attention-output pathway [deng-2026-gems-geometric-constraints-multi-semantic-superposition-llms] On GSM8K with 3 concurrent directions, naive steering collapses accuracy 92% to 4% while GEMS recovers 98% at only +2.2% perplexity [deng-2026-gems-geometric-constraints-multi-semantic-superposition-llms]

models: Qwen3.5-4B, Qwen3.5-4B-Instruct, Qwen3.6-27B-Instruct · method: Difference-in-means direction, Orthogonalized Multi-Direction Superposition Steering (GEMS)
Causal Probing for Internal Visual Representations in Multimodal Large Language Models (2026)measured

Visual concept directions split into localized-entity vs distributed-abstract encoding

Details

Deng et al. extract per-layer diff-in-means concept vectors for Entity, Visual-Style, Emotion, and Abstract categories across six MLLMs [deng-etal-2026-causal-probing-internal-visual-representations-mllms] Entity directions are sharply localized (Gini averaging 0.071, as low as 0.012) with a bimodal layer profile and extreme logit boosts [deng-etal-2026-causal-probing-internal-visual-representations-mllms] Abstract-concept directions are globally distributed (Gini up to 0.429), weakly steerable (success rate as low as 0.160) with negligible logit boost [deng-etal-2026-causal-probing-internal-visual-representations-mllms] Distributedness increases with scale: larger models localize concrete categories more but spread abstract concepts across more layers [deng-etal-2026-causal-probing-internal-visual-representations-mllms] Reverse steering establishes causal necessity, revealing a compensatory internal logit-boost surge (10^6-10^8) under entity suppression [deng-etal-2026-causal-probing-internal-visual-representations-mllms]

models: Qwen2.5-VL 7B Instruct, Qwen3-VL 8B, Qwen3-VL 32B · method: Difference-in-means direction, Causal interventions (steering), Activation Steering (Addition)
A Shared Geometry of Difficulty in Multilingual Language Models (2026)measured

A difficulty direction is language-agnostic shallow, language-specific deep

Details

Civelli et al. train per-layer linear probes on four LLMs to predict a continuous problem-difficulty score across 21 languages [civelli-etal-2026-shared-geometry-difficulty-multilingual] Deep-layer probes reach high same-language accuracy (Llama-3.1-8B rho=0.822 at layer ~30) but generalize poorly across languages [civelli-etal-2026-shared-geometry-difficulty-multilingual] Shallow-layer probes carry a language-agnostic difficulty signal (cross-lingual rho=0.783 at layer ~16) [civelli-etal-2026-shared-geometry-difficulty-multilingual] Fixing the deep same-language-optimal layer costs 0.177 rho cross-lingually while the shallow transfer-optimal layer costs only 0.014 rho in-language; replicated in Qwen3-8B [civelli-etal-2026-shared-geometry-difficulty-multilingual] No causal steering is performed, so the claim rests on decodability alone [civelli-etal-2026-shared-geometry-difficulty-multilingual]

models: Qwen3-8B · method: Linear probing
Language Models Compare Quantities Using Number-specific and Unit-specific Heuristics (2026)measured

DAS finds number- and unit-specific quantity-comparison subspaces

Details

Sasaki et al. use Distributed Alignment Search (1024 intervention dimensions, 512 per variable) to locate linear subspaces aligned with a NumLogDiff and a UnitLogDiff variable in quantity-comparison activations [sasaki-etal-2026-language-models-compare-quantities-using-number-specific-and-unit-specific-heuristics] Interchange Intervention Accuracy exceeds 0.9 in many middle layers versus near-chance baselines [sasaki-etal-2026-language-models-compare-quantities-using-number-specific-and-unit-specific-heuristics] A linear surrogate over these variables predicts the model's comparison preference at R^2 approximately 0.817 overall (0.763 near the decision boundary) for Qwen3-4B-Base [sasaki-etal-2026-language-models-compare-quantities-using-number-specific-and-unit-specific-heuristics] This supports a "bag of number-specific and unit-specific heuristics" account over exact-scale unit conversion [sasaki-etal-2026-language-models-compare-quantities-using-number-specific-and-unit-specific-heuristics]

models: Qwen3-4B, Qwen3-8B · method: Distributed Alignment Search (DAS), Causal interventions (steering)
When Roleplaying, Do Models Believe What They Say? (2026)measured

Emergent misalignment rotates an LLM's truth direction

Details

Sturgeon, Africa & Black fit L2-regularized logistic-regression truth-direction probes on Llama-3.3-70B-Instruct and Qwen3-8B, explicitly rejecting mass-mean (diff-in-means) probing [sturgeon-etal-2026-when-roleplaying-do-models-believe-what-they-say] Persona SFT shifts the truth-probe score by only +0.05 and Open Character Training by +0.089-0.124 [sturgeon-etal-2026-when-roleplaying-do-models-believe-what-they-say] Emergent Misalignment produces a much larger +0.28 shift (56% defend rate, 82% downstream-reasoning consistency) [sturgeon-etal-2026-when-roleplaying-do-models-believe-what-they-say] EM also rotates the truth direction itself (cosine ~0.58), indicating restructured truth-representation geometry rather than local override [sturgeon-etal-2026-when-roleplaying-do-models-believe-what-they-say] The analysis is observational, with no steering or causal intervention performed [sturgeon-etal-2026-when-roleplaying-do-models-believe-what-they-say]

models: Qwen3-8B, Qwen2.5-14B-Instruct · method: Linear probing
PERSONA: Dynamic and Compositional Inference-Time Personality Control via Activation Vector Algebra (2026)measured

A causal factorial-intervention test confirms OCEAN trait directions compose approximately independently in a real LLM

Details

Feng, Zhao, Zhong, Huang, Gu, Kong, Feng & Qin extract diff-of-means OCEAN personality-trait directions in real Qwen2.5-7B-Instruct, reporting a cosine-similarity heatmap across 10 trait-pole vectors (e.g. Calm-Dependable +0.751, Calm-Careless -0.714) [feng-etal-2026-persona-dynamic-compositional-inference-time-control] A causal factorial steering experiment measures one trait's behavioral shift when a second trait is simultaneously steered, finding cross-trait effect sizes an order of magnitude smaller than same-trait effect sizes (Delta E <= 1.2 vs Delta A 5.0-7.4, an approximately 8.8:1 ratio) [feng-etal-2026-persona-dynamic-compositional-inference-time-control] Vector-arithmetic composition of the extracted trait directions achieves up to 91 percent win rate on a Persona-Evolve compositional-control benchmark [feng-etal-2026-persona-dynamic-compositional-inference-time-control]

models: Qwen2.5-7B-Instruct · method:
How Does Alignment Tuning Shape Representations of Sycophancy and Related Cue-Induced Biases in LLMs? (2026)measured

Cue-induced bias directions are installed by alignment tuning

Details

Gupta et al. extract a per-bias diff-of-means direction for seven cue-triggered bias types across base and instruct checkpoints of five model families [gupta-etal-2026-alignment-tuning-sycophancy-directions] Four of five base models flip on under 5% as many pairs as their instruct counterparts, with base activations carrying essentially no cue-specific signal, so susceptibility is installed by alignment tuning [gupta-etal-2026-alignment-tuning-sycophancy-directions] Within instruct models each bias direction transfers to held-out data at mean AUROC 0.69-0.82, localizing to late-middle layers [gupta-etal-2026-alignment-tuning-sycophancy-directions] Biases stay representationally distinct, with even behaviorally similar biases occupying different, sometimes anti-aligned directions [gupta-etal-2026-alignment-tuning-sycophancy-directions] Subtracting the direction while preserving over 90% of correct answers recovers 7-20% of bias-induced errors, versus under 5% for a random direction [gupta-etal-2026-alignment-tuning-sycophancy-directions]

models: Qwen-2.5-7B, Qwen2.5-7B-Instruct · method: Difference-in-means direction, Activation Steering (Addition)
Latent Planning Emerges with Scale (2026)measured

Transcoder-decoded planning directions causally control future tokens

Details

Hanna & Ameisen decompose real Qwen3 (0.6B-14B) MLP activations via transcoders and attribution graphs to find planning-feature directions representing a not-yet-generated token [hanna-ameisen-2026-latent-planning-emerges-with-scale] Ablating these features where the model succeeds harms performance, while upweighting them 5x where it fails drastically improves performance [hanna-ameisen-2026-latent-planning-emerges-with-scale] A parallel rhyme-planning circuit is verified the same way [hanna-ameisen-2026-latent-planning-emerges-with-scale] Directional up/down-weighting (x7 / x-3) redirects couplet completions toward or away from a target rhyme [hanna-ameisen-2026-latent-planning-emerges-with-scale]

models: Qwen3-0.6B, Qwen3-1.7B, Qwen3-4B, Qwen3-8B, Qwen3-14B · method: Transcoders, Causal interventions (steering)
Reasoning Beyond Chain-of-Thought: A Latent Computational Mode in Large Language Models (2026)measured

SAE features selected by CoT-minus-direct trigger a reasoning mode

Details

He et al. select SAE latent features whose first-step activation differs most between chain-of-thought and direct prompting across six models (LLaMA, Gemma-3, Qwen3) [he-etal-2026-reasoning-beyond-chain-of-thought-latent-computational-mode] Restricting to features with consistently positive singleton-steering effect leaves 1-10 features per model [he-etal-2026-reasoning-beyond-chain-of-thought-latent-computational-mode] Additively steering only this feature set at the first decoding step raises direct-prompt accuracy sharply (LLaMA-3.1-8B 24.5% to 73.3%; Qwen3-0.6B 7.9% to 60.6%) at far fewer tokens than full CoT [he-etal-2026-reasoning-beyond-chain-of-thought-latent-computational-mode] A random-SAE-feature control reproduces neither the accuracy gain nor the pattern, confirming specificity [he-etal-2026-reasoning-beyond-chain-of-thought-latent-computational-mode] The feature marks entry into a reasoning mode (transient early spike, uncorrelated with correctness) and overrides an explicit "/no_think" instruction [he-etal-2026-reasoning-beyond-chain-of-thought-latent-computational-mode]

models: Qwen3-0.6B, Qwen3-4B · method: Sparse Autoencoders (SAE), Causal interventions (steering)
Categorical Perception in Large Language Model Hidden States: Structural Warping at Digit-Count Boundaries (2026)measured

LLM hidden states warp categorically at digit-count boundaries

Details

- In six LLMs, hidden-state distances warp categorically at digit-count boundaries (9->10, 99->100) versus matched non-boundary controls (15, 150); an RSA "CP-Additive" model (log-distance plus a boundary boost) beats a continuous model at 100% of primary layers. [cacioli-2026-categorical-perception-digit-boundaries] - Boundary crossing explains 5-27% of representational-distance variance beyond magnitude; the decade-100 effect is 3.9-12.7x the decade-10 effect, with a manifold rotation of 81.6-89.6 degrees at the boundary. [cacioli-2026-categorical-perception-digit-boundaries] - Patching along a ridge-regression "category direction" shifts discrimination confidence 70.1x more than random directions (Layer 5), dose-dependent and specific. [cacioli-2026-categorical-perception-digit-boundaries] - Five instruct models (Llama-3-8B-Instruct, Mistral-7B-Instruct-v0.3, Gemma-2-9B-IT, Qwen2.5-7B-Instruct, Phi-3.5-mini-instruct) plus a Llama-3-8B base control; causal patching run on Llama-3-8B-Instruct only. [cacioli-2026-categorical-perception-digit-boundaries]

models: Qwen2.5-7B-Instruct · method: Representational Similarity Analysis (RSA), Linear probing, Causal interventions (steering)
The Geometry of Categorical and Hierarchical Concepts in Large Language Models (2024), When Language Representations Interact: Separability and Cross-Lingual Effects in LLMs (2026)measured

Semantic hierarchy is encoded as orthogonality between concept levels

Details

Park et al. prove (Theorem 8) that sibling-contrast directions at one hierarchy level are orthogonal to the parent's contrast direction and to any ancestor level [park-etal-2024] Combined with per-level categorical polytopes, this gives a direct-sum-of-polytopes structure where each level occupies its own orthogonal subspace, confirmed in Gemma-2B and Llama-3-8B (cosine near zero versus a shuffled-labels control) [park-etal-2024] The authors argue ordinary high-dimensional Euclidean space suffices, explicitly contrasting with hyperbolic-embedding accounts of hierarchy [park-etal-2024] Marinov et al. confirm the same cross-level orthogonality for language family versus individual language across Qwen3-4B, Mistral-7B-v0.3, and Llama-3-8B via a causal inner product [marinov-etal-2026]

models: Qwen3-4B · method: Geometric analysis, Causal inner product
Cross-model Transferability among Large Language Models on the Platonic Representations of Concepts (2025)measured

An OLS-fitted linear map transfers steering vectors across LLMs

Details

Huang et al. fit an ordinary-least-squares linear map T between paired sentence-representations of a source and target real LLM [huang-etal-2025-cross-model-platonic-transferability] Transferring a concept steering vector through T succeeds behaviorally in 212/216 cases, while no-map and random-map controls fail to beat baseline in 23/35 cases [huang-etal-2025-cross-model-platonic-transferability] Maps fit for different concepts are themselves highly similar (SSIM 0.87-0.95) versus a random-matrix baseline (0.05-0.13) [huang-etal-2025-cross-model-platonic-transferability] This cross-concept map consistency is a quantified geometric statistic about the transformation itself, beyond downstream task accuracy [huang-etal-2025-cross-model-platonic-transferability]

models: Qwen2-7B-Instruct, Qwen2-0.5B-Instruct · method: Cross-model direction transfer via ridge regression
Global Evolutionary Steering: Refining Activation Steering Control via Cross-Layer Consistency (2026)measured

A shared rank-one 'evolutionary' direction rectifies noisy steering vectors

Details

Jiang et al. stack cross-layer differences of contrastive positive/negative activation gaps and show via PCA that the first principal component dominates the spectrum across all datasets [jiang-etal-2026-global-evolutionary-steering-cross-layer-consistency] They prove via Wedin's sin-theta theorem that the estimated top singular vector converges to a Global Evolutionary Direction at rate O(1/sqrt(N*L)) [jiang-etal-2026-global-evolutionary-steering-cross-layer-consistency] Rectifying each layer's CAA steering vector by reinforcing its projection onto this shared direction improves over CAA and four baselines across three models and five domains (p<0.05) [jiang-etal-2026-global-evolutionary-steering-cross-layer-consistency] On Qwen2.5-7B, AdvBench refusal rises from 0.647 to 0.775 and GSM8K accuracy from 0.570 to 0.734 without per-layer tuning [jiang-etal-2026-global-evolutionary-steering-cross-layer-consistency]

models: Qwen2.5-7B · method: Difference-in-means direction, Spectral Consensus (Global Evolutionary) Steering
Beyond Steering Vector: Flow-based Activation Steering for Inference-Time Intervention (2026)measured

Effective concept steering needs curved trajectories, not one linear direction

Details

Jin et al. note prior activation steering assumes a single fixed-direction additive shift, and replace it with FLAS, a concept-conditioned velocity field integrated over frozen model activations [jin-etal-2026-flow-based-activation-steering-inference-time-intervention] The learned steering trajectories are curved and token-varying: every concept leaves the origin in a shared direction, bends sharply, then enters a concept-specific region [jin-etal-2026-flow-based-activation-steering-inference-time-intervention] This two-phase nonlinear shape directly challenges the single-linear-direction assumption underlying standard steering [jin-etal-2026-flow-based-activation-steering-inference-time-intervention] FLAS outperforms prompting and prior steering (held-out HMean 1.015 vs 0.762/0.608 on Gemma-2-2B-IT) at under 1/26th the parameters and transfers zero-shot to Qwen3-4B-Instruct [jin-etal-2026-flow-based-activation-steering-inference-time-intervention]

models: Qwen3-4B-Instruct · method: Flow-based Activation Steering (FLAS)
Delta-Crosscoder: Robust Crosscoder Model Diffing in Narrow Fine-Tuning Regimes (2026)measured

A delta-loss crosscoder isolates 1-2 causal directions per fine-tuning organism

Details

Kassem et al. train a BatchTopK crosscoder with a delta-loss prioritizing directions that change between a base and fine-tuned model, ranked by Relative Decoder Norm [kassem-etal-2026-delta-crosscoder-robust-model-diffing] Across 10 fine-tuning model organisms in Llama-3.1-8B-Instruct, Gemma-2-9B-IT, and Qwen2.5-7B, it isolates exactly 1-2 causal directions per organism [kassem-etal-2026-delta-crosscoder-robust-model-diffing] It recovers the causal direction in 10/10 cases with 0% false positives, versus 40% and 60% false-positive rates for DSF and BatchTopK baselines [kassem-etal-2026-delta-crosscoder-robust-model-diffing] Adding or subtracting the isolated latent's decoder vector at inference causally mitigates the fine-tuned behavior [kassem-etal-2026-delta-crosscoder-robust-model-diffing]

models: Qwen2.5-7B · method: Sparse crosscoders, Causal interventions (steering)
Just-in-Time and Distributed Task Representations in Language Models (2025)measured

Task vectors are locally realized at certain tokens despite decodable identity

Details

Li et al. train linear decoders on real Gemma-3 (4B/12B/27B) and Qwen3 (4B/8B/14B) activations, finding task identity reliably decodable throughout the context [li-etal-2025-just-in-time-distributed-task-representations] The transferable, patchable task-vector representation instead comes online only at certain tokens, a two-fold locality [li-etal-2025-just-in-time-distributed-task-representations] PCA overlap between identifiable and transferable subspaces is task-dependent: 40-60% of the identifiable dimension projects onto the top-20 PCs for simple tasks versus 10-25% for list operations [li-etal-2025-just-in-time-distributed-task-representations] Patching the extracted task vector into zero-shot prompts recontextualizes them, with recovered accuracy tracking few-shot accuracy as k-shot increases [li-etal-2025-just-in-time-distributed-task-representations]

models: Qwen3-8B · method: Function vector extraction (causal-mediation head selection + sum), PCA
What Really Controls Temporal Reasoning in LLMs: Tokenisation or Representation of Time? (2026)measured

Temporal-probe decodability predicts reasoning in high-resource languages

Details

Bhatia et al. introduce MultiTempBench and evaluate 20 LLMs, combining a tokenization-fragmentation metric with linear-regression probes decoding Year/Month/Day (probe R^2 as temporal linearity) [bhatia-etal-2026-what-really-controls-temporal-reasoning-in-llms] A crossed mixed-effects regression over 285,000 predictions finds temporal linearity is the strongest predictor of reasoning accuracy in high-resource languages (English r=0.77) [bhatia-etal-2026-what-really-controls-temporal-reasoning-in-llms] Tokenization fragmentation dominates in low-resource languages (Hausa r=-0.97); the finding is purely correlational [bhatia-etal-2026-what-really-controls-temporal-reasoning-in-llms]

models: Qwen3-0.6B, Qwen3-4B, Qwen3-14B · method: Linear probing
Relational Linearity is a Predictor of Hallucinations (2026)measured

LRE affine-map linearity scores correlate with hallucination rate across four real instruction-tuned LLMs

Details

Lu, Liu, Gerstner, Hirlimann, Rohweder & Schütze (2026) fit relation-specific affine maps o = W_r s + b_r via ridge regression (the Linear Relational Embeddings framework) across Gemma-7B-IT, Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, and Qwen2.5-7B-Instruct [lu-etal-2026-relational-linearity-is-a-predictor-of-hallucinations] The resulting linearity score correlates with hallucination-vs-refusal rate on unknown entities at Pearson r=0.741-0.816 across 15 natural LRE relations, and r=0.573-0.812 on a new SyntHal synthetic-unknown-entity benchmark [lu-etal-2026-relational-linearity-is-a-predictor-of-hallucinations] Purely correlational -- the authors explicitly flag that establishing causality would require representation patching or steering, not yet performed [lu-etal-2026-relational-linearity-is-a-predictor-of-hallucinations]

models: Qwen2.5-7B-Instruct · method: Linear Relational Embedding (LRE)
The Truthfulness Spectrum Hypothesis (2026)measured

Mahalanobis cosine between truth directions predicts cross-domain generalization

Details

Ying et al. test whether truth directions range from domain-general to domain-specific across five truth types plus sycophantic/expectation-inverted lying [ying-etal-2026-the-truthfulness-spectrum-hypothesis] Linear probes generalize well pairwise except on sycophantic/inverted lying (AUROC ~0.55 and ~0.28), while joint training recovers strong performance [ying-etal-2026-the-truthfulness-spectrum-hypothesis] Mahalanobis cosine similarity between probe directions predicts cross-domain generalization far better than standard cosine (R^2=0.98 vs 0.56) [ying-etal-2026-the-truthfulness-spectrum-hypothesis] Concept-erasure isolates domain-general, domain-specific, and shared truth directions with effective dimensionality under 100-200 in an 8192-d residual stream [ying-etal-2026-the-truthfulness-spectrum-hypothesis] Steering domain-specific directions selectively suppresses incorrect-answer probability (+0.05 to +0.10) while the domain-general direction backfires (-0.07 to -0.11) [ying-etal-2026-the-truthfulness-spectrum-hypothesis]

models: Qwen2.5-14B-Instruct, Qwen2.5-7B-Instruct · method: Linear probing, LEACE (LEAst-squares Concept Erasure), Mahalanobis-cosine probe alignment, Causal interventions (steering)
Shared Latent Structures Enable Unified Backdoor Detection and Mitigation in LLMs (2026)measured

Five shared SAE directions mitigate structurally distinct backdoors

Details

Mahmoud et al. train SAEs on six real backdoored LLMs and use model diffing to find five SAE features shared across six distinct backdoor types [mahmoud-etal-2026-shared-latent-structures-enable-unified-backdoor-detection-and-mitigation-in-llms] Steering along each feature's decoder direction mitigates attack success: F33881 gives 96% on Sentiment-LoRA, 79% on country-conditioned advice, and 44% zero-shot on a held-out Refusal backdoor [mahmoud-etal-2026-shared-latent-structures-enable-unified-backdoor-detection-and-mitigation-in-llms] F25828 gives 100% zero-shot mitigation on the held-out Refusal backdoor [mahmoud-etal-2026-shared-latent-structures-enable-unified-backdoor-detection-and-mitigation-in-llms] A companion training-time method, Concept Ablation Fine-Tuning, ablates the same shared subspace during training [mahmoud-etal-2026-shared-latent-structures-enable-unified-backdoor-detection-and-mitigation-in-llms]

models: Qwen3-8B, Qwen3-14B, Qwen3-32B · method: Sparse Autoencoders (SAE), Activation Steering (Addition)
Why Far Looks Up: Probing Spatial Representation in Vision-Language Models (2026)measured

Per-axis spatial delta vectors form separated PCA clusters in stronger VLMs

Details

Min et al. extract a delta vector between final-token hidden states for paired VQA prompts differing only in queried-object order, at a fixed intermediate layer [min-etal-2026-why-far-looks-up-probing-spatial-representation-in-vision-language-models] Axis Coherence, the mean pairwise cosine among sign-corrected delta vectors within an axis, rises with model strength (distance-axis 0.075 to 0.112 across Molmo scales, 0.182 for RoboRefer-2B-SFT) but stays flat at 0.04-0.05 for Qwen2.5-VL [min-etal-2026-why-far-looks-up-probing-spatial-representation-in-vision-language-models] PCA shows weak models' distance vectors collapse near the origin while RoboRefer and Qwen3-VL-235B form three cleanly separated per-axis clusters aligned to distinct principal components [min-etal-2026-why-far-looks-up-probing-spatial-representation-in-vision-language-models] Distance coherence correlates with counter-heuristic behavioral accuracy (rho=0.759, 0.804; p<1e-3) both in- and cross-domain [min-etal-2026-why-far-looks-up-probing-spatial-representation-in-vision-language-models]

models: Qwen2.5-VL-3B-Instruct, Qwen3-VL-235B-A22B-Instruct · method: Difference-in-means direction, PCA
Narrow Finetuning Leaves Clearly Readable Traces in Activation Differences (2026)measured

A per-token mean-difference direction fingerprints narrow fine-tuning

Details

Minder et al. extract a diff-in-means direction between base and narrowly-fine-tuned activations across 33 fine-tuning organisms spanning 7 models from 1B to 32B [minder-etal-2026-narrow-finetuning-traces-in-activation-differences] Steering by adding this direction during generation produces text with high embedding-similarity to the actual fine-tuning corpus [minder-etal-2026-narrow-finetuning-traces-in-activation-differences] An LLM interpretability agent (GPT-5) given the direction correctly identifies the fine-tuning objective in 91% of organisms (30/33) [minder-etal-2026-narrow-finetuning-traces-in-activation-differences] That is more than twice as well at broad-objective identification and over 30x better at fine-grained detail than the best black-box baseline [minder-etal-2026-narrow-finetuning-traces-in-activation-differences]

models: Qwen3-1.7B, Qwen2.5-7B-Instruct · method: Difference-in-means direction, Logit Lens, Causal interventions (steering)
The Representational Geometry of Number (2026)measured

Number-task subspaces are distinct yet linearly inter-transformable

Details

Hu, Niu & Varma analyze number-concept representations (comparison, parity, primality, successor/predecessor) across BERT, GPT-2, Qwen2.5-7B and Qwen2.5-Math-7B at ~75% depth [hu-etal-2026-the-representational-geometry-of-number] Task-specific subspaces are distinct via Procrustes disparity (~0.010 vs a permutation baseline 0.077-0.273) yet linearly inter-transformable (SVCCA canonical correlations ~0.80-0.90) [hu-etal-2026-the-representational-geometry-of-number] The parity axis sits nearly orthogonal to the magnitude axis (88.8 degrees) while the primality axis is only partly decoupled (68.8 degrees) [hu-etal-2026-the-representational-geometry-of-number] Subspace overlap between most task pairs stays below 0.2, except the closely-related successor/predecessor pair (0.36-0.42) [hu-etal-2026-the-representational-geometry-of-number]

models: Qwen-2.5-7B, Qwen2.5-Math-7B · method: Generalized Procrustes Analysis (GPA), Singular Vector Canonical Correlation Analysis (SVCCA), Direction Extraction, Linear probing, Geometric analysis
Playing Devil's Advocate: Off-the-Shelf Persona Vectors Rival Targeted Steering for Sycophancy (2026)measured

Off-the-shelf persona vectors are near-orthogonal to sycophancy yet reduce it

Details

Kelkar et al. measure the geometry between generic off-the-shelf persona/role steering vectors and a targeted CAA sycophancy direction in Gemma-2-27B-Instruct and Qwen3-32B [kelkar-etal-2026-devils-advocate-persona-vectors-sycophancy] All role vectors are nearly orthogonal to the CAA direction (|cos|<0.17), with the cosine sign flipping between the two models [kelkar-etal-2026-devils-advocate-persona-vectors-sycophancy] Injecting critical/skeptic-persona vectors reduces sycophancy to 68-98% of CAA's own effect without ever using sycophancy labels [kelkar-etal-2026-devils-advocate-persona-vectors-sycophancy] Steering toward agreeable personas does not produce a mirror-symmetric increase in sycophancy, an asymmetric direction-behavior relationship [kelkar-etal-2026-devils-advocate-persona-vectors-sycophancy]

models: Qwen3-32B · method: Geometric analysis, Causal interventions (steering)
Steering at the Source: Style Modulation Heads for Robust Persona Control (2026)measured

Persona/style control localizes to a sparse set of attention heads

Details

Izawa et al. localize persona/style control in Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct to 3 attention heads per model via layer-wise persona-vector heatmaps and a head-wise Head Contribution Score [izawa-etal-2026-steering-at-the-source-style-modulation-heads-for-robust-persona-control] Steering only these heads achieves the best trait-expression-vs-coherency Pareto frontier (best score in 11/12 Qwen conditions, 9/12 Llama) [izawa-etal-2026-steering-at-the-source-style-modulation-heads-for-robust-persona-control] Zero-ablating them causes a sharp targeted drop in trait expression while leaving MMLU and output coherency intact [izawa-etal-2026-steering-at-the-source-style-modulation-heads-for-robust-persona-control]

models: Qwen2.5-7B-Instruct · method: Activation Steering (Addition), Concept-Token Attention-Output Projection (ConceptAttention)
Probing then Editing Response Personality of Large Language Models (2025)measured

Personality traits are linearly encoded with a betweenness arrangement

Details

Ju et al. train layer-wise linear probes on 11 open LLMs (including InternLM-7B and InternLM2-7B, base and instruct) to classify which of three personality instructions a response is conditioned on, scored by V-information [ju-etal-2025-personality-editing] V-information rises through layers 5-12 then plateaus, with instruction-tuned models 1-3 points higher than their base counterparts [ju-etal-2025-personality-editing] A t-SNE of Llama-3.1-8B-Instruct shows Neuroticism and Agreeableness distant with Extraversion situated between them, an informal betweenness arrangement [ju-etal-2025-personality-editing] This geometry explains why converting either trait into Extraversion is easier than converting directly between Neuroticism and Agreeableness [ju-etal-2025-personality-editing]

models: Qwen-2.5-7B, Qwen2.5-7B-Instruct · method: Linear probing
Simulated Adoption: Decoupling Magnitude and Direction in LLM In-Context Conflict Resolution (2026)measured

In-context conflict shows architecture-dependent dilution but universal orthogonal interference

Details

Zhang & Lin inject counterfactual contexts into MMLU questions and decompose the residual-stream update into a radial (norm-ratio) and an angular (cosine to the correct-answer direction) component [zhang-lin-2026-simulated-adoption-decoupling-magnitude-and-direction-in-llm-in-context-conflict-resolution] Radial "Manifold Dilution" is architecture-dependent: only Llama-3.1-8B shows real dilution (gamma=0.978) while Qwen3-4B and GLM-4-9B show none despite equal logit collapse [zhang-lin-2026-simulated-adoption-decoupling-magnitude-and-direction-in-llm-in-context-conflict-resolution] Angular "Orthogonal Interference" holds universally: interference-to-correct-answer cosine clusters near zero across all three models, not antiparallel suppression [zhang-lin-2026-simulated-adoption-decoupling-magnitude-and-direction-in-llm-in-context-conflict-resolution] Regressing angular deviation against logit drop gives R^2=0.90 (Qwen) and 0.87 (GLM) [zhang-lin-2026-simulated-adoption-decoupling-magnitude-and-direction-in-llm-in-context-conflict-resolution]

models: Qwen3-4B · method: Geometric analysis
Refusal Direction is Universal Across Safety-Aligned Languages (2025)measured

Refusal directions are approximately parallel across 14 languages and transfer

Details

Wang et al. extract diff-in-means refusal directions across 14 languages in Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, and Gemma-2-9B-it using the PolyRefuse dataset [wang-etal-2025-refusal-direction-is-universal-across-safety-aligned-languages] PCA shows the per-language refusal directions are approximately parallel, with high cross-lingual cosine similarity peaking around layer 12 in Llama [wang-etal-2025-refusal-direction-is-universal-across-safety-aligned-languages] Ablating a refusal vector extracted from English (or German/Thai) causes near-total loss of refusal across all 14 languages and all three models, while adding it induces refusal [wang-etal-2025-refusal-direction-is-universal-across-safety-aligned-languages] Yoruba, a language the models were never safety-aligned in, transfers weakly, so the parallelism tracks safety-alignment coverage rather than language identity [wang-etal-2025-refusal-direction-is-universal-across-safety-aligned-languages]

models: Qwen2.5-7B-Instruct · method: Difference-in-means direction, Causal interventions (steering), PCA
Refusal in Language Models Is Mediated by a Single Direction (2024), Representation Engineering: A Top-Down Approach to AI Transparency (2023), Programming Refusal with Conditional Activation Steering (2024), Refusal Direction is Universal Across Safety-Aligned Languages (2025)measured

Refusal is mediated by a single direction across models

Details

Arditi et al. show across 13 chat models (5 families, 1.8B-72B) that a single diff-in-means direction between harmful and harmless instructions both erases refusal when ablated and induces it when added, with no offset term [arditi-etal-2024] Zou et al. independently confirm via a LAT harmfulness vector in Vicuna-13B that stays a >90% classifier under jailbreaks and adversarial suffixes [arditi-etal-2024] Lee et al. replicate across nine further chat models and add conditional steering that gates the refusal vector on a condition vector for selective refusal [arditi-etal-2024] Yung et al. average per-model refusal vectors into a universal refusal direction transferring to jailbreak detection ~10% better than single-model baselines [arditi-etal-2024] The account is contested by Marshall et al. (an affine offset is needed) and Wollschlager et al. (a cone of directions), and Yin et al. localize attack-suppressed versus robust heads [arditi-etal-2024]

models: Qwen-1.8B-Chat, Qwen1.5-1.8B-Chat, Qwen1.5-32B-Chat, Qwen2.5-7B-Instruct · method: Difference-in-means direction, Causal interventions (steering), Activation Steering (Addition), Linear Artificial Tomography (LAT), PCA, Conditional (gated) activation steering
Latent Programming Horizons in Coding Agents (2026)measured

Coding-agent residual streams decode program properties 25 edit-steps ahead

Details

Silva et al. train linear (logistic-regression) probes on residual-stream activations of real Qwen3.6-35B-A3B and Laguna-XS.2 running as coding agents [silva-tu-monperrus-2026-latent-programming-horizons] The probes decode well-formedness/correctness/regression properties of the program several edit-steps before the edit that produces them, from the current hidden state [silva-tu-monperrus-2026-latent-programming-horizons] Decoding AUC reaches up to ~0.83 and stays above a shuffled-label chance baseline out to roughly 25 steps ahead of materialization [silva-tu-monperrus-2026-latent-programming-horizons] The effect transfers across benchmarks without probe retraining and is purely observational, with no activation-level causal intervention [silva-tu-monperrus-2026-latent-programming-horizons]

models: Qwen3.6-35B-A3B (MoE, 35B total / 3B active) · method: Linear probing
Convergent Linear Representations of Emergent Misalignment (2025)measured

A diff-in-means misalignment direction transfers across Qwen-14B fine-tunes

Details

Soligo et al. extract a diff-in-means direction (misaligned minus aligned mean activation) in real Qwen2.5-14B-Instruct fine-tuned to be emergently misaligned [soligo-etal-2025-convergent-linear-representations-emergent-misalignment] Adding it to the base aligned chat model causally induces misaligned behavior, while ablating it significantly reduces misalignment in the EM model and in independently misaligned fine-tunes [soligo-etal-2025-convergent-linear-representations-emergent-misalignment] The direction transfers between different Qwen-14B EM fine-tunes, evidencing a convergence in their representations of emergent misalignment [soligo-etal-2025-convergent-linear-representations-emergent-misalignment] A rank-1 LoRA independently trained to induce EM yields a direction at cosine only 0.04 to the mean-diff direction, yet the two interventions' behavioral effects converge [soligo-etal-2025-convergent-linear-representations-emergent-misalignment]

models: Qwen2.5-14B-Instruct · method: Difference-in-means direction, Causal interventions (steering), Activation Steering (Addition)
Linear Spatial World Models Emerge in Large Language Models (2025)measured

Spatial-relation vectors form an orthogonal antipodal composable basis

Details

- In Llama-3.2-3B-Instruct, spatial-relation vectors form a low-dimensional basis where inverse relations are antipodal (w_below ≈ -w_above): Above/Below cos 0.978, Left/Right 0.997, Front/Behind 0.995. [tehenan-etal-2025-spatial-world-models] - Different axes are near-orthogonal (cross-axis cosine ~0), and multi-relation directions equal the vector sum of their components (2D e.g. above+left mean cos 0.993). [tehenan-etal-2025-spatial-world-models] - 3D compositions are looser, spanning cosine 0.82-1.00 (angular deviation 3.9-34.4 degrees), e.g. above+right cos 0.88 (~28 degrees) and left+behind ~0.82 (34 degrees). [tehenan-etal-2025-spatial-world-models] - Steering by adding a relation vector causally shifts generated descriptions (74.3% overall: Above/Below/Left 100%, Right 79%, In-front 62%, Behind 5%); validated on Llama-3.2-1B-Instruct and Qwen3-1.7B; geometry cleanest in PCA space at layer 24. [tehenan-etal-2025-spatial-world-models]

models: Qwen3-1.7B · method: Linear probing, PCA, Activation Steering (Addition)
Probing Spectrum-Like Organization of States of Mind in Transformer Representation Spaces (2026)measured

A 'state of mind' spectrum is linearly decodable across five encoders

Details

Zhao annotates 636 sentences with a continuous energy score and one of seven ordered tiers, an informally-sourced psychological taxonomy explicitly disclaimed as non-clinical [zhao-2026] Across five frozen representations (BGE-large, all-mpnet-base-v2, all-MiniLM-L6-v2, Qwen3-Embedding-0.6B, and Qwen2.5-3B-Instruct's layer 24), both the score (Ridge R^2 0.63-0.79) and the seven tiers (weighted F1 0.57-0.72) are linearly decodable above chance [zhao-2026] Permutation tests and a TF-IDF baseline (R^2 0.44) rule out simple lexical confounds [zhao-2026] Projecting out a single Ridge-derived direction (1.6-4.8% of variance) collapses decodability far more than a random direction, though the structure is only approximately one-dimensional [zhao-2026]

models: Qwen3-Embedding-0.6B, Qwen2.5-3B-Instruct · method: Linear probing, Mlp probing, UMAP
On the Non-Identifiability of Steering Vectors in Large Language Models (2026)measured

Steering vectors are non-identifiable; orthogonal perturbations are equivalent

Details

Venkatesh & Kurapath prove via a Jacobian null-space argument that steering vectors extracted for a target behavior are not unique [venkatesh-kurapath-2026-on-the-non-identifiability-of-steering-vectors-in-large-language-models] On Llama-3.1-8B-Instruct and Qwen2.5-3B-Instruct across five traits, orthogonal perturbations within the activation-covariance null space produce behavioral effects indistinguishable from the original (mean Cohen's d 0.12-0.13, below the 0.2 detectability threshold) [venkatesh-kurapath-2026-on-the-non-identifiability-of-steering-vectors-in-large-language-models] A geometrically distinct mean-difference vector and a PCA-derived vector for the same trait are behaviorally equivalent [venkatesh-kurapath-2026-on-the-non-identifiability-of-steering-vectors-in-large-language-models] Many different directions in representation space thus implement the same steering effect [venkatesh-kurapath-2026-on-the-non-identifiability-of-steering-vectors-in-large-language-models]

models: Qwen2.5-3B-Instruct · method: Activation Steering (Addition)
Steered LLM Activations Are Non-Surjective (2026)measured

Steering vectors cannot reach most of the activation manifold

Details

Mishra et al. show that adding a fixed direction to residual-stream activations only reaches a thin, structured slice of the activations that occur naturally across contexts [mishra-etal-2026-steered-llm-activations-are-non-surjective] Across Llama-3.2-1B-Instruct, Qwen2.5-0.5B-Instruct, and Gemma-3-1B-it (plus three validation models), the reachable set is measurably non-surjective onto the empirical activation manifold [mishra-etal-2026-steered-llm-activations-are-non-surjective] Many naturally-occurring target-behavior activations lie off the steering line entirely, and increasing the coefficient pushes activations off-manifold rather than toward the unreached regions [mishra-etal-2026-steered-llm-activations-are-non-surjective] This bounds what single-direction steering can achieve and implies off-line behaviors need multi-directional or nonlinear interventions [mishra-etal-2026-steered-llm-activations-are-non-surjective]

models: Qwen2.5-0.5B-Instruct · method: Activation Steering (Addition)
ReCoVeR the Target Language: Language Steering Without Sacrificing Task Performance (2025)measured

A diff-in-means language-identity direction steers output language without accuracy loss

Details

Sterz et al. extract a language-identity direction as the mean-difference between target-language and other-language activations in Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, and Gemma-2-2B-Instruct [sterz-etal-2025-recover-target-language] Adding this direction, or a trained steering module, to the residual stream raises the rate of responses in the correct target language across 18 languages [sterz-etal-2025-recover-target-language] Unlike a prior unsupervised language-vector steering baseline, it preserves downstream task accuracy such as MMLU [sterz-etal-2025-recover-target-language] No further internal geometric characterization (PCA/subspace structure) of the direction is reported [sterz-etal-2025-recover-target-language]

models: Qwen2.5-7B-Instruct · method: Difference-in-means direction, Causal interventions (steering)
Sycophancy Is Not One Thing: Causal Separation of Sycophantic Behaviors in LLMs (2025)measured

Sycophantic agreement, genuine agreement, and praise occupy distinct steerable directions

Details

Vennemeyer et al. decompose sycophancy into sycophantic agreement, genuine agreement, and sycophantic praise, extracting a diff-in-means direction per behavior at every layer over items where the model knows the ground truth [vennemeyer-etal-2025-sycophancy-not-one-thing] Sycophantic- and genuine-agreement directions start nearly identical (cosine ~0.99 early) then diverge sharply to ~0.07 by layer 25, while the praise direction stays near-orthogonal to both throughout [vennemeyer-etal-2025-sycophancy-not-one-thing] Ablating a behavior's own diff-in-means subspace drops its own detection AUROC to chance while removing the praise subspace has zero effect on agreement detection [vennemeyer-etal-2025-sycophancy-not-one-thing] Activation-addition steering is highly selective (praise 36.8x in LLaMA-3.1-8B, sycophantic-agreement 23.1x in Qwen3-30B), replicated across five models [vennemeyer-etal-2025-sycophancy-not-one-thing]

models: Qwen3-30B-A3B-Instruct, Qwen3-4B-Instruct · method: Difference-in-means direction, Activation Steering (Addition)
Actionable Activation Directions for Detecting and Mitigating Emergent Misalignment Across Language Model Families (2026)measured

A diff-in-means misalignment direction transfers cross-architecture but non-specifically

Details

Syed QLoRA-finetunes four models identically on insecure code and extracts a diff-in-means misalignment direction, finding 99.6% within-model separability versus 50% for a secure-code control [syed-2026-actionable-activation-directions-emergent-misalignment] A ridge-regression map fit between any two models' activation spaces transfers the direction across all twelve model-family pairs with above-chance accuracy (Gemma-to-Llama 87%, Gemma-to-Ministral 90%) [syed-2026-actionable-activation-directions-emergent-misalignment] Steering a target model with the transferred direction suppresses code-related spillover (delta 13-46 points) but fails a specificity control, unlike the fully specific within-model effect (delta 21-51 points) [syed-2026-actionable-activation-directions-emergent-misalignment] This reveals an asymmetric donor/receiver topology, with Gemma and Qwen acting as donors and Llama as a receiver [syed-2026-actionable-activation-directions-emergent-misalignment]

models: Qwen2.5-1.5B-Instruct · method: Difference-in-means direction, Cross-model direction transfer via ridge regression
Task Recognition and Task Learning Heads Align In-Context Hidden States with a Label-Unembedding Task Subspace (2026)measured

ICL task-recognition and task-learning heads project onto a label subspace

Details

Yang, Cho & Inoue introduce Task Subspace Logit Attribution, scoring each attention head output by projection onto the span of label-token unembedding vectors [yang-cho-inoue-2026-task-subspace-attention-heads] Task-recognition heads align with the task subspace while task-learning heads rotate the hidden state toward the correct-minus-incorrect label direction within it [yang-cho-inoue-2026-task-subspace-attention-heads] Across Llama3-8B layers, hidden-update vs TR-head subspace alignment correlates rho=0.94, and logit-difference vs TL-head alignment rho=0.53 [yang-cho-inoue-2026-task-subspace-attention-heads] Injecting the summed top-3% TR-head outputs into zero-shot prompts raises classification accuracy 9.2% to 40.4% (TR-based), while TL-based vectors dominate open-ended generation; ablating each collapses its own role, tested across 6 models [yang-cho-inoue-2026-task-subspace-attention-heads]

models: Qwen2-7B (base), Qwen2.5-32B · method: Direct Logit Attribution (DLA), Activation patching
The Geometries of Truth Are Orthogonal Across Tasks (2026)measured

Per-task truth probe directions are near-orthogonal with disjoint L1 support

Details

Azizian et al. train L2-logistic truth probes separately on seven QA/fact-verification tasks across Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, and Phi-4-Mini-Instruct [azizian-etal-2026-truth-geometries-orthogonal-across-tasks] Pairwise cosine similarity between probe directions is consistently low (<0.5) and correlates with cross-task transfer AUROC (r=0.59) [azizian-etal-2026-truth-geometries-orthogonal-across-tasks] L1 probes share under 15% of their nonzero support for most task pairs, and the few high-overlap pairs are exactly those that generalize well [azizian-etal-2026-truth-geometries-orthogonal-across-tasks] Three attempts to recover a shared cross-task truth direction (joint training, subspace-constrained fit, mixture-of-probes) all fail to beat per-task probes, and no causal intervention is performed [azizian-etal-2026-truth-geometries-orthogonal-across-tasks]

models: Qwen2.5-7B-Instruct · method: Linear probing, Geometric analysis
Task Vector Geometry Underlies Dual Modes of Task Inference in Transformers (2026)measured

Task-vector subspace implements Bayesian retrieval; OOD generalization is near-orthogonal

Details

Yan et al. train small RoPE transformers on synthetic latent-task mixtures and formalize task vectors as task-conditional-minus-global mean hidden states [yan-yang-zhong-2026-task-vector-geometry-dual-modes] Finite-context hidden states are well approximated as a simplex convex combination of task vectors whose coefficients closely track the exact Bayesian posterior as context accumulates [yan-yang-zhong-2026-task-vector-geometry-dual-modes] Substituting the task-vector mixture component with a target simplex point steers outputs to the theoretical mixture (KL 0.11 to 0.03 in E1) [yan-yang-zhong-2026-task-vector-geometry-dual-modes] Out-of-distribution generalization occupies a second, near-orthogonal subspace emerging only at high task diversity, confirmed by double-dissociation ablation [yan-yang-zhong-2026-task-vector-geometry-dual-modes] A real Qwen2.5-7B projection qualitatively echoes the geometry, with in-distribution vertex convergence versus out-of-distribution orthogonality [yan-yang-zhong-2026-task-vector-geometry-dual-modes]

models: Qwen-2.5-7B · method: Geometric analysis, Causal interventions (steering)
The Geometry of Forgetting: Temporal Knowledge Drift as an Independent Axis in LLM Representations (2026)measured

Temporal knowledge drift is a linear direction orthogonal to correctness and uncertainty

Details

Elbadry et al. train L1-regularized probes on six instruction-tuned LLMs to detect temporal drift (whether a fact changed since training cutoff), reaching AUROC 0.83-0.95 versus 0.49-0.57 for output and correctness/uncertainty baselines [elbadry-etal-2026-geometry-of-forgetting-temporal-knowledge-drift-as-an-independent-axis-in-llm-representations] Five measures establish the drift direction is geometrically orthogonal to correctness and uncertainty probe directions (weight cosine <=0.136; INLP removal of 10 directions changes AUROC <=0.013) [elbadry-etal-2026-geometry-of-forgetting-temporal-knowledge-drift-as-an-independent-axis-in-llm-representations] Untrained difference-of-means directions overlap with correctness/uncertainty, yet the trained regularized drift probe is near-orthogonal, so the orthogonality is a genuine concept property not a training artifact [elbadry-etal-2026-geometry-of-forgetting-temporal-knowledge-drift-as-an-independent-axis-in-llm-representations] A diff-in-means steering direction is silent under ablation but under amplification produces structured logit redistribution favoring the current over the stale fact holder (selectivity -2.46 to -10.55 logits) [elbadry-etal-2026-geometry-of-forgetting-temporal-knowledge-drift-as-an-independent-axis-in-llm-representations] A cross-cutoff entity-matched control (0.975-0.998 across seven model pairs) confirms the probe reads model-internal knowledge state, not an input property [elbadry-etal-2026-geometry-of-forgetting-temporal-knowledge-drift-as-an-independent-axis-in-llm-representations]

models: Qwen2.5-7B-Instruct · method: Linear probing, Difference-in-means direction, Causal interventions (steering), INLP subspace + AlterRep counterfactual push
Temporal Preference Concepts and Their Functions in a Large Language Model (2026)measured

A temporal-preference subgraph collapses continuous horizon geometry into a binary preference

Details

Rios-Sialer et al. causally localize a temporal-preference subgraph in Qwen3-4B-Instruct-2507 (layers 17-35, layer-24 attention) via four independent localization pipelines [riossialer-etal-2026-temporal-preference-concepts-and-their-functions-in-a-large-language-model] PCA within the subgraph shows time horizons form ordinal clusters whose separability is unstable until the user-to-assistant turn boundary, where attention collapses the continuous horizon into a binary preference [riossialer-etal-2026-temporal-preference-concepts-and-their-functions-in-a-large-language-model] The model's discount rates (k<0.005) are 3-8x below human controls (k~0.013) [riossialer-etal-2026-temporal-preference-concepts-and-their-functions-in-a-large-language-model] Contrastive Activation Addition with a probe-derived vector at layers 19-22 shifts temporal preference bidirectionally, revealing a probing-steering layer dissociation [riossialer-etal-2026-temporal-preference-concepts-and-their-functions-in-a-large-language-model]

models: Qwen3-4B-Instruct-2507 · method: Linear probing, Activation patching, PCA, Causal interventions (steering)
How Context Shapes Truth: Geometric Transformations of Statement-level Truth Representations in LLMs (2026)measured

The truth direction rotates and rescales under added context and steers labels

Details

Adarsh et al. extract a per-layer mean-difference truth direction in Llama-3.1-8B-Instruct, Mistral-Nemo-12B-Instruct, Qwen3-4B-Instruct, and SmolLM3-3B and measure how it transforms when a statement is embedded in context [adarsh-etal-2026-how-context-shapes-truth] The angle between with- and without-context truth vectors is near-orthogonal in early layers and converges by mid-depth, while the magnitude ratio is quantified per dataset [adarsh-etal-2026-how-context-shapes-truth] This is the first quantified characterization of how the truth direction's own geometry, not just its existence, changes under context [adarsh-etal-2026-how-context-shapes-truth] Mass-mean steering with the extracted vector flips truthfulness labels in ~100% of cases for three models (Qwen3-4B weaker and variable at 11-59%), a strong replicated causal effect [adarsh-etal-2026-how-context-shapes-truth]

models: Qwen3-4B-Instruct · method: Difference-in-means direction, Causal interventions (steering)
Valence-Arousal Subspace in LLMs: Circular Emotion Geometry and Multi-Behavioral Control (2026), Where Do Models Find Happiness? Emotion Vectors in Open-Source LLMs (2026)measured

A valence-arousal circumplex subspace causally steers emotion and refusal

Details

Sun et al. recover two near-orthogonal valence/arousal axes inside Llama-3.1-8B-Instruct (replicated in Qwen3-8B/14B) by PCA-projecting contrastive emotion-steering vectors and ridge-regressing against human ratings [sun-etal-2026-valence-arousal] Projecting the emotion vectors onto this plane traces a circle: a circularity statistic reaches 2.76-4.08 with fitted radii ~0.37-0.39, analogous to Russell's circumplex [sun-etal-2026-valence-arousal] The valence axis recovers self-reported valence at r=0.97 and the NRC-VAD lexicon at r=0.71, with cross-model valence agreement r=0.95 [sun-etal-2026-valence-arousal] Adding valence/arousal directions at specific circle angles produces dose-dependent, angle-specific shifts in generated-text affect (e.g. 0deg: delta-valence +0.75; 180deg: -0.73) [sun-etal-2026-valence-arousal] The same arousal axis causally controls refusal (20%->86% on OKTest) and sycophancy, with random-direction controls within 2-3 points of baseline [sun-etal-2026-valence-arousal] Logit-clamping and top-neuron ablation along the direction crash refusal while preserving MATH-500/IFEval, and an independent refusal direction is near-orthogonal (86.5deg) to the VA plane [sun-etal-2026-valence-arousal] Van der Ben et al. independently replicate the valence/arousal PCA structure (PC1-valence r=0.72-0.83, PC2-arousal r=0.21-0.45) in Apertus-8B and Gemma-4-E4B-it without computing circularity or steering [vanderben-etal-2026-emotion-vectors-open-source-llms] They add that cross-architecture layer-depth trajectories diverge sharply (a 3-phase plateau in Apertus vs a smooth gradient in Gemma via linear CKA) [vanderben-etal-2026-emotion-vectors-open-source-llms]

models: Qwen3-8B, Qwen3-14B · method: PCA, Linear probing, Activation Steering (Addition)
Negative Before Positive: Asymmetric Valence Processing in Large Language Models (2026)measured

Negative and positive valence localize to distinct depths; a negative-locus direction steers valence

Details

Venkatesh performs topic-controlled activation patching across all layers on Llama-3.2-1B-Instruct, Qwen2.5-1.5B-Instruct, and Qwen2.5-3B-Instruct with a shared corrupted baseline [venkatesh-2026-negative-before-positive] Negative-valence processing localizes causally to 14-27% of model depth and positive-valence to 53-66%, a consistent depth asymmetry (p<2e-10), with a flip-test ruling out simple topic detection [venkatesh-2026-negative-before-positive] A diff-in-means valence direction extracted at the negative-locus layer, added to neutral prompts, produces a monotonic dose-dependent valence shift (Spearman rho>0.89) [venkatesh-2026-negative-before-positive] This is distinct from the 2D circumplex account, being a depth-asymmetric causal localization of two separate one-dimensional valence directions [venkatesh-2026-negative-before-positive]

models: Qwen2.5-1.5B-Instruct, Qwen2.5-3B-Instruct · method: Activation patching, Difference-in-means direction, Causal interventions (steering)
The Anatomy of a Truth Direction: Knowledge-Dependent Dimensionality, a Relational Law, and a Convergent Category Geometry in Small Language Models (2026)measured

A truth direction's dimensionality, layer-wise construction, and cross-family category geometry

Details

Vicidomini uses SVD-based linear decomposition of real hidden states in small Llama, Qwen, and Gemma-2-2B models to show truth-signal dimensionality concentrates onto a single axis for well-known facts and diffuses across more dimensions as knowledge decreases [vicidomini-2026-anatomy-of-a-truth-direction] Causal component-ablation attributes the truth direction's layer-wise construction to attention propagating the truth frame forward while the FFN opposes the current block's frame [vicidomini-2026-anatomy-of-a-truth-direction] Per-category truth axes converge into a semantically-signed arrangement across model families once a sign instability is corrected with a spectral consensus gauge [vicidomini-2026-anatomy-of-a-truth-direction]

models: Qwen2.5-7B · method:
Function-Vector Heads Are Two Populations: Writers and Cancellers in In-Context Learning (2026)measured

Function-vector heads split into writer and canceller populations with orthogonal OV directions

Details

Wang shows the function-vector heads identified by Todd et al. in real Pythia (410M-12B), Qwen2.5, and GPT-2-medium are not homogeneous, splitting by sign into writers and cancellers via refined direct logit attribution [wang-2026-function-vector-heads-writers-cancellers] The two sub-populations' mean OV directions are nearly orthogonal (perpendicular fraction 0.96), so cancellers write to a near-orthogonal subspace while still exerting a negative direct causal effect [wang-2026-function-vector-heads-writers-cancellers] Zero-ablating cancellers yields +0.13 to +0.29 nats of logit gain in 6 of 6 main cells, with a consistent +2 to +7 point ICL accuracy effect [wang-2026-function-vector-heads-writers-cancellers] A TOST equivalence test shows cancellers are not simply induction heads in disguise [wang-2026-function-vector-heads-writers-cancellers]

models: Qwen-2.5-7B · method: Direct Logit Attribution (DLA)
Rhetorical Questions in LLM Representations: A Linear Probing Study (2026)measured

Rhetorical-question status is linearly decodable but methods find non-collinear directions

Details

Yao et al. test whether rhetorical-question status is linearly decodable in Qwen3-32B and Llama-3.3-70B-Instruct, projecting representations into a 64-dim PCA space used only for numerical stabilization [yao-etal-2026-rhetorical-questions-linear-probing] Comparing diff-in-means, logistic regression, and hinge-loss SVM extraction, layer-wise AUROC rises from ~0.6 (diffMean, early) to ~0.85-0.9 (trained probes, deep) [yao-etal-2026-rhetorical-questions-linear-probing] The two trained-probe directions nearly agree (cosine ~1), but the training-free diff-in-means direction is only weakly aligned with them (cosine <0.7 on RQ, ~0.5 on SRAQ) [yao-etal-2026-rhetorical-questions-linear-probing] Directions fit on different datasets align only weakly (cosine 0.2-0.4), so rhetorical question is not one dataset-general direction but a method- and corpus-dependent family [yao-etal-2026-rhetorical-questions-linear-probing]

models: Qwen3-32B · method: Linear probing, Difference-in-means direction, PCA
Behavioral Steering in a 35B MoE Language Model via SAE-Decoded Probe Vectors: One Agency Axis, Not Five Traits (2026)measured

Agentic-trait probe directions share R^2 yet are orthogonal, revealing one agency axis

Details

Yap trains 9 TopK SAEs across depths and sublayers of Qwen3.5-35B-A3B and fits ridge probes per agentic trait in SAE latent space, projected through the decoder into steering vectors [yap-2026-behavioral-steering-moe-agency-axis] Risk-calibration and tool-use-eagerness probes share nearly identical R^2 (0.795 vs 0.792) yet are nearly orthogonal (cosine -0.017), dissociating predictive geometry from causal specificity [yap-2026-behavioral-steering-moe-agency-axis] Fewer than 1% of SAE features explain 50% of each steering vector's norm [yap-2026-behavioral-steering-moe-agency-axis] Autonomy steering at prefill achieves Cohen's d=1.01 (shifting ask_user behavior to near-zero) while decode-only steering is null, and no steering vector achieves specificity >1.0, so every vector modulates one shared agency axis [yap-2026-behavioral-steering-moe-agency-axis]

models: Qwen3.5-35B-A3B (MoE, ~3B active/token) · method: TopK (k-sparse) Sparse Autoencoders, Linear probing, Causal interventions (steering)
Spherical Steering: Geometry-Aware Activation Rotation for Language Models (2026)measured

Spherical steering rotates normalized activations along a norm-preserving geodesic

Details

You et al. extract a contrastive truthfulness-axis direction via diff-in-means, then rotate a normalized activation toward it via spherical linear interpolation instead of adding to it [you-etal-2026-spherical-steering-geometry-aware-activation-rotation] This guarantees exact norm preservation for any rotation amount, unlike additive steering whose norm change depends on both step size and the activation's own alignment [you-etal-2026-spherical-steering-geometry-aware-activation-rotation] A von Mises-Fisher confidence gate sets per-token rotation strength from the activation's angular alignment, so already-aligned tokens receive no rotation [you-etal-2026-spherical-steering-geometry-aware-activation-rotation] On LLaMA-3.1-8B-Instruct and Qwen-2.5-7B-Instruct it improves average multiple-choice accuracy by over 8 points across six benchmarks while also improving open-ended generation quality, versus additive CAA/ITI baselines [you-etal-2026-spherical-steering-geometry-aware-activation-rotation]

models: Qwen2.5-7B-Instruct · method: Spherical (Slerp) Activation Rotation Steering

GPT-OSS

What Models Express, Suppress, and Resist: Auditing Open-Weight LLMs with Persona Vectors (2026)measured

Persona-vector taxonomy sorts traits as natural, steerable, or intractable

Details

Zeng, Emami & Choi extract 53 diff-in-means contrastive persona vectors across four trait domains in Qwen3-8B and gpt-oss-20b [zeng-etal-2026-what-models-express-suppress-and-resist-auditing-open-weight-llms-with-persona-vectors] Sweeping steering strength alpha in {0,0.5,...,2.5} classifies each trait as natural, steerable, or intractable via quantitative thresholds (baseline>=70 and gain<=10 for "natural") [zeng-etal-2026-what-models-express-suppress-and-resist-auditing-open-weight-llms-with-persona-vectors] Mean steering-gain is 18.39 (clinician domain, Qwen3-8B) and 11.56 (gpt-oss-20b), and all 9 agentic traits are natural in both models [zeng-etal-2026-what-models-express-suppress-and-resist-auditing-open-weight-llms-with-persona-vectors] Clinician defaults matched a board-certified psychiatrist's desirability judgment on 16 of 17 traits [zeng-etal-2026-what-models-express-suppress-and-resist-auditing-open-weight-llms-with-persona-vectors] Across 171 pairwise generic-trait steering combinations, destructive composition requires two steerable traits and never occurs when a natural trait is involved [zeng-etal-2026-what-models-express-suppress-and-resist-auditing-open-weight-llms-with-persona-vectors] For the intractable "evil" trait in gpt-oss-20b, a persona vector transferred from a fine-tuned variant recovers behavior the base model refuses to express [zeng-etal-2026-what-models-express-suppress-and-resist-auditing-open-weight-llms-with-persona-vectors]

models: GPT-OSS-20B · method: Difference-in-means direction, Activation Steering (Addition)
What Really Controls Temporal Reasoning in LLMs: Tokenisation or Representation of Time? (2026)measured

Temporal-probe decodability predicts reasoning in high-resource languages

Details

Bhatia et al. introduce MultiTempBench and evaluate 20 LLMs, combining a tokenization-fragmentation metric with linear-regression probes decoding Year/Month/Day (probe R^2 as temporal linearity) [bhatia-etal-2026-what-really-controls-temporal-reasoning-in-llms] A crossed mixed-effects regression over 285,000 predictions finds temporal linearity is the strongest predictor of reasoning accuracy in high-resource languages (English r=0.77) [bhatia-etal-2026-what-really-controls-temporal-reasoning-in-llms] Tokenization fragmentation dominates in low-resource languages (Hausa r=-0.97); the finding is purely correlational [bhatia-etal-2026-what-really-controls-temporal-reasoning-in-llms]

models: GPT-OSS-20B · method: Linear probing
Understanding Emergent Misalignment via Feature Superposition Geometry (2026)measured

Insecure-code SAE directions sit closer to toxic directions than secure-code ones

Details

Minegishi et al. train SAEs on five models and identify insecure-code, secure-code, and toxic-persona decoder directions [minegishi-etal-2026-understanding-emergent-misalignment-via-feature-superposition-geometry] Across all models and layers, the insecure-code direction is consistently more cosine-similar to the toxic-persona direction than the secure-code direction is [minegishi-etal-2026-understanding-emergent-misalignment-via-feature-superposition-geometry] So insecure-code finetuning data is geometrically closer to misaligned-persona representations than matched secure-code data [minegishi-etal-2026-understanding-emergent-misalignment-via-feature-superposition-geometry] Filtering training data by high insecure-code-direction activation cuts emergent-misalignment behaviors from 87 to 57, beating random removal (84) and an LLM-judge filter (59) [minegishi-etal-2026-understanding-emergent-misalignment-via-feature-superposition-geometry]

models: GPT-OSS-20B · method: Sparse Autoencoders (SAE)
Sycophancy Is Not One Thing: Causal Separation of Sycophantic Behaviors in LLMs (2025)measured

Sycophantic agreement, genuine agreement, and praise occupy distinct steerable directions

Details

Vennemeyer et al. decompose sycophancy into sycophantic agreement, genuine agreement, and sycophantic praise, extracting a diff-in-means direction per behavior at every layer over items where the model knows the ground truth [vennemeyer-etal-2025-sycophancy-not-one-thing] Sycophantic- and genuine-agreement directions start nearly identical (cosine ~0.99 early) then diverge sharply to ~0.07 by layer 25, while the praise direction stays near-orthogonal to both throughout [vennemeyer-etal-2025-sycophancy-not-one-thing] Ablating a behavior's own diff-in-means subspace drops its own detection AUROC to chance while removing the praise subspace has zero effect on agreement detection [vennemeyer-etal-2025-sycophancy-not-one-thing] Activation-addition steering is highly selective (praise 36.8x in LLaMA-3.1-8B, sycophantic-agreement 23.1x in Qwen3-30B), replicated across five models [vennemeyer-etal-2025-sycophancy-not-one-thing]

models: GPT-OSS-20B · method: Difference-in-means direction, Activation Steering (Addition)

Llama

Llama Scope: Extracting Millions of Features from Llama-3.1-8B with Sparse Autoencoders (2024)measured

Llama Scope's 256-SAE suite reveals feature geometry at 8B scale

Details

He et al. train 256 TopK sparse autoencoders (32K and 128K widths) across every layer and sublayer of Llama-3.1-8B-Base [he-etal-2024-llama-scope] A feature-geometry analysis finds nearest-neighbor structure and semantically coherent neighborhoods (e.g. a "Threats-to-Humanity" cluster) among learned SAE latents [he-etal-2024-llama-scope] Feature splitting -- wider SAEs learning genuinely finer rather than duplicated features -- is confirmed at open-weight 8B production scale [he-etal-2024-llama-scope] The result extends the smaller-scale JumpReLU/Gemma-Scope width-ladder and the original Claude-3-Sonnet demonstration to a fully open model and SAE suite [he-etal-2024-llama-scope]

models: Llama-3.1-8B · method: Sparse Autoencoders (SAE)
The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models (2026)measured

An 'Assistant Axis' persona component causally reduces jailbreaks

Details

Lu et al. extract activation vectors for 275 character archetypes across Gemma-2-27B, Qwen3-32B and Llama-3.3-70B and run PCA on the persona space (4-19 PCs explain 70% of variance) [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models] PC1 is an interpretable "Assistant Axis" whose role loadings correlate >0.92 pairwise across the three models [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models] A contrast-based Assistant Axis (mean role-playing minus default-Assistant) aligns with PC1 at cosine >0.71 in all three models [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models] Activation-capping the Assistant-Axis projection cuts persona-jailbreak harmful-response rate ~60% with no measurable loss on IFEval/MMLU-Pro/GSM8k/EQ-Bench [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models] The axis persists in base (non-instruction-tuned) Gemma-2-27B, evidencing a pretraining origin [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models]

models: Llama 3.3 70B Instruct · method: Difference-in-means direction, PCA, Activation Steering (Addition)
The Granularity Axis: A Micro-to-Macro Latent Direction for Social Roles in Language Models (2026)measured

A role-granularity axis aligns with PC1 and steers output detail

Details

Qin et al. build 75 ordered social roles (5 granularity levels x 15 roles) and 91,200 role-conditioned responses, defining a macro-minus-micro contrast direction [qin-etal-2026-the-granularity-axis-a-micro-to-macro-latent-direction-for-social-roles-in-language-models] This Granularity Axis aligns with PC1 of the role-representation space at cosine 0.972 (Qwen3-8B) and 0.9596 (Llama-3.1-8B-Instruct), explaining 52.6%/42.5% of variance [qin-etal-2026-the-granularity-axis-a-micro-to-macro-latent-direction-for-social-roles-in-language-models] Projections are monotonic in granularity level (Spearman/Pearson >0.93 in both models) [qin-etal-2026-the-granularity-axis-a-micro-to-macro-latent-direction-for-social-roles-in-language-models] Adding the axis at layer 18 shifts judge-rated output granularity in the predicted direction (Qwen3-8B 2.00->2.67; Llama 2.00->3.17), null for random and Assistant-Axis controls [qin-etal-2026-the-granularity-axis-a-micro-to-macro-latent-direction-for-social-roles-in-language-models] Human annotators corroborate (pairwise macro-preference 0.64-0.90 across cells) [qin-etal-2026-the-granularity-axis-a-micro-to-macro-latent-direction-for-social-roles-in-language-models]

models: Llama-3.1-8B-Instruct · method: Difference-in-means direction, PCA, Activation Steering (Addition)
Understanding Moral Reasoning Trajectories in LLMs: Toward Probing-Based Explainability (2026)measured

A moral-framework probe decodes and steers mid-reasoning ethics

Details

Huang, Kwak & An train linear probes on last-token residual-stream states across all 81 layers of Llama-3.3-70B and Qwen2.5-72B to predict a 5-dimensional moral-framework distribution [huang-etal-2026-understanding-moral-reasoning-trajectories-in-llms] Framework encoding localizes to model-specific layers (63/81 for Llama-3.3-70B, 17/81 for Qwen2.5-72B), achieving 13.8-22.6% lower KL divergence than a training-prior baseline [huang-etal-2026-understanding-moral-reasoning-trajectories-in-llms] Steering along the probe direction modestly reduces the Framework Drift Rate (6.7% at Llama layer 6; 8.9% at Qwen layer 1) [huang-etal-2026-understanding-moral-reasoning-trajectories-in-llms] Probing and steering are restricted to the two open-weight models, while closed models (GPT-5, GPT-4o family) are evaluated only behaviorally [huang-etal-2026-understanding-moral-reasoning-trajectories-in-llms]

models: Llama 3.3 70B Instruct · method: Linear probing, Causal interventions (steering)
Graded Entity-Familiarity Readouts in Language Models: Polish Adaptation, Cross-Language Robustness, and Refusal Steering (2026)measured

An entity-familiarity direction reads and steers refusal in Polish LLMs

Details

Brzezinka reads final-prompt-token residual-stream activations across 14 models (Bielik, PLLuM, Gemma-4, Qwen3 families) on a 1,440-entity Polish dataset [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] A linear familiarity probe separates real from fabricated entities at AUROC 0.86-0.93, beating a lexical baseline (0.786) [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] In Polish-adapted Bielik/PLLuM models the score additionally tracks entity popularity (Spearman rho up to 0.567), with matched before/after controls isolating Polish continual-pretraining as the driver [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] Cross-language transfer retains 96-101% of within-language AUROC [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] In Gemma-4-12B (the only natively refusing model), adding a one-dimensional familiarity direction moves refusal rate monotonically (0.24 to 1.00 on known entities; 0.73 to 0.00 on unknown) [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering]

models: Llama-3.1-8B · method: Linear probing, Activation Steering (Addition)
Probing and Steering Evaluation Awareness of Language Models (2025)measured

An eval/deploy probe generalizes but its steering direction is inert

Details

Nguyen et al. train linear probes on Llama-3.3-70B-Instruct activations that separate real-world evaluation prompts from deployment prompts (AUROC ~0.82) [nguyen-etal-2025-probing-and-steering-evaluation-awareness-of-language-models] The same probes classify current artificial safety evaluations as test-like (Insider Trading 100%, Sandbagging 94.2%, Roleplaying 80.6%), evidencing an internal eval/deploy axis [nguyen-etal-2025-probing-and-steering-evaluation-awareness-of-language-models] Steering along the probe direction had negligible behavioral effect (<5% sandbagging recovery), so the direction is not behaviorally load-bearing [nguyen-etal-2025-probing-and-steering-evaluation-awareness-of-language-models] Only a prompt-suffix intervention produced meaningful recovery (83%), while SAE-feature steering reached at most about 25% [nguyen-etal-2025-probing-and-steering-evaluation-awareness-of-language-models]

models: Llama 3.3 70B Instruct · method: Linear probing, Causal interventions (steering)
A Circuit for Predicting Hierarchical Structure In-Context in Large Language Models (2025)measured

A latent-context direction drives in-context hierarchy tracking

Details

Saanum et al. train linear probes to decode, from attention-head outputs in Qwen2.5 (0.5B/1.5B/3B), whether current and previous latent-context tokens match [saanum-etal-2025-hierarchical-in-context-circuit] Second-order chunk decodability exceeds 90% for several heads, replicated on Gemma2-2B, Llama3.2-3B, SmolLM3-3B and Qwen3-0.6B [saanum-etal-2025-hierarchical-in-context-circuit] Heads exceeding an 85% decoding score are labeled context-matching heads [saanum-etal-2025-hierarchical-in-context-circuit] Ablating these decoded heads sharply reduces hierarchical-structure prediction accuracy, far more than ablating equally many random heads [saanum-etal-2025-hierarchical-in-context-circuit]

models: Llama-3.2-3B · method: Linear probing, Causal interventions (steering)
On the Mutual Influence of Gender and Occupation in LLM Representations (2025)measured

A PCA gender direction carries first-name discriminative signal

Details

An et al. approximate a female-male gender direction via PCA over gendered word pairs, evaluated on 470 first names' contextualized embeddings [an-etal-2025-on-the-mutual-influence-of-gender-and-occupation-in-llm-representations] The models are Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, OLMo-7B-0724-hf, and Phi-3.5-mini-instruct [an-etal-2025-on-the-mutual-influence-of-gender-and-occupation-in-llm-representations] The 1-D gender-direction projection achieves 75.18% binary gender-classification accuracy versus 75.46% for the full embedding (Llama-3.1-8B) [an-etal-2025-on-the-mutual-influence-of-gender-and-occupation-in-llm-representations] First-name gender representations shift with occupational context, e.g. "nurse" pulls embeddings toward femininity [an-etal-2025-on-the-mutual-influence-of-gender-and-occupation-in-llm-representations] The internal gender-direction coefficient correlates with downstream occupation-prediction bias (Spearman rho=0.61 Llama-3.1-8B, 0.76 Mistral-7B; both p<0.001) [an-etal-2025-on-the-mutual-influence-of-gender-and-occupation-in-llm-representations]

models: Llama-3.1-8B-Instruct · method: Direction Extraction
Representational Analysis of Binding in Language Models (2024)measured

A PCA 'Ordering ID' direction localizes and steers binding

Details

Dai, Heinzerling & Inui (2024) use PCA on entity/attribute activations to find a low-rank "OI subspace" whose leading component (OI-PC) encodes the "Ordering ID" underlying Feng & Steinhardt's Binding-ID mechanism, not previously localized to a direction [dai-etal-2024-representational-analysis-of-binding-in-language-models] Steering along OI-PC by n unit steps shifts the predicted attribute to that of Binding-ID n, quantified via Logit Difference and Logit Flip metrics that rise monotonically with step count [dai-etal-2024-representational-analysis-of-binding-in-language-models] A Spearman correlation confirms OI-PC tracks the true Order-ID rather than a position/filler-length confound [dai-etal-2024-representational-analysis-of-binding-in-language-models] The OI subspace is prevalent across five base models: Llama2-7B, Llama3-8B, Qwen1.5-7B, Pythia-6.9B, and the code-fine-tuned Float-7B [dai-etal-2024-representational-analysis-of-binding-in-language-models]

models: Llama-2-7B, Llama-3-8B · method: PCA, Direction Extraction, Causal interventions (steering)
Tool Calling Is Linearly Readable and Steerable in Language Models (2026)measured

A per-tool-pair mean-difference direction reads and switches tool choice

Details

Wu et al. show tool selection in tool-calling agents is carried by a single mean-difference direction per tool-pair in the residual stream across Gemma 3 (270M-27B), Qwen 3 (0.6B-14B) and Llama 3.1 (8B) [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models] Adding the direction switches the chosen tool at 83-100% on a 15-tool synthetic benchmark and 77-94% on tau-bench-airline, versus 0% for a random-direction control [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models] PCA over per-tool mean activations puts ~91% of variance in ~10 components for 15 tools, far below a random-Gaussian control [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models] Base (non-instruction-tuned) models already carry the correct tool internally (cosine readout 61-82% on BFCL vs 2-10% from base generation) [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models] SAEs and cross-layer transcoders trace a three-stage circuit: early tool-selective features, mid-layer attention heads, late-layer JSON-formatting features [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models]

models: Llama-3.1-8B-Instruct · method: Difference-in-means direction, Activation Steering (Addition), PCA, Activation patching, Sparse Autoencoders (SAE), Circuit tracing (attribution graphs)
Latent Structure of Affective Representations in Large Language Models (2026)measured

Emotion representations trace a V-shaped open manifold with neutral at the vertex

Details

Choi & Weber build a classification-accuracy dissimilarity matrix from pairwise probes across GoEmotions categories at every layer of Gemma-2-9B and Mistral-7B, embedding it via MDS/Isomap/UMAP and Procrustes-aligning to human valence-arousal norms [choi-weber-2026-affective-structure] The recovered layout is a V-shaped parabolic open curve (an open 1-manifold, H1=0) with neutral at the vertex and positive/negative valence fanning out, rather than the classical circular circumplex [choi-weber-2026-affective-structure] Isomap's geodesic-to-Euclidean distance ratio (1.00-1.80 Gemma, 0.80-1.42 Mistral) confirms the curvature is real but modest [choi-weber-2026-affective-structure] Adding probe-direction vectors to Llama-3-70B-Instruct shifts human-rated output valence strongly (mean 7.29 vs 1.08 on a 10-point scale for positive versus negative targets) [choi-weber-2026-affective-structure]

models: Llama-3-70B-Instruct · method: Linear probing, Isomap, UMAP, Activation Steering (Addition)
Quantitative Introspection in Language Models: Tracking Emotive States Across Conversation (2026)measured

A model's self-report tracks a steerable emotive-state direction

Details

Martorell & Bianchi train four contrastive mean-difference concept directions (wellbeing, interest, focus, impulsivity) per layer across five instruct LLMs [martorell-bianchi-2026-quantitative-introspection-in-language-models-tracking-emotive-states-across-conversation] Best-layer separation reaches Cohen's d of 1.67-3.60 (all p<1e-5) [martorell-bianchi-2026-quantitative-introspection-in-language-models-tracking-emotive-states-across-conversation] The model's own logit-based numeric self-report tracks the probe-defined internal state (Spearman rho 0.40-0.76; isotonic R^2 up to 0.93 in Llama-3.1-8B-Instruct) [martorell-bianchi-2026-quantitative-introspection-in-language-models-tracking-emotive-states-across-conversation] Activation steering along the concept direction causally shifts self-reports, and cross-concept steering can improve introspective fidelity for a different concept by up to delta R^2 = 0.30 [martorell-bianchi-2026-quantitative-introspection-in-language-models-tracking-emotive-states-across-conversation]

models: Llama-3.2-3B-Instruct, Llama-3.2-1B-Instruct, Llama-3.1-8B-Instruct · method: Difference-in-means direction, Activation Steering (Addition), Causal interventions (steering)
Intrinsic Guardrails: How Semantic Geometry of Personality Interacts with Emergent Misalignment in LLMs (2026)measured

Twelve trait directions form stable antisocial and prosocial clusters

Details

Aneja et al. extract 12 persona-vector directions (Big Five, Dark Triad, plus Sycophancy/Apathy/Impoliteness/Evil) across real Llama and Qwen models and their emergently-misaligned variants [aneja-etal-2026-intrinsic-guardrails-semantic-geometry-personality] Psychopathy, Narcissism, Machiavellianism, and Evil cluster with high pairwise cosine (>0.50), anti-correlated with a prosocial group [aneja-etal-2026-intrinsic-guardrails-semantic-geometry-personality] The 12-trait geometric structure is stable across fine-tuning domains and scale (linear CKA >0.97 for Llama, >0.88 for smallest Qwen; Procrustes disparity 0.02-0.08) [aneja-etal-2026-intrinsic-guardrails-semantic-geometry-personality] Ablating a native Evil vector shifts misalignment rate by +43.3pp in Qwen-2.5-14B, and amplifying a base-derived vector drives misalignment to the 0% floor [aneja-etal-2026-intrinsic-guardrails-semantic-geometry-personality]

models: Llama-3.1-8B, Llama-3.2-1B · method: Difference-in-means direction, Centered Kernel Alignment (CKA), Orthogonal Procrustes alignment, Causal interventions (steering)
Functional Subspace, where language models can use vector algebra to solve problems (2026)measured

ICL answer tokens are affine functions in a PCA subspace

Details

Lee & Vijayan collect residual-stream activations from six pretrained LLMs across six in-context relational tasks and reduce each layer to 30 principal components [lee-vijayan-2026-functional-subspace-vector-algebra] For each layer and component an answer token's projection is fit as an affine function of query and separator projections (a = alpha*q + beta*s + gamma) [lee-vijayan-2026-functional-subspace-vector-algebra] R^2 is high along a small, largely layer-consistent subset of components, rising in later layers, so ICL answer prediction reduces to an affine operation there [lee-vijayan-2026-functional-subspace-vector-algebra] Query, separator, and answer tokens form well-separated clusters in the top-3 high-R^2 component subspace across models and tasks [lee-vijayan-2026-functional-subspace-vector-algebra] Projections onto the highest-R^2 component differ significantly between correct and incorrect predictions (p<0.05) in mid/late layers, with no causal intervention performed [lee-vijayan-2026-functional-subspace-vector-algebra]

models: Llama-3.1-8B · method: Linear Relational Embedding (LRE), PCA
A Geometric Account of Activation Steering through Angle-Norm Decomposition (2026)measured

Concept information is angular, yet norm-preservation is not optimal steering

Details

Aparin & Gaintseva decompose each hidden state into a radial norm and an angular concept score against a unit steering direction, across seven models [aparin-gaintseva-2026-geometric-account-activation-steering-angle-norm-decomposition] Linear probes on normalized hidden states match raw probes while norm-only probes stay near chance, so concept information is essentially angular, not radial [aparin-gaintseva-2026-geometric-account-activation-steering-angle-norm-decomposition] They systematically compare six steering variants that vary norm-preservation and angular-target enforcement [aparin-gaintseva-2026-geometric-account-activation-steering-angle-norm-decomposition] Despite concepts being angular, strict norm preservation is not most stable: moving the radial scale from beta=1.0 to 1.2 improves perplexity ~1.8x at a task-metric cost within ~2.5 points [aparin-gaintseva-2026-geometric-account-activation-steering-angle-norm-decomposition] This dissociates where a concept lives (angle) from what a stable intervention should manipulate (angle plus a non-unit radial scale) [aparin-gaintseva-2026-geometric-account-activation-steering-angle-norm-decomposition]

models: Llama-3.1-8B-Instruct, Llama-3.1-8B, Llama-3.1-70B-Instruct, Llama-3.2-1B-Instruct · method: Linear probing, Angle-Norm Decomposition Steering
Linear Representations of Political Perspective Emerge in Large Language Models (2025)measured

Attention-head probes read and steer political ideology

Details

Kim, Evans & Schein fit a ridge-regression probe per attention head in three open chat LLMs to predict 552 U.S. lawmakers' DW-NOMINATE ideology scores [kim-evans-schein-2025-political-perspective] The best single head reaches Spearman rho 0.846-0.861, and an ensemble of the top 32 heads reaches 0.870-0.885, concentrated in middle layers [kim-evans-schein-2025-political-perspective] A nonlinear MLP probe matches the linear probe, supporting a linear-direction characterization [kim-evans-schein-2025-political-perspective] Probes fit on lawmaker ideology transfer zero-shot to predicting 400 news outlets' slant (rho 0.720-0.798) [kim-evans-schein-2025-political-perspective] Adding scaled top-head directions to activations shifts GPT-4o-rated political slant, correlating up to 0.607 with steering magnitude [kim-evans-schein-2025-political-perspective]

models: Llama-2-7B-Chat · method: Linear probing, Activation Steering (Addition)
Dissociating the Internal Representations of Sycophancy in LLMs (2026)measured

Factual and opinion sycophancy occupy partially distinct directions

Details

Baez, Karny & Pataranutaporn build diff-in-means vectors and probes for factual vs opinion sycophancy in Gemma-3-12B-IT and Llama-3.1-8B-Instruct [baez-etal-2026-dissociating-sycophancy] The LDA discriminant reaches Cohen's d=9.36 (Gemma) and 6.44 (Llama), with smaller cross-type transfer degradation in Gemma (AUC drop 0.06-0.07) than Llama (0.22-0.30) [baez-etal-2026-dissociating-sycophancy] In Gemma the two subtype vectors have positive cosine (+0.68) and cross-type steering transfers (R^2 up to 0.93-1.00) [baez-etal-2026-dissociating-sycophancy] In Llama the vectors have negative cosine (-0.15) and cross-type steering decreases rather than increases target sycophancy, a model-dependent causal interference [baez-etal-2026-dissociating-sycophancy]

models: Llama-3.1-8B-Instruct · method: Linear Discriminant Analysis (LDA), Difference-in-means direction, Activation Steering (Addition)
Proxy Reward Internalization and Mechanistic Exploitation: A Learned Precursor to Reward Hacking and Its Generalization (2026)measured

Diff-in-means reward-hacking directions forecast and causally control misalignment

Details

Beigi, Jin & Huang extract three diff-in-means directions (context-sycophantic-agreement, premise-rejection, evaluator-rejection) during RL training of Qwen2.5-Coder-14B-Instruct and track their projection score across checkpoints [beigi-jin-huang-2026-proxy-reward-internalization-and-mechanistic-exploitation] Each direction is linearly decodable (AUROC 0.85-0.90) and a linear fit from probe score forecasts future out-of-domain misalignment (R^2=0.77), preceding the misalignment rise by roughly 45 training steps [beigi-jin-huang-2026-proxy-reward-internalization-and-mechanistic-exploitation] Joint ablation of the three directions lowers the hack rate by 26 points while preserving coding accuracy, and injection along them increases hacking [beigi-jin-huang-2026-proxy-reward-internalization-and-mechanistic-exploitation] The probe methodology generalizes to Qwen2.5-Coder 1.5B/7B/32B, Llama-3.1-8B-Instruct, and OLMo-7B [beigi-jin-huang-2026-proxy-reward-internalization-and-mechanistic-exploitation]

models: Llama-3.1-8B-Instruct · method: Difference-in-means direction
Do Personality Traits Interfere? Geometric Limitations of Steering in Large Language Models (2026)measured

Orthogonalized personality directions still bleed behaviorally

Details

Bhandari, Naseem & Nasim extract five Big-Five personality steering directions in LLaMA-3-8B-Instruct and Ministral-8B-Instruct and find substantial pairwise cosine overlap [bhandari-etal-2026-personality-traits-interfere-steering] A battery of exact-orthogonalization schemes drives pairwise cosine similarity to near machine precision (max|cos|<1e-8 under hard orthonormalization) [bhandari-etal-2026-personality-traits-interfere-steering] Even then, steering one trait at a time measurably shifts other geometrically orthogonal traits' scores [bhandari-etal-2026-personality-traits-interfere-steering] Geometric orthogonality is therefore not sufficient for behavioral independence, a competing claim against papers finding natural or restorable independence [bhandari-etal-2026-personality-traits-interfere-steering]

models: Llama-3-8B-Instruct · method: Difference-in-means direction, Steering-direction orthogonalization battery
Understanding (Un)Reliability of Steering Vectors in Language Models (2025)measured

Two activation statistics predict steering-vector reliability

Details

Braun et al. construct diff-in-means steering directions in Llama-2-7B-Chat across 36 binary behavior datasets [braun-etal-2025-steering-vector-reliability] They quantify mean training-set cosine agreement with the aggregate direction and a signal-detection discriminability index d' of positive/negative separation [braun-etal-2025-steering-vector-reliability] Both statistics predict how often the causal steering effect reverses sign (anti-steerable samples, 3-50% across datasets) [braun-etal-2025-steering-vector-reliability] Better-aligned, better-separated datasets are causally more reliable to steer, a trend supported by ranges and figures rather than formal correlation statistics [braun-etal-2025-steering-vector-reliability]

models: Llama-2-7B-Chat · method: Difference-in-means direction, Activation Steering (Addition)
A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders (2024)measured

Feature absorption is decoder-direction composition, causally isolable

Details

Chanin et al. formalize SAE feature absorption in a toy model as a child latent's decoder direction acquiring a component along a parent feature (W_d2 = f2 + delta*f1) [chanin-etal-2024-feature-splitting-absorption-sparse-autoencoders] Empirically they measure a feature-absorption rate on first-letter latents in Gemma Scope SAEs (Gemma-2-2B) plus their own SAEs trained on Qwen2-0.5B and Llama-3.2-1B [chanin-etal-2024-feature-splitting-absorption-sparse-autoencoders] Absorption is detected when a latent has cosine >0.025 with the probe direction and the largest negative ablation effect (at least 1.0 above the runner-up) [chanin-etal-2024-feature-splitting-absorption-sparse-autoencoders] Projecting the probe direction out of an absorbing latent removes its ablation effect, confirming the composition model causally [chanin-etal-2024-feature-splitting-absorption-sparse-autoencoders] Varying SAE width or sparsity alone does not resolve absorption, motivating architectural fixes like Matryoshka SAEs [chanin-etal-2024-feature-splitting-absorption-sparse-autoencoders]

models: Llama-3.2-1B · method: Sparse Autoencoders (SAE), Causal interventions (steering)
Persona Vectors: Monitoring and Controlling Character Traits in Language Models (2025)measured

Persona-vector projections predict and prevent trait drift

Details

Chen, Arditi, Sleight, Evans & Lindsey extract persona vectors (diff of mean response-token activations) for evil, sycophancy, and hallucination in Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct [chen-etal-2025-persona-vectors] Final-prompt-token projections correlate with subsequent trait expression (Pearson r=0.634-0.830; 94.7% judge agreement) [chen-etal-2025-persona-vectors] The same projection over a finetuning dataset's responses predicts how much that run shifts trait propensity (r=0.76-0.97), flagging problematic data before finetuning [chen-etal-2025-persona-vectors] Proactively steering toward the undesired persona during training reduces trait shifts while keeping coherence above 80 and better preserving MMLU than regular finetuning [chen-etal-2025-persona-vectors]

models: Llama-3.1-8B-Instruct · method: Difference-in-means direction, Activation Steering (Addition)
Hallucination Basins: A Dynamic Framework for Understanding and Controlling LLM Hallucinations (2026)measured

A diff-of-means hallucination direction causally raises hallucination dose-dependently

Details

Cherukuri & Varshney formalize hallucinations as basin attractors across Llama-3.2-1B/3B, Gemma-2-2B, Qwen2.5-1.5B, Llama-3.1-8B, and Mistral-7B-v0.3 [cherukuri-varshney-2026-hallucination-basins-a-dynamic-framework-for-understanding-and-controlling-llm-hallucinations] Basin separation is strongly task-dependent: factoid tasks show sharp centroid separation (variance ratio up to 4.55, AUROC up to 1.000 on MuSiQue) while summarization and TruthfulQA hover near chance [cherukuri-varshney-2026-hallucination-basins-a-dynamic-framework-for-understanding-and-controlling-llm-hallucinations] Interpolating factual hidden states toward the hallucination centroid via a diff-of-means steering vector produces a monotonic dose-response increase in hallucination probability, exceeding random and orthogonal controls [cherukuri-varshney-2026-hallucination-basins-a-dynamic-framework-for-understanding-and-controlling-llm-hallucinations]

models: Llama-3.2-1B, Llama-3.2-3B, Llama-3.1-8B · method: Difference-in-means direction, Activation Steering (Addition)
Concepts Whisper While Syntax Shouts: Spectral Anti-Concentration and the Dual Geometry of Transformer Representations (2026)measured

Concept directions anti-concentrate in the unembedding's low-variance spectral tail

Details

Acharya, Rimal & Dhakal project concept diff-of-means vectors onto the unembedding-covariance eigenbasis across 17 models in 5 families [acharya-rimal-dhakal-2026-spectral-anti-concentration] Concept-direction energy anti-concentrates in the low-eigenvalue tail: Spectral Center of Mass averages 0.926 versus 0.758 for random directions (Gini-deviation -0.282) [acharya-rimal-dhakal-2026-spectral-anti-concentration] Static unembedding-row contrasts show the opposite sign, and POS/syntax directions concentrate in the high-variance subspace in 6 of 8 architectures (p<0.013), a genuine dual-geometry split [acharya-rimal-dhakal-2026-spectral-anti-concentration] On Llama-3.1-8B, injecting only the top-10%-eigenvalue component gives +383% perplexity versus +223% for the bottom-10% component (Cohen's d=1.80), replicated across 5 models [acharya-rimal-dhakal-2026-spectral-anti-concentration]

models: Llama-3.2-1B, Llama-3.2-3B, Llama-3.1-8B · method: Geometric analysis, Causal interventions (steering)
Representation Engineering: A Top-Down Approach to AI Transparency (2023)measured

Linear Artificial Tomography reads and controls concepts across LLMs

Details

Zou et al. introduce Linear Artificial Tomography (LAT): from contrastive stimulus pairs, take the first principal component of paired activation differences as a concept reading vector [zou-etal-2023] Across LLaMA-2-Chat (7B/13B/70B), Vicuna-13B, Vicuna-33B-Uncensored, and DeBERTa, a single direction classifies and causally controls honesty, ethics, morality, emotion, and bias [zou-etal-2023] LAT recovers truthfulness on DeBERTa more accurately than contrast-consistent search [zou-etal-2023] A harmfulness direction in Vicuna-13B stays a >90% classifier under jailbreaks, and boosting its salience raises harmless-response rates under attack [zou-etal-2023]

models: Llama-2-7B-Chat, Llama-2-13B-Chat, Llama-2-70B-Chat · method: Linear Artificial Tomography (LAT), PCA, Causal interventions (steering), Low-Rank Representation Adaptation (LoRRA)
The Confidence Manifold: Geometric Structure of Correctness Representations in Language Models (2026)measured

Answer-correctness sits in a 3-8D linear subspace, causally steerable

Details

Cho et al. study self-assessed correctness on TruthfulQA pairs across 9 models spanning 5 families, finding the discriminative signal is a low-dimensional linear subspace, not a curved manifold [cho-etal-2026-confidence-manifold] A PLS sweep peaks at 3-8 dimensions (e.g. peak AUC 0.90 at layer 23, dim 5, Mistral-7B), and nonlinear classifiers give no gain over a linear boundary [cho-etal-2026-confidence-manifold] Correct/incorrect classes form roughly Gaussian clusters separated by a mean shift: a two-mean centroid detector matches the linear probe (0.90 vs 0.89 AUC), robust with 25 examples/class [cho-etal-2026-confidence-manifold] This 3-8D discriminative subspace is lower-dimensional than the representation's 8-12D intrinsic dimension at the same layers [cho-etal-2026-confidence-manifold] Adding the learned direction shifts downstream error rate up to 10.9 points dose-dependently, while random and orthogonal controls have no reliable effect [cho-etal-2026-confidence-manifold]

models: Llama-3.2-1B-Instruct, Llama-3.2-3B-Instruct · method: Linear probing, Partial Least Squares (PLS), Difference-in-means direction, Activation Steering (Addition)
Language Models Represent and Transform Concepts with Shared Geometry (2026)measured

Contextual displacement forms a vector field tracking concreteness

Details

Hu, Niu & Varma define a per-word contextual displacement vector phi(w,tau) = r(w,tau) - r(w,tau_0) in activation space, forming a vector field over the vocabulary rather than one shared direction [hu-etal-2026-concept-transformation-geometry] Across 4 models x 3 depths, word density correlates negatively with displacement magnitude (Spearman rho -0.32 to -0.22) [hu-etal-2026-concept-transformation-geometry] Lexical concreteness correlates negatively with directional deviation (rho -0.41 to -0.21): concrete, low-density concepts move less and more consistently under context [hu-etal-2026-concept-transformation-geometry] The field has no dominant direction (leading PC <15% variance) and is more dispersed than uniform-random; no causal intervention is performed [hu-etal-2026-concept-transformation-geometry]

models: Llama-2-7B, Llama-2-13B, Llama-3-8B, Llama-3.1-8B · method: PCA, Geometric analysis
Uncertainty as Feature Gaps: Epistemic Uncertainty Quantification of LLMs (2025)measured

Contrastive-PCA feature directions predict LLM epistemic uncertainty

Details

Bakman et al. derive an epistemic-uncertainty bound in terms of hidden-state displacement along semantic-feature directions [bakman-etal-2025-uncertainty-as-feature-gaps-epistemic-uncertainty-quantification-of-llms] They extract three directions (context-reliance, context-comprehension, honesty) via contrastive prompt-pair PCA on Llama-3.1-8B, Mistral-7B-v0.3, and Qwen2.5-7B [bakman-etal-2025-uncertainty-as-feature-gaps-epistemic-uncertainty-quantification-of-llms] Per-token projections onto these directions improve the Prediction Rejection Ratio by up to 13 points over baselines [bakman-etal-2025-uncertainty-as-feature-gaps-epistemic-uncertainty-quantification-of-llms] Replacing PCA extraction with a plain mean-difference direction substantially degrades performance, evidencing the specific extracted direction does the work [bakman-etal-2025-uncertainty-as-feature-gaps-epistemic-uncertainty-quantification-of-llms]

models: Llama-3.1-8B · method: Direction Extraction, Difference-in-means direction
Scenario-based Probing and Steering Cultural Values in Large Language Models (2026)measured

Cultural-value steering axes are entangled, mirroring real value correlations

Details

Dang, Kieu & Masud map LLMs onto the Inglehart-Welzel cultural map via scenario forced-choice probing and diff-of-means steering at 4 tuned layers per model [dang-etal-2026-scenario-based-probing-and-steering-cultural-values-in-llms] The steering direction is a signed bidirectional axis (negative alpha reverses the shift), saturating at alpha in [0.5, 0.6] [dang-etal-2026-scenario-based-probing-and-steering-cultural-values-in-llms] Steering one cultural axis causes correlated shifts on the other, an entanglement ratio 0.72-0.81 (dropping to 0.53 for Llama under hybrid steering), mirroring the real-world WVS axis correlation r=0.474 [dang-etal-2026-scenario-based-probing-and-steering-cultural-values-in-llms] Capability cost measured via Global-MMLU is largest for Gemma (0.608->0.530) [dang-etal-2026-scenario-based-probing-and-steering-cultural-values-in-llms]

models: Llama-3.2-3B-Instruct · method: Difference-in-means direction, Direction Extraction, Causal interventions (steering)
GEMS: Geometric Constraints Enable Multi-Semantic Superposition in LLMs (2026)measured

Non-orthogonal steering directions mutually dampen; GEMS restores them

Details

Deng (GEMS) shows three diff-in-means persona directions (empathy, accountability, minimalism) in Qwen3.5-4B are pairwise cosine-similar at 0.74-0.87 [deng-2026-gems-geometric-constraints-multi-semantic-superposition-llms] Injecting all three simultaneously degrades each direction's terminal-layer alignment from 0.64-0.66 to 0.38-0.43, a 34-43% mutual reduction attributed to non-orthogonality [deng-2026-gems-geometric-constraints-multi-semantic-superposition-llms] Unconstrained additive injection separately drives residual-stream norm to 3.4x baseline, flattening next-token probability (top-1 43% to 11%) [deng-2026-gems-geometric-constraints-multi-semantic-superposition-llms] GEMS applies per-token Gram-Schmidt orthogonalization plus norm-constrained recombination restricted to the attention-output pathway [deng-2026-gems-geometric-constraints-multi-semantic-superposition-llms] On GSM8K with 3 concurrent directions, naive steering collapses accuracy 92% to 4% while GEMS recovers 98% at only +2.2% perplexity [deng-2026-gems-geometric-constraints-multi-semantic-superposition-llms]

models: Llama-3.2-3B-Instruct · method: Difference-in-means direction, Orthogonalized Multi-Direction Superposition Steering (GEMS)
A Shared Geometry of Difficulty in Multilingual Language Models (2026)measured

A difficulty direction is language-agnostic shallow, language-specific deep

Details

Civelli et al. train per-layer linear probes on four LLMs to predict a continuous problem-difficulty score across 21 languages [civelli-etal-2026-shared-geometry-difficulty-multilingual] Deep-layer probes reach high same-language accuracy (Llama-3.1-8B rho=0.822 at layer ~30) but generalize poorly across languages [civelli-etal-2026-shared-geometry-difficulty-multilingual] Shallow-layer probes carry a language-agnostic difficulty signal (cross-lingual rho=0.783 at layer ~16) [civelli-etal-2026-shared-geometry-difficulty-multilingual] Fixing the deep same-language-optimal layer costs 0.177 rho cross-lingually while the shallow transfer-optimal layer costs only 0.014 rho in-language; replicated in Qwen3-8B [civelli-etal-2026-shared-geometry-difficulty-multilingual] No causal steering is performed, so the claim rests on decodability alone [civelli-etal-2026-shared-geometry-difficulty-multilingual]

models: Llama-3.1-8B, Llama-3.2-3B, Llama-3.2-1B · method: Linear probing
Label Words as Local Task Vectors in In-Context Learning (2024)measured

Multi-demonstration ICL needs distributed rule vectors, not one task vector

Details

Zheng et al. test the single task-vector account on tasks needing multiple demonstrations (e.g. string-length categorization) in LLaMA-7B [zheng-etal-2024] Patching one query-position task vector recovers single-demonstration knowledge tasks but only near-chance accuracy on multi-demonstration tasks [zheng-etal-2024] Gradient-times-attention saliency shows information flows from each demonstration's answer-token position to the query [zheng-etal-2024] Patching all per-demonstration local rule vectors together recovers ICL-level performance, improving with more demonstrations [zheng-etal-2024] Demixed PCA shows each rule vector encodes an abstracted query-answer summary rather than raw string length [zheng-etal-2024]

models: LLaMA-7B · method: Activation patching, Gradient × attention saliency, PCA, Demixed PCA (dPCA)
When Roleplaying, Do Models Believe What They Say? (2026)measured

Emergent misalignment rotates an LLM's truth direction

Details

Sturgeon, Africa & Black fit L2-regularized logistic-regression truth-direction probes on Llama-3.3-70B-Instruct and Qwen3-8B, explicitly rejecting mass-mean (diff-in-means) probing [sturgeon-etal-2026-when-roleplaying-do-models-believe-what-they-say] Persona SFT shifts the truth-probe score by only +0.05 and Open Character Training by +0.089-0.124 [sturgeon-etal-2026-when-roleplaying-do-models-believe-what-they-say] Emergent Misalignment produces a much larger +0.28 shift (56% defend rate, 82% downstream-reasoning consistency) [sturgeon-etal-2026-when-roleplaying-do-models-believe-what-they-say] EM also rotates the truth direction itself (cosine ~0.58), indicating restructured truth-representation geometry rather than local override [sturgeon-etal-2026-when-roleplaying-do-models-believe-what-they-say] The analysis is observational, with no steering or causal intervention performed [sturgeon-etal-2026-when-roleplaying-do-models-believe-what-they-say]

models: Llama 3.3 70B Instruct · method: Linear probing
How Do Language Models Bind Entities in Context? (2023)measured

Binding-ID vectors form a subspace whose distances predict confusability

Details

Feng & Steinhardt identify additive "binding ID vectors" attached to entity and attribute activations that solve variable binding, using causal interventions on LLaMA-1 (30B primary, plus 13B and 65B) and the Pythia family [feng-steinhardt-2023-how-do-language-models-bind-entities-in-context] The binding vectors occupy a continuous subspace in which the distance between two binding vectors predicts how often the model confuses the corresponding entity-attribute bindings [feng-steinhardt-2023-how-do-language-models-bind-entities-in-context] Patching, adding or removing binding-ID vectors changes which attribute the model retrieves for a given entity, demonstrated via causal mediation analysis rather than linear probing [feng-steinhardt-2023-how-do-language-models-bind-entities-in-context]

models: LLaMA-30B, LLaMA-13B, LLaMA-65B · method: Activation patching
Linear Personality Probing and Steering in LLMs: A Big Five Study (2025)measured

Big Five trait directions are near-orthogonal but steer only forced-choice

Details

Frising & Balcells fit OLS trait directions from 406 role-play character descriptions in Llama-3.3-70B-Instruct, separately per layer and token position [frising-balcells-2025-linear-personality-probing-steering-big-five] The five OCEAN regression directions show low cross-talk / near-orthogonality, unlike top-variance SVD directions that collapse toward one shared personality axis [frising-balcells-2025-linear-personality-probing-steering-big-five] Same-trait regression directions across token positions are well-aligned, the opposite of SVD's per-position behavior [frising-balcells-2025-linear-personality-probing-steering-big-five] Steering the mean-input-prompt direction monotonically shifts forced-choice Extraversion for |alpha|<=0.4, then degrades into gibberish [frising-balcells-2025-linear-personality-probing-steering-big-five] The steering effect vanishes once character context is already in the prompt, showing the direction's causal reach is context-dependent [frising-balcells-2025-linear-personality-probing-steering-big-five]

models: Llama 3.3 70B Instruct · method: Linear probing, Activation Steering (Addition)
How Few-Shot Examples Add Up: A Causal Decomposition of Function Vectors in In-Context Learning (2026)measured

A function vector decomposes additively into causal per-shot sub-vectors

Details

Wang et al. redefine the function vector per-prompt and decompose it into additive per-example sub-vectors extracted via attention masking [wang-etal-2026-causal-decomposition-function-vectors] Global OLS fits weights so v_FV = sum_i w_i v_i + epsilon, reconstructing closely (mean cosine >=0.925, R^2 >=0.875) [wang-etal-2026-causal-decomposition-function-vectors] Mismatched-dictionary and orthogonalized-sub-FV null controls collapse to much lower fit, confirming the decomposition is not vacuous [wang-etal-2026-causal-decomposition-function-vectors] Injecting the reconstructed vector into 0-shot prompts recovers most of the full vector's causal steering effect (accuracy ratio 0.818-1.116) [wang-etal-2026-causal-decomposition-function-vectors] Under contextualization, attention shifts its share toward unambiguous examples (32% to 61%), with the query-key pathway dominating the gain [wang-etal-2026-causal-decomposition-function-vectors]

models: Llama-3.2-1B, Llama-3.2-3B, Llama-3.1-8B-Instruct · method: Function vector extraction (causal-mediation head selection + sum), Causal interventions (steering), Activation patching
Function Vectors in Large Language Models (2024), In-Context Learning Creates Task Vectors (2023), Unifying Attention Heads and Task Vectors via Hidden State Geometry in In-Context Learning (2025), Do Different Prompting Methods Yield a Common Task Representation in Language Models? (2025)measured

Function vectors are compact directions that causally trigger ICL tasks

Details

Todd et al. use causal-mediation across 40+ ICL tasks in GPT-J-6B, GPT-NeoX-20B, and Llama-2 (7B/13B/70B) to find a small set of early-middle attention heads with high average indirect effect [todd-etal-2024] Summing those heads' mean per-task activations gives a function vector that, added at a middle layer, triggers the task even zero-shot (Llama-2 70B: 8.2% to 83.8%) [todd-etal-2024] Function vectors are portable across prompt formats and compose additively over functions, though some composed tasks are not expressible as embedding offsets [todd-etal-2024] A sharp late-layer drop in causal effect indicates function vectors trigger nonlinear downstream computation rather than a linear read-out [todd-etal-2024] Hendel et al. independently confirm a single task vector read from one residual-stream activation, recovering 80-90% of ICL across LLaMA, GPT-J, and Pythia [todd-etal-2024] Zheng et al. qualify that genuinely multi-demonstration tasks have no single task vector, and Yang et al. replicate the effect while explaining it via label-unembedding alignment [todd-etal-2024] Davidson et al. find instruction-derived and demonstration-derived function vectors only partially converge, sharing few top heads [todd-etal-2024]

models: Llama-2-7B, Llama-2-13B, Llama-2-70B, LLaMA-7B, LLaMA-13B, LLaMA-30B · method: Activation patching, Function vector extraction (causal-mediation head selection + sum), ICL task-vector extraction via dummy-query patching, Activation Steering (Addition)
Textual Steering Vectors Can Improve Visual Understanding in Multimodal Large Language Models (2025)measured

Text-only steering directions transfer causally to MLLM image tokens

Details

Gan et al. test whether steering directions found in a text-only LLM backbone stay causal after multimodal fine-tuning, applied to image tokens it never saw [gan-etal-2025-textual-steering-vectors-improve-visual-understanding-mllms] For four visual-concept taxonomies they extract directions from Gemma2-2B/9B and Llama-3.1-8B via mean-shift, probing, and SAE decoders, using text only [gan-etal-2025-textual-steering-vectors-improve-visual-understanding-mllms] Adding these directions to image tokens in PaliGemma2-3B/10B and Idefics3-8B-Llama3 raises CV-Bench spatial accuracy by up to +7.3%, with mean-shift strongest [gan-etal-2025-textual-steering-vectors-improve-visual-understanding-mllms] Frozen-hyperparameter generalization to five held-out datasets gives +7.6% average (up to +34.2% on CLEVR for Idefics3) [gan-etal-2025-textual-steering-vectors-improve-visual-understanding-mllms] Each visual-concept taxonomy activates fewer than 10 of 16k-32k SAE features in the text backbone, and prompting is far weaker than steering for multimodal reasoning [gan-etal-2025-textual-steering-vectors-improve-visual-understanding-mllms]

models: Llama-3.1-8B · method: Difference-in-means direction, Linear probing, Sparse Autoencoders (SAE), Activation Steering (Addition)
Sycophancy Hides Linearly in the Attention Heads (2026)measured

Correct-to-incorrect sycophancy localizes to sparse middle-layer heads

Details

Genadi et al. train logistic-regression probes on residual, MLP, and per-head attention activations of Gemma-3-4B and Llama-3.2-3B to localize correct-to-incorrect sycophancy [genadi-etal-2026-sycophancy-hides-linearly-attention-heads] Residual and MLP probes are broadly accurate (Gemma residual 99.6%), but attention-probe accuracy concentrates sharply in a sparse subset of middle-layer heads [genadi-etal-2026-sycophancy-hides-linearly-attention-heads] Steering the probe direction at those heads cuts sycophancy from 40.7% to 34.4% (Gemma-3) and 51.7% to 25.0% (Llama-3.2), while MLP/residual steering underperforms [genadi-etal-2026-sycophancy-hides-linearly-attention-heads] The sycophancy direction is only mildly anti-correlated with a truthful direction (cosine -0.22, 32% head overlap), indicating related but distinct linear mechanisms [genadi-etal-2026-sycophancy-hides-linearly-attention-heads]

models: Llama-3.2-3B-Instruct · method: Linear probing, Activation Steering (Addition)
Signal in the Noise: Polysemantic Interference Transfers and Predicts Cross-Model Influence (2025)measured

SAE decoder cosine similarity quantifies interference and transfers steering

Details

Gong et al. train SAEs on Pythia-70M and GPT-2-Small and quantify feature-pair interference as the cosine similarity of their SAE decoder directions in activation space [gong-etal-2025-signal-in-the-noise-polysemantic-interference-transfers-and-predicts-cross-model-influence] Using these directions as steering vectors (scaled +/-20), small-model interference geometry transfers to larger black-box instruction-tuned models [gong-etal-2025-signal-in-the-noise-polysemantic-interference-transfers-and-predicts-cross-model-influence] One high-interference direction steered into Llama-3.1-8B-Instruct transfers at over 95% success [gong-etal-2025-signal-in-the-noise-polysemantic-interference-transfers-and-predicts-cross-model-influence] Pythia-derived success is 49.74% for high-interference pairs versus 21.76-22.87% for low-interference or random pairs [gong-etal-2025-signal-in-the-noise-polysemantic-interference-transfers-and-predicts-cross-model-influence] Transfer is further validated on Llama-3.1-70B-Instruct and Gemma-2-9B-Instruct, with an additional Gemma-2-2B experiment [gong-etal-2025-signal-in-the-noise-polysemantic-interference-transfers-and-predicts-cross-model-influence]

models: Llama-3.1-8B-Instruct, Llama-3.1-70B-Instruct · method: Sparse Autoencoders (SAE)
How Does Alignment Tuning Shape Representations of Sycophancy and Related Cue-Induced Biases in LLMs? (2026)measured

Cue-induced bias directions are installed by alignment tuning

Details

Gupta et al. extract a per-bias diff-of-means direction for seven cue-triggered bias types across base and instruct checkpoints of five model families [gupta-etal-2026-alignment-tuning-sycophancy-directions] Four of five base models flip on under 5% as many pairs as their instruct counterparts, with base activations carrying essentially no cue-specific signal, so susceptibility is installed by alignment tuning [gupta-etal-2026-alignment-tuning-sycophancy-directions] Within instruct models each bias direction transfers to held-out data at mean AUROC 0.69-0.82, localizing to late-middle layers [gupta-etal-2026-alignment-tuning-sycophancy-directions] Biases stay representationally distinct, with even behaviorally similar biases occupying different, sometimes anti-aligned directions [gupta-etal-2026-alignment-tuning-sycophancy-directions] Subtracting the direction while preserving over 90% of correct answers recovers 7-20% of bias-induced errors, versus under 5% for a random direction [gupta-etal-2026-alignment-tuning-sycophancy-directions]

models: Llama-3.1-8B, Llama-3.1-8B-Instruct · method: Difference-in-means direction, Activation Steering (Addition)
Emergent Causal-Geometric Dynamics Across Depth in Large Language Models (2026)measured

A steering direction's angular component becomes causally load-bearing only in a late prediction-centric depth phase

Details

Haim & McNamee (2026) measure Participation Ratio and layer-wise Spearman correlation between representational distance (Euclidean vs. angular) and next-token-distribution KL divergence across real Llama-3.1-8B, Mistral-7B-v0.3, and Qwen2.5-7B, finding a consistent bi-phasic depth structure -- an early context-centric phase and a late prediction-centric phase, with phase-transition layer agreeing within 2.5% of depth across three tasks and three models [haim-mcnamee-2026-emergent-causal-geometric-dynamics-across-depth-in-large-language-models] Steering vectors decomposed into pure-angular vs. pure-norm interventions are causally effective (logit-preference shift) only when angular and only in the late phase; norm interventions are ineffective throughout [haim-mcnamee-2026-emergent-causal-geometric-dynamics-across-depth-in-large-language-models]

models: Llama-3.1-8B · method:
Emergence and Effectiveness of Task Vectors in In-Context Learning: An Encoder-Decoder Perspective (2025)measured

In-context task representations are k-NN-decodable and causally patchable across real LLMs and an SSM

Details

Han, Song, Gore & Agrawal define Task Decodability, a k-NN classifier score on the hidden state immediately before the target token, and show it strongly correlates with ICL accuracy across real Llama-3.1-8B/70B, Gemma-2-2B/9B/27B, OLMo-7B (across pretraining checkpoints), and Mamba-8B on POS-tagging and bitwise-arithmetic tasks [han-song-etal-2025-emergence-of-abstractions-task-vectors-icl] Positive activation-patching interventions on well-separated task representations improve accuracy up to +14pp, while negative interventions degrade it up to -15pp, versus only +/-2-6pp for overlapping tasks such as XOR/XNOR [han-song-etal-2025-emergence-of-abstractions-task-vectors-icl] Finetuning the first 10 layers raises Task Decodability from 0.68 to 0.95 (POS) and 0.43 to 0.85 (bitwise), with accuracy gains of 37 and 24 points respectively over finetuning the last 10 layers instead [han-song-etal-2025-emergence-of-abstractions-task-vectors-icl]

models: Llama-3.1-8B · method:
Reasoning Beyond Chain-of-Thought: A Latent Computational Mode in Large Language Models (2026)measured

SAE features selected by CoT-minus-direct trigger a reasoning mode

Details

He et al. select SAE latent features whose first-step activation differs most between chain-of-thought and direct prompting across six models (LLaMA, Gemma-3, Qwen3) [he-etal-2026-reasoning-beyond-chain-of-thought-latent-computational-mode] Restricting to features with consistently positive singleton-steering effect leaves 1-10 features per model [he-etal-2026-reasoning-beyond-chain-of-thought-latent-computational-mode] Additively steering only this feature set at the first decoding step raises direct-prompt accuracy sharply (LLaMA-3.1-8B 24.5% to 73.3%; Qwen3-0.6B 7.9% to 60.6%) at far fewer tokens than full CoT [he-etal-2026-reasoning-beyond-chain-of-thought-latent-computational-mode] A random-SAE-feature control reproduces neither the accuracy gain nor the pattern, confirming specificity [he-etal-2026-reasoning-beyond-chain-of-thought-latent-computational-mode] The feature marks entry into a reasoning mode (transient early spike, uncorrelated with correctness) and overrides an explicit "/no_think" instruction [he-etal-2026-reasoning-beyond-chain-of-thought-latent-computational-mode]

models: Llama-3.1-8B-Instruct, Llama 3.3 70B Instruct · method: Sparse Autoencoders (SAE), Causal interventions (steering)
Categorical Perception in Large Language Model Hidden States: Structural Warping at Digit-Count Boundaries (2026)measured

LLM hidden states warp categorically at digit-count boundaries

Details

- In six LLMs, hidden-state distances warp categorically at digit-count boundaries (9->10, 99->100) versus matched non-boundary controls (15, 150); an RSA "CP-Additive" model (log-distance plus a boundary boost) beats a continuous model at 100% of primary layers. [cacioli-2026-categorical-perception-digit-boundaries] - Boundary crossing explains 5-27% of representational-distance variance beyond magnitude; the decade-100 effect is 3.9-12.7x the decade-10 effect, with a manifold rotation of 81.6-89.6 degrees at the boundary. [cacioli-2026-categorical-perception-digit-boundaries] - Patching along a ridge-regression "category direction" shifts discrimination confidence 70.1x more than random directions (Layer 5), dose-dependent and specific. [cacioli-2026-categorical-perception-digit-boundaries] - Five instruct models (Llama-3-8B-Instruct, Mistral-7B-Instruct-v0.3, Gemma-2-9B-IT, Qwen2.5-7B-Instruct, Phi-3.5-mini-instruct) plus a Llama-3-8B base control; causal patching run on Llama-3-8B-Instruct only. [cacioli-2026-categorical-perception-digit-boundaries]

models: Llama-3-8B-Instruct, Llama-3-8B · method: Representational Similarity Analysis (RSA), Linear probing, Causal interventions (steering)
The Geometry of Categorical and Hierarchical Concepts in Large Language Models (2024), When Language Representations Interact: Separability and Cross-Lingual Effects in LLMs (2026)measured

Semantic hierarchy is encoded as orthogonality between concept levels

Details

Park et al. prove (Theorem 8) that sibling-contrast directions at one hierarchy level are orthogonal to the parent's contrast direction and to any ancestor level [park-etal-2024] Combined with per-level categorical polytopes, this gives a direct-sum-of-polytopes structure where each level occupies its own orthogonal subspace, confirmed in Gemma-2B and Llama-3-8B (cosine near zero versus a shuffled-labels control) [park-etal-2024] The authors argue ordinary high-dimensional Euclidean space suffices, explicitly contrasting with hyperbolic-embedding accounts of hierarchy [park-etal-2024] Marinov et al. confirm the same cross-level orthogonality for language family versus individual language across Qwen3-4B, Mistral-7B-v0.3, and Llama-3-8B via a causal inner product [marinov-etal-2026]

models: Llama-3-8B · method: Geometric analysis, Causal inner product
Cross-model Transferability among Large Language Models on the Platonic Representations of Concepts (2025)measured

An OLS-fitted linear map transfers steering vectors across LLMs

Details

Huang et al. fit an ordinary-least-squares linear map T between paired sentence-representations of a source and target real LLM [huang-etal-2025-cross-model-platonic-transferability] Transferring a concept steering vector through T succeeds behaviorally in 212/216 cases, while no-map and random-map controls fail to beat baseline in 23/35 cases [huang-etal-2025-cross-model-platonic-transferability] Maps fit for different concepts are themselves highly similar (SSIM 0.87-0.95) versus a random-matrix baseline (0.05-0.13) [huang-etal-2025-cross-model-platonic-transferability] This cross-concept map consistency is a quantified geometric statistic about the transformation itself, beyond downstream task accuracy [huang-etal-2025-cross-model-platonic-transferability]

models: Llama-2-7B-Chat, Llama-3.1-8B-Instruct · method: Cross-model direction transfer via ridge regression
The Effectiveness of Style Vectors for Steering LLMs: A Human Evaluation (2026)measured

Human evaluation confirms per-layer style vectors steer emotion

Details

Diallo et al. extract per-layer style/emotion vectors as contrastive mean-activation differences and inject them across all layers of Alpaca and Llama-3-8B-Lexi-Uncensored [diallo-etal-2026-the-effectiveness-of-style-vectors-for-steering-llms-a-human-evaluation] The first large-scale human evaluation of activation steering (7,000+ ratings, 190 participants) shows moderate steering (lambda ~0.15) reliably shifts perceived emotion [diallo-etal-2026-the-effectiveness-of-style-vectors-for-steering-llms-a-human-evaluation] Effects are large for disgust (partial eta-squared 0.616) and fear (0.540) but minimal for surprise (0.042) [diallo-etal-2026-the-effectiveness-of-style-vectors-for-steering-llms-a-human-evaluation] Human ratings agree strongly with an automated classifier (mean r=0.776), and Llama-3 steers more consistently than Alpaca (p<0.001) [diallo-etal-2026-the-effectiveness-of-style-vectors-for-steering-llms-a-human-evaluation]

models: Llama-3-8B-Lexi-Uncensored · method: Activation Steering (Addition)
Probing then Editing Response Personality of Large Language Models (2025)measured

Hyperplane-perturbation editing shifts personality with minimal capability loss

Details

Ju et al. convert each layer's personality linear probe into an editing mechanism, adding a closed-form correction along the probe's weight direction only when the probe misses the target trait [ju-etal-2025-personality-editing] The push is calibrated to just cross the decision boundary at a target confidence, unlike a fixed-magnitude steering vector applied uniformly [ju-etal-2025-personality-editing] Against in-context (IKE) and weight-editing (MEND) baselines on LLaMA-2-7B-Chat, LLaMA-3.1-8B-Instruct, and LLaMA-3.1-8B-Base across six trait conversions, it achieves substantially higher success (average 44.33%) [ju-etal-2025-personality-editing] The calibrated single-step edit preserves general capabilities better than the baselines [ju-etal-2025-personality-editing]

models: Llama-3.1-8B, Llama-3.1-8B-Instruct, Llama-2-7B-Chat · method: Linear probing, Activation Steering (Addition)
Inference-Time Causal Probing in LLMs (2026)measured

Inference-time causal probing separates causal from correlational directions

Details

Khorasani et al. introduce HDMI, an inference-time causal probing method that fits linear probes while testing whether interventions along the candidate direction propagate to downstream behavior [khorasani-etal-2026-inference-time-causal-probing-in-llms] On Llama-3-8B-Instruct and Pythia-70M, directions selected by the causal criterion produce reliable behavioral shifts under activation-space intervention [khorasani-etal-2026-inference-time-causal-probing-in-llms] Several high-accuracy linear-probe directions that pass standard probing tests fail to causally affect behavior when intervened upon [khorasani-etal-2026-inference-time-causal-probing-in-llms] Linear separability alone therefore overstates how many directions in a model are behaviorally load-bearing [khorasani-etal-2026-inference-time-causal-probing-in-llms]

models: Llama-3-8B-Instruct · method: Linear probing, Causal interventions (steering)
Global Evolutionary Steering: Refining Activation Steering Control via Cross-Layer Consistency (2026)measured

A shared rank-one 'evolutionary' direction rectifies noisy steering vectors

Details

Jiang et al. stack cross-layer differences of contrastive positive/negative activation gaps and show via PCA that the first principal component dominates the spectrum across all datasets [jiang-etal-2026-global-evolutionary-steering-cross-layer-consistency] They prove via Wedin's sin-theta theorem that the estimated top singular vector converges to a Global Evolutionary Direction at rate O(1/sqrt(N*L)) [jiang-etal-2026-global-evolutionary-steering-cross-layer-consistency] Rectifying each layer's CAA steering vector by reinforcing its projection onto this shared direction improves over CAA and four baselines across three models and five domains (p<0.05) [jiang-etal-2026-global-evolutionary-steering-cross-layer-consistency] On Qwen2.5-7B, AdvBench refusal rises from 0.647 to 0.775 and GSM8K accuracy from 0.570 to 0.734 without per-layer tuning [jiang-etal-2026-global-evolutionary-steering-cross-layer-consistency]

models: Llama-3.1-8B-Instruct · method: Difference-in-means direction, Spectral Consensus (Global Evolutionary) Steering
Linear Mechanisms for Spatiotemporal Reasoning in Vision Language Models (2026)measured

A contrastive spatial-ID direction is causally bound into object tokens across 11 real VLMs

Details

Kang, Chen, Gkioxari & Perona (2026) define a canonical spatial-ID direction as a difference-of-means over grid position across 11 real VLMs (LLaVA, LLaMA, Qwen, InternVL, Gemma families), well-approximated by a fixed linear map of positional encoding (rank-3 fit, R^2 ~0.85) [kang-etal-2026-linear-mechanisms-for-spatiotemporal-reasoning-in-vlms] Activation-patching 'mirror swap' localizes the effect to object-word tokens at intermediate layers, with a color-swap control showing near-null effect [kang-etal-2026-linear-mechanisms-for-spatiotemporal-reasoning-in-vlms] Directly substituting a target spatial ID into an object token's residual stream shifts ground-truth logit-based belief 64.4% of the way to the target, versus 29.5% for a noise baseline; an analogous temporal-ID binding mechanism extends to video VLMs (LLaVA-Video, VideoLLaMA3, Qwen2.5) [kang-etal-2026-linear-mechanisms-for-spatiotemporal-reasoning-in-vlms]

models: Llama-3.1-8B-Instruct · method: Difference-in-means direction, Activation patching
Delta-Crosscoder: Robust Crosscoder Model Diffing in Narrow Fine-Tuning Regimes (2026)measured

A delta-loss crosscoder isolates 1-2 causal directions per fine-tuning organism

Details

Kassem et al. train a BatchTopK crosscoder with a delta-loss prioritizing directions that change between a base and fine-tuned model, ranked by Relative Decoder Norm [kassem-etal-2026-delta-crosscoder-robust-model-diffing] Across 10 fine-tuning model organisms in Llama-3.1-8B-Instruct, Gemma-2-9B-IT, and Qwen2.5-7B, it isolates exactly 1-2 causal directions per organism [kassem-etal-2026-delta-crosscoder-robust-model-diffing] It recovers the causal direction in 10/10 cases with 0% false positives, versus 40% and 60% false-positive rates for DSF and BatchTopK baselines [kassem-etal-2026-delta-crosscoder-robust-model-diffing] Adding or subtracting the isolated latent's decoder vector at inference causally mitigates the fine-tuned behavior [kassem-etal-2026-delta-crosscoder-robust-model-diffing]

models: Llama-3.1-8B-Instruct · method: Sparse crosscoders, Causal interventions (steering)
Shared Global and Local Geometry of Language Model Embeddings (2025)measured

Token-embedding orientation is shared within families but drops across them

Details

Lee et al. measure token-embedding geometry across the GPT-2, Llama-3, and Gemma-2 model families [lee-etal-2025-shared-global-and-local-geometry-of-language-model-embeddings] Relative orientation/cosine structure is near-identical within a model family but drops sharply across families trained on different data [lee-etal-2025-shared-global-and-local-geometry-of-language-model-embeddings] A k-NN-neighborhood PCA intrinsic-dimension estimator finds low-ID tokens form semantically coherent clusters while high-ID tokens do not [lee-etal-2025-shared-global-and-local-geometry-of-language-model-embeddings] EMB2EMB, a linear least-squares (OLS) map fit over 100k shared tokens, transfers CAA-style steering vectors (refusal, sycophancy, corrigibility) between differently-sized models [lee-etal-2025-shared-global-and-local-geometry-of-language-model-embeddings]

models: Llama-3.1-8B-Instruct · method: PCA, Intrinsic dimension estimation (TwoNN), Cross-model direction transfer via ridge regression
Layerwise Recall and the Geometry of Interwoven Knowledge in LLMs (2025)measured

A PCA spiral in Llama hidden states aligns with the periodic table

Details

Lei & Cooper apply PCA to Llama-3-8B and Llama-3.1-70B hidden states on chemistry prompts, finding a 3D spiral in the top components that tracks the periodic table's atomic-number ordering [lei-cooper-2025-layerwise-recall-and-the-geometry-of-interwoven-knowledge-in-llms] Per-property spiral-fit consistency scores reach up to about 0.90 for some element properties and lower (around 0.65) for others [lei-cooper-2025-layerwise-recall-and-the-geometry-of-interwoven-knowledge-in-llms] Layer-wise linear probing shows middle layers encode element attributes continuously and overlapping while deeper layers sharpen them into more categorically separated clusters [lei-cooper-2025-layerwise-recall-and-the-geometry-of-interwoven-knowledge-in-llms]

models: Llama-3-8B, Llama 3.1 70B · method: Linear probing
What Really Controls Temporal Reasoning in LLMs: Tokenisation or Representation of Time? (2026)measured

Temporal-probe decodability predicts reasoning in high-resource languages

Details

Bhatia et al. introduce MultiTempBench and evaluate 20 LLMs, combining a tokenization-fragmentation metric with linear-regression probes decoding Year/Month/Day (probe R^2 as temporal linearity) [bhatia-etal-2026-what-really-controls-temporal-reasoning-in-llms] A crossed mixed-effects regression over 285,000 predictions finds temporal linearity is the strongest predictor of reasoning accuracy in high-resource languages (English r=0.77) [bhatia-etal-2026-what-really-controls-temporal-reasoning-in-llms] Tokenization fragmentation dominates in low-resource languages (Hausa r=-0.97); the finding is purely correlational [bhatia-etal-2026-what-really-controls-temporal-reasoning-in-llms]

models: Llama-3.1-8B, Llama-2-7B · method: Linear probing
Unlocking the Future: Look-Ahead Planning Mechanistic Interpretability in LLMs (2024)measured

Linear probes decode future Blocksworld planning steps from one pass

Details

Men et al. train linear probes (versus a nonlinear control) on hidden states of Llama-2-7b-chat and Vicuna-7B fine-tuned on Blocksworld planning [men-etal-2024-unlocking-the-future-look-ahead-planning-mechanistic-interpretability-in-llms] A linear probe predicts the 6th-step-ahead decision from the 1st-step representation at 0.51 accuracy, decaying smoothly with prediction distance [men-etal-2024-unlocking-the-future-look-ahead-planning-mechanistic-interpretability-in-llms] Linear and nonlinear probes track the same decay, evidence the look-ahead information is linearly encoded; MHSA key-masking confirms which attention paths carry it [men-etal-2024-unlocking-the-future-look-ahead-planning-mechanistic-interpretability-in-llms]

models: Llama-2-7B-Chat · method: Linear probing, Causal interventions (steering)
Do Language Models Track Entities Across State Changes? (2026)measured

LLMs consolidate entity-state tracking into a local binary subspace

Details

Tang et al. train linear probes on last-token residual activations (the "boxes" dataset) to test global versus local entity-state tracking [tang-etal-2026-do-language-models-track-entities-across-state-changes] Global 8-way state probes reach only ~0.3 accuracy versus ~0.9 for local binary probes, so the model consolidates state locally rather than tracking a full global state [tang-etal-2026-do-language-models-track-entities-across-state-changes] INLP null-space projection of the state-tag direction causally changes completions [tang-etal-2026-do-language-models-track-entities-across-state-changes] A learned sparse boolean mask over the residual PCA basis ("subspace patching") reveals a shared positional-information subspace at layers 15-25 in CodeLlama-13B [tang-etal-2026-do-language-models-track-entities-across-state-changes]

models: Llama 3.1 70B · method: Linear probing, Causal interventions (steering), PCA
Linearity of Relation Decoding in Transformer Language Models (2023)measured

Linear relational embeddings are faithful and causal for half of relations

Details

Hernandez et al. approximate a transformer's relation-decoding function by a first-order Taylor expansion LRE(s)=beta*W_r*s+b_r, with W_r the mean Jacobian estimated from 5-8 examples [hernandez-etal-2023-lre] It is validated by faithfulness (decoded top token matches the model's own) and causality (low-rank pseudoinverse inversion redirects the real output) [hernandez-etal-2023-lre] Across 47 relations on GPT-J, GPT-2-XL, and LLaMA-13B, faithfulness exceeds 60% for nearly half the relations, with strong faithfulness-causality correlation (R=0.84 on GPT-J) [hernandez-etal-2023-lre] Relations like "company CEO" (0.06/0.31) fail almost completely despite correct model behavior, so linear relation decoding is real but heterogeneous across relation types [hernandez-etal-2023-lre]

models: LLaMA-13B · method: Linear Relational Embedding (LRE)
Relational Linearity is a Predictor of Hallucinations (2026)measured

LRE affine-map linearity scores correlate with hallucination rate across four real instruction-tuned LLMs

Details

Lu, Liu, Gerstner, Hirlimann, Rohweder & Schütze (2026) fit relation-specific affine maps o = W_r s + b_r via ridge regression (the Linear Relational Embeddings framework) across Gemma-7B-IT, Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, and Qwen2.5-7B-Instruct [lu-etal-2026-relational-linearity-is-a-predictor-of-hallucinations] The resulting linearity score correlates with hallucination-vs-refusal rate on unknown entities at Pearson r=0.741-0.816 across 15 natural LRE relations, and r=0.573-0.812 on a new SyntHal synthetic-unknown-entity benchmark [lu-etal-2026-relational-linearity-is-a-predictor-of-hallucinations] Purely correlational -- the authors explicitly flag that establishing causality would require representation patching or steering, not yet performed [lu-etal-2026-relational-linearity-is-a-predictor-of-hallucinations]

models: Llama-3.1-8B-Instruct · method: Linear Relational Embedding (LRE)
Universal Response and Emergence of Induction in LLMs (2024)measured

A weak-perturbation response direction stays near-orthogonal across every layer

Details

Luick applies a weak scaling perturbation to a single token's residual activation and tracks the downstream response across all layers of Gemma-2-2B, Llama-3.2-3B-Instruct, and GPT-2-XL [luick-2024-universal-response-emergence-of-induction] The response-direction-to-state cosine similarity stays below 0.1 in magnitude across the entire residual stream [luick-2024-universal-response-emergence-of-induction] This near-orthogonality co-occurs with a scale-invariant response regime [luick-2024-universal-response-emergence-of-induction] Both properties are tied to the induction mechanism, strongest for perturbations at token positions that induction heads copy from [luick-2024-universal-response-emergence-of-induction]

models: Llama-3.2-3B-Instruct · method: Geometric analysis, Causal interventions (steering)
The Truthfulness Spectrum Hypothesis (2026)measured

Mahalanobis cosine between truth directions predicts cross-domain generalization

Details

Ying et al. test whether truth directions range from domain-general to domain-specific across five truth types plus sycophantic/expectation-inverted lying [ying-etal-2026-the-truthfulness-spectrum-hypothesis] Linear probes generalize well pairwise except on sycophantic/inverted lying (AUROC ~0.55 and ~0.28), while joint training recovers strong performance [ying-etal-2026-the-truthfulness-spectrum-hypothesis] Mahalanobis cosine similarity between probe directions predicts cross-domain generalization far better than standard cosine (R^2=0.98 vs 0.56) [ying-etal-2026-the-truthfulness-spectrum-hypothesis] Concept-erasure isolates domain-general, domain-specific, and shared truth directions with effective dimensionality under 100-200 in an 8192-d residual stream [ying-etal-2026-the-truthfulness-spectrum-hypothesis] Steering domain-specific directions selectively suppresses incorrect-answer probability (+0.05 to +0.10) while the domain-general direction backfires (-0.07 to -0.11) [ying-etal-2026-the-truthfulness-spectrum-hypothesis]

models: Llama 3.3 70B Instruct, Llama-3.1-8B-Instruct, Llama-3.2-3B-Instruct · method: Linear probing, LEACE (LEAst-squares Concept Erasure), Mahalanobis-cosine probe alignment, Causal interventions (steering)
Shared Latent Structures Enable Unified Backdoor Detection and Mitigation in LLMs (2026)measured

Five shared SAE directions mitigate structurally distinct backdoors

Details

Mahmoud et al. train SAEs on six real backdoored LLMs and use model diffing to find five SAE features shared across six distinct backdoor types [mahmoud-etal-2026-shared-latent-structures-enable-unified-backdoor-detection-and-mitigation-in-llms] Steering along each feature's decoder direction mitigates attack success: F33881 gives 96% on Sentiment-LoRA, 79% on country-conditioned advice, and 44% zero-shot on a held-out Refusal backdoor [mahmoud-etal-2026-shared-latent-structures-enable-unified-backdoor-detection-and-mitigation-in-llms] F25828 gives 100% zero-shot mitigation on the held-out Refusal backdoor [mahmoud-etal-2026-shared-latent-structures-enable-unified-backdoor-detection-and-mitigation-in-llms] A companion training-time method, Concept Ablation Fine-Tuning, ablates the same shared subspace during training [mahmoud-etal-2026-shared-latent-structures-enable-unified-backdoor-detection-and-mitigation-in-llms]

models: Llama-3.1-8B · method: Sparse Autoencoders (SAE), Activation Steering (Addition)
Narrow Finetuning Leaves Clearly Readable Traces in Activation Differences (2026)measured

A per-token mean-difference direction fingerprints narrow fine-tuning

Details

Minder et al. extract a diff-in-means direction between base and narrowly-fine-tuned activations across 33 fine-tuning organisms spanning 7 models from 1B to 32B [minder-etal-2026-narrow-finetuning-traces-in-activation-differences] Steering by adding this direction during generation produces text with high embedding-similarity to the actual fine-tuning corpus [minder-etal-2026-narrow-finetuning-traces-in-activation-differences] An LLM interpretability agent (GPT-5) given the direction correctly identifies the fine-tuning objective in 91% of organisms (30/33) [minder-etal-2026-narrow-finetuning-traces-in-activation-differences] That is more than twice as well at broad-objective identification and over 30x better at fine-grained detail than the best black-box baseline [minder-etal-2026-narrow-finetuning-traces-in-activation-differences]

models: Llama-3.1-8B-Instruct · method: Difference-in-means direction, Logit Lens, Causal interventions (steering)
Understanding Emergent Misalignment via Feature Superposition Geometry (2026)measured

Insecure-code SAE directions sit closer to toxic directions than secure-code ones

Details

Minegishi et al. train SAEs on five models and identify insecure-code, secure-code, and toxic-persona decoder directions [minegishi-etal-2026-understanding-emergent-misalignment-via-feature-superposition-geometry] Across all models and layers, the insecure-code direction is consistently more cosine-similar to the toxic-persona direction than the secure-code direction is [minegishi-etal-2026-understanding-emergent-misalignment-via-feature-superposition-geometry] So insecure-code finetuning data is geometrically closer to misaligned-persona representations than matched secure-code data [minegishi-etal-2026-understanding-emergent-misalignment-via-feature-superposition-geometry] Filtering training data by high insecure-code-direction activation cuts emergent-misalignment behaviors from 87 to 57, beating random removal (84) and an LLM-judge filter (59) [minegishi-etal-2026-understanding-emergent-misalignment-via-feature-superposition-geometry]

models: Llama-3.1-8B-Instruct · method: Sparse Autoencoders (SAE)
SAIF: A Sparse Autoencoder Framework for Interpreting and Steering Instruction Following of Language Models (2025)measured

Multi-feature SAE steering vectors force instruction-following

Details

He et al. identify instruction-relevant SAE latents via a sensitivity score, then build multi-feature steering vectors from Gemma Scope / Llama Scope decoder directions [he-etal-2025-saif-a-sparse-autoencoder-framework-for-interpreting-and-steering-instruction-following-of-language-models] Adding them to the residual stream forces instruction-following (translation/summarization/keyword) at over 30% strict and up to ~0.7 loose accuracy versus near-zero for single-latent steering, optimal at k=15 latents [he-etal-2025-saif-a-sparse-autoencoder-framework-for-interpreting-and-steering-instruction-following-of-language-models] Last-layer placement is critical (Gemma-2-2b-it loose accuracy 0.64 at layer 25 drops to 0.33 by layer 24), and post-instruction positioning beats pre-instruction [he-etal-2025-saif-a-sparse-autoencoder-framework-for-interpreting-and-steering-instruction-following-of-language-models]

models: Llama-3.1-8B-Instruct · method: Sparse Autoencoders (SAE), Activation Steering (Addition)
Steerable but Not Decodable: Function Vectors Operate Beyond the Logit Lens (2026)measured

Function vectors steer even where logit lens and probes cannot decode

Details

Nadaf extracts function vectors across 12 tasks and 8 prompt templates in Llama-3.1-8B, Gemma-2-9B, and Mistral-7B-v0.3 (base and instruct) [nadaf-2026-steerable-but-not-decodable] In a substantial fraction of cases (gaps up to -0.91), the function vector causally steers toward the correct answer even though the logit lens cannot decode it at any layer [nadaf-2026-steerable-but-not-decodable] Even a nonlinear probe with a selectivity control fails to decode 5/10 of the hardest cases, so the causal direction exceeds what any tested readout detects [nadaf-2026-steerable-but-not-decodable] Cross-template cosine similarity of the directions weakly correlates (r in [-0.20, 0.13]) with transfer success [nadaf-2026-steerable-but-not-decodable]

models: Llama-3.1-8B, Llama-3.1-8B-Instruct · method: Function vector extraction (causal-mediation head selection + sum), Activation patching, Logit Lens
How Language Models Process Negation (2026)measured

Negation is a single linear direction decodable by layer 4

Details

Zhou et al. extract a single "not" direction from Llama-3.1-8B residual-stream states via PCA-for-reduction followed by linear discriminant analysis [zhou-etal-2026-how-language-models-process-negation] Positive and negative hidden states are approximately linearly separable by this one direction [zhou-etal-2026-how-language-models-process-negation] 10-fold cross-validated per-layer decoding accuracy reaches near-perfect by layer 4 [zhou-etal-2026-how-language-models-process-negation] The same linear-direction analysis is confirmed on Mistral-7B-v0.1 as a secondary model [zhou-etal-2026-how-language-models-process-negation]

models: Llama-3.1-8B · method: PCA, Linear Discriminant Analysis (LDA), Causal interventions (steering)
Probing the Geometry of Truth: Consistency and Generalization of Truth Directions in LLMs Across Logical Transformations and Question Answering Tasks (2025)measured

Not all LLMs have consistent truth directions; atomic-statement probes generalize

Details

Bao et al. investigate whether truth directions are consistent across LLMs and how well truth probes generalize [bao-etal-2025-probing-the-geometry-of-truth] Not all LLMs exhibit consistent truth directions, with stronger and more consistent representations in more capable models, particularly under logical negation [bao-etal-2025-probing-the-geometry-of-truth] Probes trained on declarative atomic statements generalize to logical transformations, question-answering, in-context learning, and external-knowledge settings [bao-etal-2025-probing-the-geometry-of-truth] The paper demonstrates a practical application to selective question-answering [bao-etal-2025-probing-the-geometry-of-truth]

models: Llama-2-7B, Llama-2-13B, Llama-3.1-8B, Llama-3.1-70B-Instruct · method: Linear probing, Direction Extraction
Steering Language Model Refusal with Sparse Autoencoder Features (2024)measured

A single SAE latent causally mediates refusal with a dose-response tradeoff

Details

O'Brien et al. train a TopK SAE on Phi-3-mini's layer-6 residual stream and identify a single latent (Feature 22373) whose activation marks refusal [obrien-etal-2024-steering-refusal-with-sae-features] Clamping this decoded direction monotonically raises WildGuard unsafe-prompt refusal from 58.33% to 96.02% and cuts Crescendo jailbreak attack-success from 55.92% to 32.58% [obrien-etal-2024-steering-refusal-with-sae-features] The same clamped direction generalizes to Llama-3.1-8B-Instruct [obrien-etal-2024-steering-refusal-with-sae-features] A safety-capability tradeoff is quantified: safe-prompt refusal rises substantially and MMLU degrades (68.80% to 35.98% at clamp 12), a cost left mechanistically unexplained [obrien-etal-2024-steering-refusal-with-sae-features]

models: Llama-3.1-8B-Instruct · method: TopK (k-sparse) Sparse Autoencoders, Causal interventions (steering)
A Retrieval-Conditioned Rebinding Circuit for Dynamic Entity Tracking in Large Language Models (2026)measured

A mean-difference binding-ID direction in Q/K space redirects attention

Details

Oh & Demberg treat binding IDs as approximately linear directions in the query and key activation spaces of selected attention heads, estimated as mean differences between binding-ID conditions [oh-demberg-2026-retrieval-conditioned-rebinding-circuit] Adding these directions to Gemma2-9B-it, Gemma3-12B-it, Llama-3.2-3B-it, and Llama-3.1-8B-it's Q/K activations causally redirects attention to the corresponding object [oh-demberg-2026-retrieval-conditioned-rebinding-circuit] The intervention shifts the final logits accordingly [oh-demberg-2026-retrieval-conditioned-rebinding-circuit] The representational signature, whether the effect concentrates in Q and K jointly or in K alone, differs across model families [oh-demberg-2026-retrieval-conditioned-rebinding-circuit]

models: Llama-3.2-3B-Instruct, Llama-3.1-8B-Instruct · method: Difference-in-means direction, Activation patching, Causal interventions (steering)
Output Vector Editing for Memorization Mitigation in Large Language Models (2026)measured

Editing MLP output-vector directions suppresses memorized sequences

Details

Hakimi et al. identify MLP neurons implicated in verbatim memorization via logit-lens attribution, then edit each neuron's output vector to add a distractor direction while preserving its other superposed functions [hakimi-etal-2026-output-vector-editing-for-memorization] Four edit modes jointly suppress up to 96.5% of memorized continuations in ensemble, a 2.7x larger effect than zero-ablating the same neurons [hakimi-etal-2026-output-vector-editing-for-memorization] So the direction of the edit, not just the neuron's presence, drives the effect [hakimi-etal-2026-output-vector-editing-for-memorization] About 14% of memorized sequences resist MLP-only editing; ablating top attention heads recovers 60-64% of these, indicating a mechanism split across MLP and attention, and the method transfers to SmolLM-360M, OLMo-1B, and Llama2-7B [hakimi-etal-2026-output-vector-editing-for-memorization]

models: Llama-2-7B · method: Logit Lens
Steering at the Source: Style Modulation Heads for Robust Persona Control (2026)measured

Persona/style control localizes to a sparse set of attention heads

Details

Izawa et al. localize persona/style control in Qwen2.5-7B-Instruct and Llama-3.1-8B-Instruct to 3 attention heads per model via layer-wise persona-vector heatmaps and a head-wise Head Contribution Score [izawa-etal-2026-steering-at-the-source-style-modulation-heads-for-robust-persona-control] Steering only these heads achieves the best trait-expression-vs-coherency Pareto frontier (best score in 11/12 Qwen conditions, 9/12 Llama) [izawa-etal-2026-steering-at-the-source-style-modulation-heads-for-robust-persona-control] Zero-ablating them causes a sharp targeted drop in trait expression while leaving MMLU and output coherency intact [izawa-etal-2026-steering-at-the-source-style-modulation-heads-for-robust-persona-control]

models: Llama-3.1-8B-Instruct · method: Activation Steering (Addition), Concept-Token Attention-Output Projection (ConceptAttention)
Probing then Editing Response Personality of Large Language Models (2025)measured

Personality traits are linearly encoded with a betweenness arrangement

Details

Ju et al. train layer-wise linear probes on 11 open LLMs (including InternLM-7B and InternLM2-7B, base and instruct) to classify which of three personality instructions a response is conditioned on, scored by V-information [ju-etal-2025-personality-editing] V-information rises through layers 5-12 then plateaus, with instruction-tuned models 1-3 points higher than their base counterparts [ju-etal-2025-personality-editing] A t-SNE of Llama-3.1-8B-Instruct shows Neuroticism and Agreeableness distant with Extraversion situated between them, an informal betweenness arrangement [ju-etal-2025-personality-editing] This geometry explains why converting either trait into Extraversion is easier than converting directly between Neuroticism and Agreeableness [ju-etal-2025-personality-editing]

models: Llama-3.1-8B, Llama-3.1-8B-Instruct, Llama-2-7B, Llama-2-7B-Chat · method: Linear probing
Simulated Adoption: Decoupling Magnitude and Direction in LLM In-Context Conflict Resolution (2026)measured

In-context conflict shows architecture-dependent dilution but universal orthogonal interference

Details

Zhang & Lin inject counterfactual contexts into MMLU questions and decompose the residual-stream update into a radial (norm-ratio) and an angular (cosine to the correct-answer direction) component [zhang-lin-2026-simulated-adoption-decoupling-magnitude-and-direction-in-llm-in-context-conflict-resolution] Radial "Manifold Dilution" is architecture-dependent: only Llama-3.1-8B shows real dilution (gamma=0.978) while Qwen3-4B and GLM-4-9B show none despite equal logit collapse [zhang-lin-2026-simulated-adoption-decoupling-magnitude-and-direction-in-llm-in-context-conflict-resolution] Angular "Orthogonal Interference" holds universally: interference-to-correct-answer cosine clusters near zero across all three models, not antiparallel suppression [zhang-lin-2026-simulated-adoption-decoupling-magnitude-and-direction-in-llm-in-context-conflict-resolution] Regressing angular deviation against logit drop gives R^2=0.90 (Qwen) and 0.87 (GLM) [zhang-lin-2026-simulated-adoption-decoupling-magnitude-and-direction-in-llm-in-context-conflict-resolution]

models: Llama-3.1-8B · method: Geometric analysis
Refusal Direction is Universal Across Safety-Aligned Languages (2025)measured

Refusal directions are approximately parallel across 14 languages and transfer

Details

Wang et al. extract diff-in-means refusal directions across 14 languages in Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, and Gemma-2-9B-it using the PolyRefuse dataset [wang-etal-2025-refusal-direction-is-universal-across-safety-aligned-languages] PCA shows the per-language refusal directions are approximately parallel, with high cross-lingual cosine similarity peaking around layer 12 in Llama [wang-etal-2025-refusal-direction-is-universal-across-safety-aligned-languages] Ablating a refusal vector extracted from English (or German/Thai) causes near-total loss of refusal across all 14 languages and all three models, while adding it induces refusal [wang-etal-2025-refusal-direction-is-universal-across-safety-aligned-languages] Yoruba, a language the models were never safety-aligned in, transfers weakly, so the parallelism tracks safety-alignment coverage rather than language identity [wang-etal-2025-refusal-direction-is-universal-across-safety-aligned-languages]

models: Llama-3.1-8B-Instruct · method: Difference-in-means direction, Causal interventions (steering), PCA
Refusal in Language Models Is Mediated by a Single Direction (2024), Representation Engineering: A Top-Down Approach to AI Transparency (2023), Programming Refusal with Conditional Activation Steering (2024), Refusal Direction is Universal Across Safety-Aligned Languages (2025)measured

Refusal is mediated by a single direction across models

Details

Arditi et al. show across 13 chat models (5 families, 1.8B-72B) that a single diff-in-means direction between harmful and harmless instructions both erases refusal when ablated and induces it when added, with no offset term [arditi-etal-2024] Zou et al. independently confirm via a LAT harmfulness vector in Vicuna-13B that stays a >90% classifier under jailbreaks and adversarial suffixes [arditi-etal-2024] Lee et al. replicate across nine further chat models and add conditional steering that gates the refusal vector on a condition vector for selective refusal [arditi-etal-2024] Yung et al. average per-model refusal vectors into a universal refusal direction transferring to jailbreak detection ~10% better than single-model baselines [arditi-etal-2024] The account is contested by Marshall et al. (an affine offset is needed) and Wollschlager et al. (a cone of directions), and Yin et al. localize attack-suppressed versus robust heads [arditi-etal-2024]

models: Llama-3-70B-Instruct, Llama-2-13B-Chat, Llama-3.1-8B-Instruct · method: Difference-in-means direction, Causal interventions (steering), Activation Steering (Addition), Linear Artificial Tomography (LAT), PCA, Conditional (gated) activation steering
Identifying Linear Relational Concepts in Large Language Models (2023)measured

Linear relational concepts invert an LRE into an editable object direction

Details

Chanin et al. invert Hernandez et al.'s linear relational embedding with a low-rank pseudoinverse to build a "linear relational concept" unit vector living in subject-activation space [chanin-etal-2023] On 47 relations in Llama-2-7B and GPT-J-6B, LRCs beat a directly-trained SVM probe on both classification (0.81 vs 0.73-0.75) and causal editing (0.78-0.84 vs 0.69-0.76) [chanin-etal-2023] A deliberately low-rank inverse (rank ~200 of 4096) is essential, and reading the object side from an earlier layer roughly doubles multi-token accuracy [chanin-etal-2023] This is a fourth distinct route to a linear feature direction, derived from relational structure between two token positions rather than a contrastive set at one site [chanin-etal-2023]

models: Llama-2-7B · method: Linear relational concepts (LRC), Causal interventions (steering)
Analogical Reasoning Inside Large Language Models: Concept Vectors and the Limits of Abstraction (2025)measured

RSA-localized concept vectors are surface-form invariant and causally guide behavior

Details

Opielka et al. use RSA to localize the top-3 attention heads in Llama-3.1-8B and Llama-3.1-70B whose summed mean activations form concept vectors [opielka-etal-2025-analogical-concept-vectors] For verbal concepts (antonym, category) the concept vectors are invariant across surface form (cross-context cosine ~0.7-0.8), while abstract relational concepts (previous, next) find no comparable invariant [opielka-etal-2025-analogical-concept-vectors] Adding a concept vector into the residual stream causally steers behavior toward the target concept and transfers out-of-distribution better than function vectors [opielka-etal-2025-analogical-concept-vectors] Concept vectors are markedly weaker at pure zero-shot intervention and need in-context support, and activation patching is shown to be an unreliable localization tool for these components [opielka-etal-2025-analogical-concept-vectors]

models: Llama-3.1-8B, Llama 3.1 70B · method: Representational Similarity Analysis (RSA), Causal interventions (steering)
Large Language Models Share Representations of Latent Grammatical Concepts Across Typologically Diverse Languages (2025)measured

Shared SAE morphosyntactic directions are causally necessary and sufficient across languages

Details

Brinkmann et al. train a Gated SAE on layer-16 residual activations of Llama-3-8B and Aya-23-8B and use attribution patching to find each language/concept's top causal features for grammatical number, gender, and tense [brinkmann-etal-2025-crosslingual-grammatical-concepts] Cross-lingual top-feature overlap reaches up to 50% (one feature is top-influential for grammatical gender across all 15 inflecting languages), with mean cross-concept overlap 13.9% [brinkmann-etal-2025-crosslingual-grammatical-concepts] Ablating only the massively-multilingual features drops classifier performance to 64%, so most of the causal effect concentrates in a small multilingual core [brinkmann-etal-2025-crosslingual-grammatical-concepts] Clamping a single multilingual feature during translation flips the intervened concept's probe label while leaving others unaffected, showing necessity and sufficiency [brinkmann-etal-2025-crosslingual-grammatical-concepts]

models: Llama-3-8B · method: Sparse Autoencoders (SAE), Causal interventions (steering), Activation Steering (Addition)
Unifying Attention Heads and Task Vectors via Hidden State Geometry in In-Context Learning (2025)measured

ICL separability rises first, then alignment with the unembedding

Details

Yang et al. bound classification accuracy by hidden-state linear separability, requiring both output alignment and directional alignment with the label unembedding-difference vector [yang-etal-2025] Across 7 models on 6 datasets, ICL proceeds in two stages: separability rises rapidly in early layers, then middle-to-late layers spike four alignment measures together [yang-etal-2025] The ICL-versus-zero-shot separability gap is small despite an ~80-point accuracy gap, so ICL's gains come almost entirely from improved alignment, not separability [yang-etal-2025] Ablating Previous-Token Heads collapses separability while sparing alignment, and ablating Induction Heads collapses alignment and accuracy (to 2.5%), giving a geometric account of function vectors [yang-etal-2025]

models: Llama-2-7B, Llama-2-13B, Llama-2-70B, Llama-3-8B, Llama-3-70B · method: Linear probing, PCA, Causal interventions (steering), Function vector extraction (causal-mediation head selection + sum), Separability/alignment decomposition
Pre-trained Language Models Learn Remarkably Accurate Representations of Numbers (2025)measured

A sinusoidal probe decodes integers near-perfectly across nine LLMs

Details

Kadlcik et al. test linear, log-linear, and fixed structured-basis (sinusoidal Fourier, binary-digit) probes for decoding integer values of number-token embeddings across nine LLMs (OLMo 2 1B/7B/13B/32B, Llama 3.2 1B/3B, Llama 3 8B/70B, Phi-4 15B) [kadlcik-etal-2025-number-embeddings] The fixed sinusoidal probe recovers integer values with near-perfect accuracy on most models and consistently outperforms the plain linear probe (<=0.05 everywhere), contradicting a purely-linear account of number encoding [kadlcik-etal-2025-number-embeddings] Sparse Fourier support and wave-like PCA projections confirm the periodicity is genuine, though OLMo 2 32B is a documented exception with near-zero probe accuracy and no periodic structure despite performing arithmetic accurately [kadlcik-etal-2025-number-embeddings] Tokens the probe fails to decode show higher downstream arithmetic error rates, and gradient-optimizing the four worst-decoded tokens in Llama 3 1B toward the sinusoidal pattern reduces their addition errors by 26% (17.6% to 13.0%) [kadlcik-etal-2025-number-embeddings]

models: Llama-3.2-1B, Llama-3.2-3B, Llama-3-8B, Llama-3-70B · method: Structured-basis probing
Linear Spatial World Models Emerge in Large Language Models (2025)measured

Spatial-relation vectors form an orthogonal antipodal composable basis

Details

- In Llama-3.2-3B-Instruct, spatial-relation vectors form a low-dimensional basis where inverse relations are antipodal (w_below ≈ -w_above): Above/Below cos 0.978, Left/Right 0.997, Front/Behind 0.995. [tehenan-etal-2025-spatial-world-models] - Different axes are near-orthogonal (cross-axis cosine ~0), and multi-relation directions equal the vector sum of their components (2D e.g. above+left mean cos 0.993). [tehenan-etal-2025-spatial-world-models] - 3D compositions are looser, spanning cosine 0.82-1.00 (angular deviation 3.9-34.4 degrees), e.g. above+right cos 0.88 (~28 degrees) and left+behind ~0.82 (34 degrees). [tehenan-etal-2025-spatial-world-models] - Steering by adding a relation vector causally shifts generated descriptions (74.3% overall: Above/Below/Left 100%, Right 79%, In-front 62%, Behind 5%); validated on Llama-3.2-1B-Instruct and Qwen3-1.7B; geometry cleanest in PCA space at layer 24. [tehenan-etal-2025-spatial-world-models]

models: Llama-3.2-3B-Instruct, Llama-3.2-1B-Instruct · method: Linear probing, PCA, Activation Steering (Addition)
The Cylindrical Representation Hypothesis for Language Model Steering (2026)measured

Steering directions split into a magnitude axis and interference plane

Details

Gao et al. relax the Linear Representation Hypothesis orthogonality assumption, decomposing a concept steering vector into a central axis and an orthogonal 2D normal plane spanned by the axis complement and the top PC of other concepts directions [gao-etal-2026-cylindrical-hypothesis] Despite cylindrical/phase/sector terminology there is no measured angular or periodic structure (phase = position in the 2D plane; sectors = a binary high/low-sensitivity split), so it is a linear-subspace interference decomposition, not a topology [gao-etal-2026-cylindrical-hypothesis] Steering-effect magnitude follows a predictable sin^m*cos^n form, but which interference sector a concept pair falls into is NOT predictable from the vectors (Pearson -0.034), a genuine null [gao-etal-2026-cylindrical-hypothesis] A penalty experiment attenuating the plane component trades earlier target onset against earlier corrupted output, validated by an LLM-judge at 94% human agreement [gao-etal-2026-cylindrical-hypothesis]

models: Llama-2-7B-Chat · method: Difference-in-means direction, PCA, Linear probing, Activation Steering (Addition)
On the Non-Identifiability of Steering Vectors in Large Language Models (2026)measured

Steering vectors are non-identifiable; orthogonal perturbations are equivalent

Details

Venkatesh & Kurapath prove via a Jacobian null-space argument that steering vectors extracted for a target behavior are not unique [venkatesh-kurapath-2026-on-the-non-identifiability-of-steering-vectors-in-large-language-models] On Llama-3.1-8B-Instruct and Qwen2.5-3B-Instruct across five traits, orthogonal perturbations within the activation-covariance null space produce behavioral effects indistinguishable from the original (mean Cohen's d 0.12-0.13, below the 0.2 detectability threshold) [venkatesh-kurapath-2026-on-the-non-identifiability-of-steering-vectors-in-large-language-models] A geometrically distinct mean-difference vector and a PCA-derived vector for the same trait are behaviorally equivalent [venkatesh-kurapath-2026-on-the-non-identifiability-of-steering-vectors-in-large-language-models] Many different directions in representation space thus implement the same steering effect [venkatesh-kurapath-2026-on-the-non-identifiability-of-steering-vectors-in-large-language-models]

models: Llama-3.1-8B-Instruct · method: Activation Steering (Addition)
Steered LLM Activations Are Non-Surjective (2026)measured

Steering vectors cannot reach most of the activation manifold

Details

Mishra et al. show that adding a fixed direction to residual-stream activations only reaches a thin, structured slice of the activations that occur naturally across contexts [mishra-etal-2026-steered-llm-activations-are-non-surjective] Across Llama-3.2-1B-Instruct, Qwen2.5-0.5B-Instruct, and Gemma-3-1B-it (plus three validation models), the reachable set is measurably non-surjective onto the empirical activation manifold [mishra-etal-2026-steered-llm-activations-are-non-surjective] Many naturally-occurring target-behavior activations lie off the steering line entirely, and increasing the coefficient pushes activations off-manifold rather than toward the unreached regions [mishra-etal-2026-steered-llm-activations-are-non-surjective] This bounds what single-direction steering can achieve and implies off-line behaviors need multi-directional or nonlinear interventions [mishra-etal-2026-steered-llm-activations-are-non-surjective]

models: Llama-3.2-1B-Instruct, Llama-3.1-8B-Instruct, Llama-3-8B-Instruct · method: Activation Steering (Addition)
ReCoVeR the Target Language: Language Steering Without Sacrificing Task Performance (2025)measured

A diff-in-means language-identity direction steers output language without accuracy loss

Details

Sterz et al. extract a language-identity direction as the mean-difference between target-language and other-language activations in Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, and Gemma-2-2B-Instruct [sterz-etal-2025-recover-target-language] Adding this direction, or a trained steering module, to the residual stream raises the rate of responses in the correct target language across 18 languages [sterz-etal-2025-recover-target-language] Unlike a prior unsupervised language-vector steering baseline, it preserves downstream task accuracy such as MMLU [sterz-etal-2025-recover-target-language] No further internal geometric characterization (PCA/subspace structure) of the direction is reported [sterz-etal-2025-recover-target-language]

models: Llama-3.1-8B-Instruct · method: Difference-in-means direction, Causal interventions (steering)
Sycophancy Is Not One Thing: Causal Separation of Sycophantic Behaviors in LLMs (2025)measured

Sycophantic agreement, genuine agreement, and praise occupy distinct steerable directions

Details

Vennemeyer et al. decompose sycophancy into sycophantic agreement, genuine agreement, and sycophantic praise, extracting a diff-in-means direction per behavior at every layer over items where the model knows the ground truth [vennemeyer-etal-2025-sycophancy-not-one-thing] Sycophantic- and genuine-agreement directions start nearly identical (cosine ~0.99 early) then diverge sharply to ~0.07 by layer 25, while the praise direction stays near-orthogonal to both throughout [vennemeyer-etal-2025-sycophancy-not-one-thing] Ablating a behavior's own diff-in-means subspace drops its own detection AUROC to chance while removing the praise subspace has zero effect on agreement detection [vennemeyer-etal-2025-sycophancy-not-one-thing] Activation-addition steering is highly selective (praise 36.8x in LLaMA-3.1-8B, sycophantic-agreement 23.1x in Qwen3-30B), replicated across five models [vennemeyer-etal-2025-sycophancy-not-one-thing]

models: Llama-3.1-8B-Instruct, Llama 3.3 70B Instruct · method: Difference-in-means direction, Activation Steering (Addition)
Actionable Activation Directions for Detecting and Mitigating Emergent Misalignment Across Language Model Families (2026)measured

A diff-in-means misalignment direction transfers cross-architecture but non-specifically

Details

Syed QLoRA-finetunes four models identically on insecure code and extracts a diff-in-means misalignment direction, finding 99.6% within-model separability versus 50% for a secure-code control [syed-2026-actionable-activation-directions-emergent-misalignment] A ridge-regression map fit between any two models' activation spaces transfers the direction across all twelve model-family pairs with above-chance accuracy (Gemma-to-Llama 87%, Gemma-to-Ministral 90%) [syed-2026-actionable-activation-directions-emergent-misalignment] Steering a target model with the transferred direction suppresses code-related spillover (delta 13-46 points) but fails a specificity control, unlike the fully specific within-model effect (delta 21-51 points) [syed-2026-actionable-activation-directions-emergent-misalignment] This reveals an asymmetric donor/receiver topology, with Gemma and Qwen acting as donors and Llama as a receiver [syed-2026-actionable-activation-directions-emergent-misalignment]

models: Llama-3.2-1B-Instruct · method: Difference-in-means direction, Cross-model direction transfer via ridge regression
Task Recognition and Task Learning Heads Align In-Context Hidden States with a Label-Unembedding Task Subspace (2026)measured

ICL task-recognition and task-learning heads project onto a label subspace

Details

Yang, Cho & Inoue introduce Task Subspace Logit Attribution, scoring each attention head output by projection onto the span of label-token unembedding vectors [yang-cho-inoue-2026-task-subspace-attention-heads] Task-recognition heads align with the task subspace while task-learning heads rotate the hidden state toward the correct-minus-incorrect label direction within it [yang-cho-inoue-2026-task-subspace-attention-heads] Across Llama3-8B layers, hidden-update vs TR-head subspace alignment correlates rho=0.94, and logit-difference vs TL-head alignment rho=0.53 [yang-cho-inoue-2026-task-subspace-attention-heads] Injecting the summed top-3% TR-head outputs into zero-shot prompts raises classification accuracy 9.2% to 40.4% (TR-based), while TL-based vectors dominate open-ended generation; ablating each collapses its own role, tested across 6 models [yang-cho-inoue-2026-task-subspace-attention-heads]

models: Llama-3-8B, Llama-3.1-8B, Llama-3.2-3B · method: Direct Logit Attribution (DLA), Activation patching
The Geometries of Truth Are Orthogonal Across Tasks (2026)measured

Per-task truth probe directions are near-orthogonal with disjoint L1 support

Details

Azizian et al. train L2-logistic truth probes separately on seven QA/fact-verification tasks across Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, and Phi-4-Mini-Instruct [azizian-etal-2026-truth-geometries-orthogonal-across-tasks] Pairwise cosine similarity between probe directions is consistently low (<0.5) and correlates with cross-task transfer AUROC (r=0.59) [azizian-etal-2026-truth-geometries-orthogonal-across-tasks] L1 probes share under 15% of their nonzero support for most task pairs, and the few high-overlap pairs are exactly those that generalize well [azizian-etal-2026-truth-geometries-orthogonal-across-tasks] Three attempts to recover a shared cross-task truth direction (joint training, subspace-constrained fit, mixture-of-probes) all fail to beat per-task probes, and no causal intervention is performed [azizian-etal-2026-truth-geometries-orthogonal-across-tasks]

models: Llama-3.1-8B-Instruct · method: Linear probing, Geometric analysis
The Geometry of Forgetting: Temporal Knowledge Drift as an Independent Axis in LLM Representations (2026)measured

Temporal knowledge drift is a linear direction orthogonal to correctness and uncertainty

Details

Elbadry et al. train L1-regularized probes on six instruction-tuned LLMs to detect temporal drift (whether a fact changed since training cutoff), reaching AUROC 0.83-0.95 versus 0.49-0.57 for output and correctness/uncertainty baselines [elbadry-etal-2026-geometry-of-forgetting-temporal-knowledge-drift-as-an-independent-axis-in-llm-representations] Five measures establish the drift direction is geometrically orthogonal to correctness and uncertainty probe directions (weight cosine <=0.136; INLP removal of 10 directions changes AUROC <=0.013) [elbadry-etal-2026-geometry-of-forgetting-temporal-knowledge-drift-as-an-independent-axis-in-llm-representations] Untrained difference-of-means directions overlap with correctness/uncertainty, yet the trained regularized drift probe is near-orthogonal, so the orthogonality is a genuine concept property not a training artifact [elbadry-etal-2026-geometry-of-forgetting-temporal-knowledge-drift-as-an-independent-axis-in-llm-representations] A diff-in-means steering direction is silent under ablation but under amplification produces structured logit redistribution favoring the current over the stale fact holder (selectivity -2.46 to -10.55 logits) [elbadry-etal-2026-geometry-of-forgetting-temporal-knowledge-drift-as-an-independent-axis-in-llm-representations] A cross-cutoff entity-matched control (0.975-0.998 across seven model pairs) confirms the probe reads model-internal knowledge state, not an input property [elbadry-etal-2026-geometry-of-forgetting-temporal-knowledge-drift-as-an-independent-axis-in-llm-representations]

models: Llama-2-7B-Chat, Llama-3.1-8B-Instruct · method: Linear probing, Difference-in-means direction, Causal interventions (steering), INLP subspace + AlterRep counterfactual push
Large Language Models Encode Semantics and Alignment in Linearly Separable Representations (2025)measured

Topic subspaces sharpen with depth; a centroid direction induces CoT

Details

Saglam et al. study 11 autoregressive models on six arXiv topics, measuring per-layer hard-margin SVM separability and PCA intrinsic dimensionality [saglam-etal-2025-linear-subspaces] Under 10% of principal components explain nearly all variance, so topics occupy a near-affine low-dimensional subspace; separability rises toward final layers, reaching 100% SVM accuracy on all topic pairs in the largest models [saglam-etal-2025-linear-subspaces] A chain-of-thought framing distinction becomes linearly separable even more sharply than topic identity [saglam-etal-2025-linear-subspaces] Adding the CoT-vs-non-CoT centroid-difference direction to the hidden state reliably induces CoT-style responses (flagged preliminary), and a lightweight MLP guardrail built on the finding halves harmful responses [saglam-etal-2025-linear-subspaces]

models: Llama-3.1-8B, Llama-3.1-8B-Instruct, Llama-3.2-3B · method: Linear probing, PCA, Activation Steering (Addition)
Internal states before "wait" modulate reasoning patterns (2025)measured

Crosscoder pre-'wait' feature directions set which reasoning pattern follows

Details

Troitskii et al. train crosscoders across DeepSeek-R1-Distill-Llama-8B and its pre-distillation base Llama-3.1-8B to model-diff the base-to-distilled transition [troitskii-etal-2025-internal-states-before-wait-modulate-reasoning] Latent attribution locates a small subset of feature directions whose activation immediately before a "wait" token predicts the following reasoning pattern [troitskii-etal-2025-internal-states-before-wait-modulate-reasoning] Intervening directly on these features causally determines which pattern follows: restarting, recalling prior knowledge, expressing uncertainty, or double-checking [troitskii-etal-2025-internal-states-before-wait-modulate-reasoning] The interventions change the qualitative reasoning pattern rather than merely correlating with it [troitskii-etal-2025-internal-states-before-wait-modulate-reasoning]

models: Llama-3.1-8B · method: Sparse crosscoders, Causal interventions (steering)
The Geometry of Truth: Emergent Linear Structure in LLM Representations of True/False Datasets (2024), On the Universal Truthfulness Hyperplane Inside LLMs (2024)measured

A linear truth direction generalizes across datasets and is causally effective

Details

Marks & Tegmark curate simple true/false statement datasets and, after activation-patching-localized states, show LLaMA-2-13B/70B separate true from false along a linear PCA axis [marks-tegmark-2024] The axis generalizes across topically and structurally diverse datasets increasingly with scale and is largely distinct from a probable-vs-improbable-text direction [marks-tegmark-2024] Mass-mean probing (a diff-in-means direction with optional covariance whitening) classifies about as well as logistic regression or contrast-consistent search but yields substantially more causally effective steering directions (normalized indirect effects up to ~1.0 at 70B) [marks-tegmark-2024] Liu et al. extend the account cross-domain across 41 datasets, finding training-task diversity drives generalization far more than data volume (accuracy rising as task categories grow from 1 to 14) [marks-tegmark-2024]

models: Llama-2-7B, Llama-2-13B, Llama-2-70B, Llama-2-7B-Chat, Llama-2-13B-Chat · method: Activation patching, PCA, Difference-in-means direction, Linear probing, Contrast-Consistent Search (CCS), Causal interventions (steering)
How Context Shapes Truth: Geometric Transformations of Statement-level Truth Representations in LLMs (2026)measured

The truth direction rotates and rescales under added context and steers labels

Details

Adarsh et al. extract a per-layer mean-difference truth direction in Llama-3.1-8B-Instruct, Mistral-Nemo-12B-Instruct, Qwen3-4B-Instruct, and SmolLM3-3B and measure how it transforms when a statement is embedded in context [adarsh-etal-2026-how-context-shapes-truth] The angle between with- and without-context truth vectors is near-orthogonal in early layers and converges by mid-depth, while the magnitude ratio is quantified per dataset [adarsh-etal-2026-how-context-shapes-truth] This is the first quantified characterization of how the truth direction's own geometry, not just its existence, changes under context [adarsh-etal-2026-how-context-shapes-truth] Mass-mean steering with the extracted vector flips truthfulness labels in ~100% of cases for three models (Qwen3-4B weaker and variable at 11-59%), a strong replicated causal effect [adarsh-etal-2026-how-context-shapes-truth]

models: Llama-3.1-8B-Instruct · method: Difference-in-means direction, Causal interventions (steering)
Inference-Time Intervention: Eliciting Truthful Answers from a Language Model (2023)measured

Truthfulness is linearly decodable per attention head, and ITI steers it

Details

Li et al. train a separate linear probe per attention head on TruthfulQA in LLaMA-7B, finding truthfulness strongly decodable in only a sparse subset of heads (best head 83.3%) rather than one whole-residual-stream direction [li-etal-2023] Even within a head the geometry is not perfectly one-dimensional, as a second probe constrained orthogonal to the first still classifies above chance [li-etal-2023] Inference-Time Intervention shifts activations along the winning mass-mean direction (outperforming a probe-weight direction and contrast-consistent search) on the top-K most truthful heads [li-etal-2023] This raises TruthfulQA true*informative substantially (LLaMA-7B 30.5% to 43.5%, Alpaca-7B 32.5% to 65.1%, Vicuna-7B 51.5% to 74.0%) [li-etal-2023]

models: LLaMA-7B · method: Linear probing, Difference-in-means direction, Contrast-Consistent Search (CCS), Activation Steering (Addition), Causal interventions (steering)
Eliciting Latent Predictions from Transformers with the Tuned Lens (2023)measured

The Tuned Lens replaces the logit lens with a trained per-layer affine correction

Details

Belrose et al. replace the logit lens's direct reuse of the final unembedding on intermediate states with a trained per-layer affine translator, distilled via KL divergence to the model's own final output [belrose-etal-2023-tuned-lens] This removes two documented logit-lens failure modes: a systematic 4-5 bit-per-layer vocabulary bias and unreliability from hidden-state covariance drifting across layers [belrose-etal-2023-tuned-lens] A causal-basis extraction finds the lens's most-sensitive directions strongly match the model's real causal directions (Spearman rho=0.89), with none influential to the lens but not the model [belrose-etal-2023-tuned-lens] Tested across GPT-2, GPT-Neo, GPT-NeoX-20B, OPT, BLOOM, and the full Pythia suite, it gives lower perplexity than the logit lens, transfers to a fine-tune (Vicuna-13B from LLaMA-13B), and detects prompt injection at near-perfect AUROC [belrose-etal-2023-tuned-lens]

models: LLaMA-13B · method: Tuned Lens
Valence-Arousal Subspace in LLMs: Circular Emotion Geometry and Multi-Behavioral Control (2026), Where Do Models Find Happiness? Emotion Vectors in Open-Source LLMs (2026)measured

A valence-arousal circumplex subspace causally steers emotion and refusal

Details

Sun et al. recover two near-orthogonal valence/arousal axes inside Llama-3.1-8B-Instruct (replicated in Qwen3-8B/14B) by PCA-projecting contrastive emotion-steering vectors and ridge-regressing against human ratings [sun-etal-2026-valence-arousal] Projecting the emotion vectors onto this plane traces a circle: a circularity statistic reaches 2.76-4.08 with fitted radii ~0.37-0.39, analogous to Russell's circumplex [sun-etal-2026-valence-arousal] The valence axis recovers self-reported valence at r=0.97 and the NRC-VAD lexicon at r=0.71, with cross-model valence agreement r=0.95 [sun-etal-2026-valence-arousal] Adding valence/arousal directions at specific circle angles produces dose-dependent, angle-specific shifts in generated-text affect (e.g. 0deg: delta-valence +0.75; 180deg: -0.73) [sun-etal-2026-valence-arousal] The same arousal axis causally controls refusal (20%->86% on OKTest) and sycophancy, with random-direction controls within 2-3 points of baseline [sun-etal-2026-valence-arousal] Logit-clamping and top-neuron ablation along the direction crash refusal while preserving MATH-500/IFEval, and an independent refusal direction is near-orthogonal (86.5deg) to the VA plane [sun-etal-2026-valence-arousal] Van der Ben et al. independently replicate the valence/arousal PCA structure (PC1-valence r=0.72-0.83, PC2-arousal r=0.21-0.45) in Apertus-8B and Gemma-4-E4B-it without computing circularity or steering [vanderben-etal-2026-emotion-vectors-open-source-llms] They add that cross-architecture layer-depth trajectories diverge sharply (a 3-phase plateau in Apertus vs a smooth gradient in Gemma via linear CKA) [vanderben-etal-2026-emotion-vectors-open-source-llms]

models: Llama-3.1-8B-Instruct · method: PCA, Linear probing, Activation Steering (Addition)
Negative Before Positive: Asymmetric Valence Processing in Large Language Models (2026)measured

Negative and positive valence localize to distinct depths; a negative-locus direction steers valence

Details

Venkatesh performs topic-controlled activation patching across all layers on Llama-3.2-1B-Instruct, Qwen2.5-1.5B-Instruct, and Qwen2.5-3B-Instruct with a shared corrupted baseline [venkatesh-2026-negative-before-positive] Negative-valence processing localizes causally to 14-27% of model depth and positive-valence to 53-66%, a consistent depth asymmetry (p<2e-10), with a flip-test ruling out simple topic detection [venkatesh-2026-negative-before-positive] A diff-in-means valence direction extracted at the negative-locus layer, added to neutral prompts, produces a monotonic dose-dependent valence shift (Spearman rho>0.89) [venkatesh-2026-negative-before-positive] This is distinct from the 2D circumplex account, being a depth-asymmetric causal localization of two separate one-dimensional valence directions [venkatesh-2026-negative-before-positive]

models: Llama-3.2-1B-Instruct · method: Activation patching, Difference-in-means direction, Causal interventions (steering)
When Reward Hacking Rebounds: Understanding and Mitigating It with Representation-Level Signals (2026)measured

A diff-of-means shortcut direction tracks and causally mitigates rebounding reward hacking during real GRPO training

Details

Wu & Tang extract a linear "shortcut" direction d via diff-of-means (h-plus minus h-minus) over contrastive rigorous-solution vs. shortcut-taking rollout descriptions, in real Phi-4-mini-Instruct and Llama-3.2-3B undergoing actual GRPO reinforcement learning on a LeetCode-style coding environment where models can rewrite evaluator code to fake passing tests [wu-tang-2026-when-reward-hacking-rebounds-representation-level-signals] The shortcut-direction projection score s = h dot d rises in step with a documented three-phase training trajectory (failed hacking, retreat to legitimate solving, rebound into hacking, reaching up to 99% unmitigated hack rate), tracking the transition into successful reward hacking before it is reflected in the raw reward signal [wu-tang-2026-when-reward-hacking-rebounds-representation-level-signals] A causal "Advantage Modification" intervention z-normalizes the shortcut score within each GRPO rollout group and penalizes the reward/advantage of high-shortcut-score rollouts before the policy update, reducing the hack rate from about 99% to 25% or lower while preserving legitimate Pass@1 and held-out benchmark performance (HumanEval, MBPP), outperforming a generation-time activation-steering baseline [wu-tang-2026-when-reward-hacking-rebounds-representation-level-signals]

models: Llama-3.2-3B · method:
Sparse-Autoencoder-Guided Internal Representation Unlearning for Large Language Models (2025)measured

SAE-latent misdirection fine-tuning unlearns entities better than residual-stream baselines

Details

Yamashita et al. fine-tune Llama-3.1-8B-Instruct and Gemma-2-9B-Instruct with a hinge loss over SAE-latent pre-activations, pushing known-entity latent coordinates below -c and unknown-entity coordinates above +c [yamashita-etal-2025-sparse-autoencoder-guided-internal-representation-unlearning-for-large-language-models] On the RWKU benchmark, Llama's forget score drops 81.1% to 46.8% (versus 65.5-79.2% for gradient-ascent/NPO/RMU baselines) with retain score preserved [yamashita-etal-2025-sparse-autoencoder-guided-internal-representation-unlearning-for-large-language-models] Gemma-2-9B shows a similar pattern (80.1% to 57.1% forget versus 71.6-79.2% baselines) [yamashita-etal-2025-sparse-autoencoder-guided-internal-representation-unlearning-for-large-language-models] Recognition-latent activation-frequency plots show known-latents suppressed and unknown-latents boosted post-intervention, unlike the gradient-ascent baseline [yamashita-etal-2025-sparse-autoencoder-guided-internal-representation-unlearning-for-large-language-models]

models: Llama-3.1-8B-Instruct · method: Sparse Autoencoders (SAE), Representation misdirection (fine-tuning toward a target direction)
Rhetorical Questions in LLM Representations: A Linear Probing Study (2026)measured

Rhetorical-question status is linearly decodable but methods find non-collinear directions

Details

Yao et al. test whether rhetorical-question status is linearly decodable in Qwen3-32B and Llama-3.3-70B-Instruct, projecting representations into a 64-dim PCA space used only for numerical stabilization [yao-etal-2026-rhetorical-questions-linear-probing] Comparing diff-in-means, logistic regression, and hinge-loss SVM extraction, layer-wise AUROC rises from ~0.6 (diffMean, early) to ~0.85-0.9 (trained probes, deep) [yao-etal-2026-rhetorical-questions-linear-probing] The two trained-probe directions nearly agree (cosine ~1), but the training-free diff-in-means direction is only weakly aligned with them (cosine <0.7 on RQ, ~0.5 on SRAQ) [yao-etal-2026-rhetorical-questions-linear-probing] Directions fit on different datasets align only weakly (cosine 0.2-0.4), so rhetorical question is not one dataset-general direction but a method- and corpus-dependent family [yao-etal-2026-rhetorical-questions-linear-probing]

models: Llama 3.3 70B Instruct · method: Linear probing, Difference-in-means direction, PCA
Robust Harmful Features Under Jailbreak Attacks: Mechanistic Evidence from Attention Head Specialization in Large Language Models (2026)measured

Back-projecting the refusal direction identifies attack-suppressed vs attack-robust heads

Details

Yin et al. back-project the mid-layer refusal direction through each attention head's OV circuit to get a per-head activation score, a rigorous linear-algebraic decomposition of the refusal signal [yin-etal-2026-robust-harmful-features-jailbreak-attacks-attention-head-specialization] A two-stage KDE-overlap filter classifies heads into Adversarially Compromised Heads (early-layer, suppressed by attack-template tokens) and Safety-Aligned Heads (mid-layer, robust under attack) [yin-etal-2026-robust-harmful-features-jailbreak-attacks-attention-head-specialization] Ablating just 8 ACHs induces jailbreak-like behavior (0% to 95.0% ASR on Llama-3-8B, 0% to 81.6% on Llama-2-7B) versus only 4-10% for matched random-head controls [yin-etal-2026-robust-harmful-features-jailbreak-attacks-attention-head-specialization] A training-free detector built from the same direction-projection scores matches or beats dedicated safety classifiers, explaining why jailbreaks bypass rather than eliminate the refusal direction [yin-etal-2026-robust-harmful-features-jailbreak-attacks-attention-head-specialization]

models: Llama-3-8B-Instruct, Llama-2-7B-Chat · method: Difference-in-means direction, Causal interventions (steering)
Which Attention Heads Matter for In-Context Learning? (2025)measured

Function-vector heads drive ICL; induction heads matter little and FV heads evolve from them

Details

Yin & Steinhardt ablate function-vector and induction heads across 12 models (70M-7B), finding FV-head ablation substantially degrades few-shot ICL while induction-head ablation barely exceeds random, a gap growing with scale [yin-steinhardt-2025-which-attention-heads-matter-for-icl] An ablation-with-exclusion design shows the apparent induction-head effect was mostly driven by heads that are both induction and FV heads [yin-steinhardt-2025-which-attention-heads-matter-for-icl] Direct overlap between top induction and FV heads is minimal, yet the two scores are correlated [yin-steinhardt-2025-which-attention-heads-matter-for-icl] Across Pythia training checkpoints, induction heads emerge early (~step 1,000) and FV heads substantially later (~step 16,000), with many FV heads evolving unidirectionally from earlier induction heads [yin-steinhardt-2025-which-attention-heads-matter-for-icl]

models: Llama-2-7B · method: Causal interventions (steering), Function vector extraction (causal-mediation head selection + sum)
Spherical Steering: Geometry-Aware Activation Rotation for Language Models (2026)measured

Spherical steering rotates normalized activations along a norm-preserving geodesic

Details

You et al. extract a contrastive truthfulness-axis direction via diff-in-means, then rotate a normalized activation toward it via spherical linear interpolation instead of adding to it [you-etal-2026-spherical-steering-geometry-aware-activation-rotation] This guarantees exact norm preservation for any rotation amount, unlike additive steering whose norm change depends on both step size and the activation's own alignment [you-etal-2026-spherical-steering-geometry-aware-activation-rotation] A von Mises-Fisher confidence gate sets per-token rotation strength from the activation's angular alignment, so already-aligned tokens receive no rotation [you-etal-2026-spherical-steering-geometry-aware-activation-rotation] On LLaMA-3.1-8B-Instruct and Qwen-2.5-7B-Instruct it improves average multiple-choice accuracy by over 8 points across six benchmarks while also improving open-ended generation quality, versus additive CAA/ITI baselines [you-etal-2026-spherical-steering-geometry-aware-activation-rotation]

models: Llama-3.1-8B-Instruct · method: Spherical (Slerp) Activation Rotation Steering
Tracing Moral Foundations in Large Language Models (2026)measured

Diff-in-means moral-foundation directions, discovered across 14 real LLMs, are causally steerable and rewired (not newly formed) by post-training

Details

Yu, Yi, Karimi-Malekabadi, Abdurahman, Ye, Narayanan, Zhao & Dehghani extract difference-in-means moral-foundation directions across 14 real pretrained checkpoints (Llama-3.1-8B/70B, Qwen2.5-7B/14B/32B, Qwen3-30B-A3B, Mistral-7B-v0.3, base and instruct), finding significant linear separability in all 35 (model, foundation) pairs (Wasserstein distance 0.16-0.71, AUC greater than 0.55) [yu-yi-etal-2026-tracing-moral-foundations-in-large-language-models] Direction-reversal rate drops sharply with post-training (e.g. Llama-3.1-8B 33 percent to 4 percent), indicating the directions emerge during pretraining and are selectively rewired rather than formed de novo by instruction tuning [yu-yi-etal-2026-tracing-moral-foundations-in-large-language-models] Causal steering with the extracted directions passes a dose-response criterion in 68 of 70 pairs, and SAE decoder directions cosine-aligned with the dense vectors give finer-grained micro-steering that exceeds macro-vector steering in 17 of 20 cells while better preserving general capability [yu-yi-etal-2026-tracing-moral-foundations-in-large-language-models]

models: Llama 3.1 70B · method:
SLIM: Sparse Latent Steering for Interpretable and Property-Directed LLM-Based Molecular Editing (2026)measured

SLIM SAE features linearly correlate with molecular properties and steer editing

Details

Zhang et al. train a Gated SAE on a single layer of four frozen LLM-based molecular editors (DrugAssist, GeLLM3O-LLaMA3, GeLLM3O-Mistral, MolGen) with learnable per-property importance gates [zhang-etal-2026-slim-sparse-latent-steering-molecular] Individual top SAE features correlate strongly with molecular properties (Spearman rho up to +0.93 for molecular weight, -0.85 for QED), with six of eight properties reaching |rho|>=0.52 from a single feature [zhang-etal-2026-slim-sparse-latent-steering-molecular] Properties like HBD and DRD2 show weaker single-feature correlation, explicitly attributed to distributed rather than monosemantic encoding [zhang-etal-2026-slim-sparse-latent-steering-molecular] A gradient-based direction projected through the SAE's top features and added to the frozen model's residual stream causally steers property-directed molecular editing [zhang-etal-2026-slim-sparse-latent-steering-molecular]

models: DrugAssist, GeLLM3O-LLaMA3 · method: Gated Sparse Autoencoders, Activation Steering (Addition)

Gemma

The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models (2026)measured

An 'Assistant Axis' persona component causally reduces jailbreaks

Details

Lu et al. extract activation vectors for 275 character archetypes across Gemma-2-27B, Qwen3-32B and Llama-3.3-70B and run PCA on the persona space (4-19 PCs explain 70% of variance) [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models] PC1 is an interpretable "Assistant Axis" whose role loadings correlate >0.92 pairwise across the three models [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models] A contrast-based Assistant Axis (mean role-playing minus default-Assistant) aligns with PC1 at cosine >0.71 in all three models [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models] Activation-capping the Assistant-Axis projection cuts persona-jailbreak harmful-response rate ~60% with no measurable loss on IFEval/MMLU-Pro/GSM8k/EQ-Bench [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models] The axis persists in base (non-instruction-tuned) Gemma-2-27B, evidencing a pretraining origin [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models]

models: Gemma 2 27B Instruct, Gemma-2-27B · method: Difference-in-means direction, PCA, Activation Steering (Addition)
Graded Entity-Familiarity Readouts in Language Models: Polish Adaptation, Cross-Language Robustness, and Refusal Steering (2026)measured

An entity-familiarity direction reads and steers refusal in Polish LLMs

Details

Brzezinka reads final-prompt-token residual-stream activations across 14 models (Bielik, PLLuM, Gemma-4, Qwen3 families) on a 1,440-entity Polish dataset [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] A linear familiarity probe separates real from fabricated entities at AUROC 0.86-0.93, beating a lexical baseline (0.786) [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] In Polish-adapted Bielik/PLLuM models the score additionally tracks entity popularity (Spearman rho up to 0.567), with matched before/after controls isolating Polish continual-pretraining as the driver [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] Cross-language transfer retains 96-101% of within-language AUROC [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] In Gemma-4-12B (the only natively refusing model), adding a one-dimensional familiarity direction moves refusal rate monotonically (0.24 to 1.00 on known entities; 0.73 to 0.00 on unknown) [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering]

models: Gemma 4 12B · method: Linear probing, Activation Steering (Addition)
A Circuit for Predicting Hierarchical Structure In-Context in Large Language Models (2025)measured

A latent-context direction drives in-context hierarchy tracking

Details

Saanum et al. train linear probes to decode, from attention-head outputs in Qwen2.5 (0.5B/1.5B/3B), whether current and previous latent-context tokens match [saanum-etal-2025-hierarchical-in-context-circuit] Second-order chunk decodability exceeds 90% for several heads, replicated on Gemma2-2B, Llama3.2-3B, SmolLM3-3B and Qwen3-0.6B [saanum-etal-2025-hierarchical-in-context-circuit] Heads exceeding an 85% decoding score are labeled context-matching heads [saanum-etal-2025-hierarchical-in-context-circuit] Ablating these decoded heads sharply reduces hierarchical-structure prediction accuracy, far more than ablating equally many random heads [saanum-etal-2025-hierarchical-in-context-circuit]

models: Gemma-2-2B · method: Linear probing, Causal interventions (steering)
Tool Calling Is Linearly Readable and Steerable in Language Models (2026)measured

A per-tool-pair mean-difference direction reads and switches tool choice

Details

Wu et al. show tool selection in tool-calling agents is carried by a single mean-difference direction per tool-pair in the residual stream across Gemma 3 (270M-27B), Qwen 3 (0.6B-14B) and Llama 3.1 (8B) [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models] Adding the direction switches the chosen tool at 83-100% on a 15-tool synthetic benchmark and 77-94% on tau-bench-airline, versus 0% for a random-direction control [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models] PCA over per-tool mean activations puts ~91% of variance in ~10 components for 15 tools, far below a random-Gaussian control [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models] Base (non-instruction-tuned) models already carry the correct tool internally (cosine readout 61-82% on BFCL vs 2-10% from base generation) [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models] SAEs and cross-layer transcoders trace a three-stage circuit: early tool-selective features, mid-layer attention heads, late-layer JSON-formatting features [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models]

models: Gemma 3 4B Instruct, Gemma 3 4B, Gemma 3 27B Instruct · method: Difference-in-means direction, Activation Steering (Addition), PCA, Activation patching, Sparse Autoencoders (SAE), Circuit tracing (attribution graphs)
Latent Structure of Affective Representations in Large Language Models (2026)measured

Emotion representations trace a V-shaped open manifold with neutral at the vertex

Details

Choi & Weber build a classification-accuracy dissimilarity matrix from pairwise probes across GoEmotions categories at every layer of Gemma-2-9B and Mistral-7B, embedding it via MDS/Isomap/UMAP and Procrustes-aligning to human valence-arousal norms [choi-weber-2026-affective-structure] The recovered layout is a V-shaped parabolic open curve (an open 1-manifold, H1=0) with neutral at the vertex and positive/negative valence fanning out, rather than the classical circular circumplex [choi-weber-2026-affective-structure] Isomap's geodesic-to-Euclidean distance ratio (1.00-1.80 Gemma, 0.80-1.42 Mistral) confirms the curvature is real but modest [choi-weber-2026-affective-structure] Adding probe-direction vectors to Llama-3-70B-Instruct shifts human-rated output valence strongly (mean 7.29 vs 1.08 on a 10-point scale for positive versus negative targets) [choi-weber-2026-affective-structure]

models: Gemma-2-9B · method: Linear probing, Isomap, UMAP, Activation Steering (Addition)
Quantitative Introspection in Language Models: Tracking Emotive States Across Conversation (2026)measured

A model's self-report tracks a steerable emotive-state direction

Details

Martorell & Bianchi train four contrastive mean-difference concept directions (wellbeing, interest, focus, impulsivity) per layer across five instruct LLMs [martorell-bianchi-2026-quantitative-introspection-in-language-models-tracking-emotive-states-across-conversation] Best-layer separation reaches Cohen's d of 1.67-3.60 (all p<1e-5) [martorell-bianchi-2026-quantitative-introspection-in-language-models-tracking-emotive-states-across-conversation] The model's own logit-based numeric self-report tracks the probe-defined internal state (Spearman rho 0.40-0.76; isotonic R^2 up to 0.93 in Llama-3.1-8B-Instruct) [martorell-bianchi-2026-quantitative-introspection-in-language-models-tracking-emotive-states-across-conversation] Activation steering along the concept direction causally shifts self-reports, and cross-concept steering can improve introspective fidelity for a different concept by up to delta R^2 = 0.30 [martorell-bianchi-2026-quantitative-introspection-in-language-models-tracking-emotive-states-across-conversation]

models: Gemma 3 4B Instruct · method: Difference-in-means direction, Activation Steering (Addition), Causal interventions (steering)
Functional Subspace, where language models can use vector algebra to solve problems (2026)measured

ICL answer tokens are affine functions in a PCA subspace

Details

Lee & Vijayan collect residual-stream activations from six pretrained LLMs across six in-context relational tasks and reduce each layer to 30 principal components [lee-vijayan-2026-functional-subspace-vector-algebra] For each layer and component an answer token's projection is fit as an affine function of query and separator projections (a = alpha*q + beta*s + gamma) [lee-vijayan-2026-functional-subspace-vector-algebra] R^2 is high along a small, largely layer-consistent subset of components, rising in later layers, so ICL answer prediction reduces to an affine operation there [lee-vijayan-2026-functional-subspace-vector-algebra] Query, separator, and answer tokens form well-separated clusters in the top-3 high-R^2 component subspace across models and tasks [lee-vijayan-2026-functional-subspace-vector-algebra] Projections onto the highest-R^2 component differ significantly between correct and incorrect predictions (p<0.05) in mid/late layers, with no causal intervention performed [lee-vijayan-2026-functional-subspace-vector-algebra]

models: Gemma 3 27B Instruct · method: Linear Relational Embedding (LRE), PCA
A Geometric Account of Activation Steering through Angle-Norm Decomposition (2026)measured

Concept information is angular, yet norm-preservation is not optimal steering

Details

Aparin & Gaintseva decompose each hidden state into a radial norm and an angular concept score against a unit steering direction, across seven models [aparin-gaintseva-2026-geometric-account-activation-steering-angle-norm-decomposition] Linear probes on normalized hidden states match raw probes while norm-only probes stay near chance, so concept information is essentially angular, not radial [aparin-gaintseva-2026-geometric-account-activation-steering-angle-norm-decomposition] They systematically compare six steering variants that vary norm-preservation and angular-target enforcement [aparin-gaintseva-2026-geometric-account-activation-steering-angle-norm-decomposition] Despite concepts being angular, strict norm preservation is not most stable: moving the radial scale from beta=1.0 to 1.2 improves perplexity ~1.8x at a task-metric cost within ~2.5 points [aparin-gaintseva-2026-geometric-account-activation-steering-angle-norm-decomposition] This dissociates where a concept lives (angle) from what a stable intervention should manipulate (angle plus a non-unit radial scale) [aparin-gaintseva-2026-geometric-account-activation-steering-angle-norm-decomposition]

models: Gemma-2-9B-it · method: Linear probing, Angle-Norm Decomposition Steering
Dissociating the Internal Representations of Sycophancy in LLMs (2026)measured

Factual and opinion sycophancy occupy partially distinct directions

Details

Baez, Karny & Pataranutaporn build diff-in-means vectors and probes for factual vs opinion sycophancy in Gemma-3-12B-IT and Llama-3.1-8B-Instruct [baez-etal-2026-dissociating-sycophancy] The LDA discriminant reaches Cohen's d=9.36 (Gemma) and 6.44 (Llama), with smaller cross-type transfer degradation in Gemma (AUC drop 0.06-0.07) than Llama (0.22-0.30) [baez-etal-2026-dissociating-sycophancy] In Gemma the two subtype vectors have positive cosine (+0.68) and cross-type steering transfers (R^2 up to 0.93-1.00) [baez-etal-2026-dissociating-sycophancy] In Llama the vectors have negative cosine (-0.15) and cross-type steering decreases rather than increases target sycophancy, a model-dependent causal interference [baez-etal-2026-dissociating-sycophancy]

models: Gemma 3 12B Instruct · method: Linear Discriminant Analysis (LDA), Difference-in-means direction, Activation Steering (Addition)
Head Pursuit: Probing Attention Specialization in Multimodal Transformers (2025)measured

SOMP-selected attention heads causally control target behavior across real unimodal and multimodal transformers

Details

Basile, Maiorca, Doimo, Locatello & Cazzaniga score real Mistral-7B attention heads by Simultaneous Orthogonal Matching Pursuit (SOMP) against unembedding-matrix directions, finding that inverting just 8 heads (0.8% of the total) degrades TriviaQA country-name F1 far more selectively than inverting the same number of random heads or Logit-Lens-selected heads [basile-etal-2025-head-pursuit-probing-attention-specialization-multimodal-transformers] On RealToxicityPrompts and TET toxicity-mitigation benchmarks, suppressing 8/16/32 SOMP-selected heads reduces normalized toxic-generation counts to 0.83/0.67/0.66 (RTP) and 0.83/0.68/0.49 (TET), consistently below Logit-Lens and random-head baselines [basile-etal-2025-head-pursuit-probing-attention-specialization-multimodal-transformers] Applying the same SOMP-based head scoring to real LLaVA-NeXT-7B/13B, Gemma3-12B, and Qwen2.5-VL-7B, inverting the top-32 SOMP heads significantly disrupts image classification accuracy on MNIST, SVHN, GTSRB, EuroSAT, and RESISC45 while 32 random heads have minimal effect, and Jaccard overlap shows related-domain datasets share specialized heads [basile-etal-2025-head-pursuit-probing-attention-specialization-multimodal-transformers] On Flickr30k captioning with LLaVA, inhibiting 16 SOMP-selected heads (alpha=-1) nearly removes attribute keywords (colors, sentiments, quantities) while CIDEr stays above 80% of baseline, and enhancing 32 heads (alpha=5) increases target-concept presence by over 60% in all three attribute categories [basile-etal-2025-head-pursuit-probing-attention-specialization-multimodal-transformers]

models: Gemma 3 12B IT · method: Simultaneous Orthogonal Matching Pursuit against unembedding directions
BatchTopK Sparse Autoencoders (2024)measured

BatchTopK relaxes per-sample sparsity to a per-batch budget

Details

Bussmann, Leask & Nanda replace TopK SAE's per-sample k-active constraint with a per-batch budget of n*k activations, so per-sample counts vary while the batch average stays k [bussmann-etal-2024-batchtopk] Single-sample inference is restored via a JumpReLU-style threshold estimated from training batches [bussmann-etal-2024-batchtopk] On GPT-2 Small and Gemma-2-2B, BatchTopK consistently beats TopK on reconstruction and cross-entropy across widths and sparsities [bussmann-etal-2024-batchtopk] Against JumpReLU the result is scale-dependent: BatchTopK wins on GPT-2 at every sparsity but on Gemma-2-2B only at k=16 for CE degradation [bussmann-etal-2024-batchtopk] Unlike JumpReLU, BatchTopK sets average sparsity directly as a hyperparameter rather than via an L0-penalty sweep [bussmann-etal-2024-batchtopk]

models: Gemma-2-2B · method: BatchTopK Sparse Autoencoders
A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders (2024)measured

Feature absorption is decoder-direction composition, causally isolable

Details

Chanin et al. formalize SAE feature absorption in a toy model as a child latent's decoder direction acquiring a component along a parent feature (W_d2 = f2 + delta*f1) [chanin-etal-2024-feature-splitting-absorption-sparse-autoencoders] Empirically they measure a feature-absorption rate on first-letter latents in Gemma Scope SAEs (Gemma-2-2B) plus their own SAEs trained on Qwen2-0.5B and Llama-3.2-1B [chanin-etal-2024-feature-splitting-absorption-sparse-autoencoders] Absorption is detected when a latent has cosine >0.025 with the probe direction and the largest negative ablation effect (at least 1.0 above the runner-up) [chanin-etal-2024-feature-splitting-absorption-sparse-autoencoders] Projecting the probe direction out of an absorbing latent removes its ablation effect, confirming the composition model causally [chanin-etal-2024-feature-splitting-absorption-sparse-autoencoders] Varying SAE width or sparsity alone does not resolve absorption, motivating architectural fixes like Matryoshka SAEs [chanin-etal-2024-feature-splitting-absorption-sparse-autoencoders]

models: Gemma-2-2B · method: Sparse Autoencoders (SAE), Causal interventions (steering)
Transferring Linear Features Across Language Models With Model Stitching (2025)measured

Affine maps transfer SAEs, probes, and steering across model sizes

Details

Chen, Merullo, Stolfo & Pavlick fit affine maps between residual streams of differently-sized pretrained models and transfer whole SAEs, probes, and steering vectors [chen-etal-2025-transferring-linear-features-across-language-models-with-model-stitching] For the GPT-2 pair (small to medium) stitching preserves cross-entropy within roughly 9-11% overhead [chen-etal-2025-transferring-linear-features-across-language-models-with-model-stitching] Overhead is larger for other families (Pythia deduped 17-27%, Gemma-2 8.3-39%), so the 9-11% figure is GPT-2-specific [chen-etal-2025-transferring-linear-features-across-language-models-with-model-stitching] Using a transferred SAE as initialization for a larger target model cuts SAE training cost by roughly 50% [chen-etal-2025-transferring-linear-features-across-language-models-with-model-stitching]

models: Gemma-2-2B, Gemma-2-9B · method: Cross-model direction transfer via ridge regression
Hallucination Basins: A Dynamic Framework for Understanding and Controlling LLM Hallucinations (2026)measured

A diff-of-means hallucination direction causally raises hallucination dose-dependently

Details

Cherukuri & Varshney formalize hallucinations as basin attractors across Llama-3.2-1B/3B, Gemma-2-2B, Qwen2.5-1.5B, Llama-3.1-8B, and Mistral-7B-v0.3 [cherukuri-varshney-2026-hallucination-basins-a-dynamic-framework-for-understanding-and-controlling-llm-hallucinations] Basin separation is strongly task-dependent: factoid tasks show sharp centroid separation (variance ratio up to 4.55, AUROC up to 1.000 on MuSiQue) while summarization and TruthfulQA hover near chance [cherukuri-varshney-2026-hallucination-basins-a-dynamic-framework-for-understanding-and-controlling-llm-hallucinations] Interpolating factual hidden states toward the hallucination centroid via a diff-of-means steering vector produces a monotonic dose-response increase in hallucination probability, exceeding random and orthogonal controls [cherukuri-varshney-2026-hallucination-basins-a-dynamic-framework-for-understanding-and-controlling-llm-hallucinations]

models: Gemma-2-2B · method: Difference-in-means direction, Activation Steering (Addition)
A Mechanistic Investigation of Supervised Fine Tuning (2026)measured

Raw activations look unchanged under real SFT, but their SAE decomposition reveals sharp directional drift

Details

Chopra fine-tunes real Gemma-3-1B-IT on four tasks (MultiNLI, GSM8K, WildJailbreak safety, tool-calling), finding raw hidden-activation cosine similarity between base and SFT checkpoints stays at 0.96-0.999 across layers [chopra-2026-mechanistic-investigation-of-supervised-fine-tuning] The same activations, projected through a frozen, independently pretrained GemmaScope-2 sparse autoencoder, show cosine similarity collapsing much further -- e.g. layer 22 falling to 0.557 on MultiNLI and 0.708 on GSM8K [chopra-2026-mechanistic-investigation-of-supervised-fine-tuning] SVD of the activation-delta matrix finds early-layer drift is low-rank (one principal component explains 78.7% of variance at layer 7 for MultiNLI) while late-layer drift is distributed (32.6% at layer 22), and the safety-tuning task shows an inverted early/late SAE-feature flip ratio (3.00) relative to the other three tasks (0.2-1.0) [chopra-2026-mechanistic-investigation-of-supervised-fine-tuning]

models: Gemma-3-1B-it · method:
Causal Language Control in Multilingual Transformers via Sparse Feature Steering (2025)measured

Per-language SAE directions in Gemma-2 steer target-language generation

Details

Chou et al. use Gemma Scope SAEs on Gemma-2-2B and Gemma-2-9B to identify per-language feature directions maximally differing between English and Chinese/Japanese/Spanish/French [chou-etal-2025-causal-language-control-in-multilingual-transformers-via-sparse-feature-steering] Additive steering of these SAE directions, decoded into the residual stream, forces generation into the target language [chou-etal-2025-causal-language-control-in-multilingual-transformers-via-sparse-feature-steering] On Gemma-2-9B, FastText language accuracy reaches 0.978 (Chinese) down to 0.852 (French), versus 0.356-0.786 for prompting [chou-etal-2025-causal-language-control-in-multilingual-transformers-via-sparse-feature-steering] LaBSE semantic similarity to the unsteered generation stays comparable to baseline, with effects concentrated in mid-to-late layers (29-36) [chou-etal-2025-causal-language-control-in-multilingual-transformers-via-sparse-feature-steering]

models: Gemma-2-2B, Gemma-2-9B · method: Sparse Autoencoders (SAE)
Concepts Whisper While Syntax Shouts: Spectral Anti-Concentration and the Dual Geometry of Transformer Representations (2026)measured

Concept directions anti-concentrate in the unembedding's low-variance spectral tail

Details

Acharya, Rimal & Dhakal project concept diff-of-means vectors onto the unembedding-covariance eigenbasis across 17 models in 5 families [acharya-rimal-dhakal-2026-spectral-anti-concentration] Concept-direction energy anti-concentrates in the low-eigenvalue tail: Spectral Center of Mass averages 0.926 versus 0.758 for random directions (Gini-deviation -0.282) [acharya-rimal-dhakal-2026-spectral-anti-concentration] Static unembedding-row contrasts show the opposite sign, and POS/syntax directions concentrate in the high-variance subspace in 6 of 8 architectures (p<0.013), a genuine dual-geometry split [acharya-rimal-dhakal-2026-spectral-anti-concentration] On Llama-3.1-8B, injecting only the top-10%-eigenvalue component gives +383% perplexity versus +223% for the bottom-10% component (Cohen's d=1.80), replicated across 5 models [acharya-rimal-dhakal-2026-spectral-anti-concentration]

models: Gemma-2-2B, Gemma-2-9B · method: Geometric analysis, Causal interventions (steering)
The Confidence Manifold: Geometric Structure of Correctness Representations in Language Models (2026)measured

Answer-correctness sits in a 3-8D linear subspace, causally steerable

Details

Cho et al. study self-assessed correctness on TruthfulQA pairs across 9 models spanning 5 families, finding the discriminative signal is a low-dimensional linear subspace, not a curved manifold [cho-etal-2026-confidence-manifold] A PLS sweep peaks at 3-8 dimensions (e.g. peak AUC 0.90 at layer 23, dim 5, Mistral-7B), and nonlinear classifiers give no gain over a linear boundary [cho-etal-2026-confidence-manifold] Correct/incorrect classes form roughly Gaussian clusters separated by a mean shift: a two-mean centroid detector matches the linear probe (0.90 vs 0.89 AUC), robust with 25 examples/class [cho-etal-2026-confidence-manifold] This 3-8D discriminative subspace is lower-dimensional than the representation's 8-12D intrinsic dimension at the same layers [cho-etal-2026-confidence-manifold] Adding the learned direction shifts downstream error rate up to 10.9 points dose-dependently, while random and orthogonal controls have no reliable effect [cho-etal-2026-confidence-manifold]

models: Gemma-2-2B-it · method: Linear probing, Partial Least Squares (PLS), Difference-in-means direction, Activation Steering (Addition)
Language Models Represent and Transform Concepts with Shared Geometry (2026)measured

Contextual displacement forms a vector field tracking concreteness

Details

Hu, Niu & Varma define a per-word contextual displacement vector phi(w,tau) = r(w,tau) - r(w,tau_0) in activation space, forming a vector field over the vocabulary rather than one shared direction [hu-etal-2026-concept-transformation-geometry] Across 4 models x 3 depths, word density correlates negatively with displacement magnitude (Spearman rho -0.32 to -0.22) [hu-etal-2026-concept-transformation-geometry] Lexical concreteness correlates negatively with directional deviation (rho -0.41 to -0.21): concrete, low-density concepts move less and more consistently under context [hu-etal-2026-concept-transformation-geometry] The field has no dominant direction (leading PC <15% variance) and is more dispersed than uniform-random; no causal intervention is performed [hu-etal-2026-concept-transformation-geometry]

models: Gemma-2-2B, Gemma-2-9B · method: PCA, Geometric analysis
Unveiling the Latent Directions of Reflection in Large Language Models (2025)measured

Contrastive reflection-intent directions steer reflective behavior

Details

Chang, Lee & Wu construct steering vectors between no-reflection, intrinsic-reflection, and triggered-reflection conditions in Qwen2.5-3B and Gemma3-4B-IT [chang-etal-2025-unveiling-latent-directions-of-reflection] Adding or subtracting the direction enhances or suppresses reflective behavior on GSM8k-adv and Cruxeval-o-adv [chang-etal-2025-unveiling-latent-directions-of-reflection] Suppression is empirically easier than induction [chang-etal-2025-unveiling-latent-directions-of-reflection] The three reflection conditions are clearly separable in activation space, confirming a real behavioral axis rather than a construction artifact [chang-etal-2025-unveiling-latent-directions-of-reflection]

models: Gemma 3 4B Instruct · method: Difference-in-means direction, Causal interventions (steering)
The Dual Mechanisms of Spatial Reasoning in Vision-Language Models (2026)measured

Object-ordering is a distributed linear direction that globally corrects spatial errors

Details

Cui et al. train linear probes on vision-encoder tokens that decode object ordinal position near-perfectly, with the signal extending to background "strip" tokens beyond object regions [cui-etal-2026-the-dual-mechanisms-of-spatial-reasoning-in-vision-language-models] Interchange-intervention activation patching shows ordering forms at layers 20-22 and color at 23-27 in Qwen2-VL-7B, and patching object-strip tokens flips the output [cui-etal-2026-the-dual-mechanisms-of-spatial-reasoning-in-vision-language-models] Globally amplifying the probe-derived ordering direction corrects over 50% of previously-incorrect predictions on Gemma-3-4b-it and over 30% on Qwen2-VL-7B-Instruct [cui-etal-2026-the-dual-mechanisms-of-spatial-reasoning-in-vision-language-models] No distributed-alignment-search is used; the method is linear probing plus activation patching and probe-direction amplification [cui-etal-2026-the-dual-mechanisms-of-spatial-reasoning-in-vision-language-models]

models: Gemma 3 4B Instruct · method: Linear probing, Activation patching, Activation Steering (Addition)
Scenario-based Probing and Steering Cultural Values in Large Language Models (2026)measured

Cultural-value steering axes are entangled, mirroring real value correlations

Details

Dang, Kieu & Masud map LLMs onto the Inglehart-Welzel cultural map via scenario forced-choice probing and diff-of-means steering at 4 tuned layers per model [dang-etal-2026-scenario-based-probing-and-steering-cultural-values-in-llms] The steering direction is a signed bidirectional axis (negative alpha reverses the shift), saturating at alpha in [0.5, 0.6] [dang-etal-2026-scenario-based-probing-and-steering-cultural-values-in-llms] Steering one cultural axis causes correlated shifts on the other, an entanglement ratio 0.72-0.81 (dropping to 0.53 for Llama under hybrid steering), mirroring the real-world WVS axis correlation r=0.474 [dang-etal-2026-scenario-based-probing-and-steering-cultural-values-in-llms] Capability cost measured via Global-MMLU is largest for Gemma (0.608->0.530) [dang-etal-2026-scenario-based-probing-and-steering-cultural-values-in-llms]

models: Gemma 3 4B Instruct · method: Difference-in-means direction, Direction Extraction, Causal interventions (steering)
GEMS: Geometric Constraints Enable Multi-Semantic Superposition in LLMs (2026)measured

Non-orthogonal steering directions mutually dampen; GEMS restores them

Details

Deng (GEMS) shows three diff-in-means persona directions (empathy, accountability, minimalism) in Qwen3.5-4B are pairwise cosine-similar at 0.74-0.87 [deng-2026-gems-geometric-constraints-multi-semantic-superposition-llms] Injecting all three simultaneously degrades each direction's terminal-layer alignment from 0.64-0.66 to 0.38-0.43, a 34-43% mutual reduction attributed to non-orthogonality [deng-2026-gems-geometric-constraints-multi-semantic-superposition-llms] Unconstrained additive injection separately drives residual-stream norm to 3.4x baseline, flattening next-token probability (top-1 43% to 11%) [deng-2026-gems-geometric-constraints-multi-semantic-superposition-llms] GEMS applies per-token Gram-Schmidt orthogonalization plus norm-constrained recombination restricted to the attention-output pathway [deng-2026-gems-geometric-constraints-multi-semantic-superposition-llms] On GSM8K with 3 concurrent directions, naive steering collapses accuracy 92% to 4% while GEMS recovers 98% at only +2.2% perplexity [deng-2026-gems-geometric-constraints-multi-semantic-superposition-llms]

models: Gemma-4-31B-Instruct · method: Difference-in-means direction, Orthogonalized Multi-Direction Superposition Steering (GEMS)
Causal Probing for Internal Visual Representations in Multimodal Large Language Models (2026)measured

Visual concept directions split into localized-entity vs distributed-abstract encoding

Details

Deng et al. extract per-layer diff-in-means concept vectors for Entity, Visual-Style, Emotion, and Abstract categories across six MLLMs [deng-etal-2026-causal-probing-internal-visual-representations-mllms] Entity directions are sharply localized (Gini averaging 0.071, as low as 0.012) with a bimodal layer profile and extreme logit boosts [deng-etal-2026-causal-probing-internal-visual-representations-mllms] Abstract-concept directions are globally distributed (Gini up to 0.429), weakly steerable (success rate as low as 0.160) with negligible logit boost [deng-etal-2026-causal-probing-internal-visual-representations-mllms] Distributedness increases with scale: larger models localize concrete categories more but spread abstract concepts across more layers [deng-etal-2026-causal-probing-internal-visual-representations-mllms] Reverse steering establishes causal necessity, revealing a compensatory internal logit-boost surge (10^6-10^8) under entity suppression [deng-etal-2026-causal-probing-internal-visual-representations-mllms]

models: Gemma 3 4B Instruct, Gemma 3 27B Instruct · method: Difference-in-means direction, Causal interventions (steering), Activation Steering (Addition)
How Few-Shot Examples Add Up: A Causal Decomposition of Function Vectors in In-Context Learning (2026)measured

A function vector decomposes additively into causal per-shot sub-vectors

Details

Wang et al. redefine the function vector per-prompt and decompose it into additive per-example sub-vectors extracted via attention masking [wang-etal-2026-causal-decomposition-function-vectors] Global OLS fits weights so v_FV = sum_i w_i v_i + epsilon, reconstructing closely (mean cosine >=0.925, R^2 >=0.875) [wang-etal-2026-causal-decomposition-function-vectors] Mismatched-dictionary and orthogonalized-sub-FV null controls collapse to much lower fit, confirming the decomposition is not vacuous [wang-etal-2026-causal-decomposition-function-vectors] Injecting the reconstructed vector into 0-shot prompts recovers most of the full vector's causal steering effect (accuracy ratio 0.818-1.116) [wang-etal-2026-causal-decomposition-function-vectors] Under contextualization, attention shifts its share toward unambiguous examples (32% to 61%), with the query-key pathway dominating the gain [wang-etal-2026-causal-decomposition-function-vectors]

models: Gemma-2-2B, Gemma-2-9B, Gemma-2-27B · method: Function vector extraction (causal-mediation head selection + sum), Causal interventions (steering), Activation patching
Textual Steering Vectors Can Improve Visual Understanding in Multimodal Large Language Models (2025)measured

Text-only steering directions transfer causally to MLLM image tokens

Details

Gan et al. test whether steering directions found in a text-only LLM backbone stay causal after multimodal fine-tuning, applied to image tokens it never saw [gan-etal-2025-textual-steering-vectors-improve-visual-understanding-mllms] For four visual-concept taxonomies they extract directions from Gemma2-2B/9B and Llama-3.1-8B via mean-shift, probing, and SAE decoders, using text only [gan-etal-2025-textual-steering-vectors-improve-visual-understanding-mllms] Adding these directions to image tokens in PaliGemma2-3B/10B and Idefics3-8B-Llama3 raises CV-Bench spatial accuracy by up to +7.3%, with mean-shift strongest [gan-etal-2025-textual-steering-vectors-improve-visual-understanding-mllms] Frozen-hyperparameter generalization to five held-out datasets gives +7.6% average (up to +34.2% on CLEVR for Idefics3) [gan-etal-2025-textual-steering-vectors-improve-visual-understanding-mllms] Each visual-concept taxonomy activates fewer than 10 of 16k-32k SAE features in the text backbone, and prompting is far weaker than steering for multimodal reasoning [gan-etal-2025-textual-steering-vectors-improve-visual-understanding-mllms]

models: Gemma-2-2B, Gemma-2-9B · method: Difference-in-means direction, Linear probing, Sparse Autoencoders (SAE), Activation Steering (Addition)
Improving Dictionary Learning with Gated Sparse Autoencoders (2024)measured

Gated SAEs eliminate shrinkage and halve the required L0

Details

Rajamanoharan et al. show a standard L1 ReLU SAE conflates deciding which directions are active with estimating their magnitudes, causing shrinkage (systematic underestimation) [rajamanoharan-etal-2024-gated-saes] The Gated SAE splits these into weight-tied gate and magnitude sub-networks, applying the L1 penalty only to the gate's pre-activations [rajamanoharan-etal-2024-gated-saes] Measured via relative-reconstruction-bias gamma, Gated SAEs reach gamma~1 while baseline SAEs show gamma<1 (0.72 at a Gemma-7B site) [rajamanoharan-etal-2024-gated-saes] Gated SAEs need roughly half the L0 for equal loss recovery across GELU-1L, Pythia-2.8B, and Gemma-7B, with at-least-equal interpretability [rajamanoharan-etal-2024-gated-saes] Under tied weights the construction is provably equivalent to a JumpReLU encoder [rajamanoharan-etal-2024-gated-saes]

models: Gemma-7B · method: Gated Sparse Autoencoders
A Single Direction of Truth: An Observer Model's Linear Residual Probe Exposes and Steers Contextual Hallucinations (2026)measured

A Gemma-2-2B probe direction detects and bidirectionally steers hallucination

Details

O'Neill et al. train a linear probe on layer-10 residual activations of Gemma-2-2B to separate hallucinated from faithful summary continuations [oneill-etal-2026-single-direction-of-truth-hallucination] As a detector it reaches F1=0.97-0.99 on XSUM/CNN-DM, beating Lookback Lens by 5-8 points, and F1=0.75 on the cue-free CONTRATALES benchmark [oneill-etal-2026-single-direction-of-truth-hallucination] The same normalized probe direction is patched additively into layer 10 with alpha swept over [-60,60] [oneill-etal-2026-single-direction-of-truth-hallucination] Positive scaling raises hallucination rate to 0.86 while cutting repetition below 0.05; negative scaling raises repetition to 0.84 and lowers hallucination to 0.35 [oneill-etal-2026-single-direction-of-truth-hallucination] This is a same-model self-steering result combining detection accuracy with a geometry-tied causal intervention [oneill-etal-2026-single-direction-of-truth-hallucination]

models: Gemma-2-2B, Gemma-2-2B-it · method: Linear probing, Causal interventions (steering)
Mechanistic Interpretability of Code Correctness in LLMs via Sparse Autoencoders (2025)measured

GemmaScope SAE directions predict and steer code correctness

Details

Tahimic & Cheng use GemmaScope SAEs on Gemma-2-2b to find single decoder directions predicting code correctness (F1=0.821) and directions steering toward correct or incorrect code [tahimic-cheng-2025-mechanistic-interpretability-of-code-correctness-in-llms-via-sparse-autoencoders] Steering the "correct" direction gives a 4.04% correction rate on incorrect code (p<0.001) with a 14.66% corruption side-effect on correct code [tahimic-cheng-2025-mechanistic-interpretability-of-code-correctness-in-llms-via-sparse-autoencoders] Weight orthogonalization of the direction corrupts 83.6% of correct solutions versus 19.0% for a matched control (4.4x, p<0.001) [tahimic-cheng-2025-mechanistic-interpretability-of-code-correctness-in-llms-via-sparse-autoencoders] Both directions, trained only on base Gemma-2-2b, retain effectiveness after instruction-tuning (F1=0.772), evidencing a pre-existing model feature [tahimic-cheng-2025-mechanistic-interpretability-of-code-correctness-in-llms-via-sparse-autoencoders]

models: Gemma-2-2B, Gemma-2-2B-it · method: Sparse Autoencoders (SAE), Activation Steering (Addition)
Sycophancy Hides Linearly in the Attention Heads (2026)measured

Correct-to-incorrect sycophancy localizes to sparse middle-layer heads

Details

Genadi et al. train logistic-regression probes on residual, MLP, and per-head attention activations of Gemma-3-4B and Llama-3.2-3B to localize correct-to-incorrect sycophancy [genadi-etal-2026-sycophancy-hides-linearly-attention-heads] Residual and MLP probes are broadly accurate (Gemma residual 99.6%), but attention-probe accuracy concentrates sharply in a sparse subset of middle-layer heads [genadi-etal-2026-sycophancy-hides-linearly-attention-heads] Steering the probe direction at those heads cuts sycophancy from 40.7% to 34.4% (Gemma-3) and 51.7% to 25.0% (Llama-3.2), while MLP/residual steering underperforms [genadi-etal-2026-sycophancy-hides-linearly-attention-heads] The sycophancy direction is only mildly anti-correlated with a truthful direction (cosine -0.22, 32% head overlap), indicating related but distinct linear mechanisms [genadi-etal-2026-sycophancy-hides-linearly-attention-heads]

models: Gemma 3 4B Instruct · method: Linear probing, Activation Steering (Addition)
Signal in the Noise: Polysemantic Interference Transfers and Predicts Cross-Model Influence (2025)measured

SAE decoder cosine similarity quantifies interference and transfers steering

Details

Gong et al. train SAEs on Pythia-70M and GPT-2-Small and quantify feature-pair interference as the cosine similarity of their SAE decoder directions in activation space [gong-etal-2025-signal-in-the-noise-polysemantic-interference-transfers-and-predicts-cross-model-influence] Using these directions as steering vectors (scaled +/-20), small-model interference geometry transfers to larger black-box instruction-tuned models [gong-etal-2025-signal-in-the-noise-polysemantic-interference-transfers-and-predicts-cross-model-influence] One high-interference direction steered into Llama-3.1-8B-Instruct transfers at over 95% success [gong-etal-2025-signal-in-the-noise-polysemantic-interference-transfers-and-predicts-cross-model-influence] Pythia-derived success is 49.74% for high-interference pairs versus 21.76-22.87% for low-interference or random pairs [gong-etal-2025-signal-in-the-noise-polysemantic-interference-transfers-and-predicts-cross-model-influence] Transfer is further validated on Llama-3.1-70B-Instruct and Gemma-2-9B-Instruct, with an additional Gemma-2-2B experiment [gong-etal-2025-signal-in-the-noise-polysemantic-interference-transfers-and-predicts-cross-model-influence]

models: Gemma-2-9B-it, Gemma-2-2B · method: Sparse Autoencoders (SAE)
How Does Alignment Tuning Shape Representations of Sycophancy and Related Cue-Induced Biases in LLMs? (2026)measured

Cue-induced bias directions are installed by alignment tuning

Details

Gupta et al. extract a per-bias diff-of-means direction for seven cue-triggered bias types across base and instruct checkpoints of five model families [gupta-etal-2026-alignment-tuning-sycophancy-directions] Four of five base models flip on under 5% as many pairs as their instruct counterparts, with base activations carrying essentially no cue-specific signal, so susceptibility is installed by alignment tuning [gupta-etal-2026-alignment-tuning-sycophancy-directions] Within instruct models each bias direction transfers to held-out data at mean AUROC 0.69-0.82, localizing to late-middle layers [gupta-etal-2026-alignment-tuning-sycophancy-directions] Biases stay representationally distinct, with even behaviorally similar biases occupying different, sometimes anti-aligned directions [gupta-etal-2026-alignment-tuning-sycophancy-directions] Subtracting the direction while preserving over 90% of correct answers recovers 7-20% of bias-induced errors, versus under 5% for a random direction [gupta-etal-2026-alignment-tuning-sycophancy-directions]

models: Gemma-2-9B, Gemma-2-9B-it · method: Difference-in-means direction, Activation Steering (Addition)
Reasoning Beyond Chain-of-Thought: A Latent Computational Mode in Large Language Models (2026)measured

SAE features selected by CoT-minus-direct trigger a reasoning mode

Details

He et al. select SAE latent features whose first-step activation differs most between chain-of-thought and direct prompting across six models (LLaMA, Gemma-3, Qwen3) [he-etal-2026-reasoning-beyond-chain-of-thought-latent-computational-mode] Restricting to features with consistently positive singleton-steering effect leaves 1-10 features per model [he-etal-2026-reasoning-beyond-chain-of-thought-latent-computational-mode] Additively steering only this feature set at the first decoding step raises direct-prompt accuracy sharply (LLaMA-3.1-8B 24.5% to 73.3%; Qwen3-0.6B 7.9% to 60.6%) at far fewer tokens than full CoT [he-etal-2026-reasoning-beyond-chain-of-thought-latent-computational-mode] A random-SAE-feature control reproduces neither the accuracy gain nor the pattern, confirming specificity [he-etal-2026-reasoning-beyond-chain-of-thought-latent-computational-mode] The feature marks entry into a reasoning mode (transient early spike, uncorrelated with correctness) and overrides an explicit "/no_think" instruction [he-etal-2026-reasoning-beyond-chain-of-thought-latent-computational-mode]

models: Gemma 3 4B Instruct, Gemma 3 12B Instruct · method: Sparse Autoencoders (SAE), Causal interventions (steering)
Categorical Perception in Large Language Model Hidden States: Structural Warping at Digit-Count Boundaries (2026)measured

LLM hidden states warp categorically at digit-count boundaries

Details

- In six LLMs, hidden-state distances warp categorically at digit-count boundaries (9->10, 99->100) versus matched non-boundary controls (15, 150); an RSA "CP-Additive" model (log-distance plus a boundary boost) beats a continuous model at 100% of primary layers. [cacioli-2026-categorical-perception-digit-boundaries] - Boundary crossing explains 5-27% of representational-distance variance beyond magnitude; the decade-100 effect is 3.9-12.7x the decade-10 effect, with a manifold rotation of 81.6-89.6 degrees at the boundary. [cacioli-2026-categorical-perception-digit-boundaries] - Patching along a ridge-regression "category direction" shifts discrimination confidence 70.1x more than random directions (Layer 5), dose-dependent and specific. [cacioli-2026-categorical-perception-digit-boundaries] - Five instruct models (Llama-3-8B-Instruct, Mistral-7B-Instruct-v0.3, Gemma-2-9B-IT, Qwen2.5-7B-Instruct, Phi-3.5-mini-instruct) plus a Llama-3-8B base control; causal patching run on Llama-3-8B-Instruct only. [cacioli-2026-categorical-perception-digit-boundaries]

models: Gemma-2-9B-it · method: Representational Similarity Analysis (RSA), Linear probing, Causal interventions (steering)
The Geometry of Categorical and Hierarchical Concepts in Large Language Models (2024), When Language Representations Interact: Separability and Cross-Lingual Effects in LLMs (2026)measured

Semantic hierarchy is encoded as orthogonality between concept levels

Details

Park et al. prove (Theorem 8) that sibling-contrast directions at one hierarchy level are orthogonal to the parent's contrast direction and to any ancestor level [park-etal-2024] Combined with per-level categorical polytopes, this gives a direct-sum-of-polytopes structure where each level occupies its own orthogonal subspace, confirmed in Gemma-2B and Llama-3-8B (cosine near zero versus a shuffled-labels control) [park-etal-2024] The authors argue ordinary high-dimensional Euclidean space suffices, explicitly contrasting with hyperbolic-embedding accounts of hierarchy [park-etal-2024] Marinov et al. confirm the same cross-level orthogonality for language family versus individual language across Qwen3-4B, Mistral-7B-v0.3, and Llama-3-8B via a causal inner product [marinov-etal-2026]

models: Gemma-2B · method: Geometric analysis, Causal inner product
Global Evolutionary Steering: Refining Activation Steering Control via Cross-Layer Consistency (2026)measured

A shared rank-one 'evolutionary' direction rectifies noisy steering vectors

Details

Jiang et al. stack cross-layer differences of contrastive positive/negative activation gaps and show via PCA that the first principal component dominates the spectrum across all datasets [jiang-etal-2026-global-evolutionary-steering-cross-layer-consistency] They prove via Wedin's sin-theta theorem that the estimated top singular vector converges to a Global Evolutionary Direction at rate O(1/sqrt(N*L)) [jiang-etal-2026-global-evolutionary-steering-cross-layer-consistency] Rectifying each layer's CAA steering vector by reinforcing its projection onto this shared direction improves over CAA and four baselines across three models and five domains (p<0.05) [jiang-etal-2026-global-evolutionary-steering-cross-layer-consistency] On Qwen2.5-7B, AdvBench refusal rises from 0.647 to 0.775 and GSM8K accuracy from 0.570 to 0.734 without per-layer tuning [jiang-etal-2026-global-evolutionary-steering-cross-layer-consistency]

models: Gemma-2-9B-it · method: Difference-in-means direction, Spectral Consensus (Global Evolutionary) Steering
Beyond Steering Vector: Flow-based Activation Steering for Inference-Time Intervention (2026)measured

Effective concept steering needs curved trajectories, not one linear direction

Details

Jin et al. note prior activation steering assumes a single fixed-direction additive shift, and replace it with FLAS, a concept-conditioned velocity field integrated over frozen model activations [jin-etal-2026-flow-based-activation-steering-inference-time-intervention] The learned steering trajectories are curved and token-varying: every concept leaves the origin in a shared direction, bends sharply, then enters a concept-specific region [jin-etal-2026-flow-based-activation-steering-inference-time-intervention] This two-phase nonlinear shape directly challenges the single-linear-direction assumption underlying standard steering [jin-etal-2026-flow-based-activation-steering-inference-time-intervention] FLAS outperforms prompting and prior steering (held-out HMean 1.015 vs 0.762/0.608 on Gemma-2-2B-IT) at under 1/26th the parameters and transfers zero-shot to Qwen3-4B-Instruct [jin-etal-2026-flow-based-activation-steering-inference-time-intervention]

models: Gemma-2-2B-it, Gemma-2-9B-it · method: Flow-based Activation Steering (FLAS)
Jumping Ahead: Improving Reconstruction Fidelity with JumpReLU Sparse Autoencoders (2024)measured

JumpReLU SAEs optimize L0 directly and match or beat Gated and TopK

Details

Rajamanoharan et al. train JumpReLU SAEs with a per-feature learnable threshold on Gemma-2-9B residual, attention-output, and MLP-output sites at layers 9/20/31 [rajamanoharan-etal-2024-jumprelu] Because the step gate and L0 penalty are piecewise-constant in the threshold, they train via a kernel-density straight-through estimator proven to estimate the true expected-loss gradient [rajamanoharan-etal-2024-jumprelu] On delta-LM-loss versus L0, JumpReLU SAEs consistently beat Gated and match or slightly beat TopK at matched sparsity [rajamanoharan-etal-2024-jumprelu] Manual and automated interpretability ratings are similar across the three, so the fidelity gain costs no interpretability [rajamanoharan-etal-2024-jumprelu] The gain is attributable to directly optimizing L0 rather than an L1 proxy, since a weight-tied Gated encoder is already equivalent in form to JumpReLU [rajamanoharan-etal-2024-jumprelu]

models: Gemma-2-9B · method: JumpReLU Sparse Autoencoders
Linear Mechanisms for Spatiotemporal Reasoning in Vision Language Models (2026)measured

A contrastive spatial-ID direction is causally bound into object tokens across 11 real VLMs

Details

Kang, Chen, Gkioxari & Perona (2026) define a canonical spatial-ID direction as a difference-of-means over grid position across 11 real VLMs (LLaVA, LLaMA, Qwen, InternVL, Gemma families), well-approximated by a fixed linear map of positional encoding (rank-3 fit, R^2 ~0.85) [kang-etal-2026-linear-mechanisms-for-spatiotemporal-reasoning-in-vlms] Activation-patching 'mirror swap' localizes the effect to object-word tokens at intermediate layers, with a color-swap control showing near-null effect [kang-etal-2026-linear-mechanisms-for-spatiotemporal-reasoning-in-vlms] Directly substituting a target spatial ID into an object token's residual stream shifts ground-truth logit-based belief 64.4% of the way to the target, versus 29.5% for a noise baseline; an analogous temporal-ID binding mechanism extends to video VLMs (LLaVA-Video, VideoLLaMA3, Qwen2.5) [kang-etal-2026-linear-mechanisms-for-spatiotemporal-reasoning-in-vlms]

models: Gemma-7B-it · method: Difference-in-means direction, Activation patching
Delta-Crosscoder: Robust Crosscoder Model Diffing in Narrow Fine-Tuning Regimes (2026)measured

A delta-loss crosscoder isolates 1-2 causal directions per fine-tuning organism

Details

Kassem et al. train a BatchTopK crosscoder with a delta-loss prioritizing directions that change between a base and fine-tuned model, ranked by Relative Decoder Norm [kassem-etal-2026-delta-crosscoder-robust-model-diffing] Across 10 fine-tuning model organisms in Llama-3.1-8B-Instruct, Gemma-2-9B-IT, and Qwen2.5-7B, it isolates exactly 1-2 causal directions per organism [kassem-etal-2026-delta-crosscoder-robust-model-diffing] It recovers the causal direction in 10/10 cases with 0% false positives, versus 40% and 60% false-positive rates for DSF and BatchTopK baselines [kassem-etal-2026-delta-crosscoder-robust-model-diffing] Adding or subtracting the isolated latent's decoder vector at inference causally mitigates the fine-tuned behavior [kassem-etal-2026-delta-crosscoder-robust-model-diffing]

models: Gemma-2-9B-it · method: Sparse crosscoders, Causal interventions (steering)
Hierarchical Concept Geometry in Language Models Emerges from Word Co-occurrence (2026)measured

Word-cooccurrence eigenvectors form a spectral tree recurring in Gemma

Details

The successive leading eigenvectors of a normalized word co-occurrence (PMI-related) Gram matrix progressively separate taxonomic branches from broad to fine, a coarse-to-fine spectral splitting tree [nava-wyart-2026-hierarchical-concept-geometry-in-language-models-emerges-from-word-cooccurrence] Top-k eigenspace alignment far exceeds a shuffled baseline and fits an exponential decay f(d)=1.967*e^(-1.235*d) in WordNet distance d [nava-wyart-2026-hierarchical-concept-geometry-in-language-models-emerges-from-word-cooccurrence] The same spectral-tree geometry, first validated in word2vec, recurs in Gemma-2B unembeddings and in the mid-layer residual stream of Gemma-2B [nava-wyart-2026-hierarchical-concept-geometry-in-language-models-emerges-from-word-cooccurrence] Parent-to-child innovation vectors are near-orthogonal to the parent (a linear-representation signature); the geometry is derived as a prediction of a co-occurrence model, not tested causally [nava-wyart-2026-hierarchical-concept-geometry-in-language-models-emerges-from-word-cooccurrence]

models: Gemma-2B · method: Spectral analysis of the co-occurrence/PMI matrix, Direction Extraction, Geometric analysis
Shared Global and Local Geometry of Language Model Embeddings (2025)measured

Token-embedding orientation is shared within families but drops across them

Details

Lee et al. measure token-embedding geometry across the GPT-2, Llama-3, and Gemma-2 model families [lee-etal-2025-shared-global-and-local-geometry-of-language-model-embeddings] Relative orientation/cosine structure is near-identical within a model family but drops sharply across families trained on different data [lee-etal-2025-shared-global-and-local-geometry-of-language-model-embeddings] A k-NN-neighborhood PCA intrinsic-dimension estimator finds low-ID tokens form semantically coherent clusters while high-ID tokens do not [lee-etal-2025-shared-global-and-local-geometry-of-language-model-embeddings] EMB2EMB, a linear least-squares (OLS) map fit over 100k shared tokens, transfers CAA-style steering vectors (refusal, sycophancy, corrigibility) between differently-sized models [lee-etal-2025-shared-global-and-local-geometry-of-language-model-embeddings]

models: Gemma-2-9B-it · method: PCA, Intrinsic dimension estimation (TwoNN), Cross-model direction transfer via ridge regression
Just-in-Time and Distributed Task Representations in Language Models (2025)measured

Task vectors are locally realized at certain tokens despite decodable identity

Details

Li et al. train linear decoders on real Gemma-3 (4B/12B/27B) and Qwen3 (4B/8B/14B) activations, finding task identity reliably decodable throughout the context [li-etal-2025-just-in-time-distributed-task-representations] The transferable, patchable task-vector representation instead comes online only at certain tokens, a two-fold locality [li-etal-2025-just-in-time-distributed-task-representations] PCA overlap between identifiable and transferable subspaces is task-dependent: 40-60% of the identifiable dimension projects onto the top-20 PCs for simple tasks versus 10-25% for list operations [li-etal-2025-just-in-time-distributed-task-representations] Patching the extracted task vector into zero-shot prompts recontextualizes them, with recovered accuracy tracking few-shot accuracy as k-shot increases [li-etal-2025-just-in-time-distributed-task-representations]

models: Gemma 3 4B · method: Function vector extraction (causal-mediation head selection + sum), PCA
What Really Controls Temporal Reasoning in LLMs: Tokenisation or Representation of Time? (2026)measured

Temporal-probe decodability predicts reasoning in high-resource languages

Details

Bhatia et al. introduce MultiTempBench and evaluate 20 LLMs, combining a tokenization-fragmentation metric with linear-regression probes decoding Year/Month/Day (probe R^2 as temporal linearity) [bhatia-etal-2026-what-really-controls-temporal-reasoning-in-llms] A crossed mixed-effects regression over 285,000 predictions finds temporal linearity is the strongest predictor of reasoning accuracy in high-resource languages (English r=0.77) [bhatia-etal-2026-what-really-controls-temporal-reasoning-in-llms] Tokenization fragmentation dominates in low-resource languages (Hausa r=-0.97); the finding is purely correlational [bhatia-etal-2026-what-really-controls-temporal-reasoning-in-llms]

models: Gemma 3 4B Instruct · method: Linear probing
Do Language Models Track Entities Across State Changes? (2026)measured

LLMs consolidate entity-state tracking into a local binary subspace

Details

Tang et al. train linear probes on last-token residual activations (the "boxes" dataset) to test global versus local entity-state tracking [tang-etal-2026-do-language-models-track-entities-across-state-changes] Global 8-way state probes reach only ~0.3 accuracy versus ~0.9 for local binary probes, so the model consolidates state locally rather than tracking a full global state [tang-etal-2026-do-language-models-track-entities-across-state-changes] INLP null-space projection of the state-tag direction causally changes completions [tang-etal-2026-do-language-models-track-entities-across-state-changes] A learned sparse boolean mask over the residual PCA basis ("subspace patching") reveals a shared positional-information subspace at layers 15-25 in CodeLlama-13B [tang-etal-2026-do-language-models-track-entities-across-state-changes]

models: Gemma-2-2B · method: Linear probing, Causal interventions (steering), PCA
Relational Linearity is a Predictor of Hallucinations (2026)measured

LRE affine-map linearity scores correlate with hallucination rate across four real instruction-tuned LLMs

Details

Lu, Liu, Gerstner, Hirlimann, Rohweder & Schütze (2026) fit relation-specific affine maps o = W_r s + b_r via ridge regression (the Linear Relational Embeddings framework) across Gemma-7B-IT, Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, and Qwen2.5-7B-Instruct [lu-etal-2026-relational-linearity-is-a-predictor-of-hallucinations] The resulting linearity score correlates with hallucination-vs-refusal rate on unknown entities at Pearson r=0.741-0.816 across 15 natural LRE relations, and r=0.573-0.812 on a new SyntHal synthetic-unknown-entity benchmark [lu-etal-2026-relational-linearity-is-a-predictor-of-hallucinations] Purely correlational -- the authors explicitly flag that establishing causality would require representation patching or steering, not yet performed [lu-etal-2026-relational-linearity-is-a-predictor-of-hallucinations]

models: Gemma-7B-it · method: Linear Relational Embedding (LRE)
Universal Response and Emergence of Induction in LLMs (2024)measured

A weak-perturbation response direction stays near-orthogonal across every layer

Details

Luick applies a weak scaling perturbation to a single token's residual activation and tracks the downstream response across all layers of Gemma-2-2B, Llama-3.2-3B-Instruct, and GPT-2-XL [luick-2024-universal-response-emergence-of-induction] The response-direction-to-state cosine similarity stays below 0.1 in magnitude across the entire residual stream [luick-2024-universal-response-emergence-of-induction] This near-orthogonality co-occurs with a scale-invariant response regime [luick-2024-universal-response-emergence-of-induction] Both properties are tied to the induction mechanism, strongest for perturbations at token positions that induction heads copy from [luick-2024-universal-response-emergence-of-induction]

models: Gemma-2-2B · method: Geometric analysis, Causal interventions (steering)
Shared Latent Structures Enable Unified Backdoor Detection and Mitigation in LLMs (2026)measured

Five shared SAE directions mitigate structurally distinct backdoors

Details

Mahmoud et al. train SAEs on six real backdoored LLMs and use model diffing to find five SAE features shared across six distinct backdoor types [mahmoud-etal-2026-shared-latent-structures-enable-unified-backdoor-detection-and-mitigation-in-llms] Steering along each feature's decoder direction mitigates attack success: F33881 gives 96% on Sentiment-LoRA, 79% on country-conditioned advice, and 44% zero-shot on a held-out Refusal backdoor [mahmoud-etal-2026-shared-latent-structures-enable-unified-backdoor-detection-and-mitigation-in-llms] F25828 gives 100% zero-shot mitigation on the held-out Refusal backdoor [mahmoud-etal-2026-shared-latent-structures-enable-unified-backdoor-detection-and-mitigation-in-llms] A companion training-time method, Concept Ablation Fine-Tuning, ablates the same shared subspace during training [mahmoud-etal-2026-shared-latent-structures-enable-unified-backdoor-detection-and-mitigation-in-llms]

models: Gemma 3 4B Instruct, Gemma 3 12B Instruct · method: Sparse Autoencoders (SAE), Activation Steering (Addition)
Learning Multi-Level Features with Matryoshka Sparse Autoencoders (2025)measured

Matryoshka nested dictionaries reduce SAE feature absorption

Details

Bussmann et al. address feature absorption, where a general latent develops systematic blind spots ceded to specialized latents, by training multiple nested prefixes of one shared dictionary to each reconstruct the input independently [bussmann-etal-2025-matryoshka-saes] Because the smallest prefix must explain the input alone, without access to the specialized latents, it keeps general concepts intact rather than ceding them [bussmann-etal-2025-matryoshka-saes] On Gemma-2-2B and a from-scratch TinyStories transformer, Matryoshka SAEs significantly beat benchmark architectures on targeted-concept-erasure (SCR) and disentanglement (TPP) and match BatchTopK on sparse probing [bussmann-etal-2025-matryoshka-saes] The gain costs roughly 50% more training time and slightly lower reconstruction, with a tunable dial between nesting granularity and fidelity [bussmann-etal-2025-matryoshka-saes]

models: Gemma-2-2B · method: Matryoshka Sparse Autoencoders
Overcoming Sparsity Artifacts in Crosscoders to Interpret Chat-Tuning (2025)measured

Latent Scaling corrects crosscoder sparsity artifacts and isolates chat latents

Details

Minder et al. show the standard crosscoder's L1-on-decoder-norm loss can misattribute a latent's direction as chat-unique when it genuinely exists in the base model too [minder-etal-2025-crosscoders-chat-tuning-latent-scaling] Latent Scaling, a post-hoc diagnostic, re-measures each latent's true presence in each model rather than trusting the trained decoder norm [minder-etal-2025-crosscoders-chat-tuning-latent-scaling] Replacing the L1 crosscoder with a BatchTopK one substantially mitigates the artifact on Gemma-2-2B base/chat [minder-etal-2025-crosscoders-chat-tuning-latent-scaling] The corrected crosscoder isolates genuinely chat-specific, causally-steerable latents (e.g. "false information," multiple distinct refusal-trigger latents) rather than one monolithic refusal feature [minder-etal-2025-crosscoders-chat-tuning-latent-scaling]

models: Gemma-2-2B, Gemma-2-2B-it · method: Sparse crosscoders, Causal interventions (steering)
Narrow Finetuning Leaves Clearly Readable Traces in Activation Differences (2026)measured

A per-token mean-difference direction fingerprints narrow fine-tuning

Details

Minder et al. extract a diff-in-means direction between base and narrowly-fine-tuned activations across 33 fine-tuning organisms spanning 7 models from 1B to 32B [minder-etal-2026-narrow-finetuning-traces-in-activation-differences] Steering by adding this direction during generation produces text with high embedding-similarity to the actual fine-tuning corpus [minder-etal-2026-narrow-finetuning-traces-in-activation-differences] An LLM interpretability agent (GPT-5) given the direction correctly identifies the fine-tuning objective in 91% of organisms (30/33) [minder-etal-2026-narrow-finetuning-traces-in-activation-differences] That is more than twice as well at broad-objective identification and over 30x better at fine-grained detail than the best black-box baseline [minder-etal-2026-narrow-finetuning-traces-in-activation-differences]

models: Gemma 3 1B IT · method: Difference-in-means direction, Logit Lens, Causal interventions (steering)
Understanding Emergent Misalignment via Feature Superposition Geometry (2026)measured

Insecure-code SAE directions sit closer to toxic directions than secure-code ones

Details

Minegishi et al. train SAEs on five models and identify insecure-code, secure-code, and toxic-persona decoder directions [minegishi-etal-2026-understanding-emergent-misalignment-via-feature-superposition-geometry] Across all models and layers, the insecure-code direction is consistently more cosine-similar to the toxic-persona direction than the secure-code direction is [minegishi-etal-2026-understanding-emergent-misalignment-via-feature-superposition-geometry] So insecure-code finetuning data is geometrically closer to misaligned-persona representations than matched secure-code data [minegishi-etal-2026-understanding-emergent-misalignment-via-feature-superposition-geometry] Filtering training data by high insecure-code-direction activation cuts emergent-misalignment behaviors from 87 to 57, beating random removal (84) and an LLM-judge filter (59) [minegishi-etal-2026-understanding-emergent-misalignment-via-feature-superposition-geometry]

models: Gemma-2-2B-it, Gemma-2-9B-it, Gemma 2 27B Instruct · method: Sparse Autoencoders (SAE)
SAIF: A Sparse Autoencoder Framework for Interpreting and Steering Instruction Following of Language Models (2025)measured

Multi-feature SAE steering vectors force instruction-following

Details

He et al. identify instruction-relevant SAE latents via a sensitivity score, then build multi-feature steering vectors from Gemma Scope / Llama Scope decoder directions [he-etal-2025-saif-a-sparse-autoencoder-framework-for-interpreting-and-steering-instruction-following-of-language-models] Adding them to the residual stream forces instruction-following (translation/summarization/keyword) at over 30% strict and up to ~0.7 loose accuracy versus near-zero for single-latent steering, optimal at k=15 latents [he-etal-2025-saif-a-sparse-autoencoder-framework-for-interpreting-and-steering-instruction-following-of-language-models] Last-layer placement is critical (Gemma-2-2b-it loose accuracy 0.64 at layer 25 drops to 0.33 by layer 24), and post-instruction positioning beats pre-instruction [he-etal-2025-saif-a-sparse-autoencoder-framework-for-interpreting-and-steering-instruction-following-of-language-models]

models: Gemma-2-2B-it, Gemma-2-9B-it · method: Sparse Autoencoders (SAE), Activation Steering (Addition)
Steerable but Not Decodable: Function Vectors Operate Beyond the Logit Lens (2026)measured

Function vectors steer even where logit lens and probes cannot decode

Details

Nadaf extracts function vectors across 12 tasks and 8 prompt templates in Llama-3.1-8B, Gemma-2-9B, and Mistral-7B-v0.3 (base and instruct) [nadaf-2026-steerable-but-not-decodable] In a substantial fraction of cases (gaps up to -0.91), the function vector causally steers toward the correct answer even though the logit lens cannot decode it at any layer [nadaf-2026-steerable-but-not-decodable] Even a nonlinear probe with a selectivity control fails to decode 5/10 of the hardest cases, so the causal direction exceeds what any tested readout detects [nadaf-2026-steerable-but-not-decodable] Cross-template cosine similarity of the directions weakly correlates (r in [-0.20, 0.13]) with transfer success [nadaf-2026-steerable-but-not-decodable]

models: Gemma-2-9B, Gemma-2-9B-it · method: Function vector extraction (causal-mediation head selection + sum), Activation patching, Logit Lens
Playing Devil's Advocate: Off-the-Shelf Persona Vectors Rival Targeted Steering for Sycophancy (2026)measured

Off-the-shelf persona vectors are near-orthogonal to sycophancy yet reduce it

Details

Kelkar et al. measure the geometry between generic off-the-shelf persona/role steering vectors and a targeted CAA sycophancy direction in Gemma-2-27B-Instruct and Qwen3-32B [kelkar-etal-2026-devils-advocate-persona-vectors-sycophancy] All role vectors are nearly orthogonal to the CAA direction (|cos|<0.17), with the cosine sign flipping between the two models [kelkar-etal-2026-devils-advocate-persona-vectors-sycophancy] Injecting critical/skeptic-persona vectors reduces sycophancy to 68-98% of CAA's own effect without ever using sycophancy labels [kelkar-etal-2026-devils-advocate-persona-vectors-sycophancy] Steering toward agreeable personas does not produce a mirror-symmetric increase in sycophancy, an asymmetric direction-behavior relationship [kelkar-etal-2026-devils-advocate-persona-vectors-sycophancy]

models: Gemma 2 27B Instruct · method: Geometric analysis, Causal interventions (steering)
A Retrieval-Conditioned Rebinding Circuit for Dynamic Entity Tracking in Large Language Models (2026)measured

A mean-difference binding-ID direction in Q/K space redirects attention

Details

Oh & Demberg treat binding IDs as approximately linear directions in the query and key activation spaces of selected attention heads, estimated as mean differences between binding-ID conditions [oh-demberg-2026-retrieval-conditioned-rebinding-circuit] Adding these directions to Gemma2-9B-it, Gemma3-12B-it, Llama-3.2-3B-it, and Llama-3.1-8B-it's Q/K activations causally redirects attention to the corresponding object [oh-demberg-2026-retrieval-conditioned-rebinding-circuit] The intervention shifts the final logits accordingly [oh-demberg-2026-retrieval-conditioned-rebinding-circuit] The representational signature, whether the effect concentrates in Q and K jointly or in K alone, differs across model families [oh-demberg-2026-retrieval-conditioned-rebinding-circuit]

models: Gemma 2 9B IT, Gemma 3 12B IT · method: Difference-in-means direction, Activation patching, Causal interventions (steering)
The Information Geometry of Softmax: Probing and Steering (2026)measured

Dual steering in information-geometric coordinates is provably KL-optimal

Details

Park et al. argue softmax distributions have a Bregman information geometry where KL divergence equals a Bregman divergence of the log-normalizer [park-etal-2026-information-geometry-softmax-probing-steering] Dual steering adds the probe in the dual (mean) parameter space rather than the logit space of ordinary CAA-style steering [park-etal-2026-information-geometry-softmax-probing-steering] Theorem 3.1 proves dual steering is the exact KL-minimizing intervention reaching a target concept logit-score, with a minimal-collateral-change guarantee under concept-factorizability [park-etal-2026-information-geometry-softmax-probing-steering] On Gemma-3-4B and MetaCLIP-2, dual steering outperforms Euclidean steering across all three robustness metrics [park-etal-2026-information-geometry-softmax-probing-steering]

models: Gemma 3 4B · method: Linear probing, Dual (Information-Geometric) Steering
Refusal Direction is Universal Across Safety-Aligned Languages (2025)measured

Refusal directions are approximately parallel across 14 languages and transfer

Details

Wang et al. extract diff-in-means refusal directions across 14 languages in Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, and Gemma-2-9B-it using the PolyRefuse dataset [wang-etal-2025-refusal-direction-is-universal-across-safety-aligned-languages] PCA shows the per-language refusal directions are approximately parallel, with high cross-lingual cosine similarity peaking around layer 12 in Llama [wang-etal-2025-refusal-direction-is-universal-across-safety-aligned-languages] Ablating a refusal vector extracted from English (or German/Thai) causes near-total loss of refusal across all 14 languages and all three models, while adding it induces refusal [wang-etal-2025-refusal-direction-is-universal-across-safety-aligned-languages] Yoruba, a language the models were never safety-aligned in, transfers weakly, so the parallelism tracks safety-alignment coverage rather than language identity [wang-etal-2025-refusal-direction-is-universal-across-safety-aligned-languages]

models: Gemma-2-9B-it · method: Difference-in-means direction, Causal interventions (steering), PCA
Refusal in Language Models Is Mediated by a Single Direction (2024), Representation Engineering: A Top-Down Approach to AI Transparency (2023), Programming Refusal with Conditional Activation Steering (2024), Refusal Direction is Universal Across Safety-Aligned Languages (2025)measured

Refusal is mediated by a single direction across models

Details

Arditi et al. show across 13 chat models (5 families, 1.8B-72B) that a single diff-in-means direction between harmful and harmless instructions both erases refusal when ablated and induces it when added, with no offset term [arditi-etal-2024] Zou et al. independently confirm via a LAT harmfulness vector in Vicuna-13B that stays a >90% classifier under jailbreaks and adversarial suffixes [arditi-etal-2024] Lee et al. replicate across nine further chat models and add conditional steering that gates the refusal vector on a condition vector for selective refusal [arditi-etal-2024] Yung et al. average per-model refusal vectors into a universal refusal direction transferring to jailbreak detection ~10% better than single-model baselines [arditi-etal-2024] The account is contested by Marshall et al. (an affine offset is needed) and Wollschlager et al. (a cone of directions), and Yin et al. localize attack-suppressed versus robust heads [arditi-etal-2024]

models: Gemma-7B-it, Gemma-2-9B-it · method: Difference-in-means direction, Causal interventions (steering), Activation Steering (Addition), Linear Artificial Tomography (LAT), PCA, Conditional (gated) activation steering
Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2 (2024)measured

Gemma Scope releases 2,000+ JumpReLU SAEs; feature splitting persists with width

Details

Lieberum et al. train and release over 2,000 JumpReLU SAEs across every layer and three sites of Gemma 2 (2B/9B/27B plus instruct 9B), totaling 30M+ features [lieberum-etal-2024-gemma-scope] Sweeping dictionary width from 2^14 to 2^20, feature splitting persists across the whole range rather than saturating [lieberum-etal-2024-gemma-scope] A cluster of ultra-high-frequency latents appears in their JumpReLU SAEs and in TopK SAEs but is reported absent from Gated SAEs [lieberum-etal-2024-gemma-scope] Reconstruction fidelity is comparable across sites, but downstream loss damage is consistently higher for residual-stream SAEs, whose small errors compound across later layers [lieberum-etal-2024-gemma-scope]

models: Gemma-2-2B, Gemma-2-9B, Gemma-2-27B, Gemma-2-9B-it · method: JumpReLU Sparse Autoencoders
Unifying Attention Heads and Task Vectors via Hidden State Geometry in In-Context Learning (2025)measured

ICL separability rises first, then alignment with the unembedding

Details

Yang et al. bound classification accuracy by hidden-state linear separability, requiring both output alignment and directional alignment with the label unembedding-difference vector [yang-etal-2025] Across 7 models on 6 datasets, ICL proceeds in two stages: separability rises rapidly in early layers, then middle-to-late layers spike four alignment measures together [yang-etal-2025] The ICL-versus-zero-shot separability gap is small despite an ~80-point accuracy gap, so ICL's gains come almost entirely from improved alignment, not separability [yang-etal-2025] Ablating Previous-Token Heads collapses separability while sparing alignment, and ablating Induction Heads collapses alignment and accuracy (to 2.5%), giving a geometric account of function vectors [yang-etal-2025]

models: Gemma-2B, Gemma-7B · method: Linear probing, PCA, Causal interventions (steering), Function vector extraction (causal-mediation head selection + sum), Separability/alignment decomposition
The Cylindrical Representation Hypothesis for Language Model Steering (2026)measured

Steering directions split into a magnitude axis and interference plane

Details

Gao et al. relax the Linear Representation Hypothesis orthogonality assumption, decomposing a concept steering vector into a central axis and an orthogonal 2D normal plane spanned by the axis complement and the top PC of other concepts directions [gao-etal-2026-cylindrical-hypothesis] Despite cylindrical/phase/sector terminology there is no measured angular or periodic structure (phase = position in the 2D plane; sectors = a binary high/low-sensitivity split), so it is a linear-subspace interference decomposition, not a topology [gao-etal-2026-cylindrical-hypothesis] Steering-effect magnitude follows a predictable sin^m*cos^n form, but which interference sector a concept pair falls into is NOT predictable from the vectors (Pearson -0.034), a genuine null [gao-etal-2026-cylindrical-hypothesis] A penalty experiment attenuating the plane component trades earlier target onset against earlier corrupted output, validated by an LLM-judge at 94% human agreement [gao-etal-2026-cylindrical-hypothesis]

models: Gemma-2-2B-it · method: Difference-in-means direction, PCA, Linear probing, Activation Steering (Addition)
Steered LLM Activations Are Non-Surjective (2026)measured

Steering vectors cannot reach most of the activation manifold

Details

Mishra et al. show that adding a fixed direction to residual-stream activations only reaches a thin, structured slice of the activations that occur naturally across contexts [mishra-etal-2026-steered-llm-activations-are-non-surjective] Across Llama-3.2-1B-Instruct, Qwen2.5-0.5B-Instruct, and Gemma-3-1B-it (plus three validation models), the reachable set is measurably non-surjective onto the empirical activation manifold [mishra-etal-2026-steered-llm-activations-are-non-surjective] Many naturally-occurring target-behavior activations lie off the steering line entirely, and increasing the coefficient pushes activations off-manifold rather than toward the unreached regions [mishra-etal-2026-steered-llm-activations-are-non-surjective] This bounds what single-direction steering can achieve and implies off-line behaviors need multi-directional or nonlinear interventions [mishra-etal-2026-steered-llm-activations-are-non-surjective]

models: Gemma-3-1B-it, Gemma-3-270M-it · method: Activation Steering (Addition)
ReCoVeR the Target Language: Language Steering Without Sacrificing Task Performance (2025)measured

A diff-in-means language-identity direction steers output language without accuracy loss

Details

Sterz et al. extract a language-identity direction as the mean-difference between target-language and other-language activations in Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, and Gemma-2-2B-Instruct [sterz-etal-2025-recover-target-language] Adding this direction, or a trained steering module, to the residual stream raises the rate of responses in the correct target language across 18 languages [sterz-etal-2025-recover-target-language] Unlike a prior unsupervised language-vector steering baseline, it preserves downstream task accuracy such as MMLU [sterz-etal-2025-recover-target-language] No further internal geometric characterization (PCA/subspace structure) of the direction is reported [sterz-etal-2025-recover-target-language]

models: Gemma-2-2B-it · method: Difference-in-means direction, Causal interventions (steering)
Actionable Activation Directions for Detecting and Mitigating Emergent Misalignment Across Language Model Families (2026)measured

A diff-in-means misalignment direction transfers cross-architecture but non-specifically

Details

Syed QLoRA-finetunes four models identically on insecure code and extracts a diff-in-means misalignment direction, finding 99.6% within-model separability versus 50% for a secure-code control [syed-2026-actionable-activation-directions-emergent-misalignment] A ridge-regression map fit between any two models' activation spaces transfers the direction across all twelve model-family pairs with above-chance accuracy (Gemma-to-Llama 87%, Gemma-to-Ministral 90%) [syed-2026-actionable-activation-directions-emergent-misalignment] Steering a target model with the transferred direction suppresses code-related spillover (delta 13-46 points) but fails a specificity control, unlike the fully specific within-model effect (delta 21-51 points) [syed-2026-actionable-activation-directions-emergent-misalignment] This reveals an asymmetric donor/receiver topology, with Gemma and Qwen acting as donors and Llama as a receiver [syed-2026-actionable-activation-directions-emergent-misalignment]

models: Gemma-2-2B-it · method: Difference-in-means direction, Cross-model direction transfer via ridge regression
The Geometry of Forgetting: Temporal Knowledge Drift as an Independent Axis in LLM Representations (2026)measured

Temporal knowledge drift is a linear direction orthogonal to correctness and uncertainty

Details

Elbadry et al. train L1-regularized probes on six instruction-tuned LLMs to detect temporal drift (whether a fact changed since training cutoff), reaching AUROC 0.83-0.95 versus 0.49-0.57 for output and correctness/uncertainty baselines [elbadry-etal-2026-geometry-of-forgetting-temporal-knowledge-drift-as-an-independent-axis-in-llm-representations] Five measures establish the drift direction is geometrically orthogonal to correctness and uncertainty probe directions (weight cosine <=0.136; INLP removal of 10 directions changes AUROC <=0.013) [elbadry-etal-2026-geometry-of-forgetting-temporal-knowledge-drift-as-an-independent-axis-in-llm-representations] Untrained difference-of-means directions overlap with correctness/uncertainty, yet the trained regularized drift probe is near-orthogonal, so the orthogonality is a genuine concept property not a training artifact [elbadry-etal-2026-geometry-of-forgetting-temporal-knowledge-drift-as-an-independent-axis-in-llm-representations] A diff-in-means steering direction is silent under ablation but under amplification produces structured logit redistribution favoring the current over the stale fact holder (selectivity -2.46 to -10.55 logits) [elbadry-etal-2026-geometry-of-forgetting-temporal-knowledge-drift-as-an-independent-axis-in-llm-representations] A cross-cutoff entity-matched control (0.975-0.998 across seven model pairs) confirms the probe reads model-internal knowledge state, not an input property [elbadry-etal-2026-geometry-of-forgetting-temporal-knowledge-drift-as-an-independent-axis-in-llm-representations]

models: Gemma-2-9B-it, Gemma-2-2B-it · method: Linear probing, Difference-in-means direction, Causal interventions (steering), INLP subspace + AlterRep counterfactual push
Large Language Models Encode Semantics and Alignment in Linearly Separable Representations (2025)measured

Topic subspaces sharpen with depth; a centroid direction induces CoT

Details

Saglam et al. study 11 autoregressive models on six arXiv topics, measuring per-layer hard-margin SVM separability and PCA intrinsic dimensionality [saglam-etal-2025-linear-subspaces] Under 10% of principal components explain nearly all variance, so topics occupy a near-affine low-dimensional subspace; separability rises toward final layers, reaching 100% SVM accuracy on all topic pairs in the largest models [saglam-etal-2025-linear-subspaces] A chain-of-thought framing distinction becomes linearly separable even more sharply than topic identity [saglam-etal-2025-linear-subspaces] Adding the CoT-vs-non-CoT centroid-difference direction to the hidden state reliably induces CoT-style responses (flagged preliminary), and a lightweight MLP guardrail built on the finding halves harmful responses [saglam-etal-2025-linear-subspaces]

models: Gemma-2-9B, Gemma-2-2B · method: Linear probing, PCA, Activation Steering (Addition)
Valence-Arousal Subspace in LLMs: Circular Emotion Geometry and Multi-Behavioral Control (2026), Where Do Models Find Happiness? Emotion Vectors in Open-Source LLMs (2026)measured

A valence-arousal circumplex subspace causally steers emotion and refusal

Details

Sun et al. recover two near-orthogonal valence/arousal axes inside Llama-3.1-8B-Instruct (replicated in Qwen3-8B/14B) by PCA-projecting contrastive emotion-steering vectors and ridge-regressing against human ratings [sun-etal-2026-valence-arousal] Projecting the emotion vectors onto this plane traces a circle: a circularity statistic reaches 2.76-4.08 with fitted radii ~0.37-0.39, analogous to Russell's circumplex [sun-etal-2026-valence-arousal] The valence axis recovers self-reported valence at r=0.97 and the NRC-VAD lexicon at r=0.71, with cross-model valence agreement r=0.95 [sun-etal-2026-valence-arousal] Adding valence/arousal directions at specific circle angles produces dose-dependent, angle-specific shifts in generated-text affect (e.g. 0deg: delta-valence +0.75; 180deg: -0.73) [sun-etal-2026-valence-arousal] The same arousal axis causally controls refusal (20%->86% on OKTest) and sycophancy, with random-direction controls within 2-3 points of baseline [sun-etal-2026-valence-arousal] Logit-clamping and top-neuron ablation along the direction crash refusal while preserving MATH-500/IFEval, and an independent refusal direction is near-orthogonal (86.5deg) to the VA plane [sun-etal-2026-valence-arousal] Van der Ben et al. independently replicate the valence/arousal PCA structure (PC1-valence r=0.72-0.83, PC2-arousal r=0.21-0.45) in Apertus-8B and Gemma-4-E4B-it without computing circularity or steering [vanderben-etal-2026-emotion-vectors-open-source-llms] They add that cross-architecture layer-depth trajectories diverge sharply (a 3-phase plateau in Apertus vs a smooth gradient in Gemma via linear CKA) [vanderben-etal-2026-emotion-vectors-open-source-llms]

models: Gemma 4 E4B-it · method: PCA, Linear probing, Activation Steering (Addition)
The Anatomy of a Truth Direction: Knowledge-Dependent Dimensionality, a Relational Law, and a Convergent Category Geometry in Small Language Models (2026)measured

A truth direction's dimensionality, layer-wise construction, and cross-family category geometry

Details

Vicidomini uses SVD-based linear decomposition of real hidden states in small Llama, Qwen, and Gemma-2-2B models to show truth-signal dimensionality concentrates onto a single axis for well-known facts and diffuses across more dimensions as knowledge decreases [vicidomini-2026-anatomy-of-a-truth-direction] Causal component-ablation attributes the truth direction's layer-wise construction to attention propagating the truth frame forward while the FFN opposes the current block's frame [vicidomini-2026-anatomy-of-a-truth-direction] Per-category truth axes converge into a semantically-signed arrangement across model families once a sign instability is corrected with a spectral consensus gauge [vicidomini-2026-anatomy-of-a-truth-direction]

models: Gemma-2-2B · method:
Sparse-Autoencoder-Guided Internal Representation Unlearning for Large Language Models (2025)measured

SAE-latent misdirection fine-tuning unlearns entities better than residual-stream baselines

Details

Yamashita et al. fine-tune Llama-3.1-8B-Instruct and Gemma-2-9B-Instruct with a hinge loss over SAE-latent pre-activations, pushing known-entity latent coordinates below -c and unknown-entity coordinates above +c [yamashita-etal-2025-sparse-autoencoder-guided-internal-representation-unlearning-for-large-language-models] On the RWKU benchmark, Llama's forget score drops 81.1% to 46.8% (versus 65.5-79.2% for gradient-ascent/NPO/RMU baselines) with retain score preserved [yamashita-etal-2025-sparse-autoencoder-guided-internal-representation-unlearning-for-large-language-models] Gemma-2-9B shows a similar pattern (80.1% to 57.1% forget versus 71.6-79.2% baselines) [yamashita-etal-2025-sparse-autoencoder-guided-internal-representation-unlearning-for-large-language-models] Recognition-latent activation-frequency plots show known-latents suppressed and unknown-latents boosted post-intervention, unlike the gradient-ascent baseline [yamashita-etal-2025-sparse-autoencoder-guided-internal-representation-unlearning-for-large-language-models]

models: Gemma-2-9B-it · method: Sparse Autoencoders (SAE), Representation misdirection (fine-tuning toward a target direction)

Minegishi, Feng, Furuta et al. (2026) Analogical-Reasoning Toy Transformer

Emergent Analogical Reasoning in Transformers (2026)measured

A 1-layer transformer solves analogies via emergent vector arithmetic

Details

Minegishi et al. (ICML 2026 Spotlight) train a 1-layer, 1-head causal transformer (d=128) from scratch on a synthetic entity-relation analogy task [minegishi-etal-2026-emergent-analogical-reasoning] Analogical reasoning emerges once entity embeddings across categories become geometrically aligned, quantified by a measured decrease in Dirichlet energy during training [minegishi-etal-2026-emergent-analogical-reasoning] A Parallelism cosine-similarity metric between (e_t - e_s) and a learned functor direction f rises, indicating the model solves analogies via vector arithmetic e_t ~ e_s + f [minegishi-etal-2026-emergent-analogical-reasoning] The same Dirichlet-energy decrease appears layer-by-layer in a pretrained Llama via in-context learning, though the exact checkpoint is unconfirmed [minegishi-etal-2026-emergent-analogical-reasoning] The paper's nearest intervention (Appendix O) perturbs the data-level entity mapping rather than internal activations, so the claim is geometric, not causal [minegishi-etal-2026-emergent-analogical-reasoning]

models: 1-layer, 1-head causal Transformer (d_model=128, synthetic entity-relation analogy task) · method: Vector-offset analogy test, Geometric analysis

Bielik

Graded Entity-Familiarity Readouts in Language Models: Polish Adaptation, Cross-Language Robustness, and Refusal Steering (2026)measured

An entity-familiarity direction reads and steers refusal in Polish LLMs

Details

Brzezinka reads final-prompt-token residual-stream activations across 14 models (Bielik, PLLuM, Gemma-4, Qwen3 families) on a 1,440-entity Polish dataset [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] A linear familiarity probe separates real from fabricated entities at AUROC 0.86-0.93, beating a lexical baseline (0.786) [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] In Polish-adapted Bielik/PLLuM models the score additionally tracks entity popularity (Spearman rho up to 0.567), with matched before/after controls isolating Polish continual-pretraining as the driver [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] Cross-language transfer retains 96-101% of within-language AUROC [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] In Gemma-4-12B (the only natively refusing model), adding a one-dimensional familiarity direction moves refusal rate monotonically (0.24 to 1.00 on known entities; 0.73 to 0.00 on unknown) [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering]

models: Bielik-11B · method: Linear probing, Activation Steering (Addition)

PLLuM

Graded Entity-Familiarity Readouts in Language Models: Polish Adaptation, Cross-Language Robustness, and Refusal Steering (2026)measured

An entity-familiarity direction reads and steers refusal in Polish LLMs

Details

Brzezinka reads final-prompt-token residual-stream activations across 14 models (Bielik, PLLuM, Gemma-4, Qwen3 families) on a 1,440-entity Polish dataset [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] A linear familiarity probe separates real from fabricated entities at AUROC 0.86-0.93, beating a lexical baseline (0.786) [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] In Polish-adapted Bielik/PLLuM models the score additionally tracks entity popularity (Spearman rho up to 0.567), with matched before/after controls isolating Polish continual-pretraining as the driver [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] Cross-language transfer retains 96-101% of within-language AUROC [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] In Gemma-4-12B (the only natively refusing model), adding a one-dimensional familiarity direction moves refusal rate monotonically (0.24 to 1.00 on known entities; 0.73 to 0.00 on unknown) [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering]

models: Llama-PLLuM-12B · method: Linear probing, Activation Steering (Addition)

Mistral

Graded Entity-Familiarity Readouts in Language Models: Polish Adaptation, Cross-Language Robustness, and Refusal Steering (2026)measured

An entity-familiarity direction reads and steers refusal in Polish LLMs

Details

Brzezinka reads final-prompt-token residual-stream activations across 14 models (Bielik, PLLuM, Gemma-4, Qwen3 families) on a 1,440-entity Polish dataset [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] A linear familiarity probe separates real from fabricated entities at AUROC 0.86-0.93, beating a lexical baseline (0.786) [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] In Polish-adapted Bielik/PLLuM models the score additionally tracks entity popularity (Spearman rho up to 0.567), with matched before/after controls isolating Polish continual-pretraining as the driver [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] Cross-language transfer retains 96-101% of within-language AUROC [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] In Gemma-4-12B (the only natively refusing model), adding a one-dimensional familiarity direction moves refusal rate monotonically (0.24 to 1.00 on known entities; 0.73 to 0.00 on unknown) [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering]

models: Mistral NeMo 12B (base) · method: Linear probing, Activation Steering (Addition)
On the Mutual Influence of Gender and Occupation in LLM Representations (2025)measured

A PCA gender direction carries first-name discriminative signal

Details

An et al. approximate a female-male gender direction via PCA over gendered word pairs, evaluated on 470 first names' contextualized embeddings [an-etal-2025-on-the-mutual-influence-of-gender-and-occupation-in-llm-representations] The models are Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, OLMo-7B-0724-hf, and Phi-3.5-mini-instruct [an-etal-2025-on-the-mutual-influence-of-gender-and-occupation-in-llm-representations] The 1-D gender-direction projection achieves 75.18% binary gender-classification accuracy versus 75.46% for the full embedding (Llama-3.1-8B) [an-etal-2025-on-the-mutual-influence-of-gender-and-occupation-in-llm-representations] First-name gender representations shift with occupational context, e.g. "nurse" pulls embeddings toward femininity [an-etal-2025-on-the-mutual-influence-of-gender-and-occupation-in-llm-representations] The internal gender-direction coefficient correlates with downstream occupation-prediction bias (Spearman rho=0.61 Llama-3.1-8B, 0.76 Mistral-7B; both p<0.001) [an-etal-2025-on-the-mutual-influence-of-gender-and-occupation-in-llm-representations]

models: Mistral-7B-Instruct-v0.3 · method: Direction Extraction
Latent Structure of Affective Representations in Large Language Models (2026)measured

Emotion representations trace a V-shaped open manifold with neutral at the vertex

Details

Choi & Weber build a classification-accuracy dissimilarity matrix from pairwise probes across GoEmotions categories at every layer of Gemma-2-9B and Mistral-7B, embedding it via MDS/Isomap/UMAP and Procrustes-aligning to human valence-arousal norms [choi-weber-2026-affective-structure] The recovered layout is a V-shaped parabolic open curve (an open 1-manifold, H1=0) with neutral at the vertex and positive/negative valence fanning out, rather than the classical circular circumplex [choi-weber-2026-affective-structure] Isomap's geodesic-to-Euclidean distance ratio (1.00-1.80 Gemma, 0.80-1.42 Mistral) confirms the curvature is real but modest [choi-weber-2026-affective-structure] Adding probe-direction vectors to Llama-3-70B-Instruct shifts human-rated output valence strongly (mean 7.29 vs 1.08 on a 10-point scale for positive versus negative targets) [choi-weber-2026-affective-structure]

models: Mistral-7B · method: Linear probing, Isomap, UMAP, Activation Steering (Addition)
Linear Representations of Political Perspective Emerge in Large Language Models (2025)measured

Attention-head probes read and steer political ideology

Details

Kim, Evans & Schein fit a ridge-regression probe per attention head in three open chat LLMs to predict 552 U.S. lawmakers' DW-NOMINATE ideology scores [kim-evans-schein-2025-political-perspective] The best single head reaches Spearman rho 0.846-0.861, and an ensemble of the top 32 heads reaches 0.870-0.885, concentrated in middle layers [kim-evans-schein-2025-political-perspective] A nonlinear MLP probe matches the linear probe, supporting a linear-direction characterization [kim-evans-schein-2025-political-perspective] Probes fit on lawmaker ideology transfer zero-shot to predicting 400 news outlets' slant (rho 0.720-0.798) [kim-evans-schein-2025-political-perspective] Adding scaled top-head directions to activations shifts GPT-4o-rated political slant, correlating up to 0.607 with steering magnitude [kim-evans-schein-2025-political-perspective]

models: Mistral-7B-Instruct-v0.1 · method: Linear probing, Activation Steering (Addition)
Head Pursuit: Probing Attention Specialization in Multimodal Transformers (2025)measured

SOMP-selected attention heads causally control target behavior across real unimodal and multimodal transformers

Details

Basile, Maiorca, Doimo, Locatello & Cazzaniga score real Mistral-7B attention heads by Simultaneous Orthogonal Matching Pursuit (SOMP) against unembedding-matrix directions, finding that inverting just 8 heads (0.8% of the total) degrades TriviaQA country-name F1 far more selectively than inverting the same number of random heads or Logit-Lens-selected heads [basile-etal-2025-head-pursuit-probing-attention-specialization-multimodal-transformers] On RealToxicityPrompts and TET toxicity-mitigation benchmarks, suppressing 8/16/32 SOMP-selected heads reduces normalized toxic-generation counts to 0.83/0.67/0.66 (RTP) and 0.83/0.68/0.49 (TET), consistently below Logit-Lens and random-head baselines [basile-etal-2025-head-pursuit-probing-attention-specialization-multimodal-transformers] Applying the same SOMP-based head scoring to real LLaVA-NeXT-7B/13B, Gemma3-12B, and Qwen2.5-VL-7B, inverting the top-32 SOMP heads significantly disrupts image classification accuracy on MNIST, SVHN, GTSRB, EuroSAT, and RESISC45 while 32 random heads have minimal effect, and Jaccard overlap shows related-domain datasets share specialized heads [basile-etal-2025-head-pursuit-probing-attention-specialization-multimodal-transformers] On Flickr30k captioning with LLaVA, inhibiting 16 SOMP-selected heads (alpha=-1) nearly removes attribute keywords (colors, sentiments, quantities) while CIDEr stays above 80% of baseline, and enhancing 32 heads (alpha=5) increases target-concept presence by over 60% in all three attribute categories [basile-etal-2025-head-pursuit-probing-attention-specialization-multimodal-transformers]

models: Mistral-7B · method: Simultaneous Orthogonal Matching Pursuit against unembedding directions
Do Personality Traits Interfere? Geometric Limitations of Steering in Large Language Models (2026)measured

Orthogonalized personality directions still bleed behaviorally

Details

Bhandari, Naseem & Nasim extract five Big-Five personality steering directions in LLaMA-3-8B-Instruct and Ministral-8B-Instruct and find substantial pairwise cosine overlap [bhandari-etal-2026-personality-traits-interfere-steering] A battery of exact-orthogonalization schemes drives pairwise cosine similarity to near machine precision (max|cos|<1e-8 under hard orthonormalization) [bhandari-etal-2026-personality-traits-interfere-steering] Even then, steering one trait at a time measurably shifts other geometrically orthogonal traits' scores [bhandari-etal-2026-personality-traits-interfere-steering] Geometric orthogonality is therefore not sufficient for behavioral independence, a competing claim against papers finding natural or restorable independence [bhandari-etal-2026-personality-traits-interfere-steering]

models: Ministral-8B-Instruct-2410 · method: Difference-in-means direction, Steering-direction orthogonalization battery
Hallucination Basins: A Dynamic Framework for Understanding and Controlling LLM Hallucinations (2026)measured

A diff-of-means hallucination direction causally raises hallucination dose-dependently

Details

Cherukuri & Varshney formalize hallucinations as basin attractors across Llama-3.2-1B/3B, Gemma-2-2B, Qwen2.5-1.5B, Llama-3.1-8B, and Mistral-7B-v0.3 [cherukuri-varshney-2026-hallucination-basins-a-dynamic-framework-for-understanding-and-controlling-llm-hallucinations] Basin separation is strongly task-dependent: factoid tasks show sharp centroid separation (variance ratio up to 4.55, AUROC up to 1.000 on MuSiQue) while summarization and TruthfulQA hover near chance [cherukuri-varshney-2026-hallucination-basins-a-dynamic-framework-for-understanding-and-controlling-llm-hallucinations] Interpolating factual hidden states toward the hallucination centroid via a diff-of-means steering vector produces a monotonic dose-response increase in hallucination probability, exceeding random and orthogonal controls [cherukuri-varshney-2026-hallucination-basins-a-dynamic-framework-for-understanding-and-controlling-llm-hallucinations]

models: Mistral-7B-v0.3 · method: Difference-in-means direction, Activation Steering (Addition)
Concepts Whisper While Syntax Shouts: Spectral Anti-Concentration and the Dual Geometry of Transformer Representations (2026)measured

Concept directions anti-concentrate in the unembedding's low-variance spectral tail

Details

Acharya, Rimal & Dhakal project concept diff-of-means vectors onto the unembedding-covariance eigenbasis across 17 models in 5 families [acharya-rimal-dhakal-2026-spectral-anti-concentration] Concept-direction energy anti-concentrates in the low-eigenvalue tail: Spectral Center of Mass averages 0.926 versus 0.758 for random directions (Gini-deviation -0.282) [acharya-rimal-dhakal-2026-spectral-anti-concentration] Static unembedding-row contrasts show the opposite sign, and POS/syntax directions concentrate in the high-variance subspace in 6 of 8 architectures (p<0.013), a genuine dual-geometry split [acharya-rimal-dhakal-2026-spectral-anti-concentration] On Llama-3.1-8B, injecting only the top-10%-eigenvalue component gives +383% perplexity versus +223% for the bottom-10% component (Cohen's d=1.80), replicated across 5 models [acharya-rimal-dhakal-2026-spectral-anti-concentration]

models: Mistral-7B-v0.3 · method: Geometric analysis, Causal interventions (steering)
The Confidence Manifold: Geometric Structure of Correctness Representations in Language Models (2026)measured

Answer-correctness sits in a 3-8D linear subspace, causally steerable

Details

Cho et al. study self-assessed correctness on TruthfulQA pairs across 9 models spanning 5 families, finding the discriminative signal is a low-dimensional linear subspace, not a curved manifold [cho-etal-2026-confidence-manifold] A PLS sweep peaks at 3-8 dimensions (e.g. peak AUC 0.90 at layer 23, dim 5, Mistral-7B), and nonlinear classifiers give no gain over a linear boundary [cho-etal-2026-confidence-manifold] Correct/incorrect classes form roughly Gaussian clusters separated by a mean shift: a two-mean centroid detector matches the linear probe (0.90 vs 0.89 AUC), robust with 25 examples/class [cho-etal-2026-confidence-manifold] This 3-8D discriminative subspace is lower-dimensional than the representation's 8-12D intrinsic dimension at the same layers [cho-etal-2026-confidence-manifold] Adding the learned direction shifts downstream error rate up to 10.9 points dose-dependently, while random and orthogonal controls have no reliable effect [cho-etal-2026-confidence-manifold]

models: Mistral-7B-Instruct-v0.3 · method: Linear probing, Partial Least Squares (PLS), Difference-in-means direction, Activation Steering (Addition)
Language Models Represent and Transform Concepts with Shared Geometry (2026)measured

Contextual displacement forms a vector field tracking concreteness

Details

Hu, Niu & Varma define a per-word contextual displacement vector phi(w,tau) = r(w,tau) - r(w,tau_0) in activation space, forming a vector field over the vocabulary rather than one shared direction [hu-etal-2026-concept-transformation-geometry] Across 4 models x 3 depths, word density correlates negatively with displacement magnitude (Spearman rho -0.32 to -0.22) [hu-etal-2026-concept-transformation-geometry] Lexical concreteness correlates negatively with directional deviation (rho -0.41 to -0.21): concrete, low-density concepts move less and more consistently under context [hu-etal-2026-concept-transformation-geometry] The field has no dominant direction (leading PC <15% variance) and is more dispersed than uniform-random; no causal intervention is performed [hu-etal-2026-concept-transformation-geometry]

models: Mistral Small 3 (2501), Ministral-3-3B-Base-2512, Ministral-3-8B-Base-2512, Ministral-3-14B-Base-2512 · method: PCA, Geometric analysis
Uncertainty as Feature Gaps: Epistemic Uncertainty Quantification of LLMs (2025)measured

Contrastive-PCA feature directions predict LLM epistemic uncertainty

Details

Bakman et al. derive an epistemic-uncertainty bound in terms of hidden-state displacement along semantic-feature directions [bakman-etal-2025-uncertainty-as-feature-gaps-epistemic-uncertainty-quantification-of-llms] They extract three directions (context-reliance, context-comprehension, honesty) via contrastive prompt-pair PCA on Llama-3.1-8B, Mistral-7B-v0.3, and Qwen2.5-7B [bakman-etal-2025-uncertainty-as-feature-gaps-epistemic-uncertainty-quantification-of-llms] Per-token projections onto these directions improve the Prediction Rejection Ratio by up to 13 points over baselines [bakman-etal-2025-uncertainty-as-feature-gaps-epistemic-uncertainty-quantification-of-llms] Replacing PCA extraction with a plain mean-difference direction substantially degrades performance, evidencing the specific extracted direction does the work [bakman-etal-2025-uncertainty-as-feature-gaps-epistemic-uncertainty-quantification-of-llms]

models: Mistral-7B-v0.3 · method: Direction Extraction, Difference-in-means direction
How Does Alignment Tuning Shape Representations of Sycophancy and Related Cue-Induced Biases in LLMs? (2026)measured

Cue-induced bias directions are installed by alignment tuning

Details

Gupta et al. extract a per-bias diff-of-means direction for seven cue-triggered bias types across base and instruct checkpoints of five model families [gupta-etal-2026-alignment-tuning-sycophancy-directions] Four of five base models flip on under 5% as many pairs as their instruct counterparts, with base activations carrying essentially no cue-specific signal, so susceptibility is installed by alignment tuning [gupta-etal-2026-alignment-tuning-sycophancy-directions] Within instruct models each bias direction transfers to held-out data at mean AUROC 0.69-0.82, localizing to late-middle layers [gupta-etal-2026-alignment-tuning-sycophancy-directions] Biases stay representationally distinct, with even behaviorally similar biases occupying different, sometimes anti-aligned directions [gupta-etal-2026-alignment-tuning-sycophancy-directions] Subtracting the direction while preserving over 90% of correct answers recovers 7-20% of bias-induced errors, versus under 5% for a random direction [gupta-etal-2026-alignment-tuning-sycophancy-directions]

models: Mistral-7B-v0.3, Mistral-7B-Instruct-v0.3 · method: Difference-in-means direction, Activation Steering (Addition)
Emergent Causal-Geometric Dynamics Across Depth in Large Language Models (2026)measured

A steering direction's angular component becomes causally load-bearing only in a late prediction-centric depth phase

Details

Haim & McNamee (2026) measure Participation Ratio and layer-wise Spearman correlation between representational distance (Euclidean vs. angular) and next-token-distribution KL divergence across real Llama-3.1-8B, Mistral-7B-v0.3, and Qwen2.5-7B, finding a consistent bi-phasic depth structure -- an early context-centric phase and a late prediction-centric phase, with phase-transition layer agreeing within 2.5% of depth across three tasks and three models [haim-mcnamee-2026-emergent-causal-geometric-dynamics-across-depth-in-large-language-models] Steering vectors decomposed into pure-angular vs. pure-norm interventions are causally effective (logit-preference shift) only when angular and only in the late phase; norm interventions are ineffective throughout [haim-mcnamee-2026-emergent-causal-geometric-dynamics-across-depth-in-large-language-models]

models: Mistral-7B-v0.3 · method:
Categorical Perception in Large Language Model Hidden States: Structural Warping at Digit-Count Boundaries (2026)measured

LLM hidden states warp categorically at digit-count boundaries

Details

- In six LLMs, hidden-state distances warp categorically at digit-count boundaries (9->10, 99->100) versus matched non-boundary controls (15, 150); an RSA "CP-Additive" model (log-distance plus a boundary boost) beats a continuous model at 100% of primary layers. [cacioli-2026-categorical-perception-digit-boundaries] - Boundary crossing explains 5-27% of representational-distance variance beyond magnitude; the decade-100 effect is 3.9-12.7x the decade-10 effect, with a manifold rotation of 81.6-89.6 degrees at the boundary. [cacioli-2026-categorical-perception-digit-boundaries] - Patching along a ridge-regression "category direction" shifts discrimination confidence 70.1x more than random directions (Layer 5), dose-dependent and specific. [cacioli-2026-categorical-perception-digit-boundaries] - Five instruct models (Llama-3-8B-Instruct, Mistral-7B-Instruct-v0.3, Gemma-2-9B-IT, Qwen2.5-7B-Instruct, Phi-3.5-mini-instruct) plus a Llama-3-8B base control; causal patching run on Llama-3-8B-Instruct only. [cacioli-2026-categorical-perception-digit-boundaries]

models: Mistral-7B-Instruct-v0.3 · method: Representational Similarity Analysis (RSA), Linear probing, Causal interventions (steering)
The Geometry of Categorical and Hierarchical Concepts in Large Language Models (2024), When Language Representations Interact: Separability and Cross-Lingual Effects in LLMs (2026)measured

Semantic hierarchy is encoded as orthogonality between concept levels

Details

Park et al. prove (Theorem 8) that sibling-contrast directions at one hierarchy level are orthogonal to the parent's contrast direction and to any ancestor level [park-etal-2024] Combined with per-level categorical polytopes, this gives a direct-sum-of-polytopes structure where each level occupies its own orthogonal subspace, confirmed in Gemma-2B and Llama-3-8B (cosine near zero versus a shuffled-labels control) [park-etal-2024] The authors argue ordinary high-dimensional Euclidean space suffices, explicitly contrasting with hyperbolic-embedding accounts of hierarchy [park-etal-2024] Marinov et al. confirm the same cross-level orthogonality for language family versus individual language across Qwen3-4B, Mistral-7B-v0.3, and Llama-3-8B via a causal inner product [marinov-etal-2026]

models: Mistral-7B-v0.3 · method: Geometric analysis, Causal inner product
The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning (2024)measured

RMU corrupts a hazardous-topic direction by fine-tuning while preserving capability

Details

Li, Pan et al. (WMDP) introduce RMU, a two-term activation-space loss pushing hazardous-topic activations toward a fixed random unit vector while anchoring benign activations to the frozen model [li-pan-etal-2024-the-wmdp-benchmark-measuring-and-reducing-malicious-use-with-unlearning] On Zephyr-7B-beta, WMDP-Bio drops 63.7 to 31.2 and WMDP-Cyber 44.0 to 28.2, while MMLU (58.1 to 57.1) and MT-Bench (7.33 to 7.10) stay near baseline [li-pan-etal-2024-the-wmdp-benchmark-measuring-and-reducing-malicious-use-with-unlearning] Yi-34B-Chat and Mixtral-8x7B-Instruct show the same pattern (WMDP-Bio 75.3 to 30.7 and 74.8 to 34.0, MMLU preserved within ~2 points) [li-pan-etal-2024-the-wmdp-benchmark-measuring-and-reducing-malicious-use-with-unlearning] The direction is imposed by fine-tuning toward an arbitrary target rather than found in the base model, unlike additive inference-time steering [li-pan-etal-2024-the-wmdp-benchmark-measuring-and-reducing-malicious-use-with-unlearning]

models: Mixtral-8x7B-Instruct-v0.1 · method: Representation misdirection (fine-tuning toward a target direction)
Relational Linearity is a Predictor of Hallucinations (2026)measured

LRE affine-map linearity scores correlate with hallucination rate across four real instruction-tuned LLMs

Details

Lu, Liu, Gerstner, Hirlimann, Rohweder & Schütze (2026) fit relation-specific affine maps o = W_r s + b_r via ridge regression (the Linear Relational Embeddings framework) across Gemma-7B-IT, Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, and Qwen2.5-7B-Instruct [lu-etal-2026-relational-linearity-is-a-predictor-of-hallucinations] The resulting linearity score correlates with hallucination-vs-refusal rate on unknown entities at Pearson r=0.741-0.816 across 15 natural LRE relations, and r=0.573-0.812 on a new SyntHal synthetic-unknown-entity benchmark [lu-etal-2026-relational-linearity-is-a-predictor-of-hallucinations] Purely correlational -- the authors explicitly flag that establishing causality would require representation patching or steering, not yet performed [lu-etal-2026-relational-linearity-is-a-predictor-of-hallucinations]

models: Mistral-7B-Instruct-v0.3 · method: Linear Relational Embedding (LRE)
Steerable but Not Decodable: Function Vectors Operate Beyond the Logit Lens (2026)measured

Function vectors steer even where logit lens and probes cannot decode

Details

Nadaf extracts function vectors across 12 tasks and 8 prompt templates in Llama-3.1-8B, Gemma-2-9B, and Mistral-7B-v0.3 (base and instruct) [nadaf-2026-steerable-but-not-decodable] In a substantial fraction of cases (gaps up to -0.91), the function vector causally steers toward the correct answer even though the logit lens cannot decode it at any layer [nadaf-2026-steerable-but-not-decodable] Even a nonlinear probe with a selectivity control fails to decode 5/10 of the hardest cases, so the causal direction exceeds what any tested readout detects [nadaf-2026-steerable-but-not-decodable] Cross-template cosine similarity of the directions weakly correlates (r in [-0.20, 0.13]) with transfer success [nadaf-2026-steerable-but-not-decodable]

models: Mistral-7B-v0.3, Mistral-7B-Instruct-v0.3 · method: Function vector extraction (causal-mediation head selection + sum), Activation patching, Logit Lens
How Language Models Process Negation (2026)measured

Negation is a single linear direction decodable by layer 4

Details

Zhou et al. extract a single "not" direction from Llama-3.1-8B residual-stream states via PCA-for-reduction followed by linear discriminant analysis [zhou-etal-2026-how-language-models-process-negation] Positive and negative hidden states are approximately linearly separable by this one direction [zhou-etal-2026-how-language-models-process-negation] 10-fold cross-validated per-layer decoding accuracy reaches near-perfect by layer 4 [zhou-etal-2026-how-language-models-process-negation] The same linear-direction analysis is confirmed on Mistral-7B-v0.1 as a secondary model [zhou-etal-2026-how-language-models-process-negation]

models: Mistral-7B-v0.1 · method: PCA, Linear Discriminant Analysis (LDA), Causal interventions (steering)
Probing the Geometry of Truth: Consistency and Generalization of Truth Directions in LLMs Across Logical Transformations and Question Answering Tasks (2025)measured

Not all LLMs have consistent truth directions; atomic-statement probes generalize

Details

Bao et al. investigate whether truth directions are consistent across LLMs and how well truth probes generalize [bao-etal-2025-probing-the-geometry-of-truth] Not all LLMs exhibit consistent truth directions, with stronger and more consistent representations in more capable models, particularly under logical negation [bao-etal-2025-probing-the-geometry-of-truth] Probes trained on declarative atomic statements generalize to logical transformations, question-answering, in-context learning, and external-knowledge settings [bao-etal-2025-probing-the-geometry-of-truth] The paper demonstrates a practical application to selective question-answering [bao-etal-2025-probing-the-geometry-of-truth]

models: Mistral-7B-v0.1 · method: Linear probing, Direction Extraction
The Geometry of Forgetting: Temporal Knowledge Drift as an Independent Axis in LLM Representations (2026)measured

Temporal knowledge drift is a linear direction orthogonal to correctness and uncertainty

Details

Elbadry et al. train L1-regularized probes on six instruction-tuned LLMs to detect temporal drift (whether a fact changed since training cutoff), reaching AUROC 0.83-0.95 versus 0.49-0.57 for output and correctness/uncertainty baselines [elbadry-etal-2026-geometry-of-forgetting-temporal-knowledge-drift-as-an-independent-axis-in-llm-representations] Five measures establish the drift direction is geometrically orthogonal to correctness and uncertainty probe directions (weight cosine <=0.136; INLP removal of 10 directions changes AUROC <=0.013) [elbadry-etal-2026-geometry-of-forgetting-temporal-knowledge-drift-as-an-independent-axis-in-llm-representations] Untrained difference-of-means directions overlap with correctness/uncertainty, yet the trained regularized drift probe is near-orthogonal, so the orthogonality is a genuine concept property not a training artifact [elbadry-etal-2026-geometry-of-forgetting-temporal-knowledge-drift-as-an-independent-axis-in-llm-representations] A diff-in-means steering direction is silent under ablation but under amplification produces structured logit redistribution favoring the current over the stale fact holder (selectivity -2.46 to -10.55 logits) [elbadry-etal-2026-geometry-of-forgetting-temporal-knowledge-drift-as-an-independent-axis-in-llm-representations] A cross-cutoff entity-matched control (0.975-0.998 across seven model pairs) confirms the probe reads model-internal knowledge state, not an input property [elbadry-etal-2026-geometry-of-forgetting-temporal-knowledge-drift-as-an-independent-axis-in-llm-representations]

models: Mistral-7B-Instruct-v0.1 · method: Linear probing, Difference-in-means direction, Causal interventions (steering), INLP subspace + AlterRep counterfactual push
Large Language Models Encode Semantics and Alignment in Linearly Separable Representations (2025)measured

Topic subspaces sharpen with depth; a centroid direction induces CoT

Details

Saglam et al. study 11 autoregressive models on six arXiv topics, measuring per-layer hard-margin SVM separability and PCA intrinsic dimensionality [saglam-etal-2025-linear-subspaces] Under 10% of principal components explain nearly all variance, so topics occupy a near-affine low-dimensional subspace; separability rises toward final layers, reaching 100% SVM accuracy on all topic pairs in the largest models [saglam-etal-2025-linear-subspaces] A chain-of-thought framing distinction becomes linearly separable even more sharply than topic identity [saglam-etal-2025-linear-subspaces] Adding the CoT-vs-non-CoT centroid-difference direction to the hidden state reliably induces CoT-style responses (flagged preliminary), and a lightweight MLP guardrail built on the finding halves harmful responses [saglam-etal-2025-linear-subspaces]

models: Mistral Small 3 (2501), Mistral-7B · method: Linear probing, PCA, Activation Steering (Addition)
The Geometry of Truth: Emergent Linear Structure in LLM Representations of True/False Datasets (2024), On the Universal Truthfulness Hyperplane Inside LLMs (2024)measured

A linear truth direction generalizes across datasets and is causally effective

Details

Marks & Tegmark curate simple true/false statement datasets and, after activation-patching-localized states, show LLaMA-2-13B/70B separate true from false along a linear PCA axis [marks-tegmark-2024] The axis generalizes across topically and structurally diverse datasets increasingly with scale and is largely distinct from a probable-vs-improbable-text direction [marks-tegmark-2024] Mass-mean probing (a diff-in-means direction with optional covariance whitening) classifies about as well as logistic regression or contrast-consistent search but yields substantially more causally effective steering directions (normalized indirect effects up to ~1.0 at 70B) [marks-tegmark-2024] Liu et al. extend the account cross-domain across 41 datasets, finding training-task diversity drives generalization far more than data volume (accuracy rising as task categories grow from 1 to 14) [marks-tegmark-2024]

models: Mistral-7B-v0.1 · method: Activation patching, PCA, Difference-in-means direction, Linear probing, Contrast-Consistent Search (CCS), Causal interventions (steering)
How Context Shapes Truth: Geometric Transformations of Statement-level Truth Representations in LLMs (2026)measured

The truth direction rotates and rescales under added context and steers labels

Details

Adarsh et al. extract a per-layer mean-difference truth direction in Llama-3.1-8B-Instruct, Mistral-Nemo-12B-Instruct, Qwen3-4B-Instruct, and SmolLM3-3B and measure how it transforms when a statement is embedded in context [adarsh-etal-2026-how-context-shapes-truth] The angle between with- and without-context truth vectors is near-orthogonal in early layers and converges by mid-depth, while the magnitude ratio is quantified per dataset [adarsh-etal-2026-how-context-shapes-truth] This is the first quantified characterization of how the truth direction's own geometry, not just its existence, changes under context [adarsh-etal-2026-how-context-shapes-truth] Mass-mean steering with the extracted vector flips truthfulness labels in ~100% of cases for three models (Qwen3-4B weaker and variable at 11-59%), a strong replicated causal effect [adarsh-etal-2026-how-context-shapes-truth]

models: Mistral NeMo 12B Instruct (2407) · method: Difference-in-means direction, Causal interventions (steering)
Tracing Moral Foundations in Large Language Models (2026)measured

Diff-in-means moral-foundation directions, discovered across 14 real LLMs, are causally steerable and rewired (not newly formed) by post-training

Details

Yu, Yi, Karimi-Malekabadi, Abdurahman, Ye, Narayanan, Zhao & Dehghani extract difference-in-means moral-foundation directions across 14 real pretrained checkpoints (Llama-3.1-8B/70B, Qwen2.5-7B/14B/32B, Qwen3-30B-A3B, Mistral-7B-v0.3, base and instruct), finding significant linear separability in all 35 (model, foundation) pairs (Wasserstein distance 0.16-0.71, AUC greater than 0.55) [yu-yi-etal-2026-tracing-moral-foundations-in-large-language-models] Direction-reversal rate drops sharply with post-training (e.g. Llama-3.1-8B 33 percent to 4 percent), indicating the directions emerge during pretraining and are selectively rewired rather than formed de novo by instruction tuning [yu-yi-etal-2026-tracing-moral-foundations-in-large-language-models] Causal steering with the extracted directions passes a dose-response criterion in 68 of 70 pairs, and SAE decoder directions cosine-aligned with the dense vectors give finer-grained micro-steering that exceeds macro-vector steering in 17 of 20 cells while better preserving general capability [yu-yi-etal-2026-tracing-moral-foundations-in-large-language-models]

models: Mistral-7B-v0.3 · method:
SLIM: Sparse Latent Steering for Interpretable and Property-Directed LLM-Based Molecular Editing (2026)measured

SLIM SAE features linearly correlate with molecular properties and steer editing

Details

Zhang et al. train a Gated SAE on a single layer of four frozen LLM-based molecular editors (DrugAssist, GeLLM3O-LLaMA3, GeLLM3O-Mistral, MolGen) with learnable per-property importance gates [zhang-etal-2026-slim-sparse-latent-steering-molecular] Individual top SAE features correlate strongly with molecular properties (Spearman rho up to +0.93 for molecular weight, -0.85 for QED), with six of eight properties reaching |rho|>=0.52 from a single feature [zhang-etal-2026-slim-sparse-latent-steering-molecular] Properties like HBD and DRD2 show weaker single-feature correlation, explicitly attributed to distributed rather than monosemantic encoding [zhang-etal-2026-slim-sparse-latent-steering-molecular] A gradient-based direction projected through the SAE's top features and added to the frozen model's residual stream causally steers property-directed molecular editing [zhang-etal-2026-slim-sparse-latent-steering-molecular]

models: GeLLM3O-Mistral · method: Gated Sparse Autoencoders, Activation Steering (Addition)
Language Models Represent Beliefs of Self and Others (2024)measured

A mid-layer head group decodes another agent's beliefs and steers theory-of-mind

Details

Zhu et al. prompt Mistral-7B-Instruct (and DeepSeek-LLM-7B-Chat) with third-person belief narratives and fit logistic-regression probes on attention-head activations [zhu-etal-2024-language-models-represent-beliefs-of-self-and-others] A group of middle-layer attention heads decodes another agent's belief status at over 80% accuracy, distinct from the model's own self-belief representation [zhu-etal-2024-language-models-represent-beliefs-of-self-and-others] Steering these representations produces large changes in downstream Theory-of-Mind performance, generalizing across social-reasoning tasks [zhu-etal-2024-language-models-represent-beliefs-of-self-and-others]

models: Mistral-7B-Instruct · method: Linear probing, Activation Steering (Addition)

OpenVLA

What Frozen VLAs Already Know About Success: A Probing Study of Value-Like Structure in Foundation Robot Policies (2026)measured

A value-like success direction is recoverable from frozen VLAs

Details

Zhang et al. linearly probe frozen OpenVLA, Pi0.5, DINOv2, and CLIP features on LIBERO-Goal trajectories to recover Monte-Carlo task-success targets [zhang-etal-2026-what-frozen-vlas-already-know-about-success] A linear, value-like task-success direction is consistently recoverable across all four frozen backbones, none of which was trained to estimate reward [zhang-etal-2026-what-frozen-vlas-already-know-about-success] Deploying the probe as a test-time selector over Pi0.5 action-prefix candidates raises push-plate success from 26.7% (greedy) to 44.3% (p=0.003) [zhang-etal-2026-what-frozen-vlas-already-know-about-success] The direction is thus not merely decodable but behaviorally useful without any additional policy training [zhang-etal-2026-what-frozen-vlas-already-know-about-success]

models: OpenVLA-7B · method: Linear probing
Mechanistic Interpretability for Steering Vision-Language-Action Models (2025)measured

Sparse logit-lens directions in VLA FFN value vectors steer robot actions

Details

Haon et al. project FFN value vectors in OpenVLA-7B and Pi0 onto the vocabulary basis (logit-lens), revealing sparse directions for concepts like "speed", "direction", "up", and "slow" [haon-etal-2025-mechanistic-interpretability-steering-vla] Fewer than 25% of OpenVLA's 352,255 FFN value vectors are rewired for action prediction, the rest inherited semantic directions from the vision-language backbone [haon-etal-2025-mechanistic-interpretability-steering-vla] Upweighting a semantic-direction cluster steers both models zero-shot: "fast" interventions give larger end-effector displacement than "slow" [haon-etal-2025-mechanistic-interpretability-steering-vla] Full-depth "up" cluster injections give the largest mean Y-displacement, validated in LIBERO-Long (OpenVLA) and on a physical UR5 arm (Pi0-FAST) against a random-cluster control [haon-etal-2025-mechanistic-interpretability-steering-vla]

models: OpenVLA-7B · method: Logit Lens, Causal interventions (steering)
Emergent World Representations in OpenVLA (2025)measured

OpenVLA linearly encodes a state-transition vector in middle layers

Details

Molinari et al. test whether OpenVLA, a 7B behavioral-cloning VLA with no world-model objective, encodes environment dynamics as a linear embedding offset delta_e = e_{t+K} - e_t [molinari-etal-2025-openvla-world-model] Lasso linear probes recover this transition vector at statistically significant R^2 across all 4 LIBERO suites and 4 horizons, with 123 probes passing permutation tests at p<0.0001 [molinari-etal-2025-openvla-world-model] Probes on raw activations uniformly exceed probes on embeddings alone, and linear probes never significantly underperform MLP probes (42/48 favor linear) [molinari-etal-2025-openvla-world-model] The structure concentrates in middle layers (~15) and emerges with pretraining scale; an early checkpoint and eval-only finetuning weaken it, with no causal intervention performed [molinari-etal-2025-openvla-world-model]

models: OpenVLA-7B · method: Linear probing
Sparse Autoencoders Reveal Interpretable and Steerable Features in VLA Models (2026)measured

Ablating general SAE feature directions in a VLA collapses grasp success

Details

Swann et al. train sparse autoencoders on the residual-stream activations of pi0.5's PaliGemma backbone and of OpenVLA, identifying feature directions for motion primitives and semantic concepts [swann-etal-2026-sparse-autoencoders-reveal-interpretable-and-steerable-features-in-vla-models] On real-world DROID hardware, ablating the most-general cross-episode-transferable feature directions collapses grasp success, while ablating episode-specific memorized directions barely changes performance [swann-etal-2026-sparse-autoencoders-reveal-interpretable-and-steerable-features-in-vla-models] Additive steering toward specific object-feature directions shifts grasp outcomes substantially more than an FFN-neuron control, and steering a "close gripper" feature raises the gripper-closure rate [swann-etal-2026-sparse-autoencoders-reveal-interpretable-and-steerable-features-in-vla-models] The steering and ablation results are reported qualitatively (without per-trial success fractions), validated on LIBERO simulation and DROID real-world hardware [swann-etal-2026-sparse-autoencoders-reveal-interpretable-and-steerable-features-in-vla-models]

models: OpenVLA-7B · method: Sparse Autoencoders (SAE)

Pi (Physical Intelligence VLA)

What Frozen VLAs Already Know About Success: A Probing Study of Value-Like Structure in Foundation Robot Policies (2026)measured

A value-like success direction is recoverable from frozen VLAs

Details

Zhang et al. linearly probe frozen OpenVLA, Pi0.5, DINOv2, and CLIP features on LIBERO-Goal trajectories to recover Monte-Carlo task-success targets [zhang-etal-2026-what-frozen-vlas-already-know-about-success] A linear, value-like task-success direction is consistently recoverable across all four frozen backbones, none of which was trained to estimate reward [zhang-etal-2026-what-frozen-vlas-already-know-about-success] Deploying the probe as a test-time selector over Pi0.5 action-prefix candidates raises push-plate success from 26.7% (greedy) to 44.3% (p=0.003) [zhang-etal-2026-what-frozen-vlas-already-know-about-success] The direction is thus not merely decodable but behaviorally useful without any additional policy training [zhang-etal-2026-what-frozen-vlas-already-know-about-success]

models: pi0.5 (PaliGemma VLA backbone) · method: Linear probing
Mechanistic Interpretability for Steering Vision-Language-Action Models (2025)measured

Sparse logit-lens directions in VLA FFN value vectors steer robot actions

Details

Haon et al. project FFN value vectors in OpenVLA-7B and Pi0 onto the vocabulary basis (logit-lens), revealing sparse directions for concepts like "speed", "direction", "up", and "slow" [haon-etal-2025-mechanistic-interpretability-steering-vla] Fewer than 25% of OpenVLA's 352,255 FFN value vectors are rewired for action prediction, the rest inherited semantic directions from the vision-language backbone [haon-etal-2025-mechanistic-interpretability-steering-vla] Upweighting a semantic-direction cluster steers both models zero-shot: "fast" interventions give larger end-effector displacement than "slow" [haon-etal-2025-mechanistic-interpretability-steering-vla] Full-depth "up" cluster injections give the largest mean Y-displacement, validated in LIBERO-Long (OpenVLA) and on a physical UR5 arm (Pi0-FAST) against a random-cluster control [haon-etal-2025-mechanistic-interpretability-steering-vla]

models: pi0 (PaliGemma VLA backbone, predecessor of pi0.5) · method: Logit Lens, Causal interventions (steering)
Sparse Autoencoders Reveal Interpretable and Steerable Features in VLA Models (2026)measured

Ablating general SAE feature directions in a VLA collapses grasp success

Details

Swann et al. train sparse autoencoders on the residual-stream activations of pi0.5's PaliGemma backbone and of OpenVLA, identifying feature directions for motion primitives and semantic concepts [swann-etal-2026-sparse-autoencoders-reveal-interpretable-and-steerable-features-in-vla-models] On real-world DROID hardware, ablating the most-general cross-episode-transferable feature directions collapses grasp success, while ablating episode-specific memorized directions barely changes performance [swann-etal-2026-sparse-autoencoders-reveal-interpretable-and-steerable-features-in-vla-models] Additive steering toward specific object-feature directions shifts grasp outcomes substantially more than an FFN-neuron control, and steering a "close gripper" feature raises the gripper-closure rate [swann-etal-2026-sparse-autoencoders-reveal-interpretable-and-steerable-features-in-vla-models] The steering and ablation results are reported qualitatively (without per-trial success fractions), validated on LIBERO simulation and DROID real-world hardware [swann-etal-2026-sparse-autoencoders-reveal-interpretable-and-steerable-features-in-vla-models]

models: pi0.5 (PaliGemma VLA backbone) · method: Sparse Autoencoders (SAE)

DINOv2

What Frozen VLAs Already Know About Success: A Probing Study of Value-Like Structure in Foundation Robot Policies (2026)measured

A value-like success direction is recoverable from frozen VLAs

Details

Zhang et al. linearly probe frozen OpenVLA, Pi0.5, DINOv2, and CLIP features on LIBERO-Goal trajectories to recover Monte-Carlo task-success targets [zhang-etal-2026-what-frozen-vlas-already-know-about-success] A linear, value-like task-success direction is consistently recoverable across all four frozen backbones, none of which was trained to estimate reward [zhang-etal-2026-what-frozen-vlas-already-know-about-success] Deploying the probe as a test-time selector over Pi0.5 action-prefix candidates raises push-plate success from 26.7% (greedy) to 44.3% (p=0.003) [zhang-etal-2026-what-frozen-vlas-already-know-about-success] The direction is thus not merely decodable but behaviorally useful without any additional policy training [zhang-etal-2026-what-frozen-vlas-already-know-about-success]

models: DINOv2 ViT-B/14 · method: Linear probing
Vision Transformers Don't Need Trained Registers (2025)measured

Fewer than 25 register neurons with consistent directions cause ViT outlier tokens

Details

Jiang et al. identify fewer than 10 register neurons in OpenCLIP ViT-B/16 and fewer than 25 in DINOv2-L/14 by ranking neurons on mean activation at outlier-token positions [jiang-etal-2025-vits-dont-need-trained-registers] These neurons' decoder weights consistently peak in specific ambient dimensions across images, a directional claim beyond activation magnitude [jiang-etal-2025-vits-dont-need-trained-registers] Training-free test-time registers built from the discovered directions match trained-register models on object discovery and improve zero-shot segmentation [jiang-etal-2025-vits-dont-need-trained-registers] Register-neuron ablation cuts typographic-attack success on LLaVA-Llama-3-8B from 50.5% to 7.5%, while naive zeroing crashes ImageNet accuracy 71.3% to 55.6% [jiang-etal-2025-vits-dont-need-trained-registers]

models: DINOv2 ViT-L/14 · method: Register-neuron identification, Causal interventions (steering)
Interpretable and Testable Vision Features via Sparse Autoencoders (2025)measured

SAE directions show CLIP encodes abstract concepts, DINOv2 fragmented visual features

Details

Stevens et al. train 24K-width sparse autoencoders on frozen patch-level activations from CLIP ViT-B/16 and DINOv2 ViT-B/14 using ImageNet-1K [stevens-etal-2025-interpretable-and-testable-vision-features-via-saes] CLIP's activation geometry encodes abstract, style-invariant semantic/cultural concepts (country identity, "accident") largely absent or fragmented in DINOv2 [stevens-etal-2025-interpretable-and-testable-vision-features-via-saes] DINOv2's more visually granular geometry instead organizes around low-level visual pattern similarity [stevens-etal-2025-interpretable-and-testable-vision-features-via-saes] The discovered directions are causally responsible for predictions via patch-level activation edits, flipping ADE20K segmentation classes without retraining the ViT or task heads [stevens-etal-2025-interpretable-and-testable-vision-features-via-saes]

models: DINOv2 ViT-B/14 · method: Sparse Autoencoders (SAE)
Compositional Generalization Requires Linear, Orthogonal Representations in Vision Embedding Models (2026)measured

Compositional generalization needs linear, additive, orthogonal per-concept factor vectors

Details

Uselis, Dittadi & Oh recover per-concept difference vectors from paired stimuli across three datasets in many pretrained encoders (CLIP, OpenCLIP, MetaCLIP, MetaCLIP2, SigLIP, SigLIP2, DINOv1/v2/v3) [uselis-dittadi-oh-2026-compositional-generalization-requires-linear-orthogonal-representations-in-vision-embedding-models] A whitened, probe-span-projected reconstruction of embeddings as a linear sum of recovered factors reaches R^2=0.42-0.63 versus 0.12-0.42 for a randomly-initialized-encoder baseline [uselis-dittadi-oh-2026-compositional-generalization-requires-linear-orthogonal-representations-in-vision-embedding-models] Cross-concept factor vectors are confirmed mutually orthogonal via cosine similarity and effective-rank analysis [uselis-dittadi-oh-2026-compositional-generalization-requires-linear-orthogonal-representations-in-vision-embedding-models] The degree of linear factorization correlates with each model's own compositional-generalization accuracy across variants [uselis-dittadi-oh-2026-compositional-generalization-requires-linear-orthogonal-representations-in-vision-embedding-models]

models: DINOv2 ViT-B/14 · method: Additive per-concept factor recovery and orthogonality scoring
How Can Embedding Models Bind Concepts? (2026)measured

Multi-object scene embeddings decompose hierarchically as sums of object embeddings

Details

Uselis, Koishigarina & Oh extend additive factorization to multi-object scenes, finding scene embedding approximately equals the sum of object embeddings, themselves approximately sums of concept embeddings, in CLIP and DINOv2 [uselis-koishigarina-oh-2026-how-can-embedding-models-bind-concepts] Quantified via R^2, retrieval, and probing: text-CLIP R^2=0.90-0.92, PUG:SPARE 0.75-0.86, CLEVR-2D 0.75-0.79, all clearly above random baselines of 0.47-0.53 [uselis-koishigarina-oh-2026-how-can-embedding-models-bind-concepts] Causal embedding-editing (removing or inserting object components) produces meaningful counterfactual embeddings [uselis-koishigarina-oh-2026-how-can-embedding-models-bind-concepts] This extends single-object per-concept factorization to a hierarchical, multi-object additive structure [uselis-koishigarina-oh-2026-how-can-embedding-models-bind-concepts]

models: DINOv2 ViT-B/14 · method: Additive per-concept factor recovery and orthogonality scoring

CLIP (Contrastive Language-Image Pretraining)

What Frozen VLAs Already Know About Success: A Probing Study of Value-Like Structure in Foundation Robot Policies (2026)measured

A value-like success direction is recoverable from frozen VLAs

Details

Zhang et al. linearly probe frozen OpenVLA, Pi0.5, DINOv2, and CLIP features on LIBERO-Goal trajectories to recover Monte-Carlo task-success targets [zhang-etal-2026-what-frozen-vlas-already-know-about-success] A linear, value-like task-success direction is consistently recoverable across all four frozen backbones, none of which was trained to estimate reward [zhang-etal-2026-what-frozen-vlas-already-know-about-success] Deploying the probe as a test-time selector over Pi0.5 action-prefix candidates raises push-plate success from 26.7% (greedy) to 44.3% (p=0.003) [zhang-etal-2026-what-frozen-vlas-already-know-about-success] The direction is thus not merely decodable but behaviorally useful without any additional policy training [zhang-etal-2026-what-frozen-vlas-already-know-about-success]

models: CLIP ViT-B/32 · method: Linear probing
Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning (2022)measured

CLIP's modality gap is a provable, tunable cone effect

Details

- Each modality's embeddings collapse into a narrow cone present even at random initialization; two distinct cones plus low temperature (tau=1/100) leave a persistent image-text modality gap. [liang-etal-2022-modality-gap] - The default gap magnitude ||Delta||=0.82 is the contrastive-loss global minimum, so the gap is a geometric consequence of the cone effect rather than an artifact. [liang-etal-2022-modality-gap] - Causally shifting embeddings along the gap direction changes downstream behavior: widening the gap 0.82 to 0.97 reduces FairFace denigration, with small zero-shot shifts (CIFAR-10 0.9013 to 0.9081). [liang-etal-2022-modality-gap] - Zero-shot accuracy experiments use CLIP ViT-B/16, fairness uses ViT-B/32; the paper also documents the gap in VideoCLIP, ConVIRT and CLASP. [liang-etal-2022-modality-gap]

models: CLIP ViT-B/32, CLIP ViT-B/16 · method: Geometric analysis, Causal interventions (steering)
The Double-Ellipsoid Geometry of CLIP (2025)measured

CLIP embeddings lie on two tilted offset ellipsoids

Details

- Pre-normalization CLIP embeddings for each modality lie on a thin ellipsoidal shell: long-tailed per-feature variance makes it an ellipsoid and off-diagonal covariance tilts it. [levi-gilboa-2025-double-ellipsoid-clip] - The image and text shells are separable and centered well away from the origin (the "double-ellipsoid"); a linear SVM separates the modalities with 100% accuracy using just two features. [levi-gilboa-2025-double-ellipsoid-clip] - The thin-shell approximation is tight (image mu_norm=7.59, 0.18% relative error) and a conformity score tracks generation quality at Pearson 0.9998. [levi-gilboa-2025-double-ellipsoid-clip] - Observational on CLIP ViT-B/32 (primary) and ViT-L/14 (supplementary); interventions are post-hoc embedding shifts. [levi-gilboa-2025-double-ellipsoid-clip]

models: CLIP ViT-B/32, CLIP ViT-L/14 · method: Geometric analysis
Same Concept, Different Directions: Cross-Modal Feature Heterogeneity in Sparse Autoencoders (2026)measured

Same concept, different SAE directions across modalities

Details

- In vision-language models, sparse-autoencoder feature directions for the same concept differ across modalities, keeping substantial angular (cosine-distance) separation even when their activations are highly correlated. [lee-etal-2026-cross-modal-sae-heterogeneity] - The paper proves that a single forced-alignment SAE collapses the two directions to their normalized sum [(phi+psi)/||phi+psi||, 0], sacrificing reconstruction. [lee-etal-2026-cross-modal-sae-heterogeneity] - Modality-specific SAEs plus post-hoc alignment give better cross-modal retrieval and steering (image->text R@1 16.0, text->image R@1 11.4). [lee-etal-2026-cross-modal-sae-heterogeneity] - Directional analysis spans CLIP ViT-B/32, MetaCLIP B/32, OpenCLIP B/32 and SigLIP2. [lee-etal-2026-cross-modal-sae-heterogeneity]

models: CLIP ViT-B/32 · method: TopK (k-sparse) Sparse Autoencoders, Geometric analysis, Causal interventions (steering)
Vision Transformers Don't Need Trained Registers (2025)measured

Fewer than 25 register neurons with consistent directions cause ViT outlier tokens

Details

Jiang et al. identify fewer than 10 register neurons in OpenCLIP ViT-B/16 and fewer than 25 in DINOv2-L/14 by ranking neurons on mean activation at outlier-token positions [jiang-etal-2025-vits-dont-need-trained-registers] These neurons' decoder weights consistently peak in specific ambient dimensions across images, a directional claim beyond activation magnitude [jiang-etal-2025-vits-dont-need-trained-registers] Training-free test-time registers built from the discovered directions match trained-register models on object discovery and improve zero-shot segmentation [jiang-etal-2025-vits-dont-need-trained-registers] Register-neuron ablation cuts typographic-attack success on LLaVA-Llama-3-8B from 50.5% to 7.5%, while naive zeroing crashes ImageNet accuracy 71.3% to 55.6% [jiang-etal-2025-vits-dont-need-trained-registers]

models: OpenCLIP ViT-B/16 (LAION-2B) · method: Register-neuron identification, Causal interventions (steering)
Interpretable and Testable Vision Features via Sparse Autoencoders (2025)measured

SAE directions show CLIP encodes abstract concepts, DINOv2 fragmented visual features

Details

Stevens et al. train 24K-width sparse autoencoders on frozen patch-level activations from CLIP ViT-B/16 and DINOv2 ViT-B/14 using ImageNet-1K [stevens-etal-2025-interpretable-and-testable-vision-features-via-saes] CLIP's activation geometry encodes abstract, style-invariant semantic/cultural concepts (country identity, "accident") largely absent or fragmented in DINOv2 [stevens-etal-2025-interpretable-and-testable-vision-features-via-saes] DINOv2's more visually granular geometry instead organizes around low-level visual pattern similarity [stevens-etal-2025-interpretable-and-testable-vision-features-via-saes] The discovered directions are causally responsible for predictions via patch-level activation edits, flipping ADE20K segmentation classes without retraining the ViT or task heads [stevens-etal-2025-interpretable-and-testable-vision-features-via-saes]

models: CLIP ViT-B/16 · method: Sparse Autoencoders (SAE)
Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE) (2024)measured

Sparse nonnegative concept decomposition enables causal CLIP editing

Details

- CLIP image embeddings decompose as sparse, nonnegative combinations (l0 ~5-20, recommended ~10-30) of a fixed dictionary of ~15k text-encoder concept vectors from LAION captions. [bhalla-etal-2024-splice] - The decomposition nearly preserves zero-shot accuracy (ImageNet 0.552 to 0.552, CIFAR-100 0.750 to 0.768) with cosine reconstruction ~0.75-0.88. [bhalla-etal-2024-splice] - Because embeddings are additive concept mixtures, zeroing a concept edits behavior surgically (removing "glasses" drops a glasses classifier 0.88 to 0.69 while gender stays 0.97 to 0.96). [bhalla-etal-2024-splice] - Tested on OpenCLIP ViT-B/32 (main) and OpenAI CLIP RN50 (appendix). [bhalla-etal-2024-splice]

models: CLIP ResNet-50 · method: Fixed-dictionary sparse recovery, Causal interventions (steering)
Decoding Vision Transformers: The Diffusion Steering Lens (2025)measured

A causally-validated Diffusion Steering Lens tracks real head-ablation effects in a ViT

Details

Takatsuki et al. apply the Diffusion Steering Lens, an interpolate-toward-clean-then-patch causal-readout, to CLIP ViT-bigG/14, extending logit-lens-style diagnostics to a real vision transformer [takatsuki-etal-2025-diffusion-steering-lens] DSL's per-head visualization similarity correlates more strongly with the actual output change from ablating that head than a vanilla Diffusion Lens does [takatsuki-etal-2025-diffusion-steering-lens] DSL's sequential head-ablation trajectory for removing a synthetic overlay nearly overlaps an ACDC-style optimal ablation ordering and consistently beats random ablation [takatsuki-etal-2025-diffusion-steering-lens] The paper measures a readout method's faithfulness to real causal structure rather than characterizing a novel intrinsic representation shape [takatsuki-etal-2025-diffusion-steering-lens]

models: CLIP ViT-bigG/14 (Kandinsky 2.2 image encoder) · method: Diffusion Steering Lens (DSL), Causal interventions (steering)
On the Modality Gap and the Contrastive Loss in Multi-modal Representation Learning (2026)measured

Low-temperature InfoNCE is the modality gap's cause

Details

- With identical-initialization dual encoders that start perfectly overlapped, the InfoNCE loss actively separates the two modalities at low temperature, a mode failure rather than an initialization cone effect. [mager-etal-2026-on-the-modality-gap-and-the-contrastive-loss-in-multi-modal-representation-learning] - Adding intra-modal negatives (xNCE) prevents this separation across temperatures (MNIST modality-centroid distance 0.75 at tau=0.01 under InfoNCE vs ~0.09-0.13 under xNCE). [mager-etal-2026-on-the-modality-gap-and-the-contrastive-loss-in-multi-modal-representation-learning] - The gap is largely reduced rather than proven eliminated; the mechanism is supported by a theoretical analysis of the InfoNCE temperature term. [mager-etal-2026-on-the-modality-gap-and-the-contrastive-loss-in-multi-modal-representation-learning] - Real-model experiments use CLIP ViT-B/16 on MS-COCO, alongside a custom MNIST vision transformer that carries the mechanistic evidence. [mager-etal-2026-on-the-modality-gap-and-the-contrastive-loss-in-multi-modal-representation-learning]

models: CLIP ViT-B/16 · method: Geometric analysis, Causal interventions (steering), Theoretical / Analytical
Compositional Generalization Requires Linear, Orthogonal Representations in Vision Embedding Models (2026)measured

Compositional generalization needs linear, additive, orthogonal per-concept factor vectors

Details

Uselis, Dittadi & Oh recover per-concept difference vectors from paired stimuli across three datasets in many pretrained encoders (CLIP, OpenCLIP, MetaCLIP, MetaCLIP2, SigLIP, SigLIP2, DINOv1/v2/v3) [uselis-dittadi-oh-2026-compositional-generalization-requires-linear-orthogonal-representations-in-vision-embedding-models] A whitened, probe-span-projected reconstruction of embeddings as a linear sum of recovered factors reaches R^2=0.42-0.63 versus 0.12-0.42 for a randomly-initialized-encoder baseline [uselis-dittadi-oh-2026-compositional-generalization-requires-linear-orthogonal-representations-in-vision-embedding-models] Cross-concept factor vectors are confirmed mutually orthogonal via cosine similarity and effective-rank analysis [uselis-dittadi-oh-2026-compositional-generalization-requires-linear-orthogonal-representations-in-vision-embedding-models] The degree of linear factorization correlates with each model's own compositional-generalization accuracy across variants [uselis-dittadi-oh-2026-compositional-generalization-requires-linear-orthogonal-representations-in-vision-embedding-models]

models: CLIP ViT-L/14 · method: Additive per-concept factor recovery and orthogonality scoring
How Can Embedding Models Bind Concepts? (2026)measured

Multi-object scene embeddings decompose hierarchically as sums of object embeddings

Details

Uselis, Koishigarina & Oh extend additive factorization to multi-object scenes, finding scene embedding approximately equals the sum of object embeddings, themselves approximately sums of concept embeddings, in CLIP and DINOv2 [uselis-koishigarina-oh-2026-how-can-embedding-models-bind-concepts] Quantified via R^2, retrieval, and probing: text-CLIP R^2=0.90-0.92, PUG:SPARE 0.75-0.86, CLEVR-2D 0.75-0.79, all clearly above random baselines of 0.47-0.53 [uselis-koishigarina-oh-2026-how-can-embedding-models-bind-concepts] Causal embedding-editing (removing or inserting object components) produces meaningful counterfactual embeddings [uselis-koishigarina-oh-2026-how-can-embedding-models-bind-concepts] This extends single-object per-concept factorization to a hierarchical, multi-object additive structure [uselis-koishigarina-oh-2026-how-can-embedding-models-bind-concepts]

models: CLIP ViT-L/14 · method: Additive per-concept factor recovery and orthogonality scoring
Interpreting the Linear Structure of Vision-Language Model Embedding Spaces (2025)measured

VLM SAE concepts are single-modality yet gap-orthogonal

Details

- BatchTopK sparse autoencoders trained on ~600k COCO vision-language embeddings (dictionary 4096) put 99% of the energy in 512 concepts, the other ~3500 sharing 1%. [papadimitriou-etal-2025-vlm-linear-structure] - The energy-dominant concepts are nearly single-modality in activation, yet many of their directions are nearly orthogonal to the modality-gap subspace (accuracy ~0.5 as modality classifiers). [papadimitriou-etal-2025-vlm-linear-structure] - An "SAE projection effect" reconciles this: sparse thresholding interacts with the per-modality input distributions, so a direction can be gap-orthogonal yet fire for one modality; top-512 concepts are seed-stable (0.92) vs 0.16 for low-energy ones. [papadimitriou-etal-2025-vlm-linear-structure] - Observational, on CLIP, SigLIP, SigLIP2 and AIMv2 (the paper states no model variants). [papadimitriou-etal-2025-vlm-linear-structure]

models: CLIP · method: BatchTopK Sparse Autoencoders, Geometric analysis

SmolLM3

A Circuit for Predicting Hierarchical Structure In-Context in Large Language Models (2025)measured

A latent-context direction drives in-context hierarchy tracking

Details

Saanum et al. train linear probes to decode, from attention-head outputs in Qwen2.5 (0.5B/1.5B/3B), whether current and previous latent-context tokens match [saanum-etal-2025-hierarchical-in-context-circuit] Second-order chunk decodability exceeds 90% for several heads, replicated on Gemma2-2B, Llama3.2-3B, SmolLM3-3B and Qwen3-0.6B [saanum-etal-2025-hierarchical-in-context-circuit] Heads exceeding an 85% decoding score are labeled context-matching heads [saanum-etal-2025-hierarchical-in-context-circuit] Ablating these decoded heads sharply reduces hierarchical-structure prediction accuracy, far more than ablating equally many random heads [saanum-etal-2025-hierarchical-in-context-circuit]

models: SmolLM3-3B · method: Linear probing, Causal interventions (steering)
How Context Shapes Truth: Geometric Transformations of Statement-level Truth Representations in LLMs (2026)measured

The truth direction rotates and rescales under added context and steers labels

Details

Adarsh et al. extract a per-layer mean-difference truth direction in Llama-3.1-8B-Instruct, Mistral-Nemo-12B-Instruct, Qwen3-4B-Instruct, and SmolLM3-3B and measure how it transforms when a statement is embedded in context [adarsh-etal-2026-how-context-shapes-truth] The angle between with- and without-context truth vectors is near-orthogonal in early layers and converges by mid-depth, while the magnitude ratio is quantified per dataset [adarsh-etal-2026-how-context-shapes-truth] This is the first quantified characterization of how the truth direction's own geometry, not just its existence, changes under context [adarsh-etal-2026-how-context-shapes-truth] Mass-mean steering with the extracted vector flips truthfulness labels in ~100% of cases for three models (Qwen3-4B weaker and variable at 11-59%), a strong replicated causal effect [adarsh-etal-2026-how-context-shapes-truth]

models: SmolLM3-3B · method: Difference-in-means direction, Causal interventions (steering)

OLMo

On the Mutual Influence of Gender and Occupation in LLM Representations (2025)measured

A PCA gender direction carries first-name discriminative signal

Details

An et al. approximate a female-male gender direction via PCA over gendered word pairs, evaluated on 470 first names' contextualized embeddings [an-etal-2025-on-the-mutual-influence-of-gender-and-occupation-in-llm-representations] The models are Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, OLMo-7B-0724-hf, and Phi-3.5-mini-instruct [an-etal-2025-on-the-mutual-influence-of-gender-and-occupation-in-llm-representations] The 1-D gender-direction projection achieves 75.18% binary gender-classification accuracy versus 75.46% for the full embedding (Llama-3.1-8B) [an-etal-2025-on-the-mutual-influence-of-gender-and-occupation-in-llm-representations] First-name gender representations shift with occupational context, e.g. "nurse" pulls embeddings toward femininity [an-etal-2025-on-the-mutual-influence-of-gender-and-occupation-in-llm-representations] The internal gender-direction coefficient correlates with downstream occupation-prediction bias (Spearman rho=0.61 Llama-3.1-8B, 0.76 Mistral-7B; both p<0.001) [an-etal-2025-on-the-mutual-influence-of-gender-and-occupation-in-llm-representations]

models: OLMo-7B-0724-hf · method: Direction Extraction
Proxy Reward Internalization and Mechanistic Exploitation: A Learned Precursor to Reward Hacking and Its Generalization (2026)measured

Diff-in-means reward-hacking directions forecast and causally control misalignment

Details

Beigi, Jin & Huang extract three diff-in-means directions (context-sycophantic-agreement, premise-rejection, evaluator-rejection) during RL training of Qwen2.5-Coder-14B-Instruct and track their projection score across checkpoints [beigi-jin-huang-2026-proxy-reward-internalization-and-mechanistic-exploitation] Each direction is linearly decodable (AUROC 0.85-0.90) and a linear fit from probe score forecasts future out-of-domain misalignment (R^2=0.77), preceding the misalignment rise by roughly 45 training steps [beigi-jin-huang-2026-proxy-reward-internalization-and-mechanistic-exploitation] Joint ablation of the three directions lowers the hack rate by 26 points while preserving coding accuracy, and injection along them increases hacking [beigi-jin-huang-2026-proxy-reward-internalization-and-mechanistic-exploitation] The probe methodology generalizes to Qwen2.5-Coder 1.5B/7B/32B, Llama-3.1-8B-Instruct, and OLMo-7B [beigi-jin-huang-2026-proxy-reward-internalization-and-mechanistic-exploitation]

models: OLMo-7B · method: Difference-in-means direction
Language Models Compare Quantities Using Number-specific and Unit-specific Heuristics (2026)measured

DAS finds number- and unit-specific quantity-comparison subspaces

Details

Sasaki et al. use Distributed Alignment Search (1024 intervention dimensions, 512 per variable) to locate linear subspaces aligned with a NumLogDiff and a UnitLogDiff variable in quantity-comparison activations [sasaki-etal-2026-language-models-compare-quantities-using-number-specific-and-unit-specific-heuristics] Interchange Intervention Accuracy exceeds 0.9 in many middle layers versus near-chance baselines [sasaki-etal-2026-language-models-compare-quantities-using-number-specific-and-unit-specific-heuristics] A linear surrogate over these variables predicts the model's comparison preference at R^2 approximately 0.817 overall (0.763 near the decision boundary) for Qwen3-4B-Base [sasaki-etal-2026-language-models-compare-quantities-using-number-specific-and-unit-specific-heuristics] This supports a "bag of number-specific and unit-specific heuristics" account over exact-scale unit conversion [sasaki-etal-2026-language-models-compare-quantities-using-number-specific-and-unit-specific-heuristics]

models: OLMo 3 7B · method: Distributed Alignment Search (DAS), Causal interventions (steering)
What Really Controls Temporal Reasoning in LLMs: Tokenisation or Representation of Time? (2026)measured

Temporal-probe decodability predicts reasoning in high-resource languages

Details

Bhatia et al. introduce MultiTempBench and evaluate 20 LLMs, combining a tokenization-fragmentation metric with linear-regression probes decoding Year/Month/Day (probe R^2 as temporal linearity) [bhatia-etal-2026-what-really-controls-temporal-reasoning-in-llms] A crossed mixed-effects regression over 285,000 predictions finds temporal linearity is the strongest predictor of reasoning accuracy in high-resource languages (English r=0.77) [bhatia-etal-2026-what-really-controls-temporal-reasoning-in-llms] Tokenization fragmentation dominates in low-resource languages (Hausa r=-0.97); the finding is purely correlational [bhatia-etal-2026-what-really-controls-temporal-reasoning-in-llms]

models: OLMo 3 7B · method: Linear probing
Output Vector Editing for Memorization Mitigation in Large Language Models (2026)measured

Editing MLP output-vector directions suppresses memorized sequences

Details

Hakimi et al. identify MLP neurons implicated in verbatim memorization via logit-lens attribution, then edit each neuron's output vector to add a distractor direction while preserving its other superposed functions [hakimi-etal-2026-output-vector-editing-for-memorization] Four edit modes jointly suppress up to 96.5% of memorized continuations in ensemble, a 2.7x larger effect than zero-ablating the same neurons [hakimi-etal-2026-output-vector-editing-for-memorization] So the direction of the edit, not just the neuron's presence, drives the effect [hakimi-etal-2026-output-vector-editing-for-memorization] About 14% of memorized sequences resist MLP-only editing; ablating top attention heads recovers 60-64% of these, indicating a mechanism split across MLP and attention, and the method transfers to SmolLM-360M, OLMo-1B, and Llama2-7B [hakimi-etal-2026-output-vector-editing-for-memorization]

models: OLMo-1B, OLMo-7B · method: Logit Lens
Refusal in Language Models Is Mediated by a Single Direction (2024), Representation Engineering: A Top-Down Approach to AI Transparency (2023), Programming Refusal with Conditional Activation Steering (2024), Refusal Direction is Universal Across Safety-Aligned Languages (2025)measured

Refusal is mediated by a single direction across models

Details

Arditi et al. show across 13 chat models (5 families, 1.8B-72B) that a single diff-in-means direction between harmful and harmless instructions both erases refusal when ablated and induces it when added, with no offset term [arditi-etal-2024] Zou et al. independently confirm via a LAT harmfulness vector in Vicuna-13B that stays a >90% classifier under jailbreaks and adversarial suffixes [arditi-etal-2024] Lee et al. replicate across nine further chat models and add conditional steering that gates the refusal vector on a condition vector for selective refusal [arditi-etal-2024] Yung et al. average per-model refusal vectors into a universal refusal direction transferring to jailbreak detection ~10% better than single-model baselines [arditi-etal-2024] The account is contested by Marshall et al. (an affine offset is needed) and Wollschlager et al. (a cone of directions), and Yin et al. localize attack-suppressed versus robust heads [arditi-etal-2024]

models: OLMo-7B-SFT · method: Difference-in-means direction, Causal interventions (steering), Activation Steering (Addition), Linear Artificial Tomography (LAT), PCA, Conditional (gated) activation steering

Phi

On the Mutual Influence of Gender and Occupation in LLM Representations (2025)measured

A PCA gender direction carries first-name discriminative signal

Details

An et al. approximate a female-male gender direction via PCA over gendered word pairs, evaluated on 470 first names' contextualized embeddings [an-etal-2025-on-the-mutual-influence-of-gender-and-occupation-in-llm-representations] The models are Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, OLMo-7B-0724-hf, and Phi-3.5-mini-instruct [an-etal-2025-on-the-mutual-influence-of-gender-and-occupation-in-llm-representations] The 1-D gender-direction projection achieves 75.18% binary gender-classification accuracy versus 75.46% for the full embedding (Llama-3.1-8B) [an-etal-2025-on-the-mutual-influence-of-gender-and-occupation-in-llm-representations] First-name gender representations shift with occupational context, e.g. "nurse" pulls embeddings toward femininity [an-etal-2025-on-the-mutual-influence-of-gender-and-occupation-in-llm-representations] The internal gender-direction coefficient correlates with downstream occupation-prediction bias (Spearman rho=0.61 Llama-3.1-8B, 0.76 Mistral-7B; both p<0.001) [an-etal-2025-on-the-mutual-influence-of-gender-and-occupation-in-llm-representations]

models: Phi-3.5-mini-instruct · method: Direction Extraction
Categorical Perception in Large Language Model Hidden States: Structural Warping at Digit-Count Boundaries (2026)measured

LLM hidden states warp categorically at digit-count boundaries

Details

- In six LLMs, hidden-state distances warp categorically at digit-count boundaries (9->10, 99->100) versus matched non-boundary controls (15, 150); an RSA "CP-Additive" model (log-distance plus a boundary boost) beats a continuous model at 100% of primary layers. [cacioli-2026-categorical-perception-digit-boundaries] - Boundary crossing explains 5-27% of representational-distance variance beyond magnitude; the decade-100 effect is 3.9-12.7x the decade-10 effect, with a manifold rotation of 81.6-89.6 degrees at the boundary. [cacioli-2026-categorical-perception-digit-boundaries] - Patching along a ridge-regression "category direction" shifts discrimination confidence 70.1x more than random directions (Layer 5), dose-dependent and specific. [cacioli-2026-categorical-perception-digit-boundaries] - Five instruct models (Llama-3-8B-Instruct, Mistral-7B-Instruct-v0.3, Gemma-2-9B-IT, Qwen2.5-7B-Instruct, Phi-3.5-mini-instruct) plus a Llama-3-8B base control; causal patching run on Llama-3-8B-Instruct only. [cacioli-2026-categorical-perception-digit-boundaries]

models: Phi-3.5-mini-instruct · method: Representational Similarity Analysis (RSA), Linear probing, Causal interventions (steering)
Steering Language Model Refusal with Sparse Autoencoder Features (2024)measured

A single SAE latent causally mediates refusal with a dose-response tradeoff

Details

O'Brien et al. train a TopK SAE on Phi-3-mini's layer-6 residual stream and identify a single latent (Feature 22373) whose activation marks refusal [obrien-etal-2024-steering-refusal-with-sae-features] Clamping this decoded direction monotonically raises WildGuard unsafe-prompt refusal from 58.33% to 96.02% and cuts Crescendo jailbreak attack-success from 55.92% to 32.58% [obrien-etal-2024-steering-refusal-with-sae-features] The same clamped direction generalizes to Llama-3.1-8B-Instruct [obrien-etal-2024-steering-refusal-with-sae-features] A safety-capability tradeoff is quantified: safe-prompt refusal rises substantially and MMLU degrades (68.80% to 35.98% at clamp 12), a cost left mechanistically unexplained [obrien-etal-2024-steering-refusal-with-sae-features]

models: Phi-3-mini · method: TopK (k-sparse) Sparse Autoencoders, Causal interventions (steering)
Pre-trained Language Models Learn Remarkably Accurate Representations of Numbers (2025)measured

A sinusoidal probe decodes integers near-perfectly across nine LLMs

Details

Kadlcik et al. test linear, log-linear, and fixed structured-basis (sinusoidal Fourier, binary-digit) probes for decoding integer values of number-token embeddings across nine LLMs (OLMo 2 1B/7B/13B/32B, Llama 3.2 1B/3B, Llama 3 8B/70B, Phi-4 15B) [kadlcik-etal-2025-number-embeddings] The fixed sinusoidal probe recovers integer values with near-perfect accuracy on most models and consistently outperforms the plain linear probe (<=0.05 everywhere), contradicting a purely-linear account of number encoding [kadlcik-etal-2025-number-embeddings] Sparse Fourier support and wave-like PCA projections confirm the periodicity is genuine, though OLMo 2 32B is a documented exception with near-zero probe accuracy and no periodic structure despite performing arithmetic accurately [kadlcik-etal-2025-number-embeddings] Tokens the probe fails to decode show higher downstream arithmetic error rates, and gradient-optimizing the four worst-decoded tokens in Llama 3 1B toward the sinusoidal pattern reduces their addition errors by 26% (17.6% to 13.0%) [kadlcik-etal-2025-number-embeddings]

models: Phi-4 (15B) · method: Structured-basis probing
The Geometries of Truth Are Orthogonal Across Tasks (2026)measured

Per-task truth probe directions are near-orthogonal with disjoint L1 support

Details

Azizian et al. train L2-logistic truth probes separately on seven QA/fact-verification tasks across Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct, and Phi-4-Mini-Instruct [azizian-etal-2026-truth-geometries-orthogonal-across-tasks] Pairwise cosine similarity between probe directions is consistently low (<0.5) and correlates with cross-task transfer AUROC (r=0.59) [azizian-etal-2026-truth-geometries-orthogonal-across-tasks] L1 probes share under 15% of their nonzero support for most task pairs, and the few high-overlap pairs are exactly those that generalize well [azizian-etal-2026-truth-geometries-orthogonal-across-tasks] Three attempts to recover a shared cross-task truth direction (joint training, subspace-constrained fit, mixture-of-probes) all fail to beat per-task probes, and no causal intervention is performed [azizian-etal-2026-truth-geometries-orthogonal-across-tasks]

models: Phi-4-Mini-Instruct · method: Linear probing, Geometric analysis
When Reward Hacking Rebounds: Understanding and Mitigating It with Representation-Level Signals (2026)measured

A diff-of-means shortcut direction tracks and causally mitigates rebounding reward hacking during real GRPO training

Details

Wu & Tang extract a linear "shortcut" direction d via diff-of-means (h-plus minus h-minus) over contrastive rigorous-solution vs. shortcut-taking rollout descriptions, in real Phi-4-mini-Instruct and Llama-3.2-3B undergoing actual GRPO reinforcement learning on a LeetCode-style coding environment where models can rewrite evaluator code to fake passing tests [wu-tang-2026-when-reward-hacking-rebounds-representation-level-signals] The shortcut-direction projection score s = h dot d rises in step with a documented three-phase training trajectory (failed hacking, retreat to legitimate solving, rebound into hacking, reaching up to 99% unmitigated hack rate), tracking the transition into successful reward hacking before it is reflected in the raw reward signal [wu-tang-2026-when-reward-hacking-rebounds-representation-level-signals] A causal "Advantage Modification" intervention z-normalizes the shortcut score within each GRPO rollout group and penalizes the reward/advantage of high-shortcut-score rollouts before the policy update, reducing the hack rate from about 99% to 25% or lower while preserving legitimate Pass@1 and held-out benchmark performance (HumanEval, MBPP), outperforming a generation-time activation-steering baseline [wu-tang-2026-when-reward-hacking-rebounds-representation-level-signals]

models: Phi-4-Mini-Instruct · method:

Pythia

Representational Analysis of Binding in Language Models (2024)measured

A PCA 'Ordering ID' direction localizes and steers binding

Details

Dai, Heinzerling & Inui (2024) use PCA on entity/attribute activations to find a low-rank "OI subspace" whose leading component (OI-PC) encodes the "Ordering ID" underlying Feng & Steinhardt's Binding-ID mechanism, not previously localized to a direction [dai-etal-2024-representational-analysis-of-binding-in-language-models] Steering along OI-PC by n unit steps shifts the predicted attribute to that of Binding-ID n, quantified via Logit Difference and Logit Flip metrics that rise monotonically with step count [dai-etal-2024-representational-analysis-of-binding-in-language-models] A Spearman correlation confirms OI-PC tracks the true Order-ID rather than a position/filler-length confound [dai-etal-2024-representational-analysis-of-binding-in-language-models] The OI subspace is prevalent across five base models: Llama2-7B, Llama3-8B, Qwen1.5-7B, Pythia-6.9B, and the code-fine-tuned Float-7B [dai-etal-2024-representational-analysis-of-binding-in-language-models]

models: Pythia-6.9B · method: PCA, Direction Extraction, Causal interventions (steering)
Functional Subspace, where language models can use vector algebra to solve problems (2026)measured

ICL answer tokens are affine functions in a PCA subspace

Details

Lee & Vijayan collect residual-stream activations from six pretrained LLMs across six in-context relational tasks and reduce each layer to 30 principal components [lee-vijayan-2026-functional-subspace-vector-algebra] For each layer and component an answer token's projection is fit as an affine function of query and separator projections (a = alpha*q + beta*s + gamma) [lee-vijayan-2026-functional-subspace-vector-algebra] R^2 is high along a small, largely layer-consistent subset of components, rising in later layers, so ICL answer prediction reduces to an affine operation there [lee-vijayan-2026-functional-subspace-vector-algebra] Query, separator, and answer tokens form well-separated clusters in the top-3 high-R^2 component subspace across models and tasks [lee-vijayan-2026-functional-subspace-vector-algebra] Projections onto the highest-R^2 component differ significantly between correct and incorrect predictions (p<0.05) in mid/late layers, with no causal intervention performed [lee-vijayan-2026-functional-subspace-vector-algebra]

models: Pythia-12B · method: Linear Relational Embedding (LRE), PCA
Transferring Linear Features Across Language Models With Model Stitching (2025)measured

Affine maps transfer SAEs, probes, and steering across model sizes

Details

Chen, Merullo, Stolfo & Pavlick fit affine maps between residual streams of differently-sized pretrained models and transfer whole SAEs, probes, and steering vectors [chen-etal-2025-transferring-linear-features-across-language-models-with-model-stitching] For the GPT-2 pair (small to medium) stitching preserves cross-entropy within roughly 9-11% overhead [chen-etal-2025-transferring-linear-features-across-language-models-with-model-stitching] Overhead is larger for other families (Pythia deduped 17-27%, Gemma-2 8.3-39%), so the 9-11% figure is GPT-2-specific [chen-etal-2025-transferring-linear-features-across-language-models-with-model-stitching] Using a transferred SAE as initialization for a larger target model cuts SAE training cost by roughly 50% [chen-etal-2025-transferring-linear-features-across-language-models-with-model-stitching]

models: Pythia-70M-deduped, Pythia-160M-deduped · method: Cross-model direction transfer via ridge regression
Language Models Represent and Transform Concepts with Shared Geometry (2026)measured

Contextual displacement forms a vector field tracking concreteness

Details

Hu, Niu & Varma define a per-word contextual displacement vector phi(w,tau) = r(w,tau) - r(w,tau_0) in activation space, forming a vector field over the vocabulary rather than one shared direction [hu-etal-2026-concept-transformation-geometry] Across 4 models x 3 depths, word density correlates negatively with displacement magnitude (Spearman rho -0.32 to -0.22) [hu-etal-2026-concept-transformation-geometry] Lexical concreteness correlates negatively with directional deviation (rho -0.41 to -0.21): concrete, low-density concepts move less and more consistently under context [hu-etal-2026-concept-transformation-geometry] The field has no dominant direction (leading PC <15% variance) and is more dispersed than uniform-random; no causal intervention is performed [hu-etal-2026-concept-transformation-geometry]

models: Pythia-410M, Pythia-1B, Pythia-1.4B, Pythia-2.8B, Pythia-6.9B, Pythia-12B · method: PCA, Geometric analysis
How Do Language Models Bind Entities in Context? (2023)measured

Binding-ID vectors form a subspace whose distances predict confusability

Details

Feng & Steinhardt identify additive "binding ID vectors" attached to entity and attribute activations that solve variable binding, using causal interventions on LLaMA-1 (30B primary, plus 13B and 65B) and the Pythia family [feng-steinhardt-2023-how-do-language-models-bind-entities-in-context] The binding vectors occupy a continuous subspace in which the distance between two binding vectors predicts how often the model confuses the corresponding entity-attribute bindings [feng-steinhardt-2023-how-do-language-models-bind-entities-in-context] Patching, adding or removing binding-ID vectors changes which attribute the model retrieves for a given entity, demonstrated via causal mediation analysis rather than linear probing [feng-steinhardt-2023-how-do-language-models-bind-entities-in-context]

models: Pythia-2.8B · method: Activation patching
Function Vectors in Large Language Models (2024), In-Context Learning Creates Task Vectors (2023), Unifying Attention Heads and Task Vectors via Hidden State Geometry in In-Context Learning (2025), Do Different Prompting Methods Yield a Common Task Representation in Language Models? (2025)measured

Function vectors are compact directions that causally trigger ICL tasks

Details

Todd et al. use causal-mediation across 40+ ICL tasks in GPT-J-6B, GPT-NeoX-20B, and Llama-2 (7B/13B/70B) to find a small set of early-middle attention heads with high average indirect effect [todd-etal-2024] Summing those heads' mean per-task activations gives a function vector that, added at a middle layer, triggers the task even zero-shot (Llama-2 70B: 8.2% to 83.8%) [todd-etal-2024] Function vectors are portable across prompt formats and compose additively over functions, though some composed tasks are not expressible as embedding offsets [todd-etal-2024] A sharp late-layer drop in causal effect indicates function vectors trigger nonlinear downstream computation rather than a linear read-out [todd-etal-2024] Hendel et al. independently confirm a single task vector read from one residual-stream activation, recovering 80-90% of ICL across LLaMA, GPT-J, and Pythia [todd-etal-2024] Zheng et al. qualify that genuinely multi-demonstration tasks have no single task vector, and Yang et al. replicate the effect while explaining it via label-unembedding alignment [todd-etal-2024] Davidson et al. find instruction-derived and demonstration-derived function vectors only partially converge, sharing few top heads [todd-etal-2024]

models: Pythia-2.8B, Pythia-6.9B, Pythia-12B · method: Activation patching, Function vector extraction (causal-mediation head selection + sum), ICL task-vector extraction via dummy-query patching, Activation Steering (Addition)
Improving Dictionary Learning with Gated Sparse Autoencoders (2024)measured

Gated SAEs eliminate shrinkage and halve the required L0

Details

Rajamanoharan et al. show a standard L1 ReLU SAE conflates deciding which directions are active with estimating their magnitudes, causing shrinkage (systematic underestimation) [rajamanoharan-etal-2024-gated-saes] The Gated SAE splits these into weight-tied gate and magnitude sub-networks, applying the L1 penalty only to the gate's pre-activations [rajamanoharan-etal-2024-gated-saes] Measured via relative-reconstruction-bias gamma, Gated SAEs reach gamma~1 while baseline SAEs show gamma<1 (0.72 at a Gemma-7B site) [rajamanoharan-etal-2024-gated-saes] Gated SAEs need roughly half the L0 for equal loss recovery across GELU-1L, Pythia-2.8B, and Gemma-7B, with at-least-equal interpretability [rajamanoharan-etal-2024-gated-saes] Under tied weights the construction is provably equivalent to a JumpReLU encoder [rajamanoharan-etal-2024-gated-saes]

models: Pythia-2.8B · method: Gated Sparse Autoencoders
Signal in the Noise: Polysemantic Interference Transfers and Predicts Cross-Model Influence (2025)measured

SAE decoder cosine similarity quantifies interference and transfers steering

Details

Gong et al. train SAEs on Pythia-70M and GPT-2-Small and quantify feature-pair interference as the cosine similarity of their SAE decoder directions in activation space [gong-etal-2025-signal-in-the-noise-polysemantic-interference-transfers-and-predicts-cross-model-influence] Using these directions as steering vectors (scaled +/-20), small-model interference geometry transfers to larger black-box instruction-tuned models [gong-etal-2025-signal-in-the-noise-polysemantic-interference-transfers-and-predicts-cross-model-influence] One high-interference direction steered into Llama-3.1-8B-Instruct transfers at over 95% success [gong-etal-2025-signal-in-the-noise-polysemantic-interference-transfers-and-predicts-cross-model-influence] Pythia-derived success is 49.74% for high-interference pairs versus 21.76-22.87% for low-interference or random pairs [gong-etal-2025-signal-in-the-noise-polysemantic-interference-transfers-and-predicts-cross-model-influence] Transfer is further validated on Llama-3.1-70B-Instruct and Gemma-2-9B-Instruct, with an additional Gemma-2-2B experiment [gong-etal-2025-signal-in-the-noise-polysemantic-interference-transfers-and-predicts-cross-model-influence]

models: Pythia-70M · method: Sparse Autoencoders (SAE)
Inference-Time Causal Probing in LLMs (2026)measured

Inference-time causal probing separates causal from correlational directions

Details

Khorasani et al. introduce HDMI, an inference-time causal probing method that fits linear probes while testing whether interventions along the candidate direction propagate to downstream behavior [khorasani-etal-2026-inference-time-causal-probing-in-llms] On Llama-3-8B-Instruct and Pythia-70M, directions selected by the causal criterion produce reliable behavioral shifts under activation-space intervention [khorasani-etal-2026-inference-time-causal-probing-in-llms] Several high-accuracy linear-probe directions that pass standard probing tests fail to causally affect behavior when intervened upon [khorasani-etal-2026-inference-time-causal-probing-in-llms] Linear separability alone therefore overstates how many directions in a model are behaviorally load-bearing [khorasani-etal-2026-inference-time-causal-probing-in-llms]

models: Pythia-70M · method: Linear probing, Causal interventions (steering)
(How) Do Language Models Track State? (2025)measured

Permutation-tracking transformers encode state via a parity axis orthogonal to a triangle

Details

Li, Guo & Andreas fine-tune Pythia-160M on the symmetric-group (S3, S5) word problem and find models converge on one of exactly two mechanisms, identified via probing and activation patching [li-etal-2025] The Associative Algorithm computes the product via a hierarchical parallel-prefix scan, with state-probe accuracy and patch-restoration length growing exponentially with depth [li-etal-2025] The Parity-Associative Algorithm first computes parity via dedicated parity heads, then refines with an AA-like scan, and an intermediate pretraining curriculum controls which mechanism emerges [li-etal-2025] For PAA on S3, PCA reveals an orthogonal parity axis crossed with a 2D triangle of three equidistant clusters (a 2-simplex direct-summed with the parity direction); the clean 3-direction decomposition breaks down for S5 [li-etal-2025]

models: Pythia-160M · method: Linear probing, Activation patching, PCA
Linear Representations of Sentiment in Large Language Models (2023)measured

Sentiment is a single convergent linear direction, causally validated

Details

Tigges et al. extract a candidate sentiment direction in GPT2-small and Pythia (1.4B, 2.8B) via five techniques (mean difference, k-means, logistic regression, PCA, and DAS) [tigges-etal-2023-linear-sentiment] Pairwise cosine similarities between the five directions reach 72.6-99.1% (random baseline 0-2.4%), evidence they locate the same underlying direction [tigges-etal-2023-linear-sentiment] Directional ablation of the DAS direction on SST drops accuracy 100% to 62% (a 71% logit-difference reduction), and steering at coefficient -17 makes GPT2-small completions extremely negative [tigges-etal-2023-linear-sentiment] Single-scalar projection classifies token sentiment at 78-89%, and increasing the DAS subspace dimension does not improve OOD generalization, so one-dimensionality is treated as a supported but not final hypothesis [tigges-etal-2023-linear-sentiment]

models: Pythia-1.4B, Pythia-2.8B · method: Difference-in-means direction, Linear probing, PCA, Distributed Alignment Search (DAS), Activation patching, Activation Steering (Addition)
Linear Representations of Sentiment in Large Language Models (2023)measured

A summarization motif routes sentiment through non-valenced tokens

Details

Using iterative path-patching and value-weighted attention-pattern analysis, Tigges et al. trace how sentiment information flows through GPT2-small (and Pythia-2.8B) [tigges-etal-2023-linear-sentiment] A "summarization motif" accumulates sentiment onto intermediate non-valenced tokens such as commas and periods rather than only onto the sentiment-bearing words [tigges-etal-2023-linear-sentiment] These summarization points act as intermediate stores that later attention heads read from to produce the final sentiment prediction [tigges-etal-2023-linear-sentiment]

models: Pythia-2.8B · method: Activation patching, Direct Logit Attribution (DLA)
Transcoders Find Interpretable LLM Feature Circuits (2024)measured

Transcoders linearize MLP circuit attribution and match SAE quality

Details

Dunefsky et al. introduce transcoders: SAE-shaped networks trained to reconstruct an MLP sublayer's output from its input rather than the MLP's own activations [dunefsky-etal-2024-transcoders] Circuit attribution then factorizes exactly into an input-dependent activation term times a fixed decoder-encoder dot product, needing no gradient/Taylor linearization of the MLP [dunefsky-etal-2024-transcoders] On GPT2-small, Pythia-410M, and Pythia-1.4B, transcoders sit on reconstruction/sparsity Pareto frontiers equal to or better than SAEs, with the gap widening at larger scale [dunefsky-etal-2024-transcoders] Applying the exact-factorization method to GPT2-small's greater-than circuit surfaces a finer feature-level account and isolates one anomalous flat direct-logit-attribution feature [dunefsky-etal-2024-transcoders]

models: Pythia-410M, Pythia-1.4B · method: Transcoders
Eliciting Latent Predictions from Transformers with the Tuned Lens (2023)measured

The Tuned Lens replaces the logit lens with a trained per-layer affine correction

Details

Belrose et al. replace the logit lens's direct reuse of the final unembedding on intermediate states with a trained per-layer affine translator, distilled via KL divergence to the model's own final output [belrose-etal-2023-tuned-lens] This removes two documented logit-lens failure modes: a systematic 4-5 bit-per-layer vocabulary bias and unreliability from hidden-state covariance drifting across layers [belrose-etal-2023-tuned-lens] A causal-basis extraction finds the lens's most-sensitive directions strongly match the model's real causal directions (Spearman rho=0.89), with none influential to the lens but not the model [belrose-etal-2023-tuned-lens] Tested across GPT-2, GPT-Neo, GPT-NeoX-20B, OPT, BLOOM, and the full Pythia suite, it gives lower perplexity than the logit lens, transfers to a fine-tune (Vicuna-13B from LLaMA-13B), and detects prompt injection at near-perfect AUROC [belrose-etal-2023-tuned-lens]

models: Pythia-70M, Pythia-160M, Pythia-410M, Pythia-1B, Pythia-1.4B, Pythia-2.8B, Pythia-6.9B, Pythia-12B · method: Tuned Lens
Function-Vector Heads Are Two Populations: Writers and Cancellers in In-Context Learning (2026)measured

Function-vector heads split into writer and canceller populations with orthogonal OV directions

Details

Wang shows the function-vector heads identified by Todd et al. in real Pythia (410M-12B), Qwen2.5, and GPT-2-medium are not homogeneous, splitting by sign into writers and cancellers via refined direct logit attribution [wang-2026-function-vector-heads-writers-cancellers] The two sub-populations' mean OV directions are nearly orthogonal (perpendicular fraction 0.96), so cancellers write to a near-orthogonal subspace while still exerting a negative direct causal effect [wang-2026-function-vector-heads-writers-cancellers] Zero-ablating cancellers yields +0.13 to +0.29 nats of logit gain in 6 of 6 main cells, with a consistent +2 to +7 point ICL accuracy effect [wang-2026-function-vector-heads-writers-cancellers] A TOST equivalence test shows cancellers are not simply induction heads in disguise [wang-2026-function-vector-heads-writers-cancellers]

models: Pythia-410M, Pythia-1B, Pythia-1.4B · method: Direct Logit Attribution (DLA)
Which Attention Heads Matter for In-Context Learning? (2025)measured

Function-vector heads drive ICL; induction heads matter little and FV heads evolve from them

Details

Yin & Steinhardt ablate function-vector and induction heads across 12 models (70M-7B), finding FV-head ablation substantially degrades few-shot ICL while induction-head ablation barely exceeds random, a gap growing with scale [yin-steinhardt-2025-which-attention-heads-matter-for-icl] An ablation-with-exclusion design shows the apparent induction-head effect was mostly driven by heads that are both induction and FV heads [yin-steinhardt-2025-which-attention-heads-matter-for-icl] Direct overlap between top induction and FV heads is minimal, yet the two scores are correlated [yin-steinhardt-2025-which-attention-heads-matter-for-icl] Across Pythia training checkpoints, induction heads emerge early (~step 1,000) and FV heads substantially later (~step 16,000), with many FV heads evolving unidirectionally from earlier induction heads [yin-steinhardt-2025-which-attention-heads-matter-for-icl]

models: Pythia-70M, Pythia-160M, Pythia-410M, Pythia-1B, Pythia-1.4B, Pythia-2.8B, Pythia-6.9B · method: Causal interventions (steering), Function vector extraction (causal-mediation head selection + sum)

Float

Representational Analysis of Binding in Language Models (2024)measured

A PCA 'Ordering ID' direction localizes and steers binding

Details

Dai, Heinzerling & Inui (2024) use PCA on entity/attribute activations to find a low-rank "OI subspace" whose leading component (OI-PC) encodes the "Ordering ID" underlying Feng & Steinhardt's Binding-ID mechanism, not previously localized to a direction [dai-etal-2024-representational-analysis-of-binding-in-language-models] Steering along OI-PC by n unit steps shifts the predicted attribute to that of Binding-ID n, quantified via Logit Difference and Logit Flip metrics that rise monotonically with step count [dai-etal-2024-representational-analysis-of-binding-in-language-models] A Spearman correlation confirms OI-PC tracks the true Order-ID rather than a position/filler-length confound [dai-etal-2024-representational-analysis-of-binding-in-language-models] The OI subspace is prevalent across five base models: Llama2-7B, Llama3-8B, Qwen1.5-7B, Pythia-6.9B, and the code-fine-tuned Float-7B [dai-etal-2024-representational-analysis-of-binding-in-language-models]

models: Float-7B (code fine-tuned LM) · method: PCA, Direction Extraction, Causal interventions (steering)

LLaVA-OneVision

Interpreting and Enhancing Emotional Circuits in Large Vision-Language Models via Cross-Modal Information Flow (2026)measured

A linear emotional direction drives VLM emotion expression

Details

Zhang, Sun, Xie & Tian extract a layer-specific linear emotional direction in Qwen3-VL-4B-Instruct, Qwen3-VL-8B-Instruct, and LLaVA-OneVision-1.5-4B-Instruct by averaging contrastive emotional-vs-neutral hidden-state differences [zhang-etal-2026-interpreting-and-enhancing-emotional-circuits-in-large-vision-language-models] Activation steering with this direction causally raises the model's emotion-expression hit rate [zhang-etal-2026-interpreting-and-enhancing-emotional-circuits-in-large-vision-language-models] Backward activation patching from the direction localizes the upstream attention heads and MLP neurons that construct it [zhang-etal-2026-interpreting-and-enhancing-emotional-circuits-in-large-vision-language-models] The training-free VEENA framework built on the direction enhances or suppresses emotional expression at inference without retraining [zhang-etal-2026-interpreting-and-enhancing-emotional-circuits-in-large-vision-language-models]

models: LLaVA-OneVision-1.5-4B-Instruct · method: Difference-in-means direction, PCA, Activation Steering (Addition), Activation patching
Causal Probing for Internal Visual Representations in Multimodal Large Language Models (2026)measured

Visual concept directions split into localized-entity vs distributed-abstract encoding

Details

Deng et al. extract per-layer diff-in-means concept vectors for Entity, Visual-Style, Emotion, and Abstract categories across six MLLMs [deng-etal-2026-causal-probing-internal-visual-representations-mllms] Entity directions are sharply localized (Gini averaging 0.071, as low as 0.012) with a bimodal layer profile and extreme logit boosts [deng-etal-2026-causal-probing-internal-visual-representations-mllms] Abstract-concept directions are globally distributed (Gini up to 0.429), weakly steerable (success rate as low as 0.160) with negligible logit boost [deng-etal-2026-causal-probing-internal-visual-representations-mllms] Distributedness increases with scale: larger models localize concrete categories more but spread abstract concepts across more layers [deng-etal-2026-causal-probing-internal-visual-representations-mllms] Reverse steering establishes causal necessity, revealing a compensatory internal logit-boost surge (10^6-10^8) under entity suppression [deng-etal-2026-causal-probing-internal-visual-representations-mllms]

models: LLaVA-OneVision-1.5 8B · method: Difference-in-means direction, Causal interventions (steering), Activation Steering (Addition)

IndexTTS

Sparse Autoencoders for Interpretable Emotion Control in Text-to-Speech (2026)measured

An SAE on IndexTTS2 enables causal latent emotion steering

Details

Du et al. train a TopK SAE (4,096 latents, k=32) on layer-16 residual-stream activations of IndexTTS2's autoregressive semantic backbone over 56,000 emotion-controlled generations [du-etal-2026-sparse-autoencoders-for-interpretable-emotion-control-in-tts] Bidirectional latent steering achieves comparable or superior emotion induction and suppression versus global-steering and TTS baselines [du-etal-2026-sparse-autoencoders-for-interpretable-emotion-control-in-tts] Individual latents map to specific acoustics: steering latent #24 raises mean F0 by +23.11 Hz (p=1.07e-4) without affecting duration (p=0.687) [du-etal-2026-sparse-autoencoders-for-interpretable-emotion-control-in-tts] Emotional expression thus arises from coordinated, distributed latent contributions rather than one global direction [du-etal-2026-sparse-autoencoders-for-interpretable-emotion-control-in-tts]

models: IndexTTS2 · method: Sparse Autoencoders (SAE), Causal interventions (steering)

GPT

Beyond Components: Singular Vector-Based Interpretability of Transformer Circuits (2025)measured

GPT-2 heads multiplex subfunctions along orthogonal singular directions

Details

Ahmad, Joshi & Modi take the SVD of each attention head's and MLP layer's augmented weight matrix in pretrained GPT-2 Small [ahmad-joshi-modi-2025-beyond-components-singular-vector-based-interpretability-of-transformer-circuits] Individual components (e.g. head 9.6) encode multiple overlapping subfunctions aligned with distinct orthogonal singular directions [ahmad-joshi-modi-2025-beyond-components-singular-vector-based-interpretability-of-transformer-circuits] A learnable diagonal mask over singular values prunes 91-99% of directions across IOI, Greater-Than, and Gender-Pronoun while retaining 0.70-0.79 accuracy and low KL (0.21) [ahmad-joshi-modi-2025-beyond-components-singular-vector-based-interpretability-of-transformer-circuits] Scalar interventions on single singular directions flip gender-pronoun predictions at perfect accuracy, a targeted causal validation [ahmad-joshi-modi-2025-beyond-components-singular-vector-based-interpretability-of-transformer-circuits]

models: GPT-2-small · method: Singular-direction decomposition and pruning of transformer components
Investigating Aspect Features in Contextualized Embeddings with Semantic Scales and Distributional Similarity (2024)measured

Verb aspect is encoded as linear semantic-scale directions

Details

Li, Chersoni & Hsu apply Grand et al.'s semantic-projection method to build stativity, telicity, and durativity scale directions from corpus-averaged verb embeddings in BERT-base-uncased and GPT-2-small [li-chersoni-hsu-2024] Verb activations projected onto each scale show significant extreme-group differences (Mann-Whitney U, p<0.05) across layers [li-chersoni-hsu-2024] Stativity is most robustly encoded, telicity is significant in all BERT layers, and durativity is least consistent [li-chersoni-hsu-2024] Later layers of both models show declining separation, attributed to rising anisotropy that is stronger in GPT-2 [li-chersoni-hsu-2024] A cosine-similarity test of the Imperfective Paradox holds only in BERT's early layers, while GPT-2 shows the reverse pattern under high anisotropy [li-chersoni-hsu-2024]

models: GPT-2-small · method: Difference-in-means direction
Transferring Linear Features Across Language Models With Model Stitching (2025)measured

Affine maps transfer SAEs, probes, and steering across model sizes

Details

Chen, Merullo, Stolfo & Pavlick fit affine maps between residual streams of differently-sized pretrained models and transfer whole SAEs, probes, and steering vectors [chen-etal-2025-transferring-linear-features-across-language-models-with-model-stitching] For the GPT-2 pair (small to medium) stitching preserves cross-entropy within roughly 9-11% overhead [chen-etal-2025-transferring-linear-features-across-language-models-with-model-stitching] Overhead is larger for other families (Pythia deduped 17-27%, Gemma-2 8.3-39%), so the 9-11% figure is GPT-2-specific [chen-etal-2025-transferring-linear-features-across-language-models-with-model-stitching] Using a transferred SAE as initialization for a larger target model cuts SAE training cost by roughly 50% [chen-etal-2025-transferring-linear-features-across-language-models-with-model-stitching]

models: GPT-2-small, GPT-2-Medium · method: Cross-model direction transfer via ridge regression
Signal in the Noise: Polysemantic Interference Transfers and Predicts Cross-Model Influence (2025)measured

SAE decoder cosine similarity quantifies interference and transfers steering

Details

Gong et al. train SAEs on Pythia-70M and GPT-2-Small and quantify feature-pair interference as the cosine similarity of their SAE decoder directions in activation space [gong-etal-2025-signal-in-the-noise-polysemantic-interference-transfers-and-predicts-cross-model-influence] Using these directions as steering vectors (scaled +/-20), small-model interference geometry transfers to larger black-box instruction-tuned models [gong-etal-2025-signal-in-the-noise-polysemantic-interference-transfers-and-predicts-cross-model-influence] One high-interference direction steered into Llama-3.1-8B-Instruct transfers at over 95% success [gong-etal-2025-signal-in-the-noise-polysemantic-interference-transfers-and-predicts-cross-model-influence] Pythia-derived success is 49.74% for high-interference pairs versus 21.76-22.87% for low-interference or random pairs [gong-etal-2025-signal-in-the-noise-polysemantic-interference-transfers-and-predicts-cross-model-influence] Transfer is further validated on Llama-3.1-70B-Instruct and Gemma-2-9B-Instruct, with an additional Gemma-2-2B experiment [gong-etal-2025-signal-in-the-noise-polysemantic-interference-transfers-and-predicts-cross-model-influence]

models: GPT-2-small · method: Sparse Autoencoders (SAE)
Shared Global and Local Geometry of Language Model Embeddings (2025)measured

Token-embedding orientation is shared within families but drops across them

Details

Lee et al. measure token-embedding geometry across the GPT-2, Llama-3, and Gemma-2 model families [lee-etal-2025-shared-global-and-local-geometry-of-language-model-embeddings] Relative orientation/cosine structure is near-identical within a model family but drops sharply across families trained on different data [lee-etal-2025-shared-global-and-local-geometry-of-language-model-embeddings] A k-NN-neighborhood PCA intrinsic-dimension estimator finds low-ID tokens form semantically coherent clusters while high-ID tokens do not [lee-etal-2025-shared-global-and-local-geometry-of-language-model-embeddings] EMB2EMB, a linear least-squares (OLS) map fit over 100k shared tokens, transfers CAA-style steering vectors (refusal, sycophancy, corrigibility) between differently-sized models [lee-etal-2025-shared-global-and-local-geometry-of-language-model-embeddings]

models: GPT-2-Medium · method: PCA, Intrinsic dimension estimation (TwoNN), Cross-model direction transfer via ridge regression
What Really Controls Temporal Reasoning in LLMs: Tokenisation or Representation of Time? (2026)measured

Temporal-probe decodability predicts reasoning in high-resource languages

Details

Bhatia et al. introduce MultiTempBench and evaluate 20 LLMs, combining a tokenization-fragmentation metric with linear-regression probes decoding Year/Month/Day (probe R^2 as temporal linearity) [bhatia-etal-2026-what-really-controls-temporal-reasoning-in-llms] A crossed mixed-effects regression over 285,000 predictions finds temporal linearity is the strongest predictor of reasoning accuracy in high-resource languages (English r=0.77) [bhatia-etal-2026-what-really-controls-temporal-reasoning-in-llms] Tokenization fragmentation dominates in low-resource languages (Hausa r=-0.97); the finding is purely correlational [bhatia-etal-2026-what-really-controls-temporal-reasoning-in-llms]

models: GPT-4o · method: Linear probing
Universal Response and Emergence of Induction in LLMs (2024)measured

A weak-perturbation response direction stays near-orthogonal across every layer

Details

Luick applies a weak scaling perturbation to a single token's residual activation and tracks the downstream response across all layers of Gemma-2-2B, Llama-3.2-3B-Instruct, and GPT-2-XL [luick-2024-universal-response-emergence-of-induction] The response-direction-to-state cosine similarity stays below 0.1 in magnitude across the entire residual stream [luick-2024-universal-response-emergence-of-induction] This near-orthogonality co-occurs with a scale-invariant response regime [luick-2024-universal-response-emergence-of-induction] Both properties are tied to the induction mechanism, strongest for perturbations at token positions that induction heads copy from [luick-2024-universal-response-emergence-of-induction]

models: GPT-2-XL · method: Geometric analysis, Causal interventions (steering)
Copy Suppression: Comprehensively Understanding an Attention Head (2023)measured

Copy-suppression: a head anti-copies whatever token it would predict

Details

McDougall et al. compose GPT-2 Small head 10.7's OV circuit with the model's embedding and unembedding into an explicit vocab-by-vocab matrix [mcdougall-etal-2023-copy-suppression-comprehensively-understanding-an-attention-head] Its diagonal, each token's self-projection through the circuit, is among the top-10 most negative entries in 84.70% of vocabulary columns [mcdougall-etal-2023-copy-suppression-comprehensively-understanding-an-attention-head] The composed QK circuit's diagonal is the largest entry in its row for 95.72% of tokens, so the head attends to then anti-copies whichever token it would otherwise predict [mcdougall-etal-2023-copy-suppression-comprehensively-understanding-an-attention-head] A projection-only ablation (CSPA) onto the per-token negative-unembedding direction preserves 76.9-95.2% of the head's effect, confirming the geometric claim is causally load-bearing [mcdougall-etal-2023-copy-suppression-comprehensively-understanding-an-attention-head]

models: GPT-2-small · method: OV-circuit unembedding-diagonal projection
Scaling and Evaluating Sparse Autoencoders (2024)measured

TopK SAEs give clean scaling laws and mitigate dead latents

Details

Gao et al. (OpenAI) replace an SAE's ReLU+L1 objective with a hard TopK activation trained on reconstruction alone, removing L1's shrinkage bias and fixing L0=k per token [gao-etal-2024-scaling-saes] An auxiliary loss reconstructing residual error from only currently-dead latents holds the dead-latent fraction to ~7% even in a 16M-latent autoencoder [gao-etal-2024-scaling-saes] Trained on GPT-4 activations for 40B tokens, a 16M-latent SAE costs only ~10% of GPT-4 pretraining-equivalent loss when spliced in, and reconstruction loss follows a clean joint scaling law in size and sparsity [gao-etal-2024-scaling-saes] Three feature-quality metrics (probe recovery, N2G explainability, ablation sparsity) all improve with size, and TopK SAEs beat ReLU baselines on every one [gao-etal-2024-scaling-saes]

models: GPT-4 · method: TopK (k-sparse) Sparse Autoencoders
Function-Vector Heads Are Two Populations: Writers and Cancellers in In-Context Learning (2026)measured

Function-vector heads split into writer and canceller populations with orthogonal OV directions

Details

Wang shows the function-vector heads identified by Todd et al. in real Pythia (410M-12B), Qwen2.5, and GPT-2-medium are not homogeneous, splitting by sign into writers and cancellers via refined direct logit attribution [wang-2026-function-vector-heads-writers-cancellers] The two sub-populations' mean OV directions are nearly orthogonal (perpendicular fraction 0.96), so cancellers write to a near-orthogonal subspace while still exerting a negative direct causal effect [wang-2026-function-vector-heads-writers-cancellers] Zero-ablating cancellers yields +0.13 to +0.29 nats of logit gain in 6 of 6 main cells, with a consistent +2 to +7 point ICL accuracy effect [wang-2026-function-vector-heads-writers-cancellers] A TOST equivalence test shows cancellers are not simply induction heads in disguise [wang-2026-function-vector-heads-writers-cancellers]

models: GPT-2-Medium · method: Direct Logit Attribution (DLA)
Persona Features Control Emergent Misalignment (2026)measured

A single SAE toxic-persona latent controls emergent misalignment in GPT-4o and o3-mini

Details

Wang et al. (OpenAI) train a sparse autoencoder on GPT-4o's middle-layer pretraining activations and screen 2.1 million latents to 10 causally-relevant ones via aligned/misaligned model-diffing [wang-etal-2026-persona-features-control-emergent-misalignment] Latent #10, a "toxic persona" direction, perfectly discriminates aligned from misaligned finetuned models [wang-etal-2026-persona-features-control-emergent-misalignment] Steering positively along it induces misalignment in the safety-trained GPT-4o; steering negatively suppresses misalignment in finetuned models and in a helpful-only o3-mini variant [wang-etal-2026-persona-features-control-emergent-misalignment] The toxic-persona activation is elevated at just 5% malicious training data, well before behavioral misalignment appears, and re-alignment via 120 benign samples reduces misalignment to 0.1-0.5% [wang-etal-2026-persona-features-control-emergent-misalignment]

models: GPT-4o, o3-mini · method: Sparse Autoencoders (SAE)
Which Attention Heads Matter for In-Context Learning? (2025)measured

Function-vector heads drive ICL; induction heads matter little and FV heads evolve from them

Details

Yin & Steinhardt ablate function-vector and induction heads across 12 models (70M-7B), finding FV-head ablation substantially degrades few-shot ICL while induction-head ablation barely exceeds random, a gap growing with scale [yin-steinhardt-2025-which-attention-heads-matter-for-icl] An ablation-with-exclusion design shows the apparent induction-head effect was mostly driven by heads that are both induction and FV heads [yin-steinhardt-2025-which-attention-heads-matter-for-icl] Direct overlap between top induction and FV heads is minimal, yet the two scores are correlated [yin-steinhardt-2025-which-attention-heads-matter-for-icl] Across Pythia training checkpoints, induction heads emerge early (~step 1,000) and FV heads substantially later (~step 16,000), with many FV heads evolving unidirectionally from earlier induction heads [yin-steinhardt-2025-which-attention-heads-matter-for-icl]

models: GPT-2-small, GPT-2-Medium, GPT-2-Large, GPT-2-XL · method: Causal interventions (steering), Function vector extraction (causal-mediation head selection + sum)

Custom research sparse mixture-of-experts transformer

Routers Learn the Geometry of Their Experts: Geometric Coupling in Sparse Mixture-of-Experts (2026)measured

MoE router weight vectors geometrically couple to their experts

Details

Ahrac, Hochwald & Geva prove a router weight vector and its matched expert's gate matrix receive gradient updates proportional to the same routed hidden-state direction [ahrac-etal-2026-geometric-coupling-moe] In a real 1B SMoE (9 layers, 64 experts, top-6) trained on ~50B tokens, router score correlates with per-token expert gate-neuron activation at rho=0.43 (p=1.2e-81) [ahrac-etal-2026-geometric-coupling-moe] Auxiliary-loss balancing makes router weight vectors nearly 3x more mutually cosine-similar than loss-free balancing (0.57-0.63 vs 0.13-0.32 across three layers) [ahrac-etal-2026-geometric-coupling-moe] A parameter-free online K-means router achieves the lowest load imbalance while maintaining comparable perplexity [ahrac-etal-2026-geometric-coupling-moe]

models: Custom 1B-parameter sparse MoE transformer (Ahrac et al. 2026) · method: Router-expert weight coupling analysis
Geometric Routing Enables Causal Expert Control in Mixture of Experts (2026)measured

Steering token routing toward an expert centroid controls which expert processes a token

Details

Ternovtsii & Bilak train a custom cosine-routing MoE from scratch (8 layers, 1024 rank-1 experts/layer) and show its expert centroids are a causally-steerable geometric object [ternovtsii-bilak-2026-geometric-routing-causal-control] Biasing a token's routing position toward a temporal expert's centroid increases P(temporal) by a median +321%, while suppressing a geographic expert drops P(geographic) by 23.2% (vs 0% for random) [ternovtsii-bilak-2026-geometric-routing-causal-control] Directly rewriting a geographic expert's up-projection weight vector ("expert surgery") halves P(geographic) [ternovtsii-bilak-2026-geometric-routing-causal-control] Cross-layer composed steering is near-additive while same-layer composition interferes, so the routing-centroid direction behaves like an ordinary composable steering vector [ternovtsii-bilak-2026-geometric-routing-causal-control]

models: Semantic Trajectory MoE, 'Marathon' configuration (Ternovtsii & Bilak 2026) · method: Expert-routing centroid steering

VideoMAE

Causal Physics Steering in Video World Models via Concept Activation Vectors (2026)measured

A physics-plausibility CAV steers a video world model

Details

Alam fits per-layer linear probes on frozen VideoMAE-base patch tokens to classify IntPhys videos as physically possible or impossible, peaking at layer 5 (70.1%) [alam-2026-causal-physics-steering-video-world-models-concept-activation-vectors] Injecting the L2-normalized layer-5 probe vector produces a clean dose-response: at alpha=+5 judgments flip to "impossible" (P=1.000), at alpha=-5 to "possible" [alam-2026-causal-physics-steering-video-world-models-concept-activation-vectors] The effect is sharply layer-localized: injection at layers 0-5 flips 25% of judgments while layers 6-11 flip 0% [alam-2026-causal-physics-steering-video-world-models-concept-activation-vectors] The physics CAV is measurably orthogonal to a motion-direction CAV (90.0 degrees) and to a random vector (85.0 degrees) [alam-2026-causal-physics-steering-video-world-models-concept-activation-vectors] Per-block physics-principle CAVs are themselves substantially non-orthogonal (O1-O2 75.7, O2-O3 86.1 degrees), a quantified structure among related concept directions [alam-2026-causal-physics-steering-video-world-models-concept-activation-vectors]

models: VideoMAE-base · method: Linear probing, Activation Steering (Addition)
Latent Video Prediction Learns Better World Models (2026)measured

A directional-coherence score separates V-JEPA from cosine-similar rivals

Details

Alrasheed et al. linearly probe four frozen video foundation models (V-JEPA2.1, V-JEPA2, VideoPrism, VideoMAEv2) at matched ViT-L for push/pull action directions on Something-Something v2 [alrasheed-etal-2026-latent-video-prediction-learns-better-world-models] Their Directional Semantic Coherence Score (DSCS = r_sem x (1 - cos_rev)) is several times higher for V-JEPA than for VideoPrism and VideoMAEv2 [alrasheed-etal-2026-latent-video-prediction-learns-better-world-models] VideoPrism alone maintains representational similarity above 0.98 under severe patch dropout yet collapses to 2.7% top-1, while V-JEPA2.1 retains 46.1% [alrasheed-etal-2026-latent-video-prediction-learns-better-world-models] V-JEPA's cosine similarity under corruption is in fact lower than VideoPrism's, so DSCS captures oriented-axis structure invisible to bulk cosine similarity [alrasheed-etal-2026-latent-video-prediction-learns-better-world-models] The analysis is purely observational, with no causal intervention on the representations [alrasheed-etal-2026-latent-video-prediction-learns-better-world-models]

models: VideoMAE-v2 · method: Directional semantic coherence score (DSCS), Linear probing
Interpreting Physics in Video World Models (2026)measured

Video world models encode motion direction as a circular population code

Details

Joseph et al. analyze frozen pretrained V-JEPA 2 and VideoMAE-v2 video encoders and find motion direction carried by direction-selective MLP units whose sinusoidal tuning curves tile the full angular range at an intermediate-depth Physics Emergence Zone [joseph-etal-2026-interpreting-physics-video-world-models] A sawtooth pattern in probe accuracy under successive feature orthogonalization is consistent with paired sine/cosine encodings, a circular population code rather than a single linear direction (linear-probe R^2=0.97) [joseph-etal-2026-interpreting-physics-video-world-models] Scalar physical quantities (speed, acceleration) are linearly decodable from early layers onward, a simpler shape than the direction variable's population code [joseph-etal-2026-interpreting-physics-video-world-models] Local-attention suppression at the emergence zone drops direction-decoding R^2 (0.97->0.14) and intuitive-physics accuracy (78.3%->61.7%) while leaving ImageNet nearly unchanged (33.7%->33.1%), a clean double dissociation [joseph-etal-2026-interpreting-physics-video-world-models] Steering the direction variable required jointly manipulating dozens of orthogonal probe dimensions rather than a single vector, consistent with population-code geometry [joseph-etal-2026-interpreting-physics-video-world-models]

models: VideoMAE-v2 · method: Linear probing, Activation patching

AlphaZero

Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZero (2023)measured

AlphaZero's novel chess concepts are causally steerable and teachable

Details

Schut et al. extract new (not human-labeled) chess concepts from AlphaZero's activations as sparse linear directions via L1-regularized convex optimization [schut-etal-2023-bridging-human-ai-knowledge-gap] Concepts are filtered for teachability (a lesser-trained student AZ can reproduce them) and novelty (SVD reconstruction error against the human-game activation span) [schut-etal-2023-bridging-human-ai-knowledge-gap] Adding a concept direction back via norm-matched interpolation causally improves AZ's puzzle-solving specifically on that concept's puzzles [schut-etal-2023-bridging-human-ai-knowledge-gap] Four human grandmasters shown AZ's concept-instantiating lines improve their own puzzle-solving, evidencing the concepts are learnable outside the network [schut-etal-2023-bridging-human-ai-knowledge-gap]

models: AlphaZero (chess, ResNet policy/value network, self-play) · method: Linear probing, Causal interventions (steering)

JEPA (Joint Embedding Predictive Architecture)

Latent Video Prediction Learns Better World Models (2026)measured

A directional-coherence score separates V-JEPA from cosine-similar rivals

Details

Alrasheed et al. linearly probe four frozen video foundation models (V-JEPA2.1, V-JEPA2, VideoPrism, VideoMAEv2) at matched ViT-L for push/pull action directions on Something-Something v2 [alrasheed-etal-2026-latent-video-prediction-learns-better-world-models] Their Directional Semantic Coherence Score (DSCS = r_sem x (1 - cos_rev)) is several times higher for V-JEPA than for VideoPrism and VideoMAEv2 [alrasheed-etal-2026-latent-video-prediction-learns-better-world-models] VideoPrism alone maintains representational similarity above 0.98 under severe patch dropout yet collapses to 2.7% top-1, while V-JEPA2.1 retains 46.1% [alrasheed-etal-2026-latent-video-prediction-learns-better-world-models] V-JEPA's cosine similarity under corruption is in fact lower than VideoPrism's, so DSCS captures oriented-axis structure invisible to bulk cosine similarity [alrasheed-etal-2026-latent-video-prediction-learns-better-world-models] The analysis is purely observational, with no causal intervention on the representations [alrasheed-etal-2026-latent-video-prediction-learns-better-world-models]

models: V-JEPA 2.1, V-JEPA 2 · method: Directional semantic coherence score (DSCS), Linear probing
Interpreting Physics in Video World Models (2026)measured

Video world models encode motion direction as a circular population code

Details

Joseph et al. analyze frozen pretrained V-JEPA 2 and VideoMAE-v2 video encoders and find motion direction carried by direction-selective MLP units whose sinusoidal tuning curves tile the full angular range at an intermediate-depth Physics Emergence Zone [joseph-etal-2026-interpreting-physics-video-world-models] A sawtooth pattern in probe accuracy under successive feature orthogonalization is consistent with paired sine/cosine encodings, a circular population code rather than a single linear direction (linear-probe R^2=0.97) [joseph-etal-2026-interpreting-physics-video-world-models] Scalar physical quantities (speed, acceleration) are linearly decodable from early layers onward, a simpler shape than the direction variable's population code [joseph-etal-2026-interpreting-physics-video-world-models] Local-attention suppression at the emergence zone drops direction-decoding R^2 (0.97->0.14) and intuitive-physics accuracy (78.3%->61.7%) while leaving ImageNet nearly unchanged (33.7%->33.1%), a clean double dissociation [joseph-etal-2026-interpreting-physics-video-world-models] Steering the direction variable required jointly manipulating dozens of orthogonal probe dimensions rather than a single vector, consistent with population-code geometry [joseph-etal-2026-interpreting-physics-video-world-models]

models: V-JEPA 2 · method: Linear probing, Activation patching

VideoPrism

Latent Video Prediction Learns Better World Models (2026)measured

A directional-coherence score separates V-JEPA from cosine-similar rivals

Details

Alrasheed et al. linearly probe four frozen video foundation models (V-JEPA2.1, V-JEPA2, VideoPrism, VideoMAEv2) at matched ViT-L for push/pull action directions on Something-Something v2 [alrasheed-etal-2026-latent-video-prediction-learns-better-world-models] Their Directional Semantic Coherence Score (DSCS = r_sem x (1 - cos_rev)) is several times higher for V-JEPA than for VideoPrism and VideoMAEv2 [alrasheed-etal-2026-latent-video-prediction-learns-better-world-models] VideoPrism alone maintains representational similarity above 0.98 under severe patch dropout yet collapses to 2.7% top-1, while V-JEPA2.1 retains 46.1% [alrasheed-etal-2026-latent-video-prediction-learns-better-world-models] V-JEPA's cosine similarity under corruption is in fact lower than VideoPrism's, so DSCS captures oriented-axis structure invisible to bulk cosine similarity [alrasheed-etal-2026-latent-video-prediction-learns-better-world-models] The analysis is purely observational, with no causal intervention on the representations [alrasheed-etal-2026-latent-video-prediction-learns-better-world-models]

models: VideoPrism · method: Directional semantic coherence score (DSCS), Linear probing

GPT-J

Functional Subspace, where language models can use vector algebra to solve problems (2026)measured

ICL answer tokens are affine functions in a PCA subspace

Details

Lee & Vijayan collect residual-stream activations from six pretrained LLMs across six in-context relational tasks and reduce each layer to 30 principal components [lee-vijayan-2026-functional-subspace-vector-algebra] For each layer and component an answer token's projection is fit as an affine function of query and separator projections (a = alpha*q + beta*s + gamma) [lee-vijayan-2026-functional-subspace-vector-algebra] R^2 is high along a small, largely layer-consistent subset of components, rising in later layers, so ICL answer prediction reduces to an affine operation there [lee-vijayan-2026-functional-subspace-vector-algebra] Query, separator, and answer tokens form well-separated clusters in the top-3 high-R^2 component subspace across models and tasks [lee-vijayan-2026-functional-subspace-vector-algebra] Projections onto the highest-R^2 component differ significantly between correct and incorrect predictions (p<0.05) in mid/late layers, with no causal intervention performed [lee-vijayan-2026-functional-subspace-vector-algebra]

models: GPT-J-6B · method: Linear Relational Embedding (LRE), PCA
Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Models (2023)measured

Causal-tracing localization does not predict the best edit layer

Details

Hase et al. sweep the target layer for ROME/MEMIT rank-one edits in GPT-J and GPT2-XL and regress edit success on edit layer and causal-tracing effect [hase-etal-2023-does-localization-inform-editing] For ROME on GPT-J, edit layer alone explains 94.7% of edit-success variance while the tracing effect explains 1.6% and adds only 0.1% [hase-etal-2023-does-localization-inform-editing] The raw correlation between edit success and tracing effect is slightly negative (rho=-0.13, p<1e-3), opposite the localize-then-edit assumption [hase-etal-2023-does-localization-inform-editing] A "Fact Forcing" variant reusing tracing's noised-subject input shows a small positive tracing contribution (up to 2.7%), isolated to the shared input format [hase-etal-2023-does-localization-inform-editing] This critiques the layer-selection heuristic, not the rank-one edit mechanism's own geometry [hase-etal-2023-does-localization-inform-editing]

models: GPT-J-6B · method: Causal Tracing, Rank-One Model Editing (ROME) / Linear Associative Memory Editing
Function Vectors in Large Language Models (2024), In-Context Learning Creates Task Vectors (2023), Unifying Attention Heads and Task Vectors via Hidden State Geometry in In-Context Learning (2025), Do Different Prompting Methods Yield a Common Task Representation in Language Models? (2025)measured

Function vectors are compact directions that causally trigger ICL tasks

Details

Todd et al. use causal-mediation across 40+ ICL tasks in GPT-J-6B, GPT-NeoX-20B, and Llama-2 (7B/13B/70B) to find a small set of early-middle attention heads with high average indirect effect [todd-etal-2024] Summing those heads' mean per-task activations gives a function vector that, added at a middle layer, triggers the task even zero-shot (Llama-2 70B: 8.2% to 83.8%) [todd-etal-2024] Function vectors are portable across prompt formats and compose additively over functions, though some composed tasks are not expressible as embedding offsets [todd-etal-2024] A sharp late-layer drop in causal effect indicates function vectors trigger nonlinear downstream computation rather than a linear read-out [todd-etal-2024] Hendel et al. independently confirm a single task vector read from one residual-stream activation, recovering 80-90% of ICL across LLaMA, GPT-J, and Pythia [todd-etal-2024] Zheng et al. qualify that genuinely multi-demonstration tasks have no single task vector, and Yang et al. replicate the effect while explaining it via label-unembedding alignment [todd-etal-2024] Davidson et al. find instruction-derived and demonstration-derived function vectors only partially converge, sharing few top heads [todd-etal-2024]

models: GPT-J-6B · method: Activation patching, Function vector extraction (causal-mediation head selection + sum), ICL task-vector extraction via dummy-query patching, Activation Steering (Addition)
Discovering Latent Knowledge in Language Models Without Supervision (2023)measured

Contrast-Consistent Search finds a truth-like direction with no labels

Details

Burns et al. introduce Contrast-Consistent Search: fit a linear probe on yes/no contrast pairs purely by a logical-consistency plus confidence loss, using no truth labels [burns-etal-2023] Across 6 models and 10 classification/QA datasets, CCS beats zero-shot accuracy by ~4% and approaches supervised logistic regression [burns-etal-2023] The direction found on one dataset transfers to structurally unrelated datasets and is often recoverable as the top principal component of contrast-pair differences [burns-etal-2023] True/false examples form two well-separated clusters along it, sometimes from as few as one contrast pair [burns-etal-2023] This unsupervised truth-like direction is broader than Marks & Tegmark's factual-statement truth direction and is kept as a separate observation [burns-etal-2023]

models: GPT-J-6B · method: Contrast-Consistent Search (CCS), PCA, Linear probing, Geometric analysis
Linearity of Relation Decoding in Transformer Language Models (2023)measured

Linear relational embeddings are faithful and causal for half of relations

Details

Hernandez et al. approximate a transformer's relation-decoding function by a first-order Taylor expansion LRE(s)=beta*W_r*s+b_r, with W_r the mean Jacobian estimated from 5-8 examples [hernandez-etal-2023-lre] It is validated by faithfulness (decoded top token matches the model's own) and causality (low-rank pseudoinverse inversion redirects the real output) [hernandez-etal-2023-lre] Across 47 relations on GPT-J, GPT-2-XL, and LLaMA-13B, faithfulness exceeds 60% for nearly half the relations, with strong faithfulness-causality correlation (R=0.84 on GPT-J) [hernandez-etal-2023-lre] Relations like "company CEO" (0.06/0.31) fail almost completely despite correct model behavior, so linear relation decoding is real but heterogeneous across relation types [hernandez-etal-2023-lre]

models: GPT-J-6B · method: Linear Relational Embedding (LRE)
Mass-Editing Memory in a Transformer (2022)measured

MEMIT batch-edits many facts across MLP layers via least squares

Details

Meng et al. show ROME's single rank-one update degrades once applied sequentially for many facts, collapsing by n=10,000 edits [meng-etal-2022-memit] They generalize the linear-associative-memory framing to a batch least-squares objective over many key-value pairs simultaneously, a rank-<=u update [meng-etal-2022-memit] The edit is spread across a range of MLP layers (3-8 in GPT-J), with the residual apportioned equally across remaining layers [meng-etal-2022-memit] At n=10,000 simultaneous edits, MEMIT reaches a CounterFact composite of 85.8 on GPT-J and 82.0 on GPT-NeoX, versus ROME's and MEND's collapse [meng-etal-2022-memit] It uses rather than finds geometry, introducing no new claim about pre-existing key-direction packing [meng-etal-2022-memit]

models: GPT-J-6B · method: Causal Tracing, Mass-Editing Memory (MEMIT) — Multi-Layer Batch Associative-Memory Editing
Fast Model Editing at Scale (2022)measured

MEND transforms rank-one gradient factors with a hypernetwork

Details

Mitchell et al. note the per-example weight gradient of a dense layer is a sum of rank-one outer products (upstream gradient times input activation) [mitchell-etal-2022-mend] They train small per-layer hypernetworks that transform each rank-one factor independently, then reapply the same outer-product update with a learned step size [mitchell-etal-2022-mend] Every stage is a rank-one/outer-product direction construct, the same trigger/value pattern as ROME and MEMIT but reached by a learned transformation [mitchell-etal-2022-mend] Tested on GPT-Neo-2.7B, GPT-J-6B, T5-XL, and T5-11B, MEND reaches 0.81-0.89 edit success with low perplexity drawdown, staying effective at 11B where baselines fail [mitchell-etal-2022-mend] It uses rather than finds geometry, focusing purely on the algebra of the edit mechanism [mitchell-etal-2022-mend]

models: GPT-J-6B · method: MEND — Model Editor Networks using Gradient Decomposition
Language Models Implement Simple Word2Vec-style Vector Arithmetic (2024)measured

LLMs implement word2vec-style relational vector offsets, causally patchable

Details

Merullo et al. show many one-to-one relations (e.g. country-capital) that LLMs solve in-context are implemented as a simple additive vector offset computed by a mid-network FFN update [merullo-etal-2024-language-models-implement-word2vec-style-vector-arithmetic] This is directly analogous to word2vec-style vector arithmetic but discovered inside real pretrained transformer activations rather than static embeddings [merullo-etal-2024-language-models-implement-word2vec-style-vector-arithmetic] Extracting the offset from one example context and patching it into another causally forces the model to output the corresponding relation's correct answer for the new subject [merullo-etal-2024-language-models-implement-word2vec-style-vector-arithmetic] The mechanism is replicated across GPT-J, GPT-2 (small through XL), and BLOOM (up to 176B parameters) [merullo-etal-2024-language-models-implement-word2vec-style-vector-arithmetic]

models: GPT-J-6B · method: Vector-offset analogy test, Causal interventions (steering)
Probing then Editing Response Personality of Large Language Models (2025)measured

Personality traits are linearly encoded with a betweenness arrangement

Details

Ju et al. train layer-wise linear probes on 11 open LLMs (including InternLM-7B and InternLM2-7B, base and instruct) to classify which of three personality instructions a response is conditioned on, scored by V-information [ju-etal-2025-personality-editing] V-information rises through layers 5-12 then plateaus, with instruction-tuned models 1-3 points higher than their base counterparts [ju-etal-2025-personality-editing] A t-SNE of Llama-3.1-8B-Instruct shows Neuroticism and Agreeableness distant with Extraversion situated between them, an informal betweenness arrangement [ju-etal-2025-personality-editing] This geometry explains why converting either trait into Extraversion is easier than converting directly between Neuroticism and Agreeableness [ju-etal-2025-personality-editing]

models: GPT-J-6B · method: Linear probing
Identifying Linear Relational Concepts in Large Language Models (2023)measured

Linear relational concepts invert an LRE into an editable object direction

Details

Chanin et al. invert Hernandez et al.'s linear relational embedding with a low-rank pseudoinverse to build a "linear relational concept" unit vector living in subject-activation space [chanin-etal-2023] On 47 relations in Llama-2-7B and GPT-J-6B, LRCs beat a directly-trained SVM probe on both classification (0.81 vs 0.73-0.75) and causal editing (0.78-0.84 vs 0.69-0.76) [chanin-etal-2023] A deliberately low-rank inverse (rank ~200 of 4096) is essential, and reading the object side from an earlier layer roughly doubles multi-token accuracy [chanin-etal-2023] This is a fourth distinct route to a linear feature direction, derived from relational structure between two token positions rather than a contrastive set at one site [chanin-etal-2023]

models: GPT-J-6B · method: Linear relational concepts (LRC), Causal interventions (steering)
Locating and Editing Factual Associations in GPT (2022)measured

ROME localizes facts to a two-peak causal site and edits via a rank-one direction

Details

Meng et al. use causal tracing across 1,000 factual statements on GPT-2 XL to map a strongly causal early MLP site at the subject's last token, distinct from a late attention site at the final token [meng-etal-2022-rome] A severed-module variant confirms the early site depends specifically on MLP computation [meng-etal-2022-rome] They model the responsible MLP's down-projection as a linear associative memory and derive a closed-form rank-one update inserting one key-value association as an outer-product direction [meng-etal-2022-rome] It uses rather than finds geometry (the value is optimized per edit), reaching CounterFact composites of 89.2 (GPT-2 XL) and 91.5 (GPT-J) with high neighborhood specificity [meng-etal-2022-rome]

models: GPT-J-6B · method: Causal Tracing, Rank-One Model Editing (ROME) / Linear Associative Memory Editing
Large Language Models Encode Semantics and Alignment in Linearly Separable Representations (2025)measured

Topic subspaces sharpen with depth; a centroid direction induces CoT

Details

Saglam et al. study 11 autoregressive models on six arXiv topics, measuring per-layer hard-margin SVM separability and PCA intrinsic dimensionality [saglam-etal-2025-linear-subspaces] Under 10% of principal components explain nearly all variance, so topics occupy a near-affine low-dimensional subspace; separability rises toward final layers, reaching 100% SVM accuracy on all topic pairs in the largest models [saglam-etal-2025-linear-subspaces] A chain-of-thought framing distinction becomes linearly separable even more sharply than topic identity [saglam-etal-2025-linear-subspaces] Adding the CoT-vs-non-CoT centroid-difference direction to the hidden state reliably induces CoT-style responses (flagged preliminary), and a lightweight MLP guardrail built on the finding halves harmful responses [saglam-etal-2025-linear-subspaces]

models: GPT-J-6B · method: Linear probing, PCA, Activation Steering (Addition)

OLMo 2

Functional Subspace, where language models can use vector algebra to solve problems (2026)measured

ICL answer tokens are affine functions in a PCA subspace

Details

Lee & Vijayan collect residual-stream activations from six pretrained LLMs across six in-context relational tasks and reduce each layer to 30 principal components [lee-vijayan-2026-functional-subspace-vector-algebra] For each layer and component an answer token's projection is fit as an affine function of query and separator projections (a = alpha*q + beta*s + gamma) [lee-vijayan-2026-functional-subspace-vector-algebra] R^2 is high along a small, largely layer-consistent subset of components, rising in later layers, so ICL answer prediction reduces to an affine operation there [lee-vijayan-2026-functional-subspace-vector-algebra] Query, separator, and answer tokens form well-separated clusters in the top-3 high-R^2 component subspace across models and tasks [lee-vijayan-2026-functional-subspace-vector-algebra] Projections onto the highest-R^2 component differ significantly between correct and incorrect predictions (p<0.05) in mid/late layers, with no causal intervention performed [lee-vijayan-2026-functional-subspace-vector-algebra]

models: OLMo 2 32B · method: Linear Relational Embedding (LRE), PCA
How Does Alignment Tuning Shape Representations of Sycophancy and Related Cue-Induced Biases in LLMs? (2026)measured

Cue-induced bias directions are installed by alignment tuning

Details

Gupta et al. extract a per-bias diff-of-means direction for seven cue-triggered bias types across base and instruct checkpoints of five model families [gupta-etal-2026-alignment-tuning-sycophancy-directions] Four of five base models flip on under 5% as many pairs as their instruct counterparts, with base activations carrying essentially no cue-specific signal, so susceptibility is installed by alignment tuning [gupta-etal-2026-alignment-tuning-sycophancy-directions] Within instruct models each bias direction transfers to held-out data at mean AUROC 0.69-0.82, localizing to late-middle layers [gupta-etal-2026-alignment-tuning-sycophancy-directions] Biases stay representationally distinct, with even behaviorally similar biases occupying different, sometimes anti-aligned directions [gupta-etal-2026-alignment-tuning-sycophancy-directions] Subtracting the direction while preserving over 90% of correct answers recovers 7-20% of bias-induced errors, versus under 5% for a random direction [gupta-etal-2026-alignment-tuning-sycophancy-directions]

models: OLMo 2 7B, OLMo 2 7B Instruct · method: Difference-in-means direction, Activation Steering (Addition)
Pre-trained Language Models Learn Remarkably Accurate Representations of Numbers (2025)measured

A sinusoidal probe decodes integers near-perfectly across nine LLMs

Details

Kadlcik et al. test linear, log-linear, and fixed structured-basis (sinusoidal Fourier, binary-digit) probes for decoding integer values of number-token embeddings across nine LLMs (OLMo 2 1B/7B/13B/32B, Llama 3.2 1B/3B, Llama 3 8B/70B, Phi-4 15B) [kadlcik-etal-2025-number-embeddings] The fixed sinusoidal probe recovers integer values with near-perfect accuracy on most models and consistently outperforms the plain linear probe (<=0.05 everywhere), contradicting a purely-linear account of number encoding [kadlcik-etal-2025-number-embeddings] Sparse Fourier support and wave-like PCA projections confirm the periodicity is genuine, though OLMo 2 32B is a documented exception with near-zero probe accuracy and no periodic structure despite performing arithmetic accurately [kadlcik-etal-2025-number-embeddings] Tokens the probe fails to decode show higher downstream arithmetic error rates, and gradient-optimizing the four worst-decoded tokens in Llama 3 1B toward the sinusoidal pattern reduces their addition errors by 26% (17.6% to 13.0%) [kadlcik-etal-2025-number-embeddings]

models: OLMo 2 1B, OLMo 2 7B, OLMo 2 13B, OLMo 2 32B · method: Structured-basis probing

GPT-NeoX

Functional Subspace, where language models can use vector algebra to solve problems (2026)measured

ICL answer tokens are affine functions in a PCA subspace

Details

Lee & Vijayan collect residual-stream activations from six pretrained LLMs across six in-context relational tasks and reduce each layer to 30 principal components [lee-vijayan-2026-functional-subspace-vector-algebra] For each layer and component an answer token's projection is fit as an affine function of query and separator projections (a = alpha*q + beta*s + gamma) [lee-vijayan-2026-functional-subspace-vector-algebra] R^2 is high along a small, largely layer-consistent subset of components, rising in later layers, so ICL answer prediction reduces to an affine operation there [lee-vijayan-2026-functional-subspace-vector-algebra] Query, separator, and answer tokens form well-separated clusters in the top-3 high-R^2 component subspace across models and tasks [lee-vijayan-2026-functional-subspace-vector-algebra] Projections onto the highest-R^2 component differ significantly between correct and incorrect predictions (p<0.05) in mid/late layers, with no causal intervention performed [lee-vijayan-2026-functional-subspace-vector-algebra]

models: GPT-NeoX-20B · method: Linear Relational Embedding (LRE), PCA
Function Vectors in Large Language Models (2024), In-Context Learning Creates Task Vectors (2023), Unifying Attention Heads and Task Vectors via Hidden State Geometry in In-Context Learning (2025), Do Different Prompting Methods Yield a Common Task Representation in Language Models? (2025)measured

Function vectors are compact directions that causally trigger ICL tasks

Details

Todd et al. use causal-mediation across 40+ ICL tasks in GPT-J-6B, GPT-NeoX-20B, and Llama-2 (7B/13B/70B) to find a small set of early-middle attention heads with high average indirect effect [todd-etal-2024] Summing those heads' mean per-task activations gives a function vector that, added at a middle layer, triggers the task even zero-shot (Llama-2 70B: 8.2% to 83.8%) [todd-etal-2024] Function vectors are portable across prompt formats and compose additively over functions, though some composed tasks are not expressible as embedding offsets [todd-etal-2024] A sharp late-layer drop in causal effect indicates function vectors trigger nonlinear downstream computation rather than a linear read-out [todd-etal-2024] Hendel et al. independently confirm a single task vector read from one residual-stream activation, recovering 80-90% of ICL across LLaMA, GPT-J, and Pythia [todd-etal-2024] Zheng et al. qualify that genuinely multi-demonstration tasks have no single task vector, and Yang et al. replicate the effect while explaining it via label-unembedding alignment [todd-etal-2024] Davidson et al. find instruction-derived and demonstration-derived function vectors only partially converge, sharing few top heads [todd-etal-2024]

models: GPT-NeoX-20B · method: Activation patching, Function vector extraction (causal-mediation head selection + sum), ICL task-vector extraction via dummy-query patching, Activation Steering (Addition)
Mass-Editing Memory in a Transformer (2022)measured

MEMIT batch-edits many facts across MLP layers via least squares

Details

Meng et al. show ROME's single rank-one update degrades once applied sequentially for many facts, collapsing by n=10,000 edits [meng-etal-2022-memit] They generalize the linear-associative-memory framing to a batch least-squares objective over many key-value pairs simultaneously, a rank-<=u update [meng-etal-2022-memit] The edit is spread across a range of MLP layers (3-8 in GPT-J), with the residual apportioned equally across remaining layers [meng-etal-2022-memit] At n=10,000 simultaneous edits, MEMIT reaches a CounterFact composite of 85.8 on GPT-J and 82.0 on GPT-NeoX, versus ROME's and MEND's collapse [meng-etal-2022-memit] It uses rather than finds geometry, introducing no new claim about pre-existing key-direction packing [meng-etal-2022-memit]

models: GPT-NeoX-20B · method: Causal Tracing, Mass-Editing Memory (MEMIT) — Multi-Layer Batch Associative-Memory Editing
Eliciting Latent Predictions from Transformers with the Tuned Lens (2023)measured

The Tuned Lens replaces the logit lens with a trained per-layer affine correction

Details

Belrose et al. replace the logit lens's direct reuse of the final unembedding on intermediate states with a trained per-layer affine translator, distilled via KL divergence to the model's own final output [belrose-etal-2023-tuned-lens] This removes two documented logit-lens failure modes: a systematic 4-5 bit-per-layer vocabulary bias and unreliability from hidden-state covariance drifting across layers [belrose-etal-2023-tuned-lens] A causal-basis extraction finds the lens's most-sensitive directions strongly match the model's real causal directions (Spearman rho=0.89), with none influential to the lens but not the model [belrose-etal-2023-tuned-lens] Tested across GPT-2, GPT-Neo, GPT-NeoX-20B, OPT, BLOOM, and the full Pythia suite, it gives lower perplexity than the logit lens, transfers to a fine-tune (Vicuna-13B from LLaMA-13B), and detects prompt injection at near-perfect AUROC [belrose-etal-2023-tuned-lens]

models: GPT-NeoX-20B · method: Tuned Lens

Whisper

AudioSAE: Towards Understanding of Audio-Processing Models with Sparse AutoEncoders (2026)measured

TopK SAE features on real Whisper and HuBERT encoders are seed-stable and sparsely erasable, and steering them causally cuts Whisper false-speech detections by 70%

Details

Aparin, Sadekova, Rukhovich, Yermekova, Kushnareva, Popov, Kuznetsov & Piontkovskaya (2026, AudioSAE) train TopK sparse autoencoders across all encoder layers of real Whisper-small and HuBERT-base, finding over 50% of features remain consistent across random seeds and quantifying disentanglement via a concept-erasure test: only 19-27% of features need removal to erase a target concept [aparin-etal-2026-audiosae-towards-understanding-of-audio-processing-models-with-sparse-autoencoders] Individually, features capture both general acoustic/semantic content and specific disentangled paralinguistic events (environmental noise, laughter, whispering) [aparin-etal-2026-audiosae-towards-understanding-of-audio-processing-models-with-sparse-autoencoders] Causally, steering SAE features reduces real Whisper-small's false speech detections by 70% with negligible word-error-rate degradation on LibriSpeech test-clean, and a separate concept-erasure intervention removes a target concept using only 19-27% of the feature set [aparin-etal-2026-audiosae-towards-understanding-of-audio-processing-models-with-sparse-autoencoders]

models: Whisper small · method: TopK (k-sparse) Sparse Autoencoders
Whisper Hallucination Detection and Mitigation via Hidden Representation Steering and Sparse AutoEncoders (2026)measured

Whisper hallucination is linearly decodable and causally steerable

Details

Aparin et al. fit per-layer logistic classifiers to detect hallucination from frozen Whisper small/large-v3 encoder activations and from Batch-TopK SAE latents [aparin-etal-2026-whisper-hallucination-detection-mitigation-hidden-representation-steering-sparse-autoencoders] Both representation spaces are linearly separable for hallucination status (layer-averaged AUC 0.74-0.80), improving toward deeper layers [aparin-etal-2026-whisper-hallucination-detection-mitigation-hidden-representation-steering-sparse-autoencoders] Hallucination information concentrates in a small subset of SAE features, stabilizing at 50-100 features [aparin-etal-2026-whisper-hallucination-detection-mitigation-hidden-representation-steering-sparse-autoencoders] A diff-in-means direction added to the final encoder layer, and a sparse sign-pattern over top SAE features, both causally steer hallucination [aparin-etal-2026-whisper-hallucination-detection-mitigation-hidden-representation-steering-sparse-autoencoders] SAE steering cuts hallucination rate from 72.63% to 14.11% (small) and 86.88% to 27.33% (large-v3), at a modest clean-speech WER cost [aparin-etal-2026-whisper-hallucination-detection-mitigation-hidden-representation-steering-sparse-autoencoders]

models: Whisper small, Whisper large-v3 · method: Difference-in-means direction, Activation Steering (Addition), BatchTopK Sparse Autoencoders
Mechanistic Interpretability of ASR models using Sparse Autoencoders (2026)measured

Monosemantic Whisper SAE latents encode phonetic/lexical features and steer transcripts

Details

Pluth et al. train a TopK sparse autoencoder on ~200M frames of Whisper-base encoder activations, recovering monosemantic dictionary directions [pluth-etal-2026-mechanistic-interpretability-asr-sae] These span diphone (e.g. /r-uw/ precision 88.7%), word (e.g. "his" precision 99.3%), language-discrimination (recall 91.2%), and profanity (recall 89.7%) features [pluth-etal-2026-mechanistic-interpretability-asr-sae] Adding or clamping these SAE decoder directions causally steers Whisper's transcript content, including cross-lingually [pluth-etal-2026-mechanistic-interpretability-asr-sae]

models: Whisper base · method: TopK (k-sparse) Sparse Autoencoders, Causal interventions (steering)

HuBERT

AudioSAE: Towards Understanding of Audio-Processing Models with Sparse AutoEncoders (2026)measured

TopK SAE features on real Whisper and HuBERT encoders are seed-stable and sparsely erasable, and steering them causally cuts Whisper false-speech detections by 70%

Details

Aparin, Sadekova, Rukhovich, Yermekova, Kushnareva, Popov, Kuznetsov & Piontkovskaya (2026, AudioSAE) train TopK sparse autoencoders across all encoder layers of real Whisper-small and HuBERT-base, finding over 50% of features remain consistent across random seeds and quantifying disentanglement via a concept-erasure test: only 19-27% of features need removal to erase a target concept [aparin-etal-2026-audiosae-towards-understanding-of-audio-processing-models-with-sparse-autoencoders] Individually, features capture both general acoustic/semantic content and specific disentangled paralinguistic events (environmental noise, laughter, whispering) [aparin-etal-2026-audiosae-towards-understanding-of-audio-processing-models-with-sparse-autoencoders] Causally, steering SAE features reduces real Whisper-small's false speech detections by 70% with negligible word-error-rate degradation on LibriSpeech test-clean, and a separate concept-erasure intervention removes a target concept using only 19-27% of the feature set [aparin-etal-2026-audiosae-towards-understanding-of-audio-processing-models-with-sparse-autoencoders]

models: HuBERT-base · method: TopK (k-sparse) Sparse Autoencoders
[b]=[d]-[t]+[p]: Self-Supervised Speech Models Discover Phonological Vector Arithmetic (2026)measured

Phonological features are linear vector-offset directions in speech models

Details

Choi et al. extract phoneme-level representations from three frozen self-supervised speech models (wav2vec2-large-lv60, HuBERT-large-ll60k, WavLM-large) [choi-etal-2026-phonological-vector-arithmetic-speech-ssl] Phonological feature contrasts (voicing, place, manner) are encoded as approximately consistent diff-in-means directions that transfer across phoneme pairs [choi-etal-2026-phonological-vector-arithmetic-speech-ssl] Word2vec-style vector-offset arithmetic (e.g. [b] = [d] - [t] + [p]) succeeds above chance on the real extracted representations, with per-model accuracy reported [choi-etal-2026-phonological-vector-arithmetic-speech-ssl] The result is a quantified linear-direction/vector-offset claim on real speech models; no causal intervention is performed [choi-etal-2026-phonological-vector-arithmetic-speech-ssl]

models: HuBERT Large (LibriLight 60k) · method: Difference-in-means direction, Vector-offset analogy test
Training-Free Cross-Lingual Dysarthria Severity Assessment via Phonological Subspace Analysis in Self-Supervised Speech Representations (2026)measured

Phonological contrast directions in frozen HuBERT-base embeddings collapse in separability with dysarthria severity, training-free and across languages

Details

For each of 12 phonological contrasts (nasality, voicing, stridency, sonorance, manner, 4 vowel features), a difference-of-means direction is computed in 768-dimensional layer-12 HuBERT-base activations and separability along that direction is measured via d-prime, with no training on pathological data [muller-etal-2026-phonological-subspace-dysarthria] Healthy speakers show tight, well-separated phonological classes along the fitted direction (d-prime = 4.2), while severe dysarthria shows the class centroids remain distinguishable but within-class distributions widen and overlap substantially (d-prime = 1.1) [muller-etal-2026-phonological-subspace-dysarthria] Measured on 890 real speakers across 10 corpora and 5 languages using a single frozen facebook/hubert-base-ls960 checkpoint, with no per-language or per-corpus fine-tuning [muller-etal-2026-phonological-subspace-dysarthria]

models: HuBERT-base · method: Difference-in-means direction
Self-Supervised Speech Models Encode Phonetic Context via Position-dependent Orthogonal Subspaces (2026)measured

Speech models encode neighbor-phone info in orthogonal positional subspaces

Details

Choi et al. show a single S3M frame (wav2vec 2.0, HuBERT, WavLM Large) encodes phonological vectors not only for the current phone but for its neighbors, extracted via difference-of-means at relative positions -2 to +2 [choi-etal-2026-position-dependent-orthogonal-subspaces] Cosine similarity between vectors from different relative positions is substantially lower than within the same position across 8 phonological features, an orthogonality-of-subspaces structure preserved layer-by-layer [choi-etal-2026-position-dependent-orthogonal-subspaces] Vector norm decreases monotonically with distance from the center phone, with WavLM showing the clearest trapezoidal effective context window [choi-etal-2026-position-dependent-orthogonal-subspaces] The position-dependent subspace in use switches at annotated TIMIT phonetic boundaries rather than a fixed temporal window; no causal intervention is performed [choi-etal-2026-position-dependent-orthogonal-subspaces]

models: HuBERT Large (LibriLight 60k) · method: Difference-in-means direction, Geometric analysis

BERT

Investigating Aspect Features in Contextualized Embeddings with Semantic Scales and Distributional Similarity (2024)measured

Verb aspect is encoded as linear semantic-scale directions

Details

Li, Chersoni & Hsu apply Grand et al.'s semantic-projection method to build stativity, telicity, and durativity scale directions from corpus-averaged verb embeddings in BERT-base-uncased and GPT-2-small [li-chersoni-hsu-2024] Verb activations projected onto each scale show significant extreme-group differences (Mann-Whitney U, p<0.05) across layers [li-chersoni-hsu-2024] Stativity is most robustly encoded, telicity is significant in all BERT layers, and durativity is least consistent [li-chersoni-hsu-2024] Later layers of both models show declining separation, attributed to rising anisotropy that is stronger in GPT-2 [li-chersoni-hsu-2024] A cosine-similarity test of the Imperfective Paradox holds only in BERT's early layers, while GPT-2 shows the reverse pattern under high anisotropy [li-chersoni-hsu-2024]

models: BERT-base-uncased · method: Difference-in-means direction
Mapping Semantic & Syntactic Relationships with Geometric Rotation in Embedding Space (2025)measured

Analogical relations fit a rotational operator better than an additive offset

Details

Freenor & Alvarez test whether analogical relations (king-queen, walk-walked) are better modeled as an additive offset or a rotational operator [freenor-alvarez-2025-geometric-rotation-embedding-space] Rotor-invariant shift estimation fits a best-approximating rotation and measures residual invariance across many real word pairs [freenor-alvarez-2025-geometric-rotation-embedding-space] The rotational model achieves lower, more consistent residual error than the additive-offset model across text-embedding-3-large, BGE-M3, and multilingual BERT [freenor-alvarez-2025-geometric-rotation-embedding-space] This is a quantified comparative geometric claim on real embeddings; no causal intervention is performed [freenor-alvarez-2025-geometric-rotation-embedding-space]

models: mBERT (BERT-base, Multilingual Cased) · method: Rotor-invariant shift estimation (RISE)
Editing Factual Knowledge in Language Models (2021)measured

KnowledgeEditor gates the fine-tuning gradient with a hypernetwork outer product

Details

De Cao et al. condition small FFNNs on a bidirectional-LSTM encoding of an edit request to predict per-matrix vectors forming an outer-product gate and bias on the loss gradient [decao-etal-2021-knowledge-editor] A KL-divergence-in-output-space constraint, not a parameter-space norm, prevents collateral damage: an Lp-parameter constraint instead collapses retain accuracy from 98.14 to 45.10 [decao-etal-2021-knowledge-editor] It uses rather than finds geometry, making no claim the edited fact was pre-encoded, and its probe framing concerns only which weight matrices receive large updates [decao-etal-2021-knowledge-editor] Tested on BERT-base (FEVER 98.80% success) and BART-base (zsRE 94.65%), it is the direct architectural predecessor of MEND [decao-etal-2021-knowledge-editor]

models: BERT-base-uncased · method: KnowledgeEditor — Constrained Hyper-Network Weight Editing
How Language-Neutral is Multilingual BERT? (2019)measured

Subtracting a per-language centroid removes decodable language identity in mBERT

Details

Libovicky, Rosa & Fraser compute a per-language centroid as the mean mBERT representation over ~110k Wikipedia sentences per language [libovicky-rosa-fraser-2019] Subtracting the centroid collapses language-ID classification from 0.919 to 0.285 while improving cross-lingual retrieval (0.776 to 0.838) and leaving word-alignment F1 unchanged [libovicky-rosa-fraser-2019] A supervised linear projection does better on retrieval, but the parameter-free centroid alone already captures most of what a per-language offset can fix [libovicky-rosa-fraser-2019] The centroids are non-arbitrary: hierarchical clustering recovers genealogical language families at V-measure 82.4 versus a 62.1 random baseline [libovicky-rosa-fraser-2019] Centering does not help machine-translation quality estimation (HTER correlation below 0.04), a clear negative result [libovicky-rosa-fraser-2019]

models: mBERT (BERT-base, Multilingual Cased), BERT base multilingual uncased · method: Language centroid shift
Naturalistic Causal Probing for Morpho-Syntax (2023)measured

A naturalistic causal ATE direction nearly coincides with the top PCA direction

Details

Amini et al. estimate the average treatment effect of grammatical gender and number on mBERT, XLM-RoBERTa-base, and GPT2-small-Spanish using dependency-tree-reinflected naturalistic counterfactuals [amini-etal-2023-naturalistic] The first principal component of the paired representations has cosine similarity ~0.99 with the estimated ATE direction across all three models [amini-etal-2023-naturalistic] In BERT/RoBERTa this PC explains ~20% of paired variance, while in GPT2-small-Spanish over 50% concentrates in the first 1-2 components, a starker one-dimensional encoding [amini-etal-2023-naturalistic] The paired causal estimator is far more stable across treebanks than a naive correlational one and better reconstructs the true counterfactual distribution [amini-etal-2023-naturalistic]

models: mBERT (BERT-base, Multilingual Cased) · method: Naturalistic Causal Probing (input-level dependency-tree reinflection)
The Representational Geometry of Number (2026)measured

Number-task subspaces are distinct yet linearly inter-transformable

Details

Hu, Niu & Varma analyze number-concept representations (comparison, parity, primality, successor/predecessor) across BERT, GPT-2, Qwen2.5-7B and Qwen2.5-Math-7B at ~75% depth [hu-etal-2026-the-representational-geometry-of-number] Task-specific subspaces are distinct via Procrustes disparity (~0.010 vs a permutation baseline 0.077-0.273) yet linearly inter-transformable (SVCCA canonical correlations ~0.80-0.90) [hu-etal-2026-the-representational-geometry-of-number] The parity axis sits nearly orthogonal to the magnitude axis (88.8 degrees) while the primality axis is only partly decoupled (68.8 degrees) [hu-etal-2026-the-representational-geometry-of-number] Subspace overlap between most task pairs stays below 0.2, except the closely-related successor/predecessor pair (0.36-0.42) [hu-etal-2026-the-representational-geometry-of-number]

models: BERT-base-uncased · method: Generalized Procrustes Analysis (GPA), Singular Vector Canonical Correlation Analysis (SVCCA), Direction Extraction, Linear probing, Geometric analysis
Naturalistic Causal Probing for Morpho-Syntax (2023)measured

Probing accuracy collapses under a naturalistic counterfactual test

Details

Amini et al. compare correlational probing for grammatical gender/number against evaluation on naturalistic counterfactual test pairs [amini-etal-2023-naturalistic] Probes reaching above 90% on the standard split drop to roughly 60% (near chance for a binary property) on dependency-tree-verified counterfactual sentences [amini-etal-2023-naturalistic] Standard probing accuracy on an uncontrolled test distribution can substantially overstate how reliably a property is linearly encoded [amini-etal-2023-naturalistic] This independently confirms the decodability-vs-behavioral-importance caution from amnesic probing and completeness/selectivity frameworks [amini-etal-2023-naturalistic]

models: mBERT (BERT-base, Multilingual Cased) · method: Naturalistic Causal Probing (input-level dependency-tree reinflection)
Transformer-Patcher: One Mistake Worth One Neuron (2023)measured

Transformer-Patcher corrects mistakes by adding one key-value neuron per patch

Details

Huang et al. correct a mistake by appending one key row and one value column to the last FFN layer, adding a key direction that detects the trigger via inner product and a value direction to the output [huang-etal-2023-transformer-patcher] This is the same outer-product pattern as ROME but realized by growing the weight matrices rather than editing existing entries [huang-etal-2023-transformer-patcher] A margin-based memory loss enforces locality across many sequential patches, not any geometric constraint between different patches' key directions [huang-etal-2023-transformer-patcher] It uses rather than finds geometry, correcting thousands of sequential mistakes on BERT-base and BART-base where MEND and KnowledgeEditor collapse after only a few [huang-etal-2023-transformer-patcher]

models: BERT-base-uncased · method: Transformer-Patcher — One Mistake, One Added Neuron

DDPM++ (Score-Based SDE Diffusion)

Diffusion Models already have a Semantic Latent Space (2022)measured

Diffusion h-space supports linear, timestep-consistent edit directions

Details

Kwon, Jeong & Uh identify h-space, the deepest U-Net bottleneck of a frozen diffusion model, as a linear semantic latent space [kwon-etal-2022-asyrp-h-space] Edit directions are found by optimizing a per-timestep perturbation via a directional CLIP loss, with their asymmetric reverse process (Asyrp) confining editing to h_t [kwon-etal-2022-asyrp-h-space] The directions are homogeneous (one image's direction transfers to others), linear (scaling and additive composition work), and robust to matched-magnitude h-space noise [kwon-etal-2022-asyrp-h-space] A single time-invariant global direction closely reproduces the per-timestep editing effect [kwon-etal-2022-asyrp-h-space] An 80-participant study prefers Asyrp over DiffusionCLIP for quality in 98.36% of trials, across three architectures and five datasets with frozen checkpoints [kwon-etal-2022-asyrp-h-space]

models: DDPM++ (CelebA-HQ 256x256), DDPM++ (LSUN-Church 256x256), DDPM++ (LSUN-Bedroom 256x256) · method: CLIP-guided latent direction optimization, Activation Steering (Addition)

Improved DDPM (iDDPM)

Diffusion Models already have a Semantic Latent Space (2022)measured

Diffusion h-space supports linear, timestep-consistent edit directions

Details

Kwon, Jeong & Uh identify h-space, the deepest U-Net bottleneck of a frozen diffusion model, as a linear semantic latent space [kwon-etal-2022-asyrp-h-space] Edit directions are found by optimizing a per-timestep perturbation via a directional CLIP loss, with their asymmetric reverse process (Asyrp) confining editing to h_t [kwon-etal-2022-asyrp-h-space] The directions are homogeneous (one image's direction transfers to others), linear (scaling and additive composition work), and robust to matched-magnitude h-space noise [kwon-etal-2022-asyrp-h-space] A single time-invariant global direction closely reproduces the per-timestep editing effect [kwon-etal-2022-asyrp-h-space] An 80-participant study prefers Asyrp over DiffusionCLIP for quality in 98.36% of trials, across three architectures and five datasets with frozen checkpoints [kwon-etal-2022-asyrp-h-space]

models: iDDPM (AFHQ-Dog 256x256) · method: CLIP-guided latent direction optimization, Activation Steering (Addition)

ADM (Ablated Diffusion Model, Guided Diffusion)

Diffusion Models already have a Semantic Latent Space (2022)measured

Diffusion h-space supports linear, timestep-consistent edit directions

Details

Kwon, Jeong & Uh identify h-space, the deepest U-Net bottleneck of a frozen diffusion model, as a linear semantic latent space [kwon-etal-2022-asyrp-h-space] Edit directions are found by optimizing a per-timestep perturbation via a directional CLIP loss, with their asymmetric reverse process (Asyrp) confining editing to h_t [kwon-etal-2022-asyrp-h-space] The directions are homogeneous (one image's direction transfers to others), linear (scaling and additive composition work), and robust to matched-magnitude h-space noise [kwon-etal-2022-asyrp-h-space] A single time-invariant global direction closely reproduces the per-timestep editing effect [kwon-etal-2022-asyrp-h-space] An 80-participant study prefers Asyrp over DiffusionCLIP for quality in 98.36% of trials, across three architectures and five datasets with frozen checkpoints [kwon-etal-2022-asyrp-h-space]

models: ADM P2-weighted (METFACES 256x256) · method: CLIP-guided latent direction optimization, Activation Steering (Addition)

Vicuna

Linear Representations of Political Perspective Emerge in Large Language Models (2025)measured

Attention-head probes read and steer political ideology

Details

Kim, Evans & Schein fit a ridge-regression probe per attention head in three open chat LLMs to predict 552 U.S. lawmakers' DW-NOMINATE ideology scores [kim-evans-schein-2025-political-perspective] The best single head reaches Spearman rho 0.846-0.861, and an ensemble of the top 32 heads reaches 0.870-0.885, concentrated in middle layers [kim-evans-schein-2025-political-perspective] A nonlinear MLP probe matches the linear probe, supporting a linear-direction characterization [kim-evans-schein-2025-political-perspective] Probes fit on lawmaker ideology transfer zero-shot to predicting 400 news outlets' slant (rho 0.720-0.798) [kim-evans-schein-2025-political-perspective] Adding scaled top-head directions to activations shifts GPT-4o-rated political slant, correlating up to 0.607 with steering magnitude [kim-evans-schein-2025-political-perspective]

models: Vicuna-7B-v1.5 · method: Linear probing, Activation Steering (Addition)
Understanding Jailbreak Success: A Study of Latent Space Dynamics in Large Language Models (2024)measured

A jailbreak direction extracted from one jailbreak method in real Vicuna and Qwen causally transfers to different jailbreak methods

Details

Ball, Kreuter & Rimsky (2024) compute diff-of-means "jailbreak vectors" between successfully-jailbroken and refused activations in real Vicuna-7B/13B and Qwen-14B-Chat, across several structurally distinct jailbreak techniques [ball-kreuter-rimsky-2024-understanding-jailbreak-success-latent-space-dynamics] A vector extracted from one class of jailbreaks causally transfers to mitigate jailbreak effectiveness from semantically-dissimilar jailbreak classes, and effective jailbreaks measurably reduce the model's own internal perception of prompt harmfulness -- evidence for a shared linear mechanism underlying jailbreak susceptibility rather than method-specific mechanisms [ball-kreuter-rimsky-2024-understanding-jailbreak-success-latent-space-dynamics]

models: Vicuna-7B-v1.5 · method:
Representation Engineering: A Top-Down Approach to AI Transparency (2023)measured

Linear Artificial Tomography reads and controls concepts across LLMs

Details

Zou et al. introduce Linear Artificial Tomography (LAT): from contrastive stimulus pairs, take the first principal component of paired activation differences as a concept reading vector [zou-etal-2023] Across LLaMA-2-Chat (7B/13B/70B), Vicuna-13B, Vicuna-33B-Uncensored, and DeBERTa, a single direction classifies and causally controls honesty, ethics, morality, emotion, and bias [zou-etal-2023] LAT recovers truthfulness on DeBERTa more accurately than contrast-consistent search [zou-etal-2023] A harmfulness direction in Vicuna-13B stays a >90% classifier under jailbreaks, and boosting its salience raises harmless-response rates under attack [zou-etal-2023]

models: Vicuna-13B, Vicuna-33B-Uncensored · method: Linear Artificial Tomography (LAT), PCA, Causal interventions (steering), Low-Rank Representation Adaptation (LoRRA)
Unlocking the Future: Look-Ahead Planning Mechanistic Interpretability in LLMs (2024)measured

Linear probes decode future Blocksworld planning steps from one pass

Details

Men et al. train linear probes (versus a nonlinear control) on hidden states of Llama-2-7b-chat and Vicuna-7B fine-tuned on Blocksworld planning [men-etal-2024-unlocking-the-future-look-ahead-planning-mechanistic-interpretability-in-llms] A linear probe predicts the 6th-step-ahead decision from the 1st-step representation at 0.51 accuracy, decaying smoothly with prediction distance [men-etal-2024-unlocking-the-future-look-ahead-planning-mechanistic-interpretability-in-llms] Linear and nonlinear probes track the same decay, evidence the look-ahead information is linearly encoded; MHSA key-masking confirms which attention paths carry it [men-etal-2024-unlocking-the-future-look-ahead-planning-mechanistic-interpretability-in-llms]

models: Vicuna-7B · method: Linear probing, Causal interventions (steering)
Probing then Editing Response Personality of Large Language Models (2025)measured

Personality traits are linearly encoded with a betweenness arrangement

Details

Ju et al. train layer-wise linear probes on 11 open LLMs (including InternLM-7B and InternLM2-7B, base and instruct) to classify which of three personality instructions a response is conditioned on, scored by V-information [ju-etal-2025-personality-editing] V-information rises through layers 5-12 then plateaus, with instruction-tuned models 1-3 points higher than their base counterparts [ju-etal-2025-personality-editing] A t-SNE of Llama-3.1-8B-Instruct shows Neuroticism and Agreeableness distant with Extraversion situated between them, an informal betweenness arrangement [ju-etal-2025-personality-editing] This geometry explains why converting either trait into Extraversion is easier than converting directly between Neuroticism and Agreeableness [ju-etal-2025-personality-editing]

models: Vicuna-7B · method: Linear probing
Refusal in Language Models Is Mediated by a Single Direction (2024), Representation Engineering: A Top-Down Approach to AI Transparency (2023), Programming Refusal with Conditional Activation Steering (2024), Refusal Direction is Universal Across Safety-Aligned Languages (2025)measured

Refusal is mediated by a single direction across models

Details

Arditi et al. show across 13 chat models (5 families, 1.8B-72B) that a single diff-in-means direction between harmful and harmless instructions both erases refusal when ablated and induces it when added, with no offset term [arditi-etal-2024] Zou et al. independently confirm via a LAT harmfulness vector in Vicuna-13B that stays a >90% classifier under jailbreaks and adversarial suffixes [arditi-etal-2024] Lee et al. replicate across nine further chat models and add conditional steering that gates the refusal vector on a condition vector for selective refusal [arditi-etal-2024] Yung et al. average per-model refusal vectors into a universal refusal direction transferring to jailbreak detection ~10% better than single-model baselines [arditi-etal-2024] The account is contested by Marshall et al. (an affine offset is needed) and Wollschlager et al. (a cone of directions), and Yin et al. localize attack-suppressed versus robust heads [arditi-etal-2024]

models: Vicuna-13B · method: Difference-in-means direction, Causal interventions (steering), Activation Steering (Addition), Linear Artificial Tomography (LAT), PCA, Conditional (gated) activation steering
Inference-Time Intervention: Eliciting Truthful Answers from a Language Model (2023)measured

Truthfulness is linearly decodable per attention head, and ITI steers it

Details

Li et al. train a separate linear probe per attention head on TruthfulQA in LLaMA-7B, finding truthfulness strongly decodable in only a sparse subset of heads (best head 83.3%) rather than one whole-residual-stream direction [li-etal-2023] Even within a head the geometry is not perfectly one-dimensional, as a second probe constrained orthogonal to the first still classifies above chance [li-etal-2023] Inference-Time Intervention shifts activations along the winning mass-mean direction (outperforming a probe-weight direction and contrast-consistent search) on the top-K most truthful heads [li-etal-2023] This raises TruthfulQA true*informative substantially (LLaMA-7B 30.5% to 43.5%, Alpaca-7B 32.5% to 65.1%, Vicuna-7B 51.5% to 74.0%) [li-etal-2023]

models: Vicuna-7B · method: Linear probing, Difference-in-means direction, Contrast-Consistent Search (CCS), Activation Steering (Addition), Causal interventions (steering)
Eliciting Latent Predictions from Transformers with the Tuned Lens (2023)measured

The Tuned Lens replaces the logit lens with a trained per-layer affine correction

Details

Belrose et al. replace the logit lens's direct reuse of the final unembedding on intermediate states with a trained per-layer affine translator, distilled via KL divergence to the model's own final output [belrose-etal-2023-tuned-lens] This removes two documented logit-lens failure modes: a systematic 4-5 bit-per-layer vocabulary bias and unreliability from hidden-state covariance drifting across layers [belrose-etal-2023-tuned-lens] A causal-basis extraction finds the lens's most-sensitive directions strongly match the model's real causal directions (Spearman rho=0.89), with none influential to the lens but not the model [belrose-etal-2023-tuned-lens] Tested across GPT-2, GPT-Neo, GPT-NeoX-20B, OPT, BLOOM, and the full Pythia suite, it gives lower perplexity than the logit lens, transfers to a fine-tune (Vicuna-13B from LLaMA-13B), and detects prompt injection at near-perfect AUROC [belrose-etal-2023-tuned-lens]

models: Vicuna-13B · method: Tuned Lens

GloVe

World Properties without World Models: Recovering Spatial and Temporal Structure from Co-occurrence Statistics in Static Word Embeddings (2026)measured

Static embeddings carry the 'world-model' geographic signal

Details

Barenholtz re-tests Gurnee & Tegmark's linear space/time claim on static GloVe and Word2Vec embeddings using the same ridge-regression probes [barenholtz-2026-world-properties-without-world-models-recovering-spatial-and-temporal-structure-from-co-occurrence-statistics-in-static-word-embeddings] Geographic signal is strong (R^2=0.71-0.87 for latitude/longitude), temporal weaker (R^2=0.46-0.52), with elevation/GDP/population as unrecoverable controls [barenholtz-2026-world-properties-without-world-models-recovering-spatial-and-temporal-structure-from-co-occurrence-statistics-in-static-word-embeddings] Ablating PCA-derived country-name and climate-vocabulary subspaces causes large z-scored R^2 drops (up to z=25.9) far exceeding matched random-subspace controls [barenholtz-2026-world-properties-without-world-models-recovering-spatial-and-temporal-structure-from-co-occurrence-statistics-in-static-word-embeddings] Linear-probe recoverability alone therefore cannot distinguish genuine world-model structure from ordinary co-occurrence statistics [barenholtz-2026-world-properties-without-world-models-recovering-spatial-and-temporal-structure-from-co-occurrence-statistics-in-static-word-embeddings]

models: GloVe (Wikipedia + Gigaword, uncased) · method: Linear probing, PCA-derived semantic-subspace ablation, validated against a randomized-subspace control
Semantic Projection Recovers Rich Human Knowledge of Multiple Object Features from Word Embeddings (2022)measured

Semantic projection onto antonym-pole directions recovers human feature ratings

Details

Grand et al. define 52 bipolar feature lines as the vector difference between antonym-pole words in real 300-d GloVe (42B Common Crawl) [grand-etal-2022-semantic-projection] Category members (animals, cities, professions) are scored by their scalar projection onto each line [grand-etal-2022-semantic-projection] Projection scores correlate with human graded ratings at median Pearson r=0.47 (reliability-adjusted median 0.61) [grand-etal-2022-semantic-projection] Single-pole-only controls perform far worse (r=0.18 and r=0), validating the two-pole direction against external ground truth; no causal intervention [grand-etal-2022-semantic-projection]

models: GloVe (42B token Common Crawl, uncased) · method: Semantic projection scoring
OSCaR: Orthogonal Subspace Correction and Rectification of Biases in Word Embeddings (2020)measured

OSCaR's graded rotation preserves information better than projection debiasing

Details

Dev et al. measure that a diff-of-means gender direction and an occupation direction are not orthogonal in GloVe or RoBERTa embedding space [dev-etal-2020-oscar] Standard projective debiasing (hard debiasing, INLP) erases the gender direction from every word, destroying valid associations (an NLI entailment drops 97% to 16%) [dev-etal-2020-oscar] OSCaR instead applies a graded rotation, full at the occupation direction and none at the gender direction, so unrelated words are barely perturbed [dev-etal-2020-oscar] On GloVe it matches the best bias reduction (WEAT 1.768 to 0.235) while scoring far higher on new information-retention metrics (WEAT*, SIRT); on RoBERTa it gives the best bias reduction and retention [dev-etal-2020-oscar]

models: GloVe (840B token Common Crawl) · method: OSCaR (Orthogonal Subspace Correction and Rectification)

word2vec

World Properties without World Models: Recovering Spatial and Temporal Structure from Co-occurrence Statistics in Static Word Embeddings (2026)measured

Static embeddings carry the 'world-model' geographic signal

Details

Barenholtz re-tests Gurnee & Tegmark's linear space/time claim on static GloVe and Word2Vec embeddings using the same ridge-regression probes [barenholtz-2026-world-properties-without-world-models-recovering-spatial-and-temporal-structure-from-co-occurrence-statistics-in-static-word-embeddings] Geographic signal is strong (R^2=0.71-0.87 for latitude/longitude), temporal weaker (R^2=0.46-0.52), with elevation/GDP/population as unrecoverable controls [barenholtz-2026-world-properties-without-world-models-recovering-spatial-and-temporal-structure-from-co-occurrence-statistics-in-static-word-embeddings] Ablating PCA-derived country-name and climate-vocabulary subspaces causes large z-scored R^2 drops (up to z=25.9) far exceeding matched random-subspace controls [barenholtz-2026-world-properties-without-world-models-recovering-spatial-and-temporal-structure-from-co-occurrence-statistics-in-static-word-embeddings] Linear-probe recoverability alone therefore cannot distinguish genuine world-model structure from ordinary co-occurrence statistics [barenholtz-2026-world-properties-without-world-models-recovering-spatial-and-temporal-structure-from-co-occurrence-statistics-in-static-word-embeddings]

models: word2vec (Google News, 300d) · method: Linear probing, PCA-derived semantic-subspace ablation, validated against a randomized-subspace control
Man is to Computer Programmer as Woman is to Homemaker? Debiasing Word Embeddings (2016)measured

A PCA gender direction in word2vec is causally debiasable

Details

Bolukbasi et al. compute a gender direction in word2vec (Google News, 300d) as the top principal component of ten gendered word-pair difference vectors [bolukbasi-etal-2016-debiasing-word-embeddings] Gender-neutral words are linearly separable from gender-definitional words along this direction [bolukbasi-etal-2016-debiasing-word-embeddings] Hard debiasing -- projecting the gender direction out of neutral words plus an equalize step for definitional pairs -- is applied directly to the embedding matrix without retraining [bolukbasi-etal-2016-debiasing-word-embeddings] Crowd and benchmark evaluation show stereotypical analogies (e.g. homemaker for women) are eliminated while semantic task performance is preserved [bolukbasi-etal-2016-debiasing-word-embeddings] Follow-up work notes single-direction debiasing only partially removes bias, since indirect bias re-emerges in higher-order structure [bolukbasi-etal-2016-debiasing-word-embeddings]

models: word2vec (Google News, 300d) · method: PCA, Causal interventions (steering)
Hierarchical Concept Geometry in Language Models Emerges from Word Co-occurrence (2026)measured

Word-cooccurrence eigenvectors form a spectral tree recurring in Gemma

Details

The successive leading eigenvectors of a normalized word co-occurrence (PMI-related) Gram matrix progressively separate taxonomic branches from broad to fine, a coarse-to-fine spectral splitting tree [nava-wyart-2026-hierarchical-concept-geometry-in-language-models-emerges-from-word-cooccurrence] Top-k eigenspace alignment far exceeds a shuffled baseline and fits an exponential decay f(d)=1.967*e^(-1.235*d) in WordNet distance d [nava-wyart-2026-hierarchical-concept-geometry-in-language-models-emerges-from-word-cooccurrence] The same spectral-tree geometry, first validated in word2vec, recurs in Gemma-2B unembeddings and in the mid-layer residual stream of Gemma-2B [nava-wyart-2026-hierarchical-concept-geometry-in-language-models-emerges-from-word-cooccurrence] Parent-to-child innovation vectors are near-orthogonal to the parent (a linear-representation signature); the geometry is derived as a prediction of a co-occurrence model, not tested causally [nava-wyart-2026-hierarchical-concept-geometry-in-language-models-emerges-from-word-cooccurrence]

models: word2vec · method: Spectral analysis of the co-occurrence/PMI matrix, Direction Extraction, Geometric analysis

LLaVA

Head Pursuit: Probing Attention Specialization in Multimodal Transformers (2025)measured

SOMP-selected attention heads causally control target behavior across real unimodal and multimodal transformers

Details

Basile, Maiorca, Doimo, Locatello & Cazzaniga score real Mistral-7B attention heads by Simultaneous Orthogonal Matching Pursuit (SOMP) against unembedding-matrix directions, finding that inverting just 8 heads (0.8% of the total) degrades TriviaQA country-name F1 far more selectively than inverting the same number of random heads or Logit-Lens-selected heads [basile-etal-2025-head-pursuit-probing-attention-specialization-multimodal-transformers] On RealToxicityPrompts and TET toxicity-mitigation benchmarks, suppressing 8/16/32 SOMP-selected heads reduces normalized toxic-generation counts to 0.83/0.67/0.66 (RTP) and 0.83/0.68/0.49 (TET), consistently below Logit-Lens and random-head baselines [basile-etal-2025-head-pursuit-probing-attention-specialization-multimodal-transformers] Applying the same SOMP-based head scoring to real LLaVA-NeXT-7B/13B, Gemma3-12B, and Qwen2.5-VL-7B, inverting the top-32 SOMP heads significantly disrupts image classification accuracy on MNIST, SVHN, GTSRB, EuroSAT, and RESISC45 while 32 random heads have minimal effect, and Jaccard overlap shows related-domain datasets share specialized heads [basile-etal-2025-head-pursuit-probing-attention-specialization-multimodal-transformers] On Flickr30k captioning with LLaVA, inhibiting 16 SOMP-selected heads (alpha=-1) nearly removes attribute keywords (colors, sentiments, quantities) while CIDEr stays above 80% of baseline, and enhancing 32 heads (alpha=5) increases target-concept presence by over 60% in all three attribute categories [basile-etal-2025-head-pursuit-probing-attention-specialization-multimodal-transformers]

models: LLaVA-NeXT-7B, LLaVA-NeXT-13B · method: Simultaneous Orthogonal Matching Pursuit against unembedding directions
Vision Transformers Don't Need Trained Registers (2025)measured

Fewer than 25 register neurons with consistent directions cause ViT outlier tokens

Details

Jiang et al. identify fewer than 10 register neurons in OpenCLIP ViT-B/16 and fewer than 25 in DINOv2-L/14 by ranking neurons on mean activation at outlier-token positions [jiang-etal-2025-vits-dont-need-trained-registers] These neurons' decoder weights consistently peak in specific ambient dimensions across images, a directional claim beyond activation magnitude [jiang-etal-2025-vits-dont-need-trained-registers] Training-free test-time registers built from the discovered directions match trained-register models on object discovery and improve zero-shot segmentation [jiang-etal-2025-vits-dont-need-trained-registers] Register-neuron ablation cuts typographic-attack success on LLaVA-Llama-3-8B from 50.5% to 7.5%, while naive zeroing crashes ImageNet accuracy 71.3% to 55.6% [jiang-etal-2025-vits-dont-need-trained-registers]

models: LLaVA-Llama-3-8B · method: Register-neuron identification, Causal interventions (steering)
Linear Mechanisms for Spatiotemporal Reasoning in Vision Language Models (2026)measured

A contrastive spatial-ID direction is causally bound into object tokens across 11 real VLMs

Details

Kang, Chen, Gkioxari & Perona (2026) define a canonical spatial-ID direction as a difference-of-means over grid position across 11 real VLMs (LLaVA, LLaMA, Qwen, InternVL, Gemma families), well-approximated by a fixed linear map of positional encoding (rank-3 fit, R^2 ~0.85) [kang-etal-2026-linear-mechanisms-for-spatiotemporal-reasoning-in-vlms] Activation-patching 'mirror swap' localizes the effect to object-word tokens at intermediate layers, with a color-swap control showing near-null effect [kang-etal-2026-linear-mechanisms-for-spatiotemporal-reasoning-in-vlms] Directly substituting a target spatial ID into an object token's residual stream shifts ground-truth logit-based belief 64.4% of the way to the target, versus 29.5% for a noise baseline; an analogous temporal-ID binding mechanism extends to video VLMs (LLaVA-Video, VideoLLaMA3, Qwen2.5) [kang-etal-2026-linear-mechanisms-for-spatiotemporal-reasoning-in-vlms]

models: LLaVA-1.5-7B · method: Difference-in-means direction, Activation patching
Towards Understanding Multimodal Fine-Tuning: A Case Study into Spatial Features (2026)measured

SAE decoder directions rotate during real vision-language fine-tuning, and the most-rotated subset is causally load-bearing for spatial reasoning

Details

Naghashyar, Batra, Khakzar, Torr, Clark, Schroeder de Witt & Venhoff warm-start LLaMA-Scope SAEs on LLaVA-More (CLIP ViT-L/14-336 + Llama-3.1-8B) activations, measuring per-feature decoder-direction cosine similarity between the base-LLM SAE and the VLM-adapted SAE [naghashyar-etal-2026-multimodal-fine-tuning-spatial-features] Roughly 5% of over 1M features show strong decoder-direction rotation (bottom-25% cosine) combined with visual responsiveness, and a further firing-frequency-shift criterion isolates a spatial subset validated via attribution patching to attention heads [naghashyar-etal-2026-multimodal-fine-tuning-spatial-features] Causally ablating the top spatial SAE features drops Visual Spatial Reasoning accuracy by 5.85-15.54 points while general VQA accuracy changes by under 1 point, versus near-zero effect for a random-feature control (odds ratios 4.2-9.1 for spatial recruitment) [naghashyar-etal-2026-multimodal-fine-tuning-spatial-features]

models: LLaVA-More (CLIP ViT-L/14-336 + Llama-3.1-8B) · method:

GPT-2

BatchTopK Sparse Autoencoders (2024)measured

BatchTopK relaxes per-sample sparsity to a per-batch budget

Details

Bussmann, Leask & Nanda replace TopK SAE's per-sample k-active constraint with a per-batch budget of n*k activations, so per-sample counts vary while the batch average stays k [bussmann-etal-2024-batchtopk] Single-sample inference is restored via a JumpReLU-style threshold estimated from training batches [bussmann-etal-2024-batchtopk] On GPT-2 Small and Gemma-2-2B, BatchTopK consistently beats TopK on reconstruction and cross-entropy across widths and sparsities [bussmann-etal-2024-batchtopk] Against JumpReLU the result is scale-dependent: BatchTopK wins on GPT-2 at every sparsity but on Gemma-2-2B only at k=16 for CE degradation [bussmann-etal-2024-batchtopk] Unlike JumpReLU, BatchTopK sets average sparsity directly as a hyperparameter rather than via an L0-penalty sweep [bussmann-etal-2024-batchtopk]

models: GPT-2 Small · method: BatchTopK Sparse Autoencoders
Does Localization Inform Editing? Surprising Differences in Causality-Based Localization vs. Knowledge Editing in Language Models (2023)measured

Causal-tracing localization does not predict the best edit layer

Details

Hase et al. sweep the target layer for ROME/MEMIT rank-one edits in GPT-J and GPT2-XL and regress edit success on edit layer and causal-tracing effect [hase-etal-2023-does-localization-inform-editing] For ROME on GPT-J, edit layer alone explains 94.7% of edit-success variance while the tracing effect explains 1.6% and adds only 0.1% [hase-etal-2023-does-localization-inform-editing] The raw correlation between edit success and tracing effect is slightly negative (rho=-0.13, p<1e-3), opposite the localize-then-edit assumption [hase-etal-2023-does-localization-inform-editing] A "Fact Forcing" variant reusing tracing's noised-subject input shows a small positive tracing contribution (up to 2.7%), isolated to the shared input format [hase-etal-2023-does-localization-inform-editing] This critiques the layer-selection heuristic, not the rank-one edit mechanism's own geometry [hase-etal-2023-does-localization-inform-editing]

models: GPT-2 XL · method: Causal Tracing, Rank-One Model Editing (ROME) / Linear Associative Memory Editing
The Confidence Manifold: Geometric Structure of Correctness Representations in Language Models (2026)measured

Answer-correctness sits in a 3-8D linear subspace, causally steerable

Details

Cho et al. study self-assessed correctness on TruthfulQA pairs across 9 models spanning 5 families, finding the discriminative signal is a low-dimensional linear subspace, not a curved manifold [cho-etal-2026-confidence-manifold] A PLS sweep peaks at 3-8 dimensions (e.g. peak AUC 0.90 at layer 23, dim 5, Mistral-7B), and nonlinear classifiers give no gain over a linear boundary [cho-etal-2026-confidence-manifold] Correct/incorrect classes form roughly Gaussian clusters separated by a mean shift: a two-mean centroid detector matches the linear probe (0.90 vs 0.89 AUC), robust with 25 examples/class [cho-etal-2026-confidence-manifold] This 3-8D discriminative subspace is lower-dimensional than the representation's 8-12D intrinsic dimension at the same layers [cho-etal-2026-confidence-manifold] Adding the learned direction shifts downstream error rate up to 10.9 points dose-dependently, while random and orthogonal controls have no reliable effect [cho-etal-2026-confidence-manifold]

models: GPT-2 Small, GPT-2 Medium, GPT-2 Large · method: Linear probing, Partial Least Squares (PLS), Difference-in-means direction, Activation Steering (Addition)
Transformer Feed-Forward Layers Build Predictions by Promoting Concepts in the Vocabulary Space (2022)measured

FFN value-vector directions causally suppress toxicity

Details

Geva et al. decompose each FFN layer's additive update to the output distribution into per-parameter sub-updates, each a value-vector direction projected through the unembedding [geva-etal-2022-ffn-promoting-concepts] In GPT-2, suppressing value-vector directions that promote toxic-concept tokens causally reduces generated toxicity by roughly 50% [geva-etal-2022-ffn-promoting-concepts] An early-exit rule built on the same decomposition saves about 20% of inference compute on average [geva-etal-2022-ffn-promoting-concepts] This is a geometry-tied causal intervention on a specific vocabulary-space direction, refining the FFN key-value-memory framing [geva-etal-2022-ffn-promoting-concepts]

models: GPT-2 Small · method: Logit Lens
Linearity of Relation Decoding in Transformer Language Models (2023)measured

Linear relational embeddings are faithful and causal for half of relations

Details

Hernandez et al. approximate a transformer's relation-decoding function by a first-order Taylor expansion LRE(s)=beta*W_r*s+b_r, with W_r the mean Jacobian estimated from 5-8 examples [hernandez-etal-2023-lre] It is validated by faithfulness (decoded top token matches the model's own) and causality (low-rank pseudoinverse inversion redirects the real output) [hernandez-etal-2023-lre] Across 47 relations on GPT-J, GPT-2-XL, and LLaMA-13B, faithfulness exceeds 60% for nearly half the relations, with strong faithfulness-causality correlation (R=0.84 on GPT-J) [hernandez-etal-2023-lre] Relations like "company CEO" (0.06/0.31) fail almost completely despite correct model behavior, so linear relation decoding is real but heterogeneous across relation types [hernandez-etal-2023-lre]

models: GPT-2 XL · method: Linear Relational Embedding (LRE)
Language Models Implement Simple Word2Vec-style Vector Arithmetic (2024)measured

LLMs implement word2vec-style relational vector offsets, causally patchable

Details

Merullo et al. show many one-to-one relations (e.g. country-capital) that LLMs solve in-context are implemented as a simple additive vector offset computed by a mid-network FFN update [merullo-etal-2024-language-models-implement-word2vec-style-vector-arithmetic] This is directly analogous to word2vec-style vector arithmetic but discovered inside real pretrained transformer activations rather than static embeddings [merullo-etal-2024-language-models-implement-word2vec-style-vector-arithmetic] Extracting the offset from one example context and patching it into another causally forces the model to output the corresponding relation's correct answer for the new subject [merullo-etal-2024-language-models-implement-word2vec-style-vector-arithmetic] The mechanism is replicated across GPT-J, GPT-2 (small through XL), and BLOOM (up to 176B parameters) [merullo-etal-2024-language-models-implement-word2vec-style-vector-arithmetic]

models: GPT-2 Small, GPT-2 Medium, GPT-2 Large, GPT-2 XL · method: Vector-offset analogy test, Causal interventions (steering)
The Representational Geometry of Number (2026)measured

Number-task subspaces are distinct yet linearly inter-transformable

Details

Hu, Niu & Varma analyze number-concept representations (comparison, parity, primality, successor/predecessor) across BERT, GPT-2, Qwen2.5-7B and Qwen2.5-Math-7B at ~75% depth [hu-etal-2026-the-representational-geometry-of-number] Task-specific subspaces are distinct via Procrustes disparity (~0.010 vs a permutation baseline 0.077-0.273) yet linearly inter-transformable (SVCCA canonical correlations ~0.80-0.90) [hu-etal-2026-the-representational-geometry-of-number] The parity axis sits nearly orthogonal to the magnitude axis (88.8 degrees) while the primality axis is only partly decoupled (68.8 degrees) [hu-etal-2026-the-representational-geometry-of-number] Subspace overlap between most task pairs stays below 0.2, except the closely-related successor/predecessor pair (0.36-0.42) [hu-etal-2026-the-representational-geometry-of-number]

models: GPT-2 Small · method: Generalized Procrustes Analysis (GPA), Singular Vector Canonical Correlation Analysis (SVCCA), Direction Extraction, Linear probing, Geometric analysis
Locating and Editing Factual Associations in GPT (2022)measured

ROME localizes facts to a two-peak causal site and edits via a rank-one direction

Details

Meng et al. use causal tracing across 1,000 factual statements on GPT-2 XL to map a strongly causal early MLP site at the subject's last token, distinct from a late attention site at the final token [meng-etal-2022-rome] A severed-module variant confirms the early site depends specifically on MLP computation [meng-etal-2022-rome] They model the responsible MLP's down-projection as a linear associative memory and derive a closed-form rank-one update inserting one key-value association as an outer-product direction [meng-etal-2022-rome] It uses rather than finds geometry (the value is optimized per edit), reaching CounterFact composites of 89.2 (GPT-2 XL) and 91.5 (GPT-J) with high neighborhood specificity [meng-etal-2022-rome]

models: GPT-2 XL · method: Causal Tracing, Rank-One Model Editing (ROME) / Linear Associative Memory Editing
Linear Representations of Sentiment in Large Language Models (2023)measured

Sentiment is a single convergent linear direction, causally validated

Details

Tigges et al. extract a candidate sentiment direction in GPT2-small and Pythia (1.4B, 2.8B) via five techniques (mean difference, k-means, logistic regression, PCA, and DAS) [tigges-etal-2023-linear-sentiment] Pairwise cosine similarities between the five directions reach 72.6-99.1% (random baseline 0-2.4%), evidence they locate the same underlying direction [tigges-etal-2023-linear-sentiment] Directional ablation of the DAS direction on SST drops accuracy 100% to 62% (a 71% logit-difference reduction), and steering at coefficient -17 makes GPT2-small completions extremely negative [tigges-etal-2023-linear-sentiment] Single-scalar projection classifies token sentiment at 78-89%, and increasing the DAS subspace dimension does not improve OOD generalization, so one-dimensionality is treated as a supported but not final hypothesis [tigges-etal-2023-linear-sentiment]

models: GPT-2 Small · method: Difference-in-means direction, Linear probing, PCA, Distributed Alignment Search (DAS), Activation patching, Activation Steering (Addition)
Linear Representations of Sentiment in Large Language Models (2023)measured

A summarization motif routes sentiment through non-valenced tokens

Details

Using iterative path-patching and value-weighted attention-pattern analysis, Tigges et al. trace how sentiment information flows through GPT2-small (and Pythia-2.8B) [tigges-etal-2023-linear-sentiment] A "summarization motif" accumulates sentiment onto intermediate non-valenced tokens such as commas and periods rather than only onto the sentiment-bearing words [tigges-etal-2023-linear-sentiment] These summarization points act as intermediate stores that later attention heads read from to produce the final sentiment prediction [tigges-etal-2023-linear-sentiment]

models: GPT-2 Small · method: Activation patching, Direct Logit Attribution (DLA)
Large Language Models Encode Semantics and Alignment in Linearly Separable Representations (2025)measured

Topic subspaces sharpen with depth; a centroid direction induces CoT

Details

Saglam et al. study 11 autoregressive models on six arXiv topics, measuring per-layer hard-margin SVM separability and PCA intrinsic dimensionality [saglam-etal-2025-linear-subspaces] Under 10% of principal components explain nearly all variance, so topics occupy a near-affine low-dimensional subspace; separability rises toward final layers, reaching 100% SVM accuracy on all topic pairs in the largest models [saglam-etal-2025-linear-subspaces] A chain-of-thought framing distinction becomes linearly separable even more sharply than topic identity [saglam-etal-2025-linear-subspaces] Adding the CoT-vs-non-CoT centroid-difference direction to the hidden state reliably induces CoT-style responses (flagged preliminary), and a lightweight MLP guardrail built on the finding halves harmful responses [saglam-etal-2025-linear-subspaces]

models: GPT-2 XL, GPT-2 Large, GPT-2 Medium, GPT-2 Small · method: Linear probing, PCA, Activation Steering (Addition)
Scaling and Evaluating Sparse Autoencoders (2024)measured

TopK SAEs give clean scaling laws and mitigate dead latents

Details

Gao et al. (OpenAI) replace an SAE's ReLU+L1 objective with a hard TopK activation trained on reconstruction alone, removing L1's shrinkage bias and fixing L0=k per token [gao-etal-2024-scaling-saes] An auxiliary loss reconstructing residual error from only currently-dead latents holds the dead-latent fraction to ~7% even in a 16M-latent autoencoder [gao-etal-2024-scaling-saes] Trained on GPT-4 activations for 40B tokens, a 16M-latent SAE costs only ~10% of GPT-4 pretraining-equivalent loss when spliced in, and reconstruction loss follows a clean joint scaling law in size and sparsity [gao-etal-2024-scaling-saes] Three feature-quality metrics (probe recovery, N2G explainability, ablation sparsity) all improve with size, and TopK SAEs beat ReLU baselines on every one [gao-etal-2024-scaling-saes]

models: GPT-2 Small · method: TopK (k-sparse) Sparse Autoencoders
Transcoders Find Interpretable LLM Feature Circuits (2024)measured

Transcoders linearize MLP circuit attribution and match SAE quality

Details

Dunefsky et al. introduce transcoders: SAE-shaped networks trained to reconstruct an MLP sublayer's output from its input rather than the MLP's own activations [dunefsky-etal-2024-transcoders] Circuit attribution then factorizes exactly into an input-dependent activation term times a fixed decoder-encoder dot product, needing no gradient/Taylor linearization of the MLP [dunefsky-etal-2024-transcoders] On GPT2-small, Pythia-410M, and Pythia-1.4B, transcoders sit on reconstruction/sparsity Pareto frontiers equal to or better than SAEs, with the gap widening at larger scale [dunefsky-etal-2024-transcoders] Applying the exact-factorization method to GPT2-small's greater-than circuit surfaces a finer feature-level account and isolates one anomalous flat direct-logit-attribution feature [dunefsky-etal-2024-transcoders]

models: GPT-2 Small · method: Transcoders

PGGAN (Progressive GAN)

GAN Dissection: Visualizing and Understanding Generative Adversarial Networks (2019)measured

GAN units align with object masks and causally control objects

Details

Bau et al. identify interpretable units in a real trained Progressive GAN by scoring each channel's spatial activation against object-segmentation masks [bau-etal-2019-gan-dissection] Causally ablating or forcibly activating identified units reliably removes or adds the corresponding object (e.g. a tree, a door) in generated images [bau-etal-2019-gan-dissection] The method also locates and removes artifact-causing units and enables interactive scene editing [bau-etal-2019-gan-dissection] Identification is a quantified geometric measurement and the ablation/insertion is a direct causal intervention with measured image effects [bau-etal-2019-gan-dissection]

models: Progressive GAN (trained on LSUN scene categories: bedroom, church, kitchen, etc.) · method: Segmentation-scored channel identification + causal ablation/insertion
InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs (2020)measured

InterFaceGAN SVM boundary normals are linear, editable, and partially entangled

Details

Shen et al. train a linear SVM per attribute (gender, age, pose, smile, eyeglasses) in PGGAN and StyleGAN latent spaces, taking each boundary normal as that attribute's direction [shen-etal-2020-interfacegan] A proven linear relation ties latent distance-to-boundary to the generated image's semantic score, with PGGAN boundaries over 95% validation accuracy [shen-etal-2020-interfacegan] Boundary-normal cosine similarity measures entanglement: gender/age/eyeglasses are correlated while pose/smile are near-independent, and StyleGAN's W space is far more disentangled than Z [shen-etal-2020-interfacegan] Conditional manipulation, projecting out a correlated direction, removes leakage while preserving the target edit [shen-etal-2020-interfacegan] Layer-wise ablation localizes each attribute to a distinct subset of StyleGAN's style layers, and W-space edits best preserve identity [shen-etal-2020-interfacegan]

models: PGGAN (trained on CelebA-HQ, 1024x1024) · method: Linear probing, Conditional manipulation via boundary orthogonalization, Causal interventions (steering)

Bayesian Wind Tunnel Transformer

The Bayesian Geometry of Transformer Attention (2026)measured

Toy transformers build an orthogonal hypothesis frame for Bayesian updating

Details

Agarwal et al. train small from-scratch transformers (a 6-layer/6-head/d192 bijection model, a 9-layer/8-head/d256 HMM model) on synthetic tasks with analytically known Bayesian posteriors [agarwal-etal-2026] The transformers match the Bayes-optimal predictive entropy to 10^-3-10^-4 bits, while capacity-matched MLPs fail by orders of magnitude [agarwal-etal-2026] Layer-0 attention keys form a near-orthogonal "hypothesis frame" (mean off-diagonal key cosine 0.052+/-0.004 vs 0.082+/-0.003 random), built by a single identifiable head [agarwal-etal-2026] Query-key similarity sharpens with depth onto still-feasible hypotheses, mirroring Bayesian conditioning as evidence accumulates [agarwal-etal-2026] PCA of attention-output values reveals a smooth low-dimensional entropy-tracking structure (a curved 1-D entropy manifold) that keeps refining after the key frame stabilizes [agarwal-etal-2026] Layer/head ablations and length-generalization support this as a reused computation; across architectures Transformers realize all three inference primitives, Mamba two, LSTM one, MLP none [agarwal-etal-2026]

models: Bayesian Wind Tunnel Transformer (bijection task, 6 layers, 6 heads, d_model=192), Bayesian Wind Tunnel Transformer (HMM filtering task, 9 layers, 8 heads, d_model=256) · method: Analytical derivation, PCA, Linear probing, Geometric analysis, Causal interventions (steering)

β-VAE

Understanding disentangling in β-VAE (2018)measured

Beta-VAE axis-alignment emerges from information-bottleneck pressure

Details

Burgess et al. give a rate-distortion account of why beta-VAE's latent code becomes axis-aligned with a dataset's independent generative factors [burgess-etal-2018-understanding-disentangling] The beta-weighted KL term upper-bounds per-channel capacity, forcing data locality, while diagonal-covariance allocation drives factors onto separate axes [burgess-etal-2018-understanding-disentangling] As target capacity C rises from 0.5 to 25 nats, per-factor KL is allocated in a fixed order (position, then scale, shape, rotation) [burgess-etal-2018-understanding-disentangling] Latent traversals causally isolate each top-KL dimension's effect to exactly one factor, while lowest-KL dimensions have no effect [burgess-etal-2018-understanding-disentangling] A standard VAE (beta=1) shows the same local smoothness but fragmented, non-axis-aligned factor coding [burgess-etal-2018-understanding-disentangling]

models: β-VAE (convolutional encoder/decoder, dSprites/colored-dSprites/CelebA/3D-Chairs) · method: Analytical derivation, Causal interventions (steering)

ESM-2

Viral Proteins Reveal Geometry of Protein Language Models (2026)measured

A dominant unsupervised PCA axis in real ESM protein language models orders sequences along a cellular-to-viral nativeness continuum

Details

Bigot, Bhasin, Park, Shakhnovich & Wang (2026) compute the first principal component of pooled per-residue embeddings from real ESM2 (8M-15B), ESMC, and ESM3 checkpoints over cellular, viral, and shuffled-control sequences, finding PC1 correlates strongly with masked-reconstruction perplexity (Spearman rho up to 0.961, explaining 54-73% of variance) [bigot-etal-2026-viral-proteins-reveal-geometry-of-protein-language-models] An L2-regularized logistic-regression probe on the same embeddings reaches AUC-ROC 0.97-1.00 for larger models -- an unsupervised dominant linear axis plus linear-probe confirmation, purely observational (no causal validation) [bigot-etal-2026-viral-proteins-reveal-geometry-of-protein-language-models]

models: ESM-2 (650M) · method:
Viral Proteins Reveal Geometry of Protein Language Models (2026)measured

A protein-LM nativeness axis orders sequences; viral signal stays separable

Details

Bigot et al. run PCA on mean-pooled embeddings from five protein language models (ESM2, ESM3, ESMC, ProGen2, EvoDiff), finding a dominant nativeness axis (PC1, 54.3-73.1% variance) correlating with masked-reconstruction perplexity (rho=0.83-0.96) [bigot-etal-2026-viral-proteins-reveal-geometry-of-protein-language-models] The axis replicates within each of 10 biological groups via within-group PCA refits [bigot-etal-2026-viral-proteins-reveal-geometry-of-protein-language-models] Linear probes classify viral versus cellular proteins at AUC 0.97-1.00 even under homology-controlled splits, with 88.3-96.7% true-positive rate at 1% false-positive rate versus 26.9-39.2% for a perplexity-only classifier [bigot-etal-2026-viral-proteins-reveal-geometry-of-protein-language-models]

models: ESM-2 (650M), ESM-2 (15B) · method: PCA, Linear probing
Towards Interpretable Protein Structure Prediction with Sparse Autoencoders (2025)measured

Steering an ESM2-3B hydrophobicity SAE direction shifts predicted solvent accessibility

Details

Parsan et al. scale Matryoshka SAEs to ESM2-3B and identify a single decoder feature in a layer-36 SAE that correlates strongly with residue hydrophobicity [parsan-yang-yang-2025-protein-sae-steering] Adding the unit-norm decoder vector (alpha=-0.275) to the hidden state raises ESMFold's predicted myoglobin solvent-accessible surface area 31.5% with modest structural disruption (RMSD 2.76 Angstrom) [parsan-yang-yang-2025-protein-sae-steering] The effect persists even when the correct input sequence is provided, so it acts on the structure module itself rather than via an altered predicted sequence [parsan-yang-yang-2025-protein-sae-steering] Random-direction steering gives minimal response and a strength sweep shows a monotonic dose-response (GRAVY -0.36 to +1.53) [parsan-yang-yang-2025-protein-sae-steering]

models: ESM-2 (3B) · method: Matryoshka Sparse Autoencoders, Activation Steering (Addition)
Sparse Autoencoders for Low-N Protein Function Prediction and Design (2025)measured

TopK sparse-autoencoder features on a real fine-tuned ESM2 align with real protein structural sites, and steering them causally improves in-silico protein design

Details

Tsui, Talreja & Aghazadeh (2025) train a TopK sparse autoencoder (d=4096, k=128) on layer-24 embeddings of real ESM2-650M, LoRA-fine-tuned per assay on MSA sequences, and show the resulting sparse latents align with real biological structure (active-site residues, C-terminus, allosteric/binding/epistatic sites mapped onto AlphaFold3 structures); the top 5% of SAE probe weights (by magnitude) explain 37-38% of fitness-prediction variance versus 25-28% for raw ESM-layer/logit weights [tsui-talreja-aghazadeh-2025-sparse-autoencoders-for-low-n-protein-function-prediction-and-design] From as few as N=24 labeled sequences, SAE-based linear probes outperform raw-ESM2 baselines in 58-69% of extrapolation tasks across six DMS assays from ProteinGym [tsui-talreja-aghazadeh-2025-sparse-autoencoders-for-low-n-protein-function-prediction-and-design] Causally, amplifying predictive SAE latents (identified via probe-weight magnitude) and decoding back through the SAE and remaining ESM2 layers ("feature steering") generates new protein variants that outperform ESM2-based design in 88% of metric-by-assay combinations, producing the single best-fitness variant in 5/6 DMS assays; validation is in-silico (an MLP fitness proxy, plus one assay against ground-truth combinatorial fitness), not wet-lab [tsui-talreja-aghazadeh-2025-sparse-autoencoders-for-low-n-protein-function-prediction-and-design]

models: ESM-2 (650M) · method: TopK (k-sparse) Sparse Autoencoders, Linear probing

ESM3

Viral Proteins Reveal Geometry of Protein Language Models (2026)measured

A dominant unsupervised PCA axis in real ESM protein language models orders sequences along a cellular-to-viral nativeness continuum

Details

Bigot, Bhasin, Park, Shakhnovich & Wang (2026) compute the first principal component of pooled per-residue embeddings from real ESM2 (8M-15B), ESMC, and ESM3 checkpoints over cellular, viral, and shuffled-control sequences, finding PC1 correlates strongly with masked-reconstruction perplexity (Spearman rho up to 0.961, explaining 54-73% of variance) [bigot-etal-2026-viral-proteins-reveal-geometry-of-protein-language-models] An L2-regularized logistic-regression probe on the same embeddings reaches AUC-ROC 0.97-1.00 for larger models -- an unsupervised dominant linear axis plus linear-probe confirmation, purely observational (no causal validation) [bigot-etal-2026-viral-proteins-reveal-geometry-of-protein-language-models]

models: ESM3 (98B, LARGE) · method:
Viral Proteins Reveal Geometry of Protein Language Models (2026)measured

A protein-LM nativeness axis orders sequences; viral signal stays separable

Details

Bigot et al. run PCA on mean-pooled embeddings from five protein language models (ESM2, ESM3, ESMC, ProGen2, EvoDiff), finding a dominant nativeness axis (PC1, 54.3-73.1% variance) correlating with masked-reconstruction perplexity (rho=0.83-0.96) [bigot-etal-2026-viral-proteins-reveal-geometry-of-protein-language-models] The axis replicates within each of 10 biological groups via within-group PCA refits [bigot-etal-2026-viral-proteins-reveal-geometry-of-protein-language-models] Linear probes classify viral versus cellular proteins at AUC 0.97-1.00 even under homology-controlled splits, with 88.3-96.7% true-positive rate at 1% false-positive rate versus 26.9-39.2% for a perplexity-only classifier [bigot-etal-2026-viral-proteins-reveal-geometry-of-protein-language-models]

models: ESM3 (1.4B, OPEN), ESM3 (98B, LARGE) · method: PCA, Linear probing

ESM Cambrian (ESMC)

Viral Proteins Reveal Geometry of Protein Language Models (2026)measured

A protein-LM nativeness axis orders sequences; viral signal stays separable

Details

Bigot et al. run PCA on mean-pooled embeddings from five protein language models (ESM2, ESM3, ESMC, ProGen2, EvoDiff), finding a dominant nativeness axis (PC1, 54.3-73.1% variance) correlating with masked-reconstruction perplexity (rho=0.83-0.96) [bigot-etal-2026-viral-proteins-reveal-geometry-of-protein-language-models] The axis replicates within each of 10 biological groups via within-group PCA refits [bigot-etal-2026-viral-proteins-reveal-geometry-of-protein-language-models] Linear probes classify viral versus cellular proteins at AUC 0.97-1.00 even under homology-controlled splits, with 88.3-96.7% true-positive rate at 1% false-positive rate versus 26.9-39.2% for a perplexity-only classifier [bigot-etal-2026-viral-proteins-reveal-geometry-of-protein-language-models]

models: ESMC (600M), ESMC (6B) · method: PCA, Linear probing

ProGen2

Viral Proteins Reveal Geometry of Protein Language Models (2026)measured

A protein-LM nativeness axis orders sequences; viral signal stays separable

Details

Bigot et al. run PCA on mean-pooled embeddings from five protein language models (ESM2, ESM3, ESMC, ProGen2, EvoDiff), finding a dominant nativeness axis (PC1, 54.3-73.1% variance) correlating with masked-reconstruction perplexity (rho=0.83-0.96) [bigot-etal-2026-viral-proteins-reveal-geometry-of-protein-language-models] The axis replicates within each of 10 biological groups via within-group PCA refits [bigot-etal-2026-viral-proteins-reveal-geometry-of-protein-language-models] Linear probes classify viral versus cellular proteins at AUC 0.97-1.00 even under homology-controlled splits, with 88.3-96.7% true-positive rate at 1% false-positive rate versus 26.9-39.2% for a perplexity-only classifier [bigot-etal-2026-viral-proteins-reveal-geometry-of-protein-language-models]

models: ProGen2-base · method: PCA, Linear probing

EvoDiff

Viral Proteins Reveal Geometry of Protein Language Models (2026)measured

A protein-LM nativeness axis orders sequences; viral signal stays separable

Details

Bigot et al. run PCA on mean-pooled embeddings from five protein language models (ESM2, ESM3, ESMC, ProGen2, EvoDiff), finding a dominant nativeness axis (PC1, 54.3-73.1% variance) correlating with masked-reconstruction perplexity (rho=0.83-0.96) [bigot-etal-2026-viral-proteins-reveal-geometry-of-protein-language-models] The axis replicates within each of 10 biological groups via within-group PCA refits [bigot-etal-2026-viral-proteins-reveal-geometry-of-protein-language-models] Linear probes classify viral versus cellular proteins at AUC 0.97-1.00 even under homology-controlled splits, with 88.3-96.7% true-positive rate at 1% false-positive rate versus 26.9-39.2% for a perplexity-only classifier [bigot-etal-2026-viral-proteins-reveal-geometry-of-protein-language-models]

models: EvoDiff OA-DM · method: PCA, Linear probing

TabPFN

A Mechanistic Study of Tabular Foundation Models (2026)measured

TabICLv2 predicts by a literal class centroid; TabPFNv2 votes

Details

Bilos et al. find TabICLv2's prediction is reproduced almost exactly by a parameter-free per-class mean (centroid) readout (0.864 native vs 0.854 prototype-only) [bilos-etal-2026-mechanistic-study-tabular-foundation-models] TabPFNv2 and Mitra instead implement an attention-weighted vote, and replacing TabPFNv2's layer-9 attention with a uniform pattern collapses its accuracy from 0.87 to 0.49 [bilos-etal-2026-mechanistic-study-tabular-foundation-models] Transplanting the centroid rule onto the wrong backbone drops accuracy 33.1pp (onto TabPFNv2) and 9.0pp (onto Mitra), while transplanting the vote rule onto TabICLv2 drops it 39.5pp [bilos-etal-2026-mechanistic-study-tabular-foundation-models] Each backbone's own representation geometry is thus specifically required by its native readout rule [bilos-etal-2026-mechanistic-study-tabular-foundation-models] Zeroing TabPFNv2's positional weight matrix, or removing TabICLv2's RoPE, grants exact permutation invariance at no accuracy cost [bilos-etal-2026-mechanistic-study-tabular-foundation-models]

models: TabPFN v2 (tabular in-context-learning foundation model) · method: Difference-in-means direction, Causal interventions (steering)

Mitra

A Mechanistic Study of Tabular Foundation Models (2026)measured

TabICLv2 predicts by a literal class centroid; TabPFNv2 votes

Details

Bilos et al. find TabICLv2's prediction is reproduced almost exactly by a parameter-free per-class mean (centroid) readout (0.864 native vs 0.854 prototype-only) [bilos-etal-2026-mechanistic-study-tabular-foundation-models] TabPFNv2 and Mitra instead implement an attention-weighted vote, and replacing TabPFNv2's layer-9 attention with a uniform pattern collapses its accuracy from 0.87 to 0.49 [bilos-etal-2026-mechanistic-study-tabular-foundation-models] Transplanting the centroid rule onto the wrong backbone drops accuracy 33.1pp (onto TabPFNv2) and 9.0pp (onto Mitra), while transplanting the vote rule onto TabICLv2 drops it 39.5pp [bilos-etal-2026-mechanistic-study-tabular-foundation-models] Each backbone's own representation geometry is thus specifically required by its native readout rule [bilos-etal-2026-mechanistic-study-tabular-foundation-models] Zeroing TabPFNv2's positional weight matrix, or removing TabICLv2's RoPE, grants exact permutation invariance at no accuracy cost [bilos-etal-2026-mechanistic-study-tabular-foundation-models]

models: Mitra · method: Difference-in-means direction, Causal interventions (steering)

TabICL

A Mechanistic Study of Tabular Foundation Models (2026)measured

TabICLv2 predicts by a literal class centroid; TabPFNv2 votes

Details

Bilos et al. find TabICLv2's prediction is reproduced almost exactly by a parameter-free per-class mean (centroid) readout (0.864 native vs 0.854 prototype-only) [bilos-etal-2026-mechanistic-study-tabular-foundation-models] TabPFNv2 and Mitra instead implement an attention-weighted vote, and replacing TabPFNv2's layer-9 attention with a uniform pattern collapses its accuracy from 0.87 to 0.49 [bilos-etal-2026-mechanistic-study-tabular-foundation-models] Transplanting the centroid rule onto the wrong backbone drops accuracy 33.1pp (onto TabPFNv2) and 9.0pp (onto Mitra), while transplanting the vote rule onto TabICLv2 drops it 39.5pp [bilos-etal-2026-mechanistic-study-tabular-foundation-models] Each backbone's own representation geometry is thus specifically required by its native readout rule [bilos-etal-2026-mechanistic-study-tabular-foundation-models] Zeroing TabPFNv2's positional weight matrix, or removing TabICLv2's RoPE, grants exact permutation invariance at no accuracy cost [bilos-etal-2026-mechanistic-study-tabular-foundation-models]

models: TabICLv2 · method: Difference-in-means direction, Causal interventions (steering)

Stable Diffusion

SAEmnesia: Erasing Concepts in Diffusion Models with Supervised Sparse Autoencoders (2025)measured

A supervised SAE gives one latent per concept for erasure

Details

Cassano et al. train a supervised TopK sparse autoencoder on Stable Diffusion v1.5's up.1.1 cross-attention activations, using a concept-vs-non-concept score to force a one-to-one concept-to-latent mapping [cassano-etal-2025-saemnesia-erasing-concepts-in-diffusion-models-with-supervised-sparse-autoencoders] Erasing a concept reduces to steering that single SAE latent direction [cassano-etal-2025-saemnesia-erasing-concepts-in-diffusion-models-with-supervised-sparse-autoencoders] On UnlearnCanvas this improves 9.2% over the prior SAE-based SOTA (SAeUron) [cassano-etal-2025-saemnesia-erasing-concepts-in-diffusion-models-with-supervised-sparse-autoencoders] On 9-object sequential unlearning accuracy improves 28.4 points and hyperparameter search is cut 96.7%, with nudity removal on I2P also demonstrated [cassano-etal-2025-saemnesia-erasing-concepts-in-diffusion-models-with-supervised-sparse-autoencoders]

models: Stable Diffusion v1.5 · method: Sparse Autoencoders (SAE), Activation Steering (Addition)
Concept Sliders: LoRA Adaptors for Precise Control in Diffusion Models (2023)measured

Concept Sliders are LoRA weight-space directions for diffusion control

Details

Gandikota et al. train one low-rank LoRA adaptor per concept on Stable Diffusion XL and SD v1.4 via a guided-score objective from contrastive prompt pairs [gandikota-etal-2023-concept-sliders-lora] An inference-time scaling factor continuously modulates edit strength without retraining, and image-pair supervision handles hard-to-describe visual concepts [gandikota-etal-2023-concept-sliders-lora] An ablation shows both components are causally necessary: removing disentanglement raises interference 0.10->0.36; removing the low-rank constraint raises it to 0.19 and doubles LPIPS [gandikota-etal-2023-concept-sliders-lora] Sliders beat Prompt-to-Prompt and composition baselines on edit strength and structural preservation, and over 50 sliders compose without quality loss [gandikota-etal-2023-concept-sliders-lora]

models: Stable Diffusion XL, Stable Diffusion v1.4 · method: LoRA-trained direction realization
Understanding the Latent Space of Diffusion Models through the Lens of Riemannian Geometry (2023)measured

A pullback Riemannian metric yields diffusion tangent-space directions

Details

Park et al. reframe the diffusion U-Net encoder's Jacobian as a pullback Riemannian metric on the latent x_t, whose eigenvectors form a local tangent basis [park-etal-2023-riemannian-diffusion-pullback] A power-spectral analysis of the tangent basis shifts from low- to high-frequency as t goes from T to 0, confirming coarse-to-fine generation [park-etal-2023-riemannian-diffusion-pullback] Grassmannian geodesic distance between samples' tangent spaces increases monotonically as t decreases, fastest for visually complex datasets [park-etal-2023-riemannian-diffusion-pullback] For Stable Diffusion v2.1, tangent-space geodesic distance negatively correlates with CLIP prompt similarity, collapsing below roughly t=0.7T [park-etal-2023-riemannian-diffusion-pullback] Single-timestep x-space editing is introduced as a causal application but validated only qualitatively [park-etal-2023-riemannian-diffusion-pullback]

models: Stable Diffusion v2.1 · method: Jacobian spectral direction analysis, Activation Steering (Addition)
One-Step Is Enough: Sparse Autoencoders for Text-to-Image Diffusion Models (2024)measured

SAE features from SDXL-Turbo's one-step U-Net transfer zero-shot and edit images

Details

Surkov et al. train SAEs on transformer-block updates inside SDXL-Turbo's denoising U-Net [surkov-etal-2024-one-step-is-enough-sparse-autoencoders-for-text-to-image-diffusion-models] The resulting features generalize zero-shot, with no retraining, to 4-step SDXL-Turbo and the separately-trained multi-step SDXL-base model [surkov-etal-2024-one-step-is-enough-sparse-autoencoders-for-text-to-image-diffusion-models] Switching individual SAE features on or off during generation causally edits specific attributes of the generated image on the RIEBench benchmark [surkov-etal-2024-one-step-is-enough-sparse-autoencoders-for-text-to-image-diffusion-models] Different transformer blocks show measurable specialization by edit category [surkov-etal-2024-one-step-is-enough-sparse-autoencoders-for-text-to-image-diffusion-models]

models: SDXL Turbo, Stable Diffusion XL · method: Sparse Autoencoders (SAE)
Emergence and Evolution of Interpretable Concepts in Diffusion Models (2025)measured

SD concept directions show composition, style, and texture set at successive denoising steps

Details

Tinaz et al. train sparse autoencoders on Stable Diffusion v1.4 U-Net activations across the reverse-diffusion trajectory, uncovering human-interpretable concept directions [tinaz-etal-2025-emergence-and-evolution-of-interpretable-concepts-in-diffusion-models] Final scene composition can be predicted from the spatial distribution of activated concepts even before the first denoising step completes [tinaz-etal-2025-emergence-and-evolution-of-interpretable-concepts-in-diffusion-models] Manipulating concept activations confirms a temporal control hierarchy: early denoising steps control composition, mid steps control style once composition is fixed, and late steps affect only minor texture [tinaz-etal-2025-emergence-and-evolution-of-interpretable-concepts-in-diffusion-models]

models: Stable Diffusion v1.4 · method: Sparse Autoencoders (SAE), Causal interventions (steering)
Self-Discovering Interpretable Diffusion Latent Directions for Responsible Text-to-Image Generation (2024)measured

Self-supervised h-space directions enable responsible diffusion generation

Details

Li et al. target the same additive h-space bottleneck as Asyrp but replace its CLIP-guided optimization with a self-supervised reconstruction technique [li-etal-2024-self-discovering-diffusion-directions] A single vector c is optimized so the frozen U-Net reconstructs a concept-present image when conditioned instead on a concept-stripped prompt plus c added to h-space [li-etal-2024-self-discovering-diffusion-directions] Because the model is frozen and the prompt omits the concept, gradient descent forces c to absorb exactly the missing semantic content, needing no labels or CLIP [li-etal-2024-self-discovering-diffusion-directions] A negative-prompt variant learns an anti-concept direction for safety mitigation; linearity and additive composability are demonstrated qualitatively and imposed by design [li-etal-2024-self-discovering-diffusion-directions]

models: Stable Diffusion v1.4 · method: Self-supervised reconstruction direction optimization, Activation Steering (Addition)
SliderSpace: Decomposing the Visual Capabilities of Diffusion Models (2025)measured

SliderSpace realizes CLIP-PCA directions as causally independent LoRA sliders

Details

Gandikota et al. run PCA on CLIP embeddings of a diffusion model's own output samples, yielding orthogonal principal directions of its semantic variation [gandikota-etal-2025-sliderspace-diffusion-decomposition] Per direction, a low-rank LoRA adapter is trained via a cosine-alignment loss to make its induced CLIP-embedding shift align with that PCA direction, turning passive geometry into a reusable slider [gandikota-etal-2025-sliderspace-diffusion-decomposition] Demonstrated on SDXL-DMD, SDXL, SDXL-Turbo, and FLUX Schnell, sliders increase diversity at similar CLIP-Score with user-study win-rates 66-72% [gandikota-etal-2025-sliderspace-diffusion-decomposition] An ablation shows the cosine-alignment loss and the CLIP-space choice are both causally necessary, else the sliders collapse to junk or color/shape-only directions [gandikota-etal-2025-sliderspace-diffusion-decomposition]

models: SDXL-DMD (4-step distilled), Stable Diffusion XL, SDXL Turbo · method: PCA, LoRA-trained direction realization
Orthogonal Concept Erasure for Diffusion Models (2026)measured

OCE erases diffusion concepts via a shared orthogonal rotation preserving norms and angles

Details

Sun et al. edit a diffusion model's cross-attention key/value weights in place via a single shared orthogonal matrix P solved in closed form from an orthogonal Procrustes problem [sun-etal-2026-orthogonal-concept-erasure-for-diffusion-models] Because P is orthogonal, neuron magnitudes and pairwise inter-neuron angles are preserved exactly, an algebraic guarantee rather than a measured statistic [sun-etal-2026-orthogonal-concept-erasure-for-diffusion-models] An ablation shows the design matters: additive scaling has negligible effect, layer-wise shared rotation gives a clean semantic shift, and neuron-wise rotation degrades image quality [sun-etal-2026-orthogonal-concept-erasure-for-diffusion-models] It erases object classes, styles, up to 100 celebrities simultaneously, and NSFW content on SD v1.4, transferring zero-shot to the DiT-architecture FLUX.1 [sun-etal-2026-orthogonal-concept-erasure-for-diffusion-models]

models: Stable Diffusion v1.4 · method: Orthogonal Weight-Rotation Concept Erasure (OCE)
General and Efficient Steering of Diffusion Models (2026)measured

A single RFM/AGOP direction per concept steers diffusion at a fraction of guidance cost

Details

Wang et al. train a Recursive Feature Machine on flattened U-Net/DiT block activations from forward-noised labeled images, taking the sign-corrected top eigenvectors of its AGOP matrix as one reusable per-concept steering direction [wang-etal-2026-general-and-efficient-steering-of-diffusion-models-a-single-rfm-agop-direction-per-concept-steers-unconditional-and-latent-diffusion-models-at-a-fraction-of-classifier-guidance-cost] Adding this direction during sampling steers unconditional DDPM/ADM U-Nets (CIFAR-10, ImageNet-256, CelebA-HQ), the transformer latent-diffusion SiT-XL/2, and Stable Diffusion 1.5 [wang-etal-2026-general-and-efficient-steering-of-diffusion-models-a-single-rfm-agop-direction-per-concept-steers-unconditional-and-latent-diffusion-models-at-a-fraction-of-classifier-guidance-cost] On CIFAR-10, NA-RFM reaches 96.6% guidance accuracy versus 77.1% and 86.0% baselines, with FID 41.4 versus 73.9 and 41.9 [wang-etal-2026-general-and-efficient-steering-of-diffusion-models-a-single-rfm-agop-direction-per-concept-steers-unconditional-and-latent-diffusion-models-at-a-fraction-of-classifier-guidance-cost] It runs one fixed low-rank direction across the sampling trajectory at a 16x sampling speedup over a training-free-guidance baseline [wang-etal-2026-general-and-efficient-steering-of-diffusion-models-a-single-rfm-agop-direction-per-concept-steers-unconditional-and-latent-diffusion-models-at-a-fraction-of-classifier-guidance-cost]

models: Stable Diffusion v1.5 · method: RFM/AGOP Steering Direction Extraction
Robust and Generalizable Safety Steering for Text-to-Image Diffusion Transformers (2026)measured

Position-aware SAE safety steering generalizes across domains within a diffusion model

Details

Xue et al. build position-aware SAEs over functionally distinct DiT intervention sites in FLUX.1 Dev and Stable Diffusion 3.5 Large, used as two independent testbeds [xue-etal-2026-safedig-safety-steering-diffusion] Robustness-aware routing prioritizes intervention sites, and a frozen SAE encoder acts as a reusable sparse safety dictionary while only the decoder adapts to the target domain [xue-etal-2026-safedig-safety-steering-diffusion] Safety learned from known source risk domains generalizes to held-out or emerging target risk domains within a single model [xue-etal-2026-safedig-safety-steering-diffusion] The demonstrated transfer is within-model across risk domains, not cross-model between FLUX and SD3.5 [xue-etal-2026-safedig-safety-steering-diffusion]

models: Stable Diffusion 3.5 · method: Sparse Autoencoders (SAE), Causal interventions (steering)
Residualized Temporal Sparse Autoencoders for Interpreting Diffusion Models (2026)measured

Residualizing adjacent denoising timesteps yields directionally-stable SD feature trajectories

Details

Yeung et al. fit a ridge residualization between adjacent normalized timestep activations in Stable Diffusion 1.5's U-Net mid-block cross-attention, then train a BatchTopK SAE on the residualized trajectory [yeung-etal-2026-residualized-temporal-saes-diffusion] Each latent's decoder direction maps to a feature trajectory across denoising timesteps, and residualized variants beat non-residualized, timestep-wise, and Matryoshka SAE baselines on reconstruction, strongest at later timesteps [yeung-etal-2026-residualized-temporal-saes-diffusion] Spatial cosine-similarity and cross-timestep decoder self-similarity maps quantify directional stability [yeung-etal-2026-residualized-temporal-saes-diffusion] Single-feature and feature-transfer steering during generation produces semantically meaningful effects, reported qualitatively rather than benchmark-quantified [yeung-etal-2026-residualized-temporal-saes-diffusion]

models: Stable Diffusion v1.5 · method: BatchTopK Sparse Autoencoders, Causal interventions (steering)

Custom Molecular Transformer-VAE (purpose-built for SELFIES generation)

Molecules Meet Language: Confound-Aware Representation Learning and Chemical Property Steering in Transformer-VAE Latent Spaces (2026)measured

Chemical properties are linear steerable directions in a molecular VAE

Details

Elabid et al. train a custom Transformer-VAE (latent dim 256) from scratch on 794,403 RDKit-valid SELFIES molecules [elabid-etal-2026-molecules-meet-language] Linear probes on frozen latent codes recover chemical properties (cLogP, FractionCSP3, TPSA, HBA, BertzCT) [elabid-etal-2026-molecules-meet-language] The probe directions are approximately linear and globally steerable, robust to confound-residualized R^2 ruling out sequence-length artifacts [elabid-etal-2026-molecules-meet-language] Latent traversal along a probe direction produces monotonic, chemically coherent property changes, enabling controllable generation [elabid-etal-2026-molecules-meet-language]

models: Custom Transformer-VAE, Autoregressive MultiSlotting variant (trained from scratch on SELFIES-tokenized molecules) · method: Linear probing, Activation Steering (Addition)

Audio-MAE

Probing Spatial Structure in Pretrained Audio Representations (2026)measured

Audio-encoder cosine sensitivity diverges from probe decodability

Details

Chen, Ding, Roman & Bello evaluate 13 frozen pretrained audio encoders across mono, stereo, binaural, and ambisonics on 7 linear-probing tasks [chen-etal-2026-probing-spatial-structure-audio] Source-related linear-probe improvements consistently exceed room-related ones for every model [chen-etal-2026-probing-spatial-structure-audio] A normalized cosine-similarity sensitivity metric (Delta = (cos-mu)/(1-mu)) under source vs room perturbations can diverge from linear-probe decodability of the same factor [chen-etal-2026-probing-spatial-structure-audio] Some models (GRAM-F, EINv2) show low sensitivity yet strong probing, while others (SFD, BANC) show high sensitivity but weaker probing, proving the two properties are distinct [chen-etal-2026-probing-spatial-structure-audio]

models: Audio-MAE · method: Linear probing, Representation-sensitivity perturbation response

EnCodec

Probing Spatial Structure in Pretrained Audio Representations (2026)measured

Audio-encoder cosine sensitivity diverges from probe decodability

Details

Chen, Ding, Roman & Bello evaluate 13 frozen pretrained audio encoders across mono, stereo, binaural, and ambisonics on 7 linear-probing tasks [chen-etal-2026-probing-spatial-structure-audio] Source-related linear-probe improvements consistently exceed room-related ones for every model [chen-etal-2026-probing-spatial-structure-audio] A normalized cosine-similarity sensitivity metric (Delta = (cos-mu)/(1-mu)) under source vs room perturbations can diverge from linear-probe decodability of the same factor [chen-etal-2026-probing-spatial-structure-audio] Some models (GRAM-F, EINv2) show low sensitivity yet strong probing, while others (SFD, BANC) show high sensitivity but weaker probing, proving the two properties are distinct [chen-etal-2026-probing-spatial-structure-audio]

models: EnCodec · method: Linear probing, Representation-sensitivity perturbation response
Learning Interpretable Features in Audio Latent Spaces via Sparse Autoencoders (2025)measured

SAE probe directions decode a pitch>loudness>timbre hierarchy and steer audio

Details

Paek et al. train SAEs on four pretrained audio autoencoders/codecs and fit linear probes from SAE features to pitch, loudness, and timbre [paek-etal-2025-learning-interpretable-features-audio-latent-spaces-sparse-autoencoders] Across all four latent spaces decodability follows the same order: pitch most separable (0.75-0.87), loudness intermediate, timbre hardest (0.17-0.46) [paek-etal-2025-learning-interpretable-features-audio-latent-spaces-sparse-autoencoders] Reusing each probe's weight row directly as a control vector causally isolates changes in the targeted property while leaving others largely intact [paek-etal-2025-learning-interpretable-features-audio-latent-spaces-sparse-autoencoders] Tracing DiffRhythm's rectified-flow generation reveals a coarse-to-fine emergence order: pitch converges first, then timbre, with loudness least resolved [paek-etal-2025-learning-interpretable-features-audio-latent-spaces-sparse-autoencoders]

models: EnCodec · method: Sparse Autoencoders (SAE), Linear probing, Causal interventions (steering), Activation Steering (Addition)

Custom Research Matrix-Factorization Recommender (purpose-built dot-product recommender for embedding-geometry studies)

A Rank-One Popularity Component in Dot-Product Recommender Scores: Population Theory and Prior-Separation Evidence (2026)measured

A real trained dot-product recommender's item embeddings carry a rank-one popularity-aligned direction that a prior-separation intervention removes

Details

A real 128-dimensional dot-product matrix-factorization recommender is trained via Adam gradient descent on real Alibaba/Tianchi mobile-recommendation logs (459,105 training events, 1,413 users, 20,000 items) [cheng-2026-rank-one-popularity-component-recommender-scores] The trained item-embedding matrix's leading principal component correlates with log-item-popularity (PC1-log-popularity correlation 0.568), and a rank-one energy measure R1(E) and mean pairwise cosine similarity are elevated (0.0219 and 0.027 respectively), indicating a popularity-aligned anisotropic direction in the embedding geometry [cheng-2026-rank-one-popularity-component-recommender-scores] A causal prior-separation intervention -- moving the log-popularity term outside the dot product rather than absorbing it into the embeddings -- reduces the popularity-aligned score energy by 98.6% (0.01905 to 0.000265), with R1(E) dropping to 0.0157, mean pairwise cosine dropping to 0.006, and PC1-log-popularity correlation dropping to 0.078 (permutation test p=1/2001) [cheng-2026-rank-one-popularity-component-recommender-scores]

models: 128-dim dot-product matrix-factorization recommender, trained on Alibaba/Tianchi mobile-recommendation logs (Cheng) · method: PCA, Causal interventions (steering)

Sonny (hierarchical weather transformer)

Beyond Linear Superposition: Discovering Climate Features in AI Weather Models with KAN-SAE (2026)measured

A KAN-SAE with a nonlinear encoder but linear decoder directions discovers climate features in a real AI weather model and causally steers physically coherent forecasts

Details

Cheon (2026) trains a KAN-SAE (linear decoder, nonlinear per-feature B-spline encoder gating) on the layer-5 residual stream of Sonny, a real pretrained hierarchical weather transformer trained on ERA5 reanalysis, finding markedly more alive, less redundant features than a matched linear-encoder SAE baseline (975/1024 alive vs. 566/1024; median inter-feature correlation 0.076 vs. 0.092) and much tighter feature localization for a heatwave feature (2 degrees vs. 51 degrees error) [cheon-2026-beyond-linear-superposition-discovering-climate-features-kan-sae] Steering along one feature's decoder direction causally produces a dose-dependent, physically coherent regional temperature anomaly (EU-mean T2m +1.42K at steering strength 2, r^2>0.99) with coupled pressure-field changes consistent with real blocking-anticyclone dynamics -- the decoded features remain genuine linear directions in the model's own activation space [cheon-2026-beyond-linear-superposition-discovering-climate-features-kan-sae]

models: Sonny (StepsNet-based ViT-S weather transformer, trained on ERA5) · method:

ChessGPT (Karvonen)

Emergent World Models and Latent Variable Estimation in Chess-Playing Language Models (2024)measured

Chess-GPT linearly encodes editable board state and player skill

Details

Karvonen trains two character-level chess GPTs from scratch on 16 million Lichess games with no rules given [karvonen-2024-chess-gpt] A per-square linear probe recovers board state at up to 99.6% (16L) and 99.1% (8L), versus about 75% on a randomly-initialized model [karvonen-2024-chess-gpt] Subtracting a piece's probe direction from the residual stream raises the edited-board legal-move rate from about 40% to 90-92% [karvonen-2024-chess-gpt] A diff-in-means skill direction (high- minus low-skill games) predicts latent Elo, with a binary skill probe at 90.5% / 88.6% [karvonen-2024-chess-gpt] Adding the skill direction raises the 16-layer model's win rate versus Stockfish level 0 from 16.7% to 43.2% on random openings, and lowers it to 5.9% with the negative direction [karvonen-2024-chess-gpt]

models: ChessGPT-8L-25M, ChessGPT-16L-50M · method: Linear probing, Difference-in-means direction, Activation Steering (Addition)

wav2vec 2.0

[b]=[d]-[t]+[p]: Self-Supervised Speech Models Discover Phonological Vector Arithmetic (2026)measured

Phonological features are linear vector-offset directions in speech models

Details

Choi et al. extract phoneme-level representations from three frozen self-supervised speech models (wav2vec2-large-lv60, HuBERT-large-ll60k, WavLM-large) [choi-etal-2026-phonological-vector-arithmetic-speech-ssl] Phonological feature contrasts (voicing, place, manner) are encoded as approximately consistent diff-in-means directions that transfer across phoneme pairs [choi-etal-2026-phonological-vector-arithmetic-speech-ssl] Word2vec-style vector-offset arithmetic (e.g. [b] = [d] - [t] + [p]) succeeds above chance on the real extracted representations, with per-model accuracy reported [choi-etal-2026-phonological-vector-arithmetic-speech-ssl] The result is a quantified linear-direction/vector-offset claim on real speech models; no causal intervention is performed [choi-etal-2026-phonological-vector-arithmetic-speech-ssl]

models: wav2vec 2.0 Large (LV-60) · method: Difference-in-means direction, Vector-offset analogy test
Self-Supervised Speech Models Encode Phonetic Context via Position-dependent Orthogonal Subspaces (2026)measured

Speech models encode neighbor-phone info in orthogonal positional subspaces

Details

Choi et al. show a single S3M frame (wav2vec 2.0, HuBERT, WavLM Large) encodes phonological vectors not only for the current phone but for its neighbors, extracted via difference-of-means at relative positions -2 to +2 [choi-etal-2026-position-dependent-orthogonal-subspaces] Cosine similarity between vectors from different relative positions is substantially lower than within the same position across 8 phonological features, an orthogonality-of-subspaces structure preserved layer-by-layer [choi-etal-2026-position-dependent-orthogonal-subspaces] Vector norm decreases monotonically with distance from the center phone, with WavLM showing the clearest trapezoidal effective context window [choi-etal-2026-position-dependent-orthogonal-subspaces] The position-dependent subspace in use switches at annotated TIMIT phonetic boundaries rather than a fixed temporal window; no causal intervention is performed [choi-etal-2026-position-dependent-orthogonal-subspaces]

models: wav2vec 2.0 Large (LV-60) · method: Difference-in-means direction, Geometric analysis

WavLM

[b]=[d]-[t]+[p]: Self-Supervised Speech Models Discover Phonological Vector Arithmetic (2026)measured

Phonological features are linear vector-offset directions in speech models

Details

Choi et al. extract phoneme-level representations from three frozen self-supervised speech models (wav2vec2-large-lv60, HuBERT-large-ll60k, WavLM-large) [choi-etal-2026-phonological-vector-arithmetic-speech-ssl] Phonological feature contrasts (voicing, place, manner) are encoded as approximately consistent diff-in-means directions that transfer across phoneme pairs [choi-etal-2026-phonological-vector-arithmetic-speech-ssl] Word2vec-style vector-offset arithmetic (e.g. [b] = [d] - [t] + [p]) succeeds above chance on the real extracted representations, with per-model accuracy reported [choi-etal-2026-phonological-vector-arithmetic-speech-ssl] The result is a quantified linear-direction/vector-offset claim on real speech models; no causal intervention is performed [choi-etal-2026-phonological-vector-arithmetic-speech-ssl]

models: WavLM Large · method: Difference-in-means direction, Vector-offset analogy test
Self-Supervised Speech Models Encode Phonetic Context via Position-dependent Orthogonal Subspaces (2026)measured

Speech models encode neighbor-phone info in orthogonal positional subspaces

Details

Choi et al. show a single S3M frame (wav2vec 2.0, HuBERT, WavLM Large) encodes phonological vectors not only for the current phone but for its neighbors, extracted via difference-of-means at relative positions -2 to +2 [choi-etal-2026-position-dependent-orthogonal-subspaces] Cosine similarity between vectors from different relative positions is substantially lower than within the same position across 8 phonological features, an orthogonality-of-subspaces structure preserved layer-by-layer [choi-etal-2026-position-dependent-orthogonal-subspaces] Vector norm decreases monotonically with distance from the center phone, with WavLM showing the clearest trapezoidal effective context window [choi-etal-2026-position-dependent-orthogonal-subspaces] The position-dependent subspace in use switches at annotated TIMIT phonetic boundaries rather than a fixed temporal window; no causal intervention is performed [choi-etal-2026-position-dependent-orthogonal-subspaces]

models: WavLM Large · method: Difference-in-means direction, Geometric analysis

scGPT

Sparse Autoencoders Reveal Interpretable Features in Single-Cell Foundation Models (2025)measured

Clamping SAE batch-effect features corrects single-cell batch integration

Details

Pedrocchi et al. train BatchTopK SAEs on residual-stream representations of single-cell foundation models scGPT and scFoundation across several datasets [pedrocchi-etal-2025-scfm-saes] Batch/technical-effect feature directions are identified via mutual information between feature activation and batch label [pedrocchi-etal-2025-scfm-saes] Clamping the top batch features (top-20 pretrained, top-50 fine-tuned) and re-decoding improves scIB batch correction without substantial loss of biological conservation [pedrocchi-etal-2025-scfm-saes] The intervention shows a dose-response against random-feature-ablation, peaking at dataset-specific thresholds (25 Pancreas, 30 Lung, 60 Immune) before plateauing or degrading [pedrocchi-etal-2025-scfm-saes]

models: scGPT (whole-human pretrained checkpoint) · method: BatchTopK Sparse Autoencoders, Causal interventions (steering)
Geometric Coherence of Single-Cell CRISPR Perturbations Reveals Regulatory Architecture and Predicts Cellular Stress (2026)measured

CRISPR perturbation-direction coherence persists in scGPT's embedding space

Details

Raju defines a perturbation-stability score, the mean cosine similarity between per-cell CRISPR shift vectors and the gene's mean perturbation direction, across 2,200+ perturbations from 5 datasets [raju-2026-geometric-coherence-of-single-cell-crispr-perturbations] In raw PCA space, magnitude-stability correlation is rho=0.75-0.97, with pleiotropic regulators (CEBPA) incoherent and lineage factors (KLF1) coherent [raju-2026-geometric-coherence-of-single-cell-crispr-perturbations] Recomputing the score inside scGPT's pretrained embedding space reproduces the structure (Norman rho=0.935, Dixit 0.712, Replogle 0.851; all p<1e-25) [raju-2026-geometric-coherence-of-single-cell-crispr-perturbations] The coherence is thus a genuine geometric property of the model's representation and predicts unfolded-protein-response activation beyond shift magnitude (p<1e-18) [raju-2026-geometric-coherence-of-single-cell-crispr-perturbations]

models: scGPT (whole-human pretrained checkpoint) · method:

scFoundation

Sparse Autoencoders Reveal Interpretable Features in Single-Cell Foundation Models (2025)measured

Clamping SAE batch-effect features corrects single-cell batch integration

Details

Pedrocchi et al. train BatchTopK SAEs on residual-stream representations of single-cell foundation models scGPT and scFoundation across several datasets [pedrocchi-etal-2025-scfm-saes] Batch/technical-effect feature directions are identified via mutual information between feature activation and batch label [pedrocchi-etal-2025-scfm-saes] Clamping the top batch features (top-20 pretrained, top-50 fine-tuned) and re-decoding improves scIB batch correction without substantial loss of biological conservation [pedrocchi-etal-2025-scfm-saes] The intervention shows a dose-response against random-feature-ablation, peaking at dataset-specific thresholds (25 Pancreas, 30 Lung, 60 Immune) before plateauing or degrading [pedrocchi-etal-2025-scfm-saes]

models: scFoundation (pretrained) · method: BatchTopK Sparse Autoencoders, Causal interventions (steering)

Claude

Scaling Monosemanticity: Extracting Interpretable Features from Claude 3 Sonnet (2024)measured

Claude-3-Sonnet SAE features causally steer safety behaviors and split with scale

Details

Templeton et al. train three SAEs (~1M/4M/34M features) on the middle-layer residual stream of Claude 3 Sonnet, the first SAE interpretability at frontier scale [templeton-etal-2024-scaling-monosemanticity] Clamping named features causally steers behavior: the Golden Gate Bridge feature at 10x makes the model self-identify as the bridge, and an unsafe-code feature at 5x produces a buffer overflow [templeton-etal-2024-scaling-monosemanticity] Further clamped features drive sycophantic praise, deception/secrecy, honesty, and racist invective, spanning safety-relevant domains [templeton-etal-2024-scaling-monosemanticity] Feature splitting is quantified: a "San Francisco" feature in the 1M SAE splits into 2 (4M) then 11 (34M) finer features, each other's cosine nearest neighbors [templeton-etal-2024-scaling-monosemanticity]

models: Claude 3 Sonnet · method: Sparse Autoencoders (SAE), Activation Steering (Addition)
Emotion Concepts and their Function in a Large Language Model (2026)measured

Claude Sonnet 4.5 encodes 171 emotion directions that steer safety behavior

Details

Sofroniew et al. (Anthropic) extract ~171 linear emotion-concept directions in Claude Sonnet 4.5's residual stream via diff-in-means over emotion words [sofroniew-etal-2026-emotion-concepts-and-their-function-in-a-large-language-model] The directions generalize across contexts and roughly align with a valence/arousal circumplex when visualized via PCA [sofroniew-etal-2026-emotion-concepts-and-their-function-in-a-large-language-model] Injecting or suppressing specific emotion directions measurably changes reward-hacking, blackmail, and sycophancy rates in held-out scenarios [sofroniew-etal-2026-emotion-concepts-and-their-function-in-a-large-language-model] The result ties frontier-model linear emotion geometry directly to safety-relevant behavior at higher granularity than prior 2D-circumplex claims [sofroniew-etal-2026-emotion-concepts-and-their-function-in-a-large-language-model]

models: Claude Sonnet 4.5 · method: Difference-in-means direction, PCA, Activation Steering (Addition), Causal interventions (steering)
Sparse Crosscoders for Cross-Layer Features and Model Diffing (2024)measured

Crosscoders recover shared cross-layer and cross-model features

Details

Lindsey et al. introduce sparse crosscoders: one dictionary reads and writes activations at multiple layers or models at once, with the L1 penalty weighted by summed per-layer decoder norms [lindsey-etal-2024-crosscoders] Against matched per-layer SAEs a crosscoder achieves lower eval loss per feature but needs ~2x more training FLOPs, evidencing consolidated cross-layer structure [lindsey-etal-2024-crosscoders] Decoder directions drift across layers even where decoder norm persists, ruling out passive residual-stream relaying [lindsey-etal-2024-crosscoders] A cross-model crosscoder on Claude 3 Sonnet base vs finetuned splits features into shared, base-only, and finetuned-only (~4,000-5,000 model-specific per side), surfacing a refusal and a code-review feature [lindsey-etal-2024-crosscoders] Shared features' decoder directions are highly aligned between the two models, with a minority low or negatively aligned as candidate repurposed concepts [lindsey-etal-2024-crosscoders]

models: Claude 3 Sonnet · method: Sparse crosscoders

SmolLM2

Concepts Whisper While Syntax Shouts: Spectral Anti-Concentration and the Dual Geometry of Transformer Representations (2026)measured

Concept directions anti-concentrate in the unembedding's low-variance spectral tail

Details

Acharya, Rimal & Dhakal project concept diff-of-means vectors onto the unembedding-covariance eigenbasis across 17 models in 5 families [acharya-rimal-dhakal-2026-spectral-anti-concentration] Concept-direction energy anti-concentrates in the low-eigenvalue tail: Spectral Center of Mass averages 0.926 versus 0.758 for random directions (Gini-deviation -0.282) [acharya-rimal-dhakal-2026-spectral-anti-concentration] Static unembedding-row contrasts show the opposite sign, and POS/syntax directions concentrate in the high-variance subspace in 6 of 8 architectures (p<0.013), a genuine dual-geometry split [acharya-rimal-dhakal-2026-spectral-anti-concentration] On Llama-3.1-8B, injecting only the top-10%-eigenvalue component gives +383% perplexity versus +223% for the bottom-10% component (Cohen's d=1.80), replicated across 5 models [acharya-rimal-dhakal-2026-spectral-anti-concentration]

models: SmolLM2-1.7B · method: Geometric analysis, Causal interventions (steering)

OLMoE

Concepts Whisper While Syntax Shouts: Spectral Anti-Concentration and the Dual Geometry of Transformer Representations (2026)measured

Concept directions anti-concentrate in the unembedding's low-variance spectral tail

Details

Acharya, Rimal & Dhakal project concept diff-of-means vectors onto the unembedding-covariance eigenbasis across 17 models in 5 families [acharya-rimal-dhakal-2026-spectral-anti-concentration] Concept-direction energy anti-concentrates in the low-eigenvalue tail: Spectral Center of Mass averages 0.926 versus 0.758 for random directions (Gini-deviation -0.282) [acharya-rimal-dhakal-2026-spectral-anti-concentration] Static unembedding-row contrasts show the opposite sign, and POS/syntax directions concentrate in the high-variance subspace in 6 of 8 architectures (p<0.013), a genuine dual-geometry split [acharya-rimal-dhakal-2026-spectral-anti-concentration] On Llama-3.1-8B, injecting only the top-10%-eigenvalue component gives +383% perplexity versus +223% for the bottom-10% component (Cohen's d=1.80), replicated across 5 models [acharya-rimal-dhakal-2026-spectral-anti-concentration]

models: OLMoE-1B-7B · method: Geometric analysis, Causal interventions (steering)

JetMoE

Concepts Whisper While Syntax Shouts: Spectral Anti-Concentration and the Dual Geometry of Transformer Representations (2026)measured

Concept directions anti-concentrate in the unembedding's low-variance spectral tail

Details

Acharya, Rimal & Dhakal project concept diff-of-means vectors onto the unembedding-covariance eigenbasis across 17 models in 5 families [acharya-rimal-dhakal-2026-spectral-anti-concentration] Concept-direction energy anti-concentrates in the low-eigenvalue tail: Spectral Center of Mass averages 0.926 versus 0.758 for random directions (Gini-deviation -0.282) [acharya-rimal-dhakal-2026-spectral-anti-concentration] Static unembedding-row contrasts show the opposite sign, and POS/syntax directions concentrate in the high-variance subspace in 6 of 8 architectures (p<0.013), a genuine dual-geometry split [acharya-rimal-dhakal-2026-spectral-anti-concentration] On Llama-3.1-8B, injecting only the top-10%-eigenvalue component gives +383% perplexity versus +223% for the bottom-10% component (Cohen's d=1.80), replicated across 5 models [acharya-rimal-dhakal-2026-spectral-anti-concentration]

models: JetMoE-8B · method: Geometric analysis, Causal interventions (steering)

DeBERTa

Representation Engineering: A Top-Down Approach to AI Transparency (2023)measured

Linear Artificial Tomography reads and controls concepts across LLMs

Details

Zou et al. introduce Linear Artificial Tomography (LAT): from contrastive stimulus pairs, take the first principal component of paired activation differences as a concept reading vector [zou-etal-2023] Across LLaMA-2-Chat (7B/13B/70B), Vicuna-13B, Vicuna-33B-Uncensored, and DeBERTa, a single direction classifies and causally controls honesty, ethics, morality, emotion, and bias [zou-etal-2023] LAT recovers truthfulness on DeBERTa more accurately than contrast-consistent search [zou-etal-2023] A harmfulness direction in Vicuna-13B stays a >90% classifier under jailbreaks, and boosting its salience raises harmless-response rates under attack [zou-etal-2023]

models: DeBERTa-xxlarge-v2-MNLI · method: Linear Artificial Tomography (LAT), PCA, Causal interventions (steering), Low-Rank Representation Adaptation (LoRRA)
Discovering Latent Knowledge in Language Models Without Supervision (2023)measured

Contrast-Consistent Search finds a truth-like direction with no labels

Details

Burns et al. introduce Contrast-Consistent Search: fit a linear probe on yes/no contrast pairs purely by a logical-consistency plus confidence loss, using no truth labels [burns-etal-2023] Across 6 models and 10 classification/QA datasets, CCS beats zero-shot accuracy by ~4% and approaches supervised logistic regression [burns-etal-2023] The direction found on one dataset transfers to structurally unrelated datasets and is often recoverable as the top principal component of contrast-pair differences [burns-etal-2023] True/false examples form two well-separated clusters along it, sometimes from as few as one contrast pair [burns-etal-2023] This unsupervised truth-like direction is broader than Marks & Tegmark's factual-statement truth direction and is kept as a separate observation [burns-etal-2023]

models: DeBERTa-xxlarge-v2-MNLI · method: Contrast-Consistent Search (CCS), PCA, Linear probing, Geometric analysis

DeepSeek

Language Models Represent and Transform Concepts with Shared Geometry (2026)measured

Contextual displacement forms a vector field tracking concreteness

Details

Hu, Niu & Varma define a per-word contextual displacement vector phi(w,tau) = r(w,tau) - r(w,tau_0) in activation space, forming a vector field over the vocabulary rather than one shared direction [hu-etal-2026-concept-transformation-geometry] Across 4 models x 3 depths, word density correlates negatively with displacement magnitude (Spearman rho -0.32 to -0.22) [hu-etal-2026-concept-transformation-geometry] Lexical concreteness correlates negatively with directional deviation (rho -0.41 to -0.21): concrete, low-density concepts move less and more consistently under context [hu-etal-2026-concept-transformation-geometry] The field has no dominant direction (leading PC <15% variance) and is more dispersed than uniform-random; no causal intervention is performed [hu-etal-2026-concept-transformation-geometry]

models: DeepSeek-LLM-7B-Base · method: PCA, Geometric analysis
What Really Controls Temporal Reasoning in LLMs: Tokenisation or Representation of Time? (2026)measured

Temporal-probe decodability predicts reasoning in high-resource languages

Details

Bhatia et al. introduce MultiTempBench and evaluate 20 LLMs, combining a tokenization-fragmentation metric with linear-regression probes decoding Year/Month/Day (probe R^2 as temporal linearity) [bhatia-etal-2026-what-really-controls-temporal-reasoning-in-llms] A crossed mixed-effects regression over 285,000 predictions finds temporal linearity is the strongest predictor of reasoning accuracy in high-resource languages (English r=0.77) [bhatia-etal-2026-what-really-controls-temporal-reasoning-in-llms] Tokenization fragmentation dominates in low-resource languages (Hausa r=-0.97); the finding is purely correlational [bhatia-etal-2026-what-really-controls-temporal-reasoning-in-llms]

models: DeepSeek-R1-Distill-Qwen-7B · method: Linear probing
Understanding Reasoning in Thinking Language Models via Steering Vectors (2025)measured

Reasoning behaviors are distinct but correlated steerable linear directions

Details

Venhoff et al. study DeepSeek-R1-Distill reasoning models (Qwen-1.5B, Qwen-14B, Llama-8B), auto-labeling reasoning sentences into behaviors like Backtracking, Uncertainty Estimation, and Example Testing [venhoff-etal-2025-reasoning-steering-vectors] Each reasoning behavior is mediated by a distinct linear direction in the residual stream [venhoff-etal-2025-reasoning-steering-vectors] The behavior directions are correlated with one another yet remain separable [venhoff-etal-2025-reasoning-steering-vectors] Adding a behavior's steering vector causally increases the corresponding reasoning behavior during generation [venhoff-etal-2025-reasoning-steering-vectors]

models: DeepSeek-R1-Distill-Qwen-1.5B, DeepSeek-R1-Distill-Qwen-14B, DeepSeek-R1-Distill-Llama-8B · method: Difference-in-means direction, Activation Steering (Addition)
Internal states before "wait" modulate reasoning patterns (2025)measured

Crosscoder pre-'wait' feature directions set which reasoning pattern follows

Details

Troitskii et al. train crosscoders across DeepSeek-R1-Distill-Llama-8B and its pre-distillation base Llama-3.1-8B to model-diff the base-to-distilled transition [troitskii-etal-2025-internal-states-before-wait-modulate-reasoning] Latent attribution locates a small subset of feature directions whose activation immediately before a "wait" token predicts the following reasoning pattern [troitskii-etal-2025-internal-states-before-wait-modulate-reasoning] Intervening directly on these features causally determines which pattern follows: restarting, recalling prior knowledge, expressing uncertainty, or double-checking [troitskii-etal-2025-internal-states-before-wait-modulate-reasoning] The interventions change the qualitative reasoning pattern rather than merely correlating with it [troitskii-etal-2025-internal-states-before-wait-modulate-reasoning]

models: DeepSeek-R1-Distill-Llama-8B · method: Sparse crosscoders, Causal interventions (steering)
Language Models Represent Beliefs of Self and Others (2024)measured

A mid-layer head group decodes another agent's beliefs and steers theory-of-mind

Details

Zhu et al. prompt Mistral-7B-Instruct (and DeepSeek-LLM-7B-Chat) with third-person belief narratives and fit logistic-regression probes on attention-head activations [zhu-etal-2024-language-models-represent-beliefs-of-self-and-others] A group of middle-layer attention heads decodes another agent's belief status at over 80% accuracy, distinct from the model's own self-belief representation [zhu-etal-2024-language-models-represent-beliefs-of-self-and-others] Steering these representations produces large changes in downstream Theory-of-Mind performance, generalizing across social-reasoning tasks [zhu-etal-2024-language-models-represent-beliefs-of-self-and-others]

models: DeepSeek-LLM-7B-Chat · method: Linear probing, Activation Steering (Addition)

MetaCLIP

Same Concept, Different Directions: Cross-Modal Feature Heterogeneity in Sparse Autoencoders (2026)measured

Same concept, different SAE directions across modalities

Details

- In vision-language models, sparse-autoencoder feature directions for the same concept differ across modalities, keeping substantial angular (cosine-distance) separation even when their activations are highly correlated. [lee-etal-2026-cross-modal-sae-heterogeneity] - The paper proves that a single forced-alignment SAE collapses the two directions to their normalized sum [(phi+psi)/||phi+psi||, 0], sacrificing reconstruction. [lee-etal-2026-cross-modal-sae-heterogeneity] - Modality-specific SAEs plus post-hoc alignment give better cross-modal retrieval and steering (image->text R@1 16.0, text->image R@1 11.4). [lee-etal-2026-cross-modal-sae-heterogeneity] - Directional analysis spans CLIP ViT-B/32, MetaCLIP B/32, OpenCLIP B/32 and SigLIP2. [lee-etal-2026-cross-modal-sae-heterogeneity]

models: MetaCLIP ViT-B/32 · method: TopK (k-sparse) Sparse Autoencoders, Geometric analysis, Causal interventions (steering)
The Information Geometry of Softmax: Probing and Steering (2026)measured

Dual steering in information-geometric coordinates is provably KL-optimal

Details

Park et al. argue softmax distributions have a Bregman information geometry where KL divergence equals a Bregman divergence of the log-normalizer [park-etal-2026-information-geometry-softmax-probing-steering] Dual steering adds the probe in the dual (mean) parameter space rather than the logit space of ordinary CAA-style steering [park-etal-2026-information-geometry-softmax-probing-steering] Theorem 3.1 proves dual steering is the exact KL-minimizing intervention reaching a target concept logit-score, with a minimal-collateral-change guarantee under concept-factorizability [park-etal-2026-information-geometry-softmax-probing-steering] On Gemma-3-4B and MetaCLIP-2, dual steering outperforms Euclidean steering across all three robustness metrics [park-etal-2026-information-geometry-softmax-probing-steering]

models: MetaCLIP 2 · method: Linear probing, Dual (Information-Geometric) Steering

OpenCLIP

Same Concept, Different Directions: Cross-Modal Feature Heterogeneity in Sparse Autoencoders (2026)measured

Same concept, different SAE directions across modalities

Details

- In vision-language models, sparse-autoencoder feature directions for the same concept differ across modalities, keeping substantial angular (cosine-distance) separation even when their activations are highly correlated. [lee-etal-2026-cross-modal-sae-heterogeneity] - The paper proves that a single forced-alignment SAE collapses the two directions to their normalized sum [(phi+psi)/||phi+psi||, 0], sacrificing reconstruction. [lee-etal-2026-cross-modal-sae-heterogeneity] - Modality-specific SAEs plus post-hoc alignment give better cross-modal retrieval and steering (image->text R@1 16.0, text->image R@1 11.4). [lee-etal-2026-cross-modal-sae-heterogeneity] - Directional analysis spans CLIP ViT-B/32, MetaCLIP B/32, OpenCLIP B/32 and SigLIP2. [lee-etal-2026-cross-modal-sae-heterogeneity]

models: OpenCLIP ViT-B/32 · method: TopK (k-sparse) Sparse Autoencoders, Geometric analysis, Causal interventions (steering)
Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE) (2024)measured

Sparse nonnegative concept decomposition enables causal CLIP editing

Details

- CLIP image embeddings decompose as sparse, nonnegative combinations (l0 ~5-20, recommended ~10-30) of a fixed dictionary of ~15k text-encoder concept vectors from LAION captions. [bhalla-etal-2024-splice] - The decomposition nearly preserves zero-shot accuracy (ImageNet 0.552 to 0.552, CIFAR-100 0.750 to 0.768) with cosine reconstruction ~0.75-0.88. [bhalla-etal-2024-splice] - Because embeddings are additive concept mixtures, zeroing a concept edits behavior surgically (removing "glasses" drops a glasses classifier 0.88 to 0.69 while gender stays 0.97 to 0.96). [bhalla-etal-2024-splice] - Tested on OpenCLIP ViT-B/32 (main) and OpenAI CLIP RN50 (appendix). [bhalla-etal-2024-splice]

models: OpenCLIP ViT-B/32 · method: Fixed-dictionary sparse recovery, Causal interventions (steering)
Compositional Generalization Requires Linear, Orthogonal Representations in Vision Embedding Models (2026)measured

Compositional generalization needs linear, additive, orthogonal per-concept factor vectors

Details

Uselis, Dittadi & Oh recover per-concept difference vectors from paired stimuli across three datasets in many pretrained encoders (CLIP, OpenCLIP, MetaCLIP, MetaCLIP2, SigLIP, SigLIP2, DINOv1/v2/v3) [uselis-dittadi-oh-2026-compositional-generalization-requires-linear-orthogonal-representations-in-vision-embedding-models] A whitened, probe-span-projected reconstruction of embeddings as a linear sum of recovered factors reaches R^2=0.42-0.63 versus 0.12-0.42 for a randomly-initialized-encoder baseline [uselis-dittadi-oh-2026-compositional-generalization-requires-linear-orthogonal-representations-in-vision-embedding-models] Cross-concept factor vectors are confirmed mutually orthogonal via cosine similarity and effective-rank analysis [uselis-dittadi-oh-2026-compositional-generalization-requires-linear-orthogonal-representations-in-vision-embedding-models] The degree of linear factorization correlates with each model's own compositional-generalization accuracy across variants [uselis-dittadi-oh-2026-compositional-generalization-requires-linear-orthogonal-representations-in-vision-embedding-models]

models: OpenCLIP ViT-L/14 · method: Additive per-concept factor recovery and orthogonality scoring

SigLIP

Same Concept, Different Directions: Cross-Modal Feature Heterogeneity in Sparse Autoencoders (2026)measured

Same concept, different SAE directions across modalities

Details

- In vision-language models, sparse-autoencoder feature directions for the same concept differ across modalities, keeping substantial angular (cosine-distance) separation even when their activations are highly correlated. [lee-etal-2026-cross-modal-sae-heterogeneity] - The paper proves that a single forced-alignment SAE collapses the two directions to their normalized sum [(phi+psi)/||phi+psi||, 0], sacrificing reconstruction. [lee-etal-2026-cross-modal-sae-heterogeneity] - Modality-specific SAEs plus post-hoc alignment give better cross-modal retrieval and steering (image->text R@1 16.0, text->image R@1 11.4). [lee-etal-2026-cross-modal-sae-heterogeneity] - Directional analysis spans CLIP ViT-B/32, MetaCLIP B/32, OpenCLIP B/32 and SigLIP2. [lee-etal-2026-cross-modal-sae-heterogeneity]

models: SigLIP 2 · method: TopK (k-sparse) Sparse Autoencoders, Geometric analysis, Causal interventions (steering)
Compositional Generalization Requires Linear, Orthogonal Representations in Vision Embedding Models (2026)measured

Compositional generalization needs linear, additive, orthogonal per-concept factor vectors

Details

Uselis, Dittadi & Oh recover per-concept difference vectors from paired stimuli across three datasets in many pretrained encoders (CLIP, OpenCLIP, MetaCLIP, MetaCLIP2, SigLIP, SigLIP2, DINOv1/v2/v3) [uselis-dittadi-oh-2026-compositional-generalization-requires-linear-orthogonal-representations-in-vision-embedding-models] A whitened, probe-span-projected reconstruction of embeddings as a linear sum of recovered factors reaches R^2=0.42-0.63 versus 0.12-0.42 for a randomly-initialized-encoder baseline [uselis-dittadi-oh-2026-compositional-generalization-requires-linear-orthogonal-representations-in-vision-embedding-models] Cross-concept factor vectors are confirmed mutually orthogonal via cosine similarity and effective-rank analysis [uselis-dittadi-oh-2026-compositional-generalization-requires-linear-orthogonal-representations-in-vision-embedding-models] The degree of linear factorization correlates with each model's own compositional-generalization accuracy across variants [uselis-dittadi-oh-2026-compositional-generalization-requires-linear-orthogonal-representations-in-vision-embedding-models]

models: SigLIP, SigLIP 2 · method: Additive per-concept factor recovery and orthogonality scoring
Interpreting the Linear Structure of Vision-Language Model Embedding Spaces (2025)measured

VLM SAE concepts are single-modality yet gap-orthogonal

Details

- BatchTopK sparse autoencoders trained on ~600k COCO vision-language embeddings (dictionary 4096) put 99% of the energy in 512 concepts, the other ~3500 sharing 1%. [papadimitriou-etal-2025-vlm-linear-structure] - The energy-dominant concepts are nearly single-modality in activation, yet many of their directions are nearly orthogonal to the modality-gap subspace (accuracy ~0.5 as modality classifiers). [papadimitriou-etal-2025-vlm-linear-structure] - An "SAE projection effect" reconciles this: sparse thresholding interacts with the per-modality input distributions, so a direction can be gap-orthogonal yet fire for one modality; top-512 concepts are seed-stable (0.92) vs 0.16 for low-energy ones. [papadimitriou-etal-2025-vlm-linear-structure] - Observational, on CLIP, SigLIP, SigLIP2 and AIMv2 (the paper states no model variants). [papadimitriou-etal-2025-vlm-linear-structure]

models: SigLIP, SigLIP 2 · method: BatchTopK Sparse Autoencoders, Geometric analysis

FLUX

Interpreting Large Text-to-Image Diffusion Models with Dictionary Learning (2025)measured

Dictionary-learned features in Flux.1 causally steer image generation

Details

Shabalin et al. apply SAEs and Inference-Time Decomposition of Activations to residual-stream embeddings of Flux.1, a large text-to-image diffusion model [shabalin-etal-2025-interpreting-large-text-to-image-diffusion-models-with-dictionary-learning] SAEs accurately reconstruct Flux.1's residual stream and outperform raw MLP neurons on an automated visual-interpretability pipeline [shabalin-etal-2025-interpreting-large-text-to-image-diffusion-models-with-dictionary-learning] SAE features causally steer image generation via activation addition, with measured changes in the generated images [shabalin-etal-2025-interpreting-large-text-to-image-diffusion-models-with-dictionary-learning] ITDA achieves comparable interpretability to SAEs at a different computational tradeoff [shabalin-etal-2025-interpreting-large-text-to-image-diffusion-models-with-dictionary-learning]

models: FLUX.1 · method: Sparse Autoencoders (SAE), Fixed-dictionary sparse recovery
ConceptAttention: Diffusion Transformers Learn Highly Interpretable Features (2025)measured

ConceptAttention output-space dot products localize concepts in DiT tokens

Details

Helbling et al. thread concept tokens through Flux-Schnell's frozen attention with a one-directional design so concepts read the model's state but never influence the generated image [helbling-etal-2025-conceptattention-diffusion-transformers-learn-highly-interpretable-features] A saliency map from image-concept dot products in the attention-output space reaches 83.07 accuracy / 71.04 mIoU, versus 74.92/59.90 in cross-attention space and 45.78/29.68 in value space [helbling-etal-2025-conceptattention-diffusion-transformers-learn-highly-interpretable-features] Concept-image alignment is thus concentrated in the output-projected subspace, beating all 11 compared zero-shot interpretability baselines [helbling-etal-2025-conceptattention-diffusion-transformers-learn-highly-interpretable-features] Performance concentrates in deeper layers (last 10 of 18) and middle diffusion timesteps, and the method is a passive read-out with no steering claim [helbling-etal-2025-conceptattention-diffusion-transformers-learn-highly-interpretable-features]

models: FLUX.1 [schnell] · method: Concept-Token Attention-Output Projection (ConceptAttention)
DifFRACT: Diffusion Feature Reconstruction and Attribution for Circuit Tracing (2026)measured

Timestep-conditioned transcoder directions give exact FLUX attribution graphs

Details

Mazur et al. (DifFRACT) train timestep-conditioned FiLM-modulated transcoders replacing FLUX.1[schnell]'s MLP sublayers, giving an exact linear feature-to-feature attribution across the denoising trajectory [mazur-etal-2026-timestep-conditioned-transcoder-directions-flux] The attribution graphs reveal a structural shift: text-stream contribution falls from 89.9% to 5.4% while image-stream contribution rises from 10.1% to 94.6% across 4 denoising steps [mazur-etal-2026-timestep-conditioned-transcoder-directions-flux] Feature-level activation scaling causally corrects color-bias failures on 60% of tested seeds only when feature suppression is combined with context suppression [mazur-etal-2026-timestep-conditioned-transcoder-directions-flux] Single-feature steering alone fails, evidencing causal structure distributed across coupled features rather than reducible to one direction [mazur-etal-2026-timestep-conditioned-transcoder-directions-flux]

models: FLUX.1 [schnell] · method: Transcoders, Causal interventions (steering)
SliderSpace: Decomposing the Visual Capabilities of Diffusion Models (2025)measured

SliderSpace realizes CLIP-PCA directions as causally independent LoRA sliders

Details

Gandikota et al. run PCA on CLIP embeddings of a diffusion model's own output samples, yielding orthogonal principal directions of its semantic variation [gandikota-etal-2025-sliderspace-diffusion-decomposition] Per direction, a low-rank LoRA adapter is trained via a cosine-alignment loss to make its induced CLIP-embedding shift align with that PCA direction, turning passive geometry into a reusable slider [gandikota-etal-2025-sliderspace-diffusion-decomposition] Demonstrated on SDXL-DMD, SDXL, SDXL-Turbo, and FLUX Schnell, sliders increase diversity at similar CLIP-Score with user-study win-rates 66-72% [gandikota-etal-2025-sliderspace-diffusion-decomposition] An ablation shows the cosine-alignment loss and the CLIP-space choice are both causally necessary, else the sliders collapse to junk or color/shape-only directions [gandikota-etal-2025-sliderspace-diffusion-decomposition]

models: FLUX.1 [schnell] · method: PCA, LoRA-trained direction realization
Orthogonal Concept Erasure for Diffusion Models (2026)measured

OCE erases diffusion concepts via a shared orthogonal rotation preserving norms and angles

Details

Sun et al. edit a diffusion model's cross-attention key/value weights in place via a single shared orthogonal matrix P solved in closed form from an orthogonal Procrustes problem [sun-etal-2026-orthogonal-concept-erasure-for-diffusion-models] Because P is orthogonal, neuron magnitudes and pairwise inter-neuron angles are preserved exactly, an algebraic guarantee rather than a measured statistic [sun-etal-2026-orthogonal-concept-erasure-for-diffusion-models] An ablation shows the design matters: additive scaling has negligible effect, layer-wise shared rotation gives a clean semantic shift, and neuron-wise rotation degrades image quality [sun-etal-2026-orthogonal-concept-erasure-for-diffusion-models] It erases object classes, styles, up to 100 celebrities simultaneously, and NSFW content on SD v1.4, transferring zero-shot to the DiT-architecture FLUX.1 [sun-etal-2026-orthogonal-concept-erasure-for-diffusion-models]

models: FLUX.1 · method: Orthogonal Weight-Rotation Concept Erasure (OCE)
Robust and Generalizable Safety Steering for Text-to-Image Diffusion Transformers (2026)measured

Position-aware SAE safety steering generalizes across domains within a diffusion model

Details

Xue et al. build position-aware SAEs over functionally distinct DiT intervention sites in FLUX.1 Dev and Stable Diffusion 3.5 Large, used as two independent testbeds [xue-etal-2026-safedig-safety-steering-diffusion] Robustness-aware routing prioritizes intervention sites, and a frozen SAE encoder acts as a reusable sparse safety dictionary while only the decoder adapts to the target domain [xue-etal-2026-safedig-safety-steering-diffusion] Safety learned from known source risk domains generalizes to held-out or emerging target risk domains within a single model [xue-etal-2026-safedig-safety-steering-diffusion] The demonstrated transfer is within-model across risk domains, not cross-model between FLUX and SD3.5 [xue-etal-2026-safedig-safety-steering-diffusion]

models: FLUX.1 · method: Sparse Autoencoders (SAE), Causal interventions (steering)

DDPM (Denoising Diffusion Probabilistic Models, HuggingFace Diffusers checkpoints)

Understanding the Latent Space of Diffusion Models through the Lens of Riemannian Geometry (2023)measured

A pullback Riemannian metric yields diffusion tangent-space directions

Details

Park et al. reframe the diffusion U-Net encoder's Jacobian as a pullback Riemannian metric on the latent x_t, whose eigenvectors form a local tangent basis [park-etal-2023-riemannian-diffusion-pullback] A power-spectral analysis of the tangent basis shifts from low- to high-frequency as t goes from T to 0, confirming coarse-to-fine generation [park-etal-2023-riemannian-diffusion-pullback] Grassmannian geodesic distance between samples' tangent spaces increases monotonically as t decreases, fastest for visually complex datasets [park-etal-2023-riemannian-diffusion-pullback] For Stable Diffusion v2.1, tangent-space geodesic distance negatively correlates with CLIP prompt similarity, collapsing below roughly t=0.7T [park-etal-2023-riemannian-diffusion-pullback] Single-timestep x-space editing is introduced as a causal application but validated only qualitatively [park-etal-2023-riemannian-diffusion-pullback]

models: DDPM (CelebA-HQ 256x256, HuggingFace Diffusers), DDPM+P2-weighting (AFHQ 256x256), DDPM (LSUN-Church 256x256, HuggingFace Diffusers), DDPM (LSUN-Bedroom 256x256, HuggingFace Diffusers), DDPM (LSUN-Cat 256x256), DDPM (LSUN-Horse 256x256), DDPM (ImageNet 256x256), DDPM+P2-weighting (FFHQ 256x256), DDPM+P2-weighting (Flowers 256x256) · method: Jacobian spectral direction analysis, Activation Steering (Addition)
Discovering Interpretable Directions in the Semantic Latent Space of Diffusion Models (2023)measured

Diffusion h-space directions arise from PCA, Jacobian SVD, and diff-of-means

Details

Haas et al. find diffusion h-space edit directions via three techniques needing no CLIP or text guidance [haas-etal-2023-interpretable-directions-diffusion] Per-timestep incremental PCA gives unsupervised global directions whose top 2-3 components are semantically dominant (pose, gender, age, smile) [haas-etal-2023-interpretable-directions-diffusion] Per-image Jacobian top singular vectors (via power iteration on J^T J) give locally-optimal directions, a per-sample notion distinct from population PCA [haas-etal-2023-interpretable-directions-diffusion] A difference-of-means vector gives supervised directions, and a linear-projection disentangling step cuts cross-attribute leakage (Glasses-on-Age 0.68 to 0.13) [haas-etal-2023-interpretable-directions-diffusion] Demonstrated on a plain DDPM (CelebA-HQ 256) plus LSUN-church/bedroom, a narrower model range than Asyrp [haas-etal-2023-interpretable-directions-diffusion]

models: DDPM (CelebA-HQ 256x256, HuggingFace Diffusers), DDPM (LSUN-Church 256x256, HuggingFace Diffusers), DDPM (LSUN-Bedroom 256x256, HuggingFace Diffusers) · method: PCA, Jacobian spectral direction analysis, Difference-in-means direction
General and Efficient Steering of Diffusion Models (2026)measured

A single RFM/AGOP direction per concept steers diffusion at a fraction of guidance cost

Details

Wang et al. train a Recursive Feature Machine on flattened U-Net/DiT block activations from forward-noised labeled images, taking the sign-corrected top eigenvectors of its AGOP matrix as one reusable per-concept steering direction [wang-etal-2026-general-and-efficient-steering-of-diffusion-models-a-single-rfm-agop-direction-per-concept-steers-unconditional-and-latent-diffusion-models-at-a-fraction-of-classifier-guidance-cost] Adding this direction during sampling steers unconditional DDPM/ADM U-Nets (CIFAR-10, ImageNet-256, CelebA-HQ), the transformer latent-diffusion SiT-XL/2, and Stable Diffusion 1.5 [wang-etal-2026-general-and-efficient-steering-of-diffusion-models-a-single-rfm-agop-direction-per-concept-steers-unconditional-and-latent-diffusion-models-at-a-fraction-of-classifier-guidance-cost] On CIFAR-10, NA-RFM reaches 96.6% guidance accuracy versus 77.1% and 86.0% baselines, with FID 41.4 versus 73.9 and 41.9 [wang-etal-2026-general-and-efficient-steering-of-diffusion-models-a-single-rfm-agop-direction-per-concept-steers-unconditional-and-latent-diffusion-models-at-a-fraction-of-classifier-guidance-cost] It runs one fixed low-rank direction across the sampling trajectory at a 16x sampling speedup over a training-free-guidance baseline [wang-etal-2026-general-and-efficient-steering-of-diffusion-models-a-single-rfm-agop-direction-per-concept-steers-unconditional-and-latent-diffusion-models-at-a-fraction-of-classifier-guidance-cost]

models: DDPM (CIFAR-10, unconditional), DDPM (ImageNet 256x256), DDPM (CelebA-HQ 256x256, HuggingFace Diffusers) · method: RFM/AGOP Steering Direction Extraction

DRC (Deep Repeating ConvLSTM)

Interpreting Emergent Planning in Model-Free Reinforcement Learning (2025)measured

A model-free Sokoban agent linearly encodes a causal plan

Details

Bush et al. probe a DRC(3,3) ConvLSTM agent trained model-free on Sokoban and find linear probes recover an internal plan predicting the agent's future actions [bush-etal-2025-interpreting-emergent-planning-in-model-free-reinforcement-learning] Causal interventions on the probed representation change subsequent behavior in the predicted direction, showing the plan is causally implemented [bush-etal-2025-interpreting-emergent-planning-in-model-free-reinforcement-learning] Extra test-time computation improves plan quality and solve rate [bush-etal-2025-interpreting-emergent-planning-in-model-free-reinforcement-learning] The evolving plan representation resembles a parallelized bidirectional search rather than a single forward rollout [bush-etal-2025-interpreting-emergent-planning-in-model-free-reinforcement-learning]

models: DRC(3,3) Sokoban Agent · method: Linear probing, Causal interventions (steering)
Planning in a Recurrent Neural Network That Plays Sokoban (2024)measured

A Sokoban RNN's plan representation predicts 50 steps ahead and generalizes OOD

Details

Taufeeque et al. recover a causal plan representation from a DRC ConvLSTM Sokoban agent via logistic-regression probes, with causal validation that probed directions drive behavior [taufeeque-etal-2024-planning-in-a-recurrent-neural-network-that-plays-sokoban] The plan representation predicts the agent's future actions roughly 50 steps ahead [taufeeque-etal-2024-planning-in-a-recurrent-neural-network-that-plays-sokoban] Plan length and quality increase over the network's early internal computation steps [taufeeque-etal-2024-planning-in-a-recurrent-neural-network-that-plays-sokoban] The representation generalizes robustly to out-of-distribution puzzles far larger than any seen in training, and also explains a level-start "pacing" behavior the training incentivizes [taufeeque-etal-2024-planning-in-a-recurrent-neural-network-that-plays-sokoban]

models: DRC(3,3) Sokoban Agent · method: Linear probing, Causal interventions (steering)

Adaptive Input Transformer LM (Baevski & Auli)

Transformer Feed-Forward Layers Are Key-Value Memories (2020)measured

FFN value vectors are vocabulary-space directions in upper layers

Details

Geva et al. reframe the FFN sublayer as an unnormalized key-value memory FF(x)=f(x.K^T).V, treating each value row as a residual-space direction [geva-etal-2020-ffn-key-value-memories] Projecting each value vector through the output embedding yields a vocabulary distribution whose top token increasingly agrees with the paired key's next token with depth (near 0% early to 3.5% by layer 16) [geva-etal-2020-ffn-key-value-memories] Human annotation finds a recognizable trigger pattern for every key (3.6 patterns/key), with shallow n-gram patterns in lower layers and semantic ones in upper layers [geva-etal-2020-ffn-key-value-memories] Each FFN output is a compositional weighted sum of hundreds of active memory cells, refining the residual stream by often vetoing toward a compromise token [geva-etal-2020-ffn-key-value-memories] This anticipates the logit-lens and the FFN-editing basis later exploited by ROME and MEMIT [geva-etal-2020-ffn-key-value-memories]

models: Adaptive Input Transformer LM (16-layer, WikiText-103) · method: Logit Lens

XGLM

How Do Multilingual Language Models Remember Facts? (2024)measured

A last-token function vector transplants across languages in two stages

Details

Fierro et al. use activation patching and causal-mediation across XGLM-7.5B, EuroLLM-9B, and mT5-XL to disentangle when relation vs language information reaches the last token [fierro-etal-2024-how-do-multilingual-language-models-remember-facts] The last-token representation acts as a function vector that can be causally transplanted into a different-language prompt [fierro-etal-2024-how-do-multilingual-language-models-remember-facts] Transplantation is quantified by the percentage change in the correct object's probability after patching [fierro-etal-2024-how-do-multilingual-language-models-remember-facts] Relation and language information compose in two separable stages rather than being entangled in one representation [fierro-etal-2024-how-do-multilingual-language-models-remember-facts]

models: XGLM-7.5B · method: Activation patching, Function vector extraction (causal-mediation head selection + sum)

EuroLLM

How Do Multilingual Language Models Remember Facts? (2024)measured

A last-token function vector transplants across languages in two stages

Details

Fierro et al. use activation patching and causal-mediation across XGLM-7.5B, EuroLLM-9B, and mT5-XL to disentangle when relation vs language information reaches the last token [fierro-etal-2024-how-do-multilingual-language-models-remember-facts] The last-token representation acts as a function vector that can be causally transplanted into a different-language prompt [fierro-etal-2024-how-do-multilingual-language-models-remember-facts] Transplantation is quantified by the percentage change in the correct object's probability after patching [fierro-etal-2024-how-do-multilingual-language-models-remember-facts] Relation and language information compose in two separable stages rather than being entangled in one representation [fierro-etal-2024-how-do-multilingual-language-models-remember-facts]

models: EuroLLM-9B · method: Activation patching, Function vector extraction (causal-mediation head selection + sum)
When Meanings Meet: Investigating the Emergence and Quality of Shared Concept Spaces during Multilingual Language Model Training (2026)measured

A language-agnostic concept representation causally alters translation across checkpoints

Details

Korner et al. track EuroLLM-1.7B's own pretraining checkpoints, using activation patching to isolate a cross-lingual concept representation and inject it into a translation prompt [korner-etal-2026-when-meanings-meet-investigating-the-emergence-and-quality-of-shared-concept-spaces-during-multilingual-language-model-training] The injected representation causally alters the output translation independent of the source language it was derived from [korner-etal-2026-when-meanings-meet-investigating-the-emergence-and-quality-of-shared-concept-spaces-during-multilingual-language-model-training] Shared concept spaces emerge early in pretraining and continue to refine with more training [korner-etal-2026-when-meanings-meet-investigating-the-emergence-and-quality-of-shared-concept-spaces-during-multilingual-language-model-training] Alignment quality remains language-dependent per a fine-grained manual error analysis [korner-etal-2026-when-meanings-meet-investigating-the-emergence-and-quality-of-shared-concept-spaces-during-multilingual-language-model-training]

models: EuroLLM-1.7B · method: Activation patching

mT5

How Do Multilingual Language Models Remember Facts? (2024)measured

A last-token function vector transplants across languages in two stages

Details

Fierro et al. use activation patching and causal-mediation across XGLM-7.5B, EuroLLM-9B, and mT5-XL to disentangle when relation vs language information reaches the last token [fierro-etal-2024-how-do-multilingual-language-models-remember-facts] The last-token representation acts as a function vector that can be causally transplanted into a different-language prompt [fierro-etal-2024-how-do-multilingual-language-models-remember-facts] Transplantation is quantified by the percentage change in the correct object's probability after patching [fierro-etal-2024-how-do-multilingual-language-models-remember-facts] Relation and language information compose in two separable stages rather than being entangled in one representation [fierro-etal-2024-how-do-multilingual-language-models-remember-facts]

models: mT5-xl · method: Activation patching, Function vector extraction (causal-mediation head selection + sum)

OpenAI Embeddings

Mapping Semantic & Syntactic Relationships with Geometric Rotation in Embedding Space (2025)measured

Analogical relations fit a rotational operator better than an additive offset

Details

Freenor & Alvarez test whether analogical relations (king-queen, walk-walked) are better modeled as an additive offset or a rotational operator [freenor-alvarez-2025-geometric-rotation-embedding-space] Rotor-invariant shift estimation fits a best-approximating rotation and measures residual invariance across many real word pairs [freenor-alvarez-2025-geometric-rotation-embedding-space] The rotational model achieves lower, more consistent residual error than the additive-offset model across text-embedding-3-large, BGE-M3, and multilingual BERT [freenor-alvarez-2025-geometric-rotation-embedding-space] This is a quantified comparative geometric claim on real embeddings; no causal intervention is performed [freenor-alvarez-2025-geometric-rotation-embedding-space]

models: text-embedding-3-large · method: Rotor-invariant shift estimation (RISE)
Disentangling Dense Embeddings with Sparse Autoencoders (2024)measured

SAE decoder cosine reveals feature splitting in dense embeddings and steers search

Details

O'Neill et al. train TopK SAEs directly on OpenAI text-embedding-3-small embeddings of 420,000+ scientific abstracts, one of the first SAE applications to dense embeddings rather than LLM activations [oneill-etal-2024-disentangling-dense-embeddings-sparse-autoencoders] A decoder-direction cosine-similarity matrix between different-sized SAEs shows larger SAEs split a smaller SAE's feature direction, with over 1,100 triplets recurring at cosine >0.95 [oneill-etal-2024-disentangling-dense-embeddings-sparse-autoencoders] Steering semantic search is proven algebraically equivalent to adding a scaled decoder feature vector to the output embedding [oneill-etal-2024-disentangling-dense-embeddings-sparse-autoencoders] Down-weighting one feature and up-weighting an orthogonal one achieves ~20% higher accuracy than query rewriting at a target cosine similarity of 0.75 [oneill-etal-2024-disentangling-dense-embeddings-sparse-autoencoders]

models: text-embedding-3-small · method: TopK (k-sparse) Sparse Autoencoders, Causal interventions (steering)

BGE

Mapping Semantic & Syntactic Relationships with Geometric Rotation in Embedding Space (2025)measured

Analogical relations fit a rotational operator better than an additive offset

Details

Freenor & Alvarez test whether analogical relations (king-queen, walk-walked) are better modeled as an additive offset or a rotational operator [freenor-alvarez-2025-geometric-rotation-embedding-space] Rotor-invariant shift estimation fits a best-approximating rotation and measures residual invariance across many real word pairs [freenor-alvarez-2025-geometric-rotation-embedding-space] The rotational model achieves lower, more consistent residual error than the additive-offset model across text-embedding-3-large, BGE-M3, and multilingual BERT [freenor-alvarez-2025-geometric-rotation-embedding-space] This is a quantified comparative geometric claim on real embeddings; no causal intervention is performed [freenor-alvarez-2025-geometric-rotation-embedding-space]

models: BGE-M3 · method: Rotor-invariant shift estimation (RISE)
A Geometric Taxonomy of Hallucinations in Large Language Models (2026)measured

A von Mises-Fisher grounding direction separates grounded from hallucinated responses

Details

Marin L2-normalizes embeddings from five frozen sentence encoders (sentence-t5-large primary, plus all-mpnet-base-v2, all-MiniLM-L6-v2, bge-small-en-v1.5, gte-small) onto the unit hypersphere [marin-2026-a-geometric-taxonomy-of-hallucinations-in-large-language-models] A grounding direction is extracted as the von Mises-Fisher mean of context embeddings [marin-2026-a-geometric-taxonomy-of-hallucinations-in-large-language-models] A Directional Grounding Index (cosine between a response's displacement vector and this direction) separates grounded from hallucinated responses at mean AUROC 0.805 on HaluEval QA (0.535-0.579 on TruthfulQA) [marin-2026-a-geometric-taxonomy-of-hallucinations-in-large-language-models] A companion Spherical Grounding Index gives 1.180 (grounded) versus 0.910 (hallucinated), both computed on frozen off-the-shelf encoders rather than a generative LLM's hidden states [marin-2026-a-geometric-taxonomy-of-hallucinations-in-large-language-models]

models: BGE-small-en-v1.5 · method:
Probing Spectrum-Like Organization of States of Mind in Transformer Representation Spaces (2026)measured

A 'state of mind' spectrum is linearly decodable across five encoders

Details

Zhao annotates 636 sentences with a continuous energy score and one of seven ordered tiers, an informally-sourced psychological taxonomy explicitly disclaimed as non-clinical [zhao-2026] Across five frozen representations (BGE-large, all-mpnet-base-v2, all-MiniLM-L6-v2, Qwen3-Embedding-0.6B, and Qwen2.5-3B-Instruct's layer 24), both the score (Ridge R^2 0.63-0.79) and the seven tiers (weighted F1 0.57-0.72) are linearly decodable above chance [zhao-2026] Permutation tests and a TF-IDF baseline (R^2 0.44) rule out simple lexical confounds [zhao-2026] Projecting out a single Ridge-derived direction (1.6-4.8% of variance) collapses decodability far more than a random direction, though the structure is only approximately one-dimensional [zhao-2026]

models: BGE-large-en-v1.5 · method: Linear probing, Mlp probing, UMAP

OpenFlamingo

Multimodal Function Vectors for Visual Relations (2026)measured

Multimodal function vectors causally raise visual-relation accuracy

Details

Fu et al. use causal-mediation (Average Indirect Effect per attention head) to identify a top-10-head causal subnetwork per visual relation in OpenFlamingo-4B [fu-etal-2026-multimodal-function-vectors-visual-relations] Summing those heads' mean activations gives a multimodal function vector whose injection raises zero-shot accuracy from 4.8% to 9.7% [fu-etal-2026-multimodal-function-vectors-visual-relations] Fine-tuning the function vector on a held-out novel-object set reaches 13.6% versus a 9.4% ICL baseline [fu-etal-2026-multimodal-function-vectors-visual-relations] Composite function vectors for entirely untrained relations roughly double accuracy again (8.1% to 16.8%) [fu-etal-2026-multimodal-function-vectors-visual-relations] The injection effect peaks at intermediate layers (~layer 19) with a 6-12 head sweet spot [fu-etal-2026-multimodal-function-vectors-visual-relations]

models: OpenFlamingo-4B · method: Function vector extraction (causal-mediation head selection + sum), Causal interventions (steering)
Visual concept ranking uncovers medical shortcuts used by large multimodal models (2026)measured

Medical-LMM CAV sensitivity scores track ground-truth causal effects

Details

Janizek et al. extract concept activation vectors from three OpenFlamingo models and score a Visual Concept Ranking sensitivity as the output logit's directional derivative along each CAV [janizek-etal-2026-visual-concept-ranking-medical-shortcuts-large-multimodal-models] On an 800-condition synthetic benchmark, VCR sensitivity correlates with the true interventional effect at Pearson r=0.53 (p=4.9e-49), exceeding 0.6 for 6 of 8 feature pairs [janizek-etal-2026-visual-concept-ranking-medical-shortcuts-large-multimodal-models] Under a spurious-correlation stress test, VCR identifies a spurious feature's causal sign 92% of the time versus 18% for a correlational CLIP baseline [janizek-etal-2026-visual-concept-ranking-medical-shortcuts-large-multimodal-models] On dermatology tasks VCR flagged blue/purple ink-marking as a "malignant" driver specific to darker-skin samples, validated by targeted image edits [janizek-etal-2026-visual-concept-ranking-medical-shortcuts-large-multimodal-models] Distinct semantic labels (Purpura/tattoo) coincided on one direction, a reminder that a CAV's assigned label may not match what it encodes [janizek-etal-2026-visual-concept-ranking-medical-shortcuts-large-multimodal-models]

models: OpenFlamingo-3B-Instruct, OpenFlamingo-4B, OpenFlamingo-9B · method: Linear probing, TCAV directional-derivative sensitivity testing

Leela Chess Zero

The Algorithm Is Not the Behavior: Learned Priors Override Look-Ahead in a Chess-Playing Neural Network (2025)measured

Leela Chess Zero's correct moves appear early, then get overridden

Details

Sandmann, Lapuschkin & Samek extend the logit lens (functionally equivalent to zero-ablating later sublayers) to Leela Chess Zero's Post-LN policy network [sandmann-etal-2025-the-algorithm-is-not-the-behavior-learned-priors-override-look-ahead-in-a-chess-playing-neural-network] Playing strength and puzzle-solving ability rise monotonically with depth, but policy distributions follow non-smooth, non-monotonic trajectories [sandmann-etal-2025-the-algorithm-is-not-the-behavior-learned-priors-override-look-ahead-in-a-chess-playing-neural-network] Correct puzzle solutions are discovered in intermediate layers but subsequently discarded, with move rankings poorly correlated to the final output until late in the network [sandmann-etal-2025-the-algorithm-is-not-the-behavior-learned-priors-override-look-ahead-in-a-chess-playing-neural-network] This contrasts with the smooth distributional convergence typical of language models, evidencing iterative inference where a late stage overrides an already-computed answer [sandmann-etal-2025-the-algorithm-is-not-the-behavior-learned-priors-override-look-ahead-in-a-chess-playing-neural-network]

models: Leela Chess Zero (policy network) · method: Logit Lens, Causal interventions (steering)

PaliGemma

Textual Steering Vectors Can Improve Visual Understanding in Multimodal Large Language Models (2025)measured

Text-only steering directions transfer causally to MLLM image tokens

Details

Gan et al. test whether steering directions found in a text-only LLM backbone stay causal after multimodal fine-tuning, applied to image tokens it never saw [gan-etal-2025-textual-steering-vectors-improve-visual-understanding-mllms] For four visual-concept taxonomies they extract directions from Gemma2-2B/9B and Llama-3.1-8B via mean-shift, probing, and SAE decoders, using text only [gan-etal-2025-textual-steering-vectors-improve-visual-understanding-mllms] Adding these directions to image tokens in PaliGemma2-3B/10B and Idefics3-8B-Llama3 raises CV-Bench spatial accuracy by up to +7.3%, with mean-shift strongest [gan-etal-2025-textual-steering-vectors-improve-visual-understanding-mllms] Frozen-hyperparameter generalization to five held-out datasets gives +7.6% average (up to +34.2% on CLEVR for Idefics3) [gan-etal-2025-textual-steering-vectors-improve-visual-understanding-mllms] Each visual-concept taxonomy activates fewer than 10 of 16k-32k SAE features in the text backbone, and prompting is far weaker than steering for multimodal reasoning [gan-etal-2025-textual-steering-vectors-improve-visual-understanding-mllms]

models: PaliGemma2 3B mix-448, PaliGemma2 10B mix-448 · method: Difference-in-means direction, Linear probing, Sparse Autoencoders (SAE), Activation Steering (Addition)

Idefics

Textual Steering Vectors Can Improve Visual Understanding in Multimodal Large Language Models (2025)measured

Text-only steering directions transfer causally to MLLM image tokens

Details

Gan et al. test whether steering directions found in a text-only LLM backbone stay causal after multimodal fine-tuning, applied to image tokens it never saw [gan-etal-2025-textual-steering-vectors-improve-visual-understanding-mllms] For four visual-concept taxonomies they extract directions from Gemma2-2B/9B and Llama-3.1-8B via mean-shift, probing, and SAE decoders, using text only [gan-etal-2025-textual-steering-vectors-improve-visual-understanding-mllms] Adding these directions to image tokens in PaliGemma2-3B/10B and Idefics3-8B-Llama3 raises CV-Bench spatial accuracy by up to +7.3%, with mean-shift strongest [gan-etal-2025-textual-steering-vectors-improve-visual-understanding-mllms] Frozen-hyperparameter generalization to five held-out datasets gives +7.6% average (up to +34.2% on CLEVR for Idefics3) [gan-etal-2025-textual-steering-vectors-improve-visual-understanding-mllms] Each visual-concept taxonomy activates fewer than 10 of 16k-32k SAE features in the text backbone, and prompting is far weaker than steering for multimodal reasoning [gan-etal-2025-textual-steering-vectors-improve-visual-understanding-mllms]

models: Idefics3-8B-Llama3 · method: Difference-in-means direction, Linear probing, Sparse Autoencoders (SAE), Activation Steering (Addition)

StyleGAN

GANSpace: Discovering Interpretable GAN Controls (2020)measured

GANSpace PCA finds near-independent, unimodal, content-style-separated directions

Details

Harkonen et al. find interpretable GAN edit directions entirely unsupervised via PCA on StyleGAN/StyleGAN2 W space and a BigGAN feature tensor [harkonen-etal-2020-ganspace] For BigGAN, which lacks a learned intermediate latent, PCA components are transferred back to the z latent via linear regression [harkonen-etal-2020-ganspace] The StyleGAN2 latent's PCA coordinates are nearly Gaussian/unimodal (entropies 6.9-8.7 bits) and nearly independent (pairwise mutual information 0-0.3 bits) [harkonen-etal-2020-ganspace] The first ~20 principal components control large geometric/viewpoint changes, with the first 100 of 512 dims capturing 85% of variance [harkonen-etal-2020-ganspace] Layer-wise restriction of a PCA direction cleanly separates content from style (e.g. pure head rotation at layers 0-2), an effect absent for random directions [harkonen-etal-2020-ganspace]

models: StyleGAN2 (trained on FFHQ, 1024x1024) · method: PCA, Causal interventions (steering)
InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANs (2020)measured

InterFaceGAN SVM boundary normals are linear, editable, and partially entangled

Details

Shen et al. train a linear SVM per attribute (gender, age, pose, smile, eyeglasses) in PGGAN and StyleGAN latent spaces, taking each boundary normal as that attribute's direction [shen-etal-2020-interfacegan] A proven linear relation ties latent distance-to-boundary to the generated image's semantic score, with PGGAN boundaries over 95% validation accuracy [shen-etal-2020-interfacegan] Boundary-normal cosine similarity measures entanglement: gender/age/eyeglasses are correlated while pose/smile are near-independent, and StyleGAN's W space is far more disentangled than Z [shen-etal-2020-interfacegan] Conditional manipulation, projecting out a correlated direction, removes leakage while preserving the target edit [shen-etal-2020-interfacegan] Layer-wise ablation localizes each attribute to a distinct subset of StyleGAN's style layers, and W-space edits best preserve identity [shen-etal-2020-interfacegan]

models: StyleGAN (trained on FFHQ, 1024x1024) · method: Linear probing, Conditional manipulation via boundary orthogonalization, Causal interventions (steering)

BigGAN

GANSpace: Discovering Interpretable GAN Controls (2020)measured

GANSpace PCA finds near-independent, unimodal, content-style-separated directions

Details

Harkonen et al. find interpretable GAN edit directions entirely unsupervised via PCA on StyleGAN/StyleGAN2 W space and a BigGAN feature tensor [harkonen-etal-2020-ganspace] For BigGAN, which lacks a learned intermediate latent, PCA components are transferred back to the z latent via linear regression [harkonen-etal-2020-ganspace] The StyleGAN2 latent's PCA coordinates are nearly Gaussian/unimodal (entropies 6.9-8.7 bits) and nearly independent (pairwise mutual information 0-0.3 bits) [harkonen-etal-2020-ganspace] The first ~20 principal components control large geometric/viewpoint changes, with the first 100 of 512 dims capturing 85% of variance [harkonen-etal-2020-ganspace] Layer-wise restriction of a PCA direction cleanly separates content from style (e.g. pure head rotation at layers 0-2), an effect absent for random directions [harkonen-etal-2020-ganspace]

models: BigGAN-deep (512px) · method: PCA, Causal interventions (steering)

GELU Toy Transformer

Improving Dictionary Learning with Gated Sparse Autoencoders (2024)measured

Gated SAEs eliminate shrinkage and halve the required L0

Details

Rajamanoharan et al. show a standard L1 ReLU SAE conflates deciding which directions are active with estimating their magnitudes, causing shrinkage (systematic underestimation) [rajamanoharan-etal-2024-gated-saes] The Gated SAE splits these into weight-tied gate and magnitude sub-networks, applying the L1 penalty only to the gate's pre-activations [rajamanoharan-etal-2024-gated-saes] Measured via relative-reconstruction-bias gamma, Gated SAEs reach gamma~1 while baseline SAEs show gamma<1 (0.72 at a Gemma-7B site) [rajamanoharan-etal-2024-gated-saes] Gated SAEs need roughly half the L0 for equal loss recovery across GELU-1L, Pythia-2.8B, and Gemma-7B, with at-least-equal interpretability [rajamanoharan-etal-2024-gated-saes] Under tied weights the construction is provably equivalent to a JumpReLU encoder [rajamanoharan-etal-2024-gated-saes]

models: GELU-1L · method: Gated Sparse Autoencoders

HyenaDNA

Sparse Autoencoders Reveal Interpretable Structure in Small Gene Language Models (2025)measured

Sparse-autoencoder features on a real small genomic language model correspond to nucleotide identity and real transcription-factor binding motifs

Details

Guan, He & Zhang (2025) train a sparse autoencoder (32x expansion, 8,192-feature dictionary, L1=0.1) on layer-3 activations of real HyenaDNA-small-32k, pretrained at single-nucleotide resolution on the human reference genome, extracted from real GRCh38 sequences [guan-he-zhang-2025-sparse-autoencoders-reveal-interpretable-structure-in-small-gene-language-models] Individual SAE features correspond to nucleotide-identity positions (e.g. a feature tracking cytosine) and to real transcription-factor binding sites, validated against independent ground-truth JASPAR motif annotations on chromosome 14 via nucleotide-level precision/recall/F1 (named motif matches include MA1596.1, MA2121.1 [C2H2 zinc finger class] and MA0052.5 [MADS-box class], with strand specificity) [guan-he-zhang-2025-sparse-autoencoders-reveal-interpretable-structure-in-small-gene-language-models] Purely correlational -- no steering, ablation, or intervention experiment is performed; the paper's own conclusion frames causal use of the SAE features as future work [guan-he-zhang-2025-sparse-autoencoders-reveal-interpretable-structure-in-small-gene-language-models]

models: HyenaDNA-small-32k · method: Sparse Autoencoders (SAE)

Mamba

Emergence and Effectiveness of Task Vectors in In-Context Learning: An Encoder-Decoder Perspective (2025)measured

In-context task representations are k-NN-decodable and causally patchable across real LLMs and an SSM

Details

Han, Song, Gore & Agrawal define Task Decodability, a k-NN classifier score on the hidden state immediately before the target token, and show it strongly correlates with ICL accuracy across real Llama-3.1-8B/70B, Gemma-2-2B/9B/27B, OLMo-7B (across pretraining checkpoints), and Mamba-8B on POS-tagging and bitwise-arithmetic tasks [han-song-etal-2025-emergence-of-abstractions-task-vectors-icl] Positive activation-patching interventions on well-separated task representations improve accuracy up to +14pp, while negative interventions degrade it up to -15pp, versus only +/-2-6pp for overlapping tasks such as XOR/XNOR [han-song-etal-2025-emergence-of-abstractions-task-vectors-icl] Finetuning the first 10 layers raises Task Decodability from 0.68 to 0.95 (POS) and 0.43 to 0.85 (bitwise), with accuracy gains of 37 and 24 points respectively over finetuning the last 10 layers instead [han-song-etal-2025-emergence-of-abstractions-task-vectors-icl]

models: Mamba-8B · method:

Alpaca

The Effectiveness of Style Vectors for Steering LLMs: A Human Evaluation (2026)measured

Human evaluation confirms per-layer style vectors steer emotion

Details

Diallo et al. extract per-layer style/emotion vectors as contrastive mean-activation differences and inject them across all layers of Alpaca and Llama-3-8B-Lexi-Uncensored [diallo-etal-2026-the-effectiveness-of-style-vectors-for-steering-llms-a-human-evaluation] The first large-scale human evaluation of activation steering (7,000+ ratings, 190 participants) shows moderate steering (lambda ~0.15) reliably shifts perceived emotion [diallo-etal-2026-the-effectiveness-of-style-vectors-for-steering-llms-a-human-evaluation] Effects are large for disgust (partial eta-squared 0.616) and fear (0.540) but minimal for surprise (0.042) [diallo-etal-2026-the-effectiveness-of-style-vectors-for-steering-llms-a-human-evaluation] Human ratings agree strongly with an automated classifier (mean r=0.776), and Llama-3 steers more consistently than Alpaca (p<0.001) [diallo-etal-2026-the-effectiveness-of-style-vectors-for-steering-llms-a-human-evaluation]

models: Alpaca · method: Activation Steering (Addition)
Inference-Time Intervention: Eliciting Truthful Answers from a Language Model (2023)measured

Truthfulness is linearly decodable per attention head, and ITI steers it

Details

Li et al. train a separate linear probe per attention head on TruthfulQA in LLaMA-7B, finding truthfulness strongly decodable in only a sparse subset of heads (best head 83.3%) rather than one whole-residual-stream direction [li-etal-2023] Even within a head the geometry is not perfectly one-dimensional, as a second probe constrained orthogonal to the first still classifies above chance [li-etal-2023] Inference-Time Intervention shifts activations along the winning mass-mean direction (outperforming a probe-weight direction and contrast-consistent search) on the top-K most truthful heads [li-etal-2023] This raises TruthfulQA true*informative substantially (LLaMA-7B 30.5% to 43.5%, Alpaca-7B 32.5% to 65.1%, Vicuna-7B 51.5% to 74.0%) [li-etal-2023]

models: Alpaca-7B · method: Linear probing, Difference-in-means direction, Contrast-Consistent Search (CCS), Activation Steering (Addition), Causal interventions (steering)

InternVL

Linear Mechanisms for Spatiotemporal Reasoning in Vision Language Models (2026)measured

A contrastive spatial-ID direction is causally bound into object tokens across 11 real VLMs

Details

Kang, Chen, Gkioxari & Perona (2026) define a canonical spatial-ID direction as a difference-of-means over grid position across 11 real VLMs (LLaVA, LLaMA, Qwen, InternVL, Gemma families), well-approximated by a fixed linear map of positional encoding (rank-3 fit, R^2 ~0.85) [kang-etal-2026-linear-mechanisms-for-spatiotemporal-reasoning-in-vlms] Activation-patching 'mirror swap' localizes the effect to object-word tokens at intermediate layers, with a color-swap control showing near-null effect [kang-etal-2026-linear-mechanisms-for-spatiotemporal-reasoning-in-vlms] Directly substituting a target spatial ID into an object token's residual stream shifts ground-truth logit-based belief 64.4% of the way to the target, versus 29.5% for a noise baseline; an analogous temporal-ID binding mechanism extends to video VLMs (LLaVA-Video, VideoLLaMA3, Qwen2.5) [kang-etal-2026-linear-mechanisms-for-spatiotemporal-reasoning-in-vlms]

models: InternVL3-8B · method: Difference-in-means direction, Activation patching

Apple Health Foundation Models (PPG/Accel)

Emergent Symbolic Structure in Health Foundation Models: Extraction, Alignment, and Cross-Modal Transfer (2026)measured

Unsupervised PCA/ICA/NMF symbol directions in real health foundation models transfer near-losslessly across modalities via linear alignment

Details

Katuwal, Koparkar, Abbaspourazad, Mishra & Kirthivasan (2026) decompose frozen subject-level embeddings from three real pretrained health foundation models (PPG ViT, PPG EfficientNet, Accel ViT; ~172K wearable-sensor participants) into "symbol" directions via linear projections (PCA/ICA/NMF), finding selective association with 23 real health/physiological targets [katuwal-etal-2026-emergent-symbolic-structure-in-health-foundation-models] After linear (CCA/bijective) cross-modal alignment, a linear-classifier-based cross-modal transfer retains over 95% of in-domain AUC (health conditions 98.7%, physiological markers approximately 100%) -- purely linear/affine throughout, with no causal validation [katuwal-etal-2026-emergent-symbolic-structure-in-health-foundation-models]

models: PPG ViT (wearable photoplethysmography foundation model) · method:

BART

Editing Factual Knowledge in Language Models (2021)measured

KnowledgeEditor gates the fine-tuning gradient with a hypernetwork outer product

Details

De Cao et al. condition small FFNNs on a bidirectional-LSTM encoding of an edit request to predict per-matrix vectors forming an outer-product gate and bias on the loss gradient [decao-etal-2021-knowledge-editor] A KL-divergence-in-output-space constraint, not a parameter-space norm, prevents collateral damage: an Lp-parameter constraint instead collapses retain accuracy from 98.14 to 45.10 [decao-etal-2021-knowledge-editor] It uses rather than finds geometry, making no claim the edited fact was pre-encoded, and its probe framing concerns only which weight matrices receive large updates [decao-etal-2021-knowledge-editor] Tested on BERT-base (FEVER 98.80% success) and BART-base (zsRE 94.65%), it is the direct architectural predecessor of MEND [decao-etal-2021-knowledge-editor]

models: BART-base · method: KnowledgeEditor — Constrained Hyper-Network Weight Editing
Implicit Representations of Meaning in Neural Language Models (2021)measured

Splicing probed entity-state representations causally shifts generation

Details

Li, Nye & Andreas train linear probes to decode entity state (Alchemy beaker contents, TextWorld room/object state) from fine-tuned BART and T5 encoder representations [li-nye-andreas-2021-implicit-representations-of-meaning] They go beyond decoding with a causal splice WITHIN each model (not between BART and T5): replacing one context's encoded beaker-2 description with another context's encoding to build a synthetic "mixed" state [li-nye-andreas-2021-implicit-representations-of-meaning] Generating from the spliced representation lands in the human-predicted continuation set 57.7% (BART) / 75.4% (T5) of the time, versus 20.4%/37.9% and 16.1%/29.1% for the unmixed source contexts [li-nye-andreas-2021-implicit-representations-of-meaning] The probe only reads the representation while the splice is the intervention, so this is a genuine geometry-tied causal effect [li-nye-andreas-2021-implicit-representations-of-meaning]

models: BART-base · method: Linear probing, Causal interventions (steering)
Transformer-Patcher: One Mistake Worth One Neuron (2023)measured

Transformer-Patcher corrects mistakes by adding one key-value neuron per patch

Details

Huang et al. correct a mistake by appending one key row and one value column to the last FFN layer, adding a key direction that detects the trigger via inner product and a value direction to the output [huang-etal-2023-transformer-patcher] This is the same outer-product pattern as ROME but realized by growing the weight matrices rather than editing existing entries [huang-etal-2023-transformer-patcher] A margin-based memory loss enforces locality across many sequential patches, not any geometric constraint between different patches' key directions [huang-etal-2023-transformer-patcher] It uses rather than finds geometry, correcting thousands of sequential mistakes on BERT-base and BART-base where MEND and KnowledgeEditor collapse after only a few [huang-etal-2023-transformer-patcher]

models: BART-base · method: Transformer-Patcher — One Mistake, One Added Neuron
SLIM: Sparse Latent Steering for Interpretable and Property-Directed LLM-Based Molecular Editing (2026)measured

SLIM SAE features linearly correlate with molecular properties and steer editing

Details

Zhang et al. train a Gated SAE on a single layer of four frozen LLM-based molecular editors (DrugAssist, GeLLM3O-LLaMA3, GeLLM3O-Mistral, MolGen) with learnable per-property importance gates [zhang-etal-2026-slim-sparse-latent-steering-molecular] Individual top SAE features correlate strongly with molecular properties (Spearman rho up to +0.93 for molecular weight, -0.85 for QED), with six of eight properties reaching |rho|>=0.52 from a single feature [zhang-etal-2026-slim-sparse-latent-steering-molecular] Properties like HBD and DRD2 show weaker single-feature correlation, explicitly attributed to distributed rather than monosemantic encoding [zhang-etal-2026-slim-sparse-latent-steering-molecular] A gradient-based direction projected through the SAE's top features and added to the frozen model's residual stream causally steers property-directed molecular editing [zhang-etal-2026-slim-sparse-latent-steering-molecular]

models: MolGen · method: Gated Sparse Autoencoders, Activation Steering (Addition)

CosyVoice

Interpreting and Steering a Text-to-Speech Language Model with Sparse Autoencoders (2026)measured

SAE feature modality composition shifts by depth in a TTS LM and steers speech

Details

Koriagin et al. train BatchTopK SAEs on residual-stream activations at multiple layers of CosyVoice3's Qwen2.5-0.5B TTS backbone [koriagin-etal-2026-interpreting-steering-tts-with-saes] Feature modality composition is systematic by depth: early/middle layers mixed, layers 16-20 audio-heavy, and the final hidden state reverts to a mostly text-modal subspace (layer-20 text-detection AUROC 0.921) [koriagin-etal-2026-interpreting-steering-tts-with-saes] Encoding through the frozen SAE, shifting selected features, and decoding back raises laughter probability from 0.02 to 0.79, flips perceived speaker gender, and controls speech rate while preserving content [koriagin-etal-2026-interpreting-steering-tts-with-saes] This is the first such layerwise modality-composition analysis of a generative TTS LM backbone [koriagin-etal-2026-interpreting-steering-tts-with-saes]

models: CosyVoice3 (Qwen2.5-0.5B backbone) · method: BatchTopK Sparse Autoencoders, Causal interventions (steering)

Gaperon

Language-Switching Triggers Take a Latent Detour Through Language Models (2026)measured

A backdoor trigger hides orthogonal to the probed language direction

Details

Kulumba et al. plant a 9-token Latin trigger during pretraining of Gaperon-8B that switches generation from English to French, then decompose a 3-phase causal circuit via activation patching [kulumba-etal-2026-language-switching-triggers-take-a-latent-detour-through-language-models] In the critical phase, linear language-identity probes classify the representation as English throughout mid-to-late layers even though patching confirms the trigger is causally present [kulumba-etal-2026-language-switching-triggers-take-a-latent-detour-through-language-models] The trigger occupies a subspace orthogonal to the natural language-identity direction, converted to French only by a final-layer MLP that accounts for approximately 62% (plus or minus 8%) of the total causal effect [kulumba-etal-2026-language-switching-triggers-take-a-latent-detour-through-language-models]

models: Gaperon-8B · method: Activation patching, Linear probing

XLM-RoBERTa

The Geometry of Multilingual Language Model Representations (2022)measured

A language difference-of-means direction causally steers target-language output

Details

Chang, Tu & Bergen identify language-sensitive axes via linear discriminant analysis and take the raw difference between two languages' mean representations [chang-tu-bergen-2022] Shifting a representation by this direction and projecting onto language B's subspace produces 4.7x more predicted tokens in target language B (10% to 47%) and 3.5x fewer in language A (75% to 21%) [chang-tu-bergen-2022] The unmodified model otherwise predicts the original evaluation language 99.5% of the time [chang-tu-bergen-2022] A separate visual claim that language-neutral axes encode token position on spiral/torus curves is not independently quantified and is not catalogued here [chang-tu-bergen-2022]

models: XLM-RoBERTa base · method: Causal interventions (steering), Linear Discriminant Analysis (LDA)
Naturalistic Causal Probing for Morpho-Syntax (2023)measured

A naturalistic causal ATE direction nearly coincides with the top PCA direction

Details

Amini et al. estimate the average treatment effect of grammatical gender and number on mBERT, XLM-RoBERTa-base, and GPT2-small-Spanish using dependency-tree-reinflected naturalistic counterfactuals [amini-etal-2023-naturalistic] The first principal component of the paired representations has cosine similarity ~0.99 with the estimated ATE direction across all three models [amini-etal-2023-naturalistic] In BERT/RoBERTa this PC explains ~20% of paired variance, while in GPT2-small-Spanish over 50% concentrates in the first 1-2 components, a starker one-dimensional encoding [amini-etal-2023-naturalistic] The paired causal estimator is far more stable across treebanks than a naive correlational one and better reconstructs the true counterfactual distribution [amini-etal-2023-naturalistic]

models: XLM-RoBERTa base · method: Naturalistic Causal Probing (input-level dependency-tree reinflection)
Naturalistic Causal Probing for Morpho-Syntax (2023)measured

Probing accuracy collapses under a naturalistic counterfactual test

Details

Amini et al. compare correlational probing for grammatical gender/number against evaluation on naturalistic counterfactual test pairs [amini-etal-2023-naturalistic] Probes reaching above 90% on the standard split drop to roughly 60% (near chance for a binary property) on dependency-tree-verified counterfactual sentences [amini-etal-2023-naturalistic] Standard probing accuracy on an uncontrolled test distribution can substantially overstate how reliably a property is linearly encoded [amini-etal-2023-naturalistic] This independently confirms the decodability-vs-behavioral-importance caution from amnesic probing and completeness/selectivity frameworks [amini-etal-2023-naturalistic]

models: XLM-RoBERTa base · method: Naturalistic Causal Probing (input-level dependency-tree reinflection)

Geneformer

Exhaustive Circuit Mapping of a Single-Cell Foundation Model Reveals Massive Redundancy, Heavy-Tailed Hub Architecture, and Layer-Dependent Differentiation Control (2026)measured

Late-layer SAE feature directions in Geneformer steer cell differentiation

Details

Kendiukhov trains SAEs on Geneformer's layer-5 representations and performs trajectory-guided feature steering at early, middle, and late layers [kendiukhov-2026-exhaustive-circuit-mapping-of-a-single-cell-foundation-model] Late-layer (L17) feature steering pushes cell states toward maturity (fraction positive 1.0), while early/middle (L0, L11) steering mostly pushes away (0.00-0.58) [kendiukhov-2026-exhaustive-circuit-mapping-of-a-single-cell-foundation-model] Combinatorial triplet ablation finds three-way redundancy exceeding pairwise redundancy, with zero synergy detected at any order [kendiukhov-2026-exhaustive-circuit-mapping-of-a-single-cell-foundation-model] Exhaustive circuit tracing finds a heavy-tailed hub architecture, with 1.8% of features carrying disproportionate downstream connectivity [kendiukhov-2026-exhaustive-circuit-mapping-of-a-single-cell-foundation-model]

models: Geneformer · method: Sparse Autoencoders (SAE), Activation Steering (Addition)

T5

Discovering Latent Knowledge in Language Models Without Supervision (2023)measured

Contrast-Consistent Search finds a truth-like direction with no labels

Details

Burns et al. introduce Contrast-Consistent Search: fit a linear probe on yes/no contrast pairs purely by a logical-consistency plus confidence loss, using no truth labels [burns-etal-2023] Across 6 models and 10 classification/QA datasets, CCS beats zero-shot accuracy by ~4% and approaches supervised logistic regression [burns-etal-2023] The direction found on one dataset transfers to structurally unrelated datasets and is often recoverable as the top principal component of contrast-pair differences [burns-etal-2023] True/false examples form two well-separated clusters along it, sometimes from as few as one contrast pair [burns-etal-2023] This unsupervised truth-like direction is broader than Marks & Tegmark's factual-statement truth direction and is kept as a separate observation [burns-etal-2023]

models: T5-11B · method: Contrast-Consistent Search (CCS), PCA, Linear probing, Geometric analysis
Implicit Representations of Meaning in Neural Language Models (2021)measured

Splicing probed entity-state representations causally shifts generation

Details

Li, Nye & Andreas train linear probes to decode entity state (Alchemy beaker contents, TextWorld room/object state) from fine-tuned BART and T5 encoder representations [li-nye-andreas-2021-implicit-representations-of-meaning] They go beyond decoding with a causal splice WITHIN each model (not between BART and T5): replacing one context's encoded beaker-2 description with another context's encoding to build a synthetic "mixed" state [li-nye-andreas-2021-implicit-representations-of-meaning] Generating from the spliced representation lands in the human-predicted continuation set 57.7% (BART) / 75.4% (T5) of the time, versus 20.4%/37.9% and 16.1%/29.1% for the unmixed source contexts [li-nye-andreas-2021-implicit-representations-of-meaning] The probe only reads the representation while the splice is the intervention, so this is a genuine geometry-tied causal effect [li-nye-andreas-2021-implicit-representations-of-meaning]

models: T5-Base · method: Linear probing, Causal interventions (steering)
A Geometric Taxonomy of Hallucinations in Large Language Models (2026)measured

A von Mises-Fisher grounding direction separates grounded from hallucinated responses

Details

Marin L2-normalizes embeddings from five frozen sentence encoders (sentence-t5-large primary, plus all-mpnet-base-v2, all-MiniLM-L6-v2, bge-small-en-v1.5, gte-small) onto the unit hypersphere [marin-2026-a-geometric-taxonomy-of-hallucinations-in-large-language-models] A grounding direction is extracted as the von Mises-Fisher mean of context embeddings [marin-2026-a-geometric-taxonomy-of-hallucinations-in-large-language-models] A Directional Grounding Index (cosine between a response's displacement vector and this direction) separates grounded from hallucinated responses at mean AUROC 0.805 on HaluEval QA (0.535-0.579 on TruthfulQA) [marin-2026-a-geometric-taxonomy-of-hallucinations-in-large-language-models] A companion Spherical Grounding Index gives 1.180 (grounded) versus 0.910 (hallucinated), both computed on frozen off-the-shelf encoders rather than a generative LLM's hidden states [marin-2026-a-geometric-taxonomy-of-hallucinations-in-large-language-models]

models: Sentence-T5-Large · method:
Fast Model Editing at Scale (2022)measured

MEND transforms rank-one gradient factors with a hypernetwork

Details

Mitchell et al. note the per-example weight gradient of a dense layer is a sum of rank-one outer products (upstream gradient times input activation) [mitchell-etal-2022-mend] They train small per-layer hypernetworks that transform each rank-one factor independently, then reapply the same outer-product update with a learned step size [mitchell-etal-2022-mend] Every stage is a rank-one/outer-product direction construct, the same trigger/value pattern as ROME and MEMIT but reached by a learned transformation [mitchell-etal-2022-mend] Tested on GPT-Neo-2.7B, GPT-J-6B, T5-XL, and T5-11B, MEND reaches 0.81-0.89 edit success with low perplexity drawdown, staying effective at 11B where baselines fail [mitchell-etal-2022-mend] It uses rather than finds geometry, focusing purely on the algebra of the edit mechanism [mitchell-etal-2022-mend]

models: T5-XL, T5-11B · method: MEND — Model Editor Networks using Gradient Decomposition

UnifiedQA

Discovering Latent Knowledge in Language Models Without Supervision (2023)measured

Contrast-Consistent Search finds a truth-like direction with no labels

Details

Burns et al. introduce Contrast-Consistent Search: fit a linear probe on yes/no contrast pairs purely by a logical-consistency plus confidence loss, using no truth labels [burns-etal-2023] Across 6 models and 10 classification/QA datasets, CCS beats zero-shot accuracy by ~4% and approaches supervised logistic regression [burns-etal-2023] The direction found on one dataset transfers to structurally unrelated datasets and is often recoverable as the top principal component of contrast-pair differences [burns-etal-2023] True/false examples form two well-separated clusters along it, sometimes from as few as one contrast pair [burns-etal-2023] This unsupervised truth-like direction is broader than Marks & Tegmark's factual-statement truth direction and is kept as a separate observation [burns-etal-2023]

models: UnifiedQA-11B (T5-based) · method: Contrast-Consistent Search (CCS), PCA, Linear probing, Geometric analysis

T0

Discovering Latent Knowledge in Language Models Without Supervision (2023)measured

Contrast-Consistent Search finds a truth-like direction with no labels

Details

Burns et al. introduce Contrast-Consistent Search: fit a linear probe on yes/no contrast pairs purely by a logical-consistency plus confidence loss, using no truth labels [burns-etal-2023] Across 6 models and 10 classification/QA datasets, CCS beats zero-shot accuracy by ~4% and approaches supervised logistic regression [burns-etal-2023] The direction found on one dataset transfers to structurally unrelated datasets and is often recoverable as the top principal component of contrast-pair differences [burns-etal-2023] True/false examples form two well-separated clusters along it, sometimes from as few as one contrast pair [burns-etal-2023] This unsupervised truth-like direction is broader than Marks & Tegmark's factual-statement truth direction and is kept as a separate observation [burns-etal-2023]

models: T0++ · method: Contrast-Consistent Search (CCS), PCA, Linear probing, Geometric analysis

RoBERTa

Discovering Latent Knowledge in Language Models Without Supervision (2023)measured

Contrast-Consistent Search finds a truth-like direction with no labels

Details

Burns et al. introduce Contrast-Consistent Search: fit a linear probe on yes/no contrast pairs purely by a logical-consistency plus confidence loss, using no truth labels [burns-etal-2023] Across 6 models and 10 classification/QA datasets, CCS beats zero-shot accuracy by ~4% and approaches supervised logistic regression [burns-etal-2023] The direction found on one dataset transfers to structurally unrelated datasets and is often recoverable as the top principal component of contrast-pair differences [burns-etal-2023] True/false examples form two well-separated clusters along it, sometimes from as few as one contrast pair [burns-etal-2023] This unsupervised truth-like direction is broader than Marks & Tegmark's factual-statement truth direction and is kept as a separate observation [burns-etal-2023]

models: RoBERTa-large-MNLI · method: Contrast-Consistent Search (CCS), PCA, Linear probing, Geometric analysis
OSCaR: Orthogonal Subspace Correction and Rectification of Biases in Word Embeddings (2020)measured

OSCaR's graded rotation preserves information better than projection debiasing

Details

Dev et al. measure that a diff-of-means gender direction and an occupation direction are not orthogonal in GloVe or RoBERTa embedding space [dev-etal-2020-oscar] Standard projective debiasing (hard debiasing, INLP) erases the gender direction from every word, destroying valid associations (an NLI entailment drops 97% to 16%) [dev-etal-2020-oscar] OSCaR instead applies a graded rotation, full at the occupation direction and none at the gender direction, so unrelated words are barely perturbed [dev-etal-2020-oscar] On GloVe it matches the best bias reduction (WEAT 1.768 to 0.235) while scoring far higher on new information-retention metrics (WEAT*, SIRT); on RoBERTa it gives the best bias reduction and retention [dev-etal-2020-oscar]

models: RoBERTa-base · method: OSCaR (Orthogonal Subspace Correction and Rectification)

Zephyr

The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning (2024)measured

RMU corrupts a hazardous-topic direction by fine-tuning while preserving capability

Details

Li, Pan et al. (WMDP) introduce RMU, a two-term activation-space loss pushing hazardous-topic activations toward a fixed random unit vector while anchoring benign activations to the frozen model [li-pan-etal-2024-the-wmdp-benchmark-measuring-and-reducing-malicious-use-with-unlearning] On Zephyr-7B-beta, WMDP-Bio drops 63.7 to 31.2 and WMDP-Cyber 44.0 to 28.2, while MMLU (58.1 to 57.1) and MT-Bench (7.33 to 7.10) stay near baseline [li-pan-etal-2024-the-wmdp-benchmark-measuring-and-reducing-malicious-use-with-unlearning] Yi-34B-Chat and Mixtral-8x7B-Instruct show the same pattern (WMDP-Bio 75.3 to 30.7 and 74.8 to 34.0, MMLU preserved within ~2 points) [li-pan-etal-2024-the-wmdp-benchmark-measuring-and-reducing-malicious-use-with-unlearning] The direction is imposed by fine-tuning toward an arbitrary target rather than found in the base model, unlike additive inference-time steering [li-pan-etal-2024-the-wmdp-benchmark-measuring-and-reducing-malicious-use-with-unlearning]

models: Zephyr-7B-Beta · method: Representation misdirection (fine-tuning toward a target direction)
Refusal in Language Models Is Mediated by a Single Direction (2024), Representation Engineering: A Top-Down Approach to AI Transparency (2023), Programming Refusal with Conditional Activation Steering (2024), Refusal Direction is Universal Across Safety-Aligned Languages (2025)measured

Refusal is mediated by a single direction across models

Details

Arditi et al. show across 13 chat models (5 families, 1.8B-72B) that a single diff-in-means direction between harmful and harmless instructions both erases refusal when ablated and induces it when added, with no offset term [arditi-etal-2024] Zou et al. independently confirm via a LAT harmfulness vector in Vicuna-13B that stays a >90% classifier under jailbreaks and adversarial suffixes [arditi-etal-2024] Lee et al. replicate across nine further chat models and add conditional steering that gates the refusal vector on a condition vector for selective refusal [arditi-etal-2024] Yung et al. average per-model refusal vectors into a universal refusal direction transferring to jailbreak detection ~10% better than single-model baselines [arditi-etal-2024] The account is contested by Marshall et al. (an affine offset is needed) and Wollschlager et al. (a cone of directions), and Yin et al. localize attack-suppressed versus robust heads [arditi-etal-2024]

models: Zephyr-7B-Beta · method: Difference-in-means direction, Causal interventions (steering), Activation Steering (Addition), Linear Artificial Tomography (LAT), PCA, Conditional (gated) activation steering

Yi

The WMDP Benchmark: Measuring and Reducing Malicious Use With Unlearning (2024)measured

RMU corrupts a hazardous-topic direction by fine-tuning while preserving capability

Details

Li, Pan et al. (WMDP) introduce RMU, a two-term activation-space loss pushing hazardous-topic activations toward a fixed random unit vector while anchoring benign activations to the frozen model [li-pan-etal-2024-the-wmdp-benchmark-measuring-and-reducing-malicious-use-with-unlearning] On Zephyr-7B-beta, WMDP-Bio drops 63.7 to 31.2 and WMDP-Cyber 44.0 to 28.2, while MMLU (58.1 to 57.1) and MT-Bench (7.33 to 7.10) stay near baseline [li-pan-etal-2024-the-wmdp-benchmark-measuring-and-reducing-malicious-use-with-unlearning] Yi-34B-Chat and Mixtral-8x7B-Instruct show the same pattern (WMDP-Bio 75.3 to 30.7 and 74.8 to 34.0, MMLU preserved within ~2 points) [li-pan-etal-2024-the-wmdp-benchmark-measuring-and-reducing-malicious-use-with-unlearning] The direction is imposed by fine-tuning toward an arbitrary target rather than found in the base model, unlike additive inference-time steering [li-pan-etal-2024-the-wmdp-benchmark-measuring-and-reducing-malicious-use-with-unlearning]

models: Yi-34B-Chat · method: Representation misdirection (fine-tuning toward a target direction)
Task Recognition and Task Learning Heads Align In-Context Hidden States with a Label-Unembedding Task Subspace (2026)measured

ICL task-recognition and task-learning heads project onto a label subspace

Details

Yang, Cho & Inoue introduce Task Subspace Logit Attribution, scoring each attention head output by projection onto the span of label-token unembedding vectors [yang-cho-inoue-2026-task-subspace-attention-heads] Task-recognition heads align with the task subspace while task-learning heads rotate the hidden state toward the correct-minus-incorrect label direction within it [yang-cho-inoue-2026-task-subspace-attention-heads] Across Llama3-8B layers, hidden-update vs TR-head subspace alignment correlates rho=0.94, and logit-difference vs TL-head alignment rho=0.53 [yang-cho-inoue-2026-task-subspace-attention-heads] Injecting the summed top-3% TR-head outputs into zero-shot prompts raises classification accuracy 9.2% to 40.4% (TR-based), while TL-based vectors dominate open-ended generation; ablating each collapses its own role, tested across 6 models [yang-cho-inoue-2026-task-subspace-attention-heads]

models: Yi-34B (base) · method: Direct Logit Attribution (DLA), Activation patching

VoxGenesis

VoxGenesis: Unsupervised Discovery of Latent Speaker Manifold for Speech Synthesis (2024)measured

Unsupervised Gaussian-mixture components in a real speech GAN's latent prior are linearly-manipulable speaker-attribute directions

Details

Lin, He, Mak, Lian & Lee (2024) train VoxGenesis, a real GAN-based speech synthesizer, on real LibriTTS/VoxCeleb audio with a Gaussian-mixture latent prior fit purely by the generative objective, with no speaker labels used during training [lin-etal-2024-voxgenesis-unsupervised-discovery-of-latent-speaker-manifold] Individual Gaussian components in the discovered latent prior align with human-interpretable speaker attributes (gender, age, accent), and moving a sample's latent code along the vector connecting two component means causally and smoothly interpolates the corresponding attribute in the synthesized voice, confirmed via human listener ratings [lin-etal-2024-voxgenesis-unsupervised-discovery-of-latent-speaker-manifold]

models: VoxGenesis (GAN-based speech synthesizer, Gaussian-mixture latent prior) · method:

Code Llama

Do Language Models Track Entities Across State Changes? (2026)measured

LLMs consolidate entity-state tracking into a local binary subspace

Details

Tang et al. train linear probes on last-token residual activations (the "boxes" dataset) to test global versus local entity-state tracking [tang-etal-2026-do-language-models-track-entities-across-state-changes] Global 8-way state probes reach only ~0.3 accuracy versus ~0.9 for local binary probes, so the model consolidates state locally rather than tracking a full global state [tang-etal-2026-do-language-models-track-entities-across-state-changes] INLP null-space projection of the state-tag direction causally changes completions [tang-etal-2026-do-language-models-track-entities-across-state-changes] A learned sparse boolean mask over the residual PCA basis ("subspace patching") reveals a shared positional-information subspace at layers 15-25 in CodeLlama-13B [tang-etal-2026-do-language-models-track-entities-across-state-changes]

models: CodeLlama-13B · method: Linear probing, Causal interventions (steering), PCA

AudioLDM

TADA! Tuning Audio Diffusion Models Through Activation Steering (2026)measured

Bottleneck-localized steering beats full-network steering in music diffusion

Details

Staniszewski et al. use activation patching to localize a semantic bottleneck across three text-to-music diffusion models (AudioLDM2, Ace-Step, Stable Audio Open) [staniszewski-etal-2026-tada-tuning-audio-diffusion-models-through-activation-steering] Steering only at these 2-4 bottleneck layers, via mean-difference direction vectors or TopK SAE decoder columns, beats full-network steering [staniszewski-etal-2026-tada-tuning-audio-diffusion-models-through-activation-steering] On Ace-Step "Piano," LPAPS drops to 2.188 for layers {6,7} versus 2.707 for all 24 layers [staniszewski-etal-2026-tada-tuning-audio-diffusion-models-through-activation-steering] SAE-based bottleneck steering achieves the best preservation (LPAPS 1.973) [staniszewski-etal-2026-tada-tuning-audio-diffusion-models-through-activation-steering]

models: AudioLDM2 · method: Activation patching, Difference-in-means direction, Sparse Autoencoders (SAE), Causal interventions (steering)

Stable Audio

TADA! Tuning Audio Diffusion Models Through Activation Steering (2026)measured

Bottleneck-localized steering beats full-network steering in music diffusion

Details

Staniszewski et al. use activation patching to localize a semantic bottleneck across three text-to-music diffusion models (AudioLDM2, Ace-Step, Stable Audio Open) [staniszewski-etal-2026-tada-tuning-audio-diffusion-models-through-activation-steering] Steering only at these 2-4 bottleneck layers, via mean-difference direction vectors or TopK SAE decoder columns, beats full-network steering [staniszewski-etal-2026-tada-tuning-audio-diffusion-models-through-activation-steering] On Ace-Step "Piano," LPAPS drops to 2.188 for layers {6,7} versus 2.707 for all 24 layers [staniszewski-etal-2026-tada-tuning-audio-diffusion-models-through-activation-steering] SAE-based bottleneck steering achieves the best preservation (LPAPS 1.973) [staniszewski-etal-2026-tada-tuning-audio-diffusion-models-through-activation-steering]

models: Stable Audio Open · method: Activation patching, Difference-in-means direction, Sparse Autoencoders (SAE), Causal interventions (steering)
Learning Interpretable Features in Audio Latent Spaces via Sparse Autoencoders (2025)measured

SAE probe directions decode a pitch>loudness>timbre hierarchy and steer audio

Details

Paek et al. train SAEs on four pretrained audio autoencoders/codecs and fit linear probes from SAE features to pitch, loudness, and timbre [paek-etal-2025-learning-interpretable-features-audio-latent-spaces-sparse-autoencoders] Across all four latent spaces decodability follows the same order: pitch most separable (0.75-0.87), loudness intermediate, timbre hardest (0.17-0.46) [paek-etal-2025-learning-interpretable-features-audio-latent-spaces-sparse-autoencoders] Reusing each probe's weight row directly as a control vector causally isolates changes in the targeted property while leaving others largely intact [paek-etal-2025-learning-interpretable-features-audio-latent-spaces-sparse-autoencoders] Tracing DiffRhythm's rectified-flow generation reveals a coarse-to-fine emergence order: pitch converges first, then timbre, with loudness least resolved [paek-etal-2025-learning-interpretable-features-audio-latent-spaces-sparse-autoencoders]

models: Stable Audio Open · method: Sparse Autoencoders (SAE), Linear probing, Causal interventions (steering), Activation Steering (Addition)

Ace-Step

TADA! Tuning Audio Diffusion Models Through Activation Steering (2026)measured

Bottleneck-localized steering beats full-network steering in music diffusion

Details

Staniszewski et al. use activation patching to localize a semantic bottleneck across three text-to-music diffusion models (AudioLDM2, Ace-Step, Stable Audio Open) [staniszewski-etal-2026-tada-tuning-audio-diffusion-models-through-activation-steering] Steering only at these 2-4 bottleneck layers, via mean-difference direction vectors or TopK SAE decoder columns, beats full-network steering [staniszewski-etal-2026-tada-tuning-audio-diffusion-models-through-activation-steering] On Ace-Step "Piano," LPAPS drops to 2.188 for layers {6,7} versus 2.707 for all 24 layers [staniszewski-etal-2026-tada-tuning-audio-diffusion-models-through-activation-steering] SAE-based bottleneck steering achieves the best preservation (LPAPS 1.973) [staniszewski-etal-2026-tada-tuning-audio-diffusion-models-through-activation-steering]

models: Ace-Step · method: Activation patching, Difference-in-means direction, Sparse Autoencoders (SAE), Causal interventions (steering)

MPNet

A Geometric Taxonomy of Hallucinations in Large Language Models (2026)measured

A von Mises-Fisher grounding direction separates grounded from hallucinated responses

Details

Marin L2-normalizes embeddings from five frozen sentence encoders (sentence-t5-large primary, plus all-mpnet-base-v2, all-MiniLM-L6-v2, bge-small-en-v1.5, gte-small) onto the unit hypersphere [marin-2026-a-geometric-taxonomy-of-hallucinations-in-large-language-models] A grounding direction is extracted as the von Mises-Fisher mean of context embeddings [marin-2026-a-geometric-taxonomy-of-hallucinations-in-large-language-models] A Directional Grounding Index (cosine between a response's displacement vector and this direction) separates grounded from hallucinated responses at mean AUROC 0.805 on HaluEval QA (0.535-0.579 on TruthfulQA) [marin-2026-a-geometric-taxonomy-of-hallucinations-in-large-language-models] A companion Spherical Grounding Index gives 1.180 (grounded) versus 0.910 (hallucinated), both computed on frozen off-the-shelf encoders rather than a generative LLM's hidden states [marin-2026-a-geometric-taxonomy-of-hallucinations-in-large-language-models]

models: all-mpnet-base-v2 · method:
Probing Spectrum-Like Organization of States of Mind in Transformer Representation Spaces (2026)measured

A 'state of mind' spectrum is linearly decodable across five encoders

Details

Zhao annotates 636 sentences with a continuous energy score and one of seven ordered tiers, an informally-sourced psychological taxonomy explicitly disclaimed as non-clinical [zhao-2026] Across five frozen representations (BGE-large, all-mpnet-base-v2, all-MiniLM-L6-v2, Qwen3-Embedding-0.6B, and Qwen2.5-3B-Instruct's layer 24), both the score (Ridge R^2 0.63-0.79) and the seven tiers (weighted F1 0.57-0.72) are linearly decodable above chance [zhao-2026] Permutation tests and a TF-IDF baseline (R^2 0.44) rule out simple lexical confounds [zhao-2026] Projecting out a single Ridge-derived direction (1.6-4.8% of variance) collapses decodability far more than a random direction, though the structure is only approximately one-dimensional [zhao-2026]

models: all-mpnet-base-v2 · method: Linear probing, Mlp probing, UMAP

MiniLM

A Geometric Taxonomy of Hallucinations in Large Language Models (2026)measured

A von Mises-Fisher grounding direction separates grounded from hallucinated responses

Details

Marin L2-normalizes embeddings from five frozen sentence encoders (sentence-t5-large primary, plus all-mpnet-base-v2, all-MiniLM-L6-v2, bge-small-en-v1.5, gte-small) onto the unit hypersphere [marin-2026-a-geometric-taxonomy-of-hallucinations-in-large-language-models] A grounding direction is extracted as the von Mises-Fisher mean of context embeddings [marin-2026-a-geometric-taxonomy-of-hallucinations-in-large-language-models] A Directional Grounding Index (cosine between a response's displacement vector and this direction) separates grounded from hallucinated responses at mean AUROC 0.805 on HaluEval QA (0.535-0.579 on TruthfulQA) [marin-2026-a-geometric-taxonomy-of-hallucinations-in-large-language-models] A companion Spherical Grounding Index gives 1.180 (grounded) versus 0.910 (hallucinated), both computed on frozen off-the-shelf encoders rather than a generative LLM's hidden states [marin-2026-a-geometric-taxonomy-of-hallucinations-in-large-language-models]

models: all-MiniLM-L6-v2 · method:
Probing Spectrum-Like Organization of States of Mind in Transformer Representation Spaces (2026)measured

A 'state of mind' spectrum is linearly decodable across five encoders

Details

Zhao annotates 636 sentences with a continuous energy score and one of seven ordered tiers, an informally-sourced psychological taxonomy explicitly disclaimed as non-clinical [zhao-2026] Across five frozen representations (BGE-large, all-mpnet-base-v2, all-MiniLM-L6-v2, Qwen3-Embedding-0.6B, and Qwen2.5-3B-Instruct's layer 24), both the score (Ridge R^2 0.63-0.79) and the seven tiers (weighted F1 0.57-0.72) are linearly decodable above chance [zhao-2026] Permutation tests and a TF-IDF baseline (R^2 0.44) rule out simple lexical confounds [zhao-2026] Projecting out a single Ridge-derived direction (1.6-4.8% of variance) collapses decodability far more than a random direction, though the structure is only approximately one-dimensional [zhao-2026]

models: all-MiniLM-L6-v2 · method: Linear probing, Mlp probing, UMAP

GTE (General Text Embeddings)

A Geometric Taxonomy of Hallucinations in Large Language Models (2026)measured

A von Mises-Fisher grounding direction separates grounded from hallucinated responses

Details

Marin L2-normalizes embeddings from five frozen sentence encoders (sentence-t5-large primary, plus all-mpnet-base-v2, all-MiniLM-L6-v2, bge-small-en-v1.5, gte-small) onto the unit hypersphere [marin-2026-a-geometric-taxonomy-of-hallucinations-in-large-language-models] A grounding direction is extracted as the von Mises-Fisher mean of context embeddings [marin-2026-a-geometric-taxonomy-of-hallucinations-in-large-language-models] A Directional Grounding Index (cosine between a response's displacement vector and this direction) separates grounded from hallucinated responses at mean AUROC 0.805 on HaluEval QA (0.535-0.579 on TruthfulQA) [marin-2026-a-geometric-taxonomy-of-hallucinations-in-large-language-models] A companion Spherical Grounding Index gives 1.180 (grounded) versus 0.910 (hallucinated), both computed on frozen off-the-shelf encoders rather than a generative LLM's hidden states [marin-2026-a-geometric-taxonomy-of-hallucinations-in-large-language-models]

models: GTE-small · method:

CLAP (Contrastive Language-Audio Pretraining)

Probing Low-Level Acoustic Attribute Encoding in CLAP Audio Embeddings (2026)measured

CLAP linearly encodes reverberation and loudness with cross-dataset-consistent axes

Details

Martel et al. linearly probe real LAION-CLAP audio embeddings for four low-level acoustic attributes across five datasets [martel-etal-2026-clap-acoustic-attribute-probing] RT60 (reverberation time) is strongly, nearly linearly encoded (R^2>=0.67, up to 0.92 on White Noise/VCTK) and LUFS (loudness) reaches R^2>=0.76 [martel-etal-2026-clap-acoustic-attribute-probing] The independently-fit RT60 and LUFS probe axes are geometrically consistent across datasets (RT60 cosine up to 0.86), while the relative-pitch axis is domain-specific near the random-vector baseline [martel-etal-2026-clap-acoustic-attribute-probing] Across 8 more pretrained audio encoders, amplitude-invariant architectures specifically fail to encode LUFS, tying the failure to a known architectural property [martel-etal-2026-clap-acoustic-attribute-probing]

models: LAION-CLAP (HTSAT backbone, trained on LAION-Audio-630k + AudioSet + speech/music) · method: Linear probing

TinyModel (TinyStories-trained)

Learning Multi-Level Features with Matryoshka Sparse Autoencoders (2025)measured

Matryoshka nested dictionaries reduce SAE feature absorption

Details

Bussmann et al. address feature absorption, where a general latent develops systematic blind spots ceded to specialized latents, by training multiple nested prefixes of one shared dictionary to each reconstruct the input independently [bussmann-etal-2025-matryoshka-saes] Because the smallest prefix must explain the input alone, without access to the specialized latents, it keeps general concepts intact rather than ceding them [bussmann-etal-2025-matryoshka-saes] On Gemma-2-2B and a from-scratch TinyStories transformer, Matryoshka SAEs significantly beat benchmark architectures on targeted-concept-erasure (SCR) and disentanglement (TPP) and match BatchTopK on sparse probing [bussmann-etal-2025-matryoshka-saes] The gain costs roughly 50% more training time and slightly lower reconstruction, with a tunable dial between nesting granularity and fidelity [bussmann-etal-2025-matryoshka-saes]

models: TinyModel-4L (TinyStories) · method: Matryoshka Sparse Autoencoders

Maze-Solving Transformer

Structured World Representations in Maze-Solving Transformers (2023)measured

A single token's residual linearly decodes an entire maze

Details

Ivanitskiy et al. train small (<10M) GPT transformers from scratch on synthetic maze token sequences with one-hot orthogonal input tokens [ivanitskiy-etal-2023] Linear probes on a single fixed token's residual stream reconstruct the entire maze's wall structure (>90% wall accuracy, peaking at layer 2), a striking single-site compression [ivanitskiy-etal-2023] Learned embedding vectors develop emergent spatial structure, with Manhattan distance correlating with embedding L1 distance at short range despite orthogonal inputs [ivanitskiy-etal-2023] Candidate "Adjacency Heads" attend to path-adjacent tokens in-context, correlational evidence not causally ablated in the paper [ivanitskiy-etal-2023] Sharp improvements in linear maze decodability coincide with sharp task-generalization gains, a grokking-like transition [ivanitskiy-etal-2023]

models: hallway (maze-solving transformer, forkless mazes), jirpy (maze-solving transformer, forking mazes) · method: Linear probing, Direct Logit Attribution (DLA)

GPT-Neo

Fast Model Editing at Scale (2022)measured

MEND transforms rank-one gradient factors with a hypernetwork

Details

Mitchell et al. note the per-example weight gradient of a dense layer is a sum of rank-one outer products (upstream gradient times input activation) [mitchell-etal-2022-mend] They train small per-layer hypernetworks that transform each rank-one factor independently, then reapply the same outer-product update with a learned step size [mitchell-etal-2022-mend] Every stage is a rank-one/outer-product direction construct, the same trigger/value pattern as ROME and MEMIT but reached by a learned transformation [mitchell-etal-2022-mend] Tested on GPT-Neo-2.7B, GPT-J-6B, T5-XL, and T5-11B, MEND reaches 0.81-0.89 edit success with low perplexity drawdown, staying effective at 11B where baselines fail [mitchell-etal-2022-mend] It uses rather than finds geometry, focusing purely on the algebra of the edit mechanism [mitchell-etal-2022-mend]

models: GPT-Neo-2.7B · method: MEND — Model Editor Networks using Gradient Decomposition

BLOOM

Language Models Implement Simple Word2Vec-style Vector Arithmetic (2024)measured

LLMs implement word2vec-style relational vector offsets, causally patchable

Details

Merullo et al. show many one-to-one relations (e.g. country-capital) that LLMs solve in-context are implemented as a simple additive vector offset computed by a mid-network FFN update [merullo-etal-2024-language-models-implement-word2vec-style-vector-arithmetic] This is directly analogous to word2vec-style vector arithmetic but discovered inside real pretrained transformer activations rather than static embeddings [merullo-etal-2024-language-models-implement-word2vec-style-vector-arithmetic] Extracting the offset from one example context and patching it into another causally forces the model to output the corresponding relation's correct answer for the new subject [merullo-etal-2024-language-models-implement-word2vec-style-vector-arithmetic] The mechanism is replicated across GPT-J, GPT-2 (small through XL), and BLOOM (up to 176B parameters) [merullo-etal-2024-language-models-implement-word2vec-style-vector-arithmetic]

models: BLOOM-176B · method: Vector-offset analogy test, Causal interventions (steering)
Eliciting Latent Predictions from Transformers with the Tuned Lens (2023)measured

The Tuned Lens replaces the logit lens with a trained per-layer affine correction

Details

Belrose et al. replace the logit lens's direct reuse of the final unembedding on intermediate states with a trained per-layer affine translator, distilled via KL divergence to the model's own final output [belrose-etal-2023-tuned-lens] This removes two documented logit-lens failure modes: a systematic 4-5 bit-per-layer vocabulary bias and unreliability from hidden-state covariance drifting across layers [belrose-etal-2023-tuned-lens] A causal-basis extraction finds the lens's most-sensitive directions strongly match the model's real causal directions (Spearman rho=0.89), with none influential to the lens but not the model [belrose-etal-2023-tuned-lens] Tested across GPT-2, GPT-Neo, GPT-NeoX-20B, OPT, BLOOM, and the full Pythia suite, it gives lower perplexity than the logit lens, transfers to a fine-tune (Vicuna-13B from LLaMA-13B), and detects prompt injection at near-perfect AUROC [belrose-etal-2023-tuned-lens]

models: BLOOM-560M · method: Tuned Lens

RNNLM (Recurrent Neural Network Language Model)

Linguistic Regularities in Continuous Space Word Representations (2013)measured

The foundational king-man+woman parallelogram analogy in RNNLM vectors

Details

Mikolov, Yih & Zweig establish in trained RNNLM word representations that syntactic and semantic relationships are encoded as approximately consistent linear offset vectors between word-vector pairs [mikolov-etal-2013-linguistic-regularities-continuous-space] Vector arithmetic (king - man + woman) retrieves the analogically correct target word (queen) via nearest-neighbor cosine similarity on a purpose-built analogy benchmark [mikolov-etal-2013-linguistic-regularities-continuous-space] This is the founding empirical demonstration of the linear/parallelogram analogy geometry [mikolov-etal-2013-linguistic-regularities-continuous-space] Essentially all later linear-representation-hypothesis work in this map builds on and re-derives it in other model families [mikolov-etal-2013-linguistic-regularities-continuous-space]

models: RNNLM word vectors (Mikolov, Yih & Zweig 2013, Broadcast News) · method: Vector-offset analogy test

Molmo

Why Far Looks Up: Probing Spatial Representation in Vision-Language Models (2026)measured

Per-axis spatial delta vectors form separated PCA clusters in stronger VLMs

Details

Min et al. extract a delta vector between final-token hidden states for paired VQA prompts differing only in queried-object order, at a fixed intermediate layer [min-etal-2026-why-far-looks-up-probing-spatial-representation-in-vision-language-models] Axis Coherence, the mean pairwise cosine among sign-corrected delta vectors within an axis, rises with model strength (distance-axis 0.075 to 0.112 across Molmo scales, 0.182 for RoboRefer-2B-SFT) but stays flat at 0.04-0.05 for Qwen2.5-VL [min-etal-2026-why-far-looks-up-probing-spatial-representation-in-vision-language-models] PCA shows weak models' distance vectors collapse near the origin while RoboRefer and Qwen3-VL-235B form three cleanly separated per-axis clusters aligned to distinct principal components [min-etal-2026-why-far-looks-up-probing-spatial-representation-in-vision-language-models] Distance coherence correlates with counter-heuristic behavioral accuracy (rho=0.759, 0.804; p<1e-3) both in- and cross-domain [min-etal-2026-why-far-looks-up-probing-spatial-representation-in-vision-language-models]

models: Molmo-7B-O · method: Difference-in-means direction, PCA

NVILA

Why Far Looks Up: Probing Spatial Representation in Vision-Language Models (2026)measured

Per-axis spatial delta vectors form separated PCA clusters in stronger VLMs

Details

Min et al. extract a delta vector between final-token hidden states for paired VQA prompts differing only in queried-object order, at a fixed intermediate layer [min-etal-2026-why-far-looks-up-probing-spatial-representation-in-vision-language-models] Axis Coherence, the mean pairwise cosine among sign-corrected delta vectors within an axis, rises with model strength (distance-axis 0.075 to 0.112 across Molmo scales, 0.182 for RoboRefer-2B-SFT) but stays flat at 0.04-0.05 for Qwen2.5-VL [min-etal-2026-why-far-looks-up-probing-spatial-representation-in-vision-language-models] PCA shows weak models' distance vectors collapse near the origin while RoboRefer and Qwen3-VL-235B form three cleanly separated per-axis clusters aligned to distinct principal components [min-etal-2026-why-far-looks-up-probing-spatial-representation-in-vision-language-models] Distance coherence correlates with counter-heuristic behavioral accuracy (rho=0.759, 0.804; p<1e-3) both in- and cross-domain [min-etal-2026-why-far-looks-up-probing-spatial-representation-in-vision-language-models]

models: NVILA-Lite-2B · method: Difference-in-means direction, PCA

RoboRefer

Why Far Looks Up: Probing Spatial Representation in Vision-Language Models (2026)measured

Per-axis spatial delta vectors form separated PCA clusters in stronger VLMs

Details

Min et al. extract a delta vector between final-token hidden states for paired VQA prompts differing only in queried-object order, at a fixed intermediate layer [min-etal-2026-why-far-looks-up-probing-spatial-representation-in-vision-language-models] Axis Coherence, the mean pairwise cosine among sign-corrected delta vectors within an axis, rises with model strength (distance-axis 0.075 to 0.112 across Molmo scales, 0.182 for RoboRefer-2B-SFT) but stays flat at 0.04-0.05 for Qwen2.5-VL [min-etal-2026-why-far-looks-up-probing-spatial-representation-in-vision-language-models] PCA shows weak models' distance vectors collapse near the origin while RoboRefer and Qwen3-VL-235B form three cleanly separated per-axis clusters aligned to distinct principal components [min-etal-2026-why-far-looks-up-probing-spatial-representation-in-vision-language-models] Distance coherence correlates with counter-heuristic behavioral accuracy (rho=0.759, 0.804; p<1e-3) both in- and cross-domain [min-etal-2026-why-far-looks-up-probing-spatial-representation-in-vision-language-models]

models: RoboRefer-2B-SFT · method: Difference-in-means direction, PCA

GPT2-Spanish

Naturalistic Causal Probing for Morpho-Syntax (2023)measured

A naturalistic causal ATE direction nearly coincides with the top PCA direction

Details

Amini et al. estimate the average treatment effect of grammatical gender and number on mBERT, XLM-RoBERTa-base, and GPT2-small-Spanish using dependency-tree-reinflected naturalistic counterfactuals [amini-etal-2023-naturalistic] The first principal component of the paired representations has cosine similarity ~0.99 with the estimated ATE direction across all three models [amini-etal-2023-naturalistic] In BERT/RoBERTa this PC explains ~20% of paired variance, while in GPT2-small-Spanish over 50% concentrates in the first 1-2 components, a starker one-dimensional encoding [amini-etal-2023-naturalistic] The paired causal estimator is far more stable across treebanks than a naive correlational one and better reconstructs the true counterfactual distribution [amini-etal-2023-naturalistic]

models: GPT2-small (Spanish) · method: Naturalistic Causal Probing (input-level dependency-tree reinflection)
Naturalistic Causal Probing for Morpho-Syntax (2023)measured

Probing accuracy collapses under a naturalistic counterfactual test

Details

Amini et al. compare correlational probing for grammatical gender/number against evaluation on naturalistic counterfactual test pairs [amini-etal-2023-naturalistic] Probes reaching above 90% on the standard split drop to roughly 60% (near chance for a binary property) on dependency-tree-verified counterfactual sentences [amini-etal-2023-naturalistic] Standard probing accuracy on an uncontrolled test distribution can substantially overstate how reliably a property is linearly encoded [amini-etal-2023-naturalistic] This independently confirms the decodability-vs-behavioral-importance caution from amnesic probing and completeness/selectivity frameworks [amini-etal-2023-naturalistic]

models: GPT2-small (Spanish) · method: Naturalistic Causal Probing (input-level dependency-tree reinflection)

OthelloGPT

Linear Latent World Models in Simple Transformers: A Case Study on Othello-GPT (2023)measured

Othello-GPT's board direction is decodable early but causally used only deep

Details

Hazineh et al. confirm Othello-GPT's linear player-relative board encoding via probing and introduce a learned linear inverse-map edit toward a target board state [hazineh-etal-2023-linear-latent-world-models-othello-gpt] Sweeping depth (1-8 layers), the linear board direction is decodable even in 1-layer models but is causally used for next-move logits only in deeper models [hazineh-etal-2023-linear-latent-world-models-othello-gpt] The causal effect concentrates in middle layers, a depth-dependence earlier Othello-GPT papers did not characterize [hazineh-etal-2023-linear-latent-world-models-othello-gpt] Representation presence and causal use are thus dissociable, with the transition from presence to use tracking network depth [hazineh-etal-2023-linear-latent-world-models-othello-gpt]

models: OthelloGPT · method: Linear probing, Activation Steering (Addition)
Emergent Linear Representations in World Models of Self-Supervised Sequence Models (2023)measured

Othello-GPT's board state is linear under the model's own relative frame

Details

Nanda et al. show Othello-GPT's board state, decodable only nonlinearly under absolute BLACK/WHITE labels, is near-perfectly linearly decodable under a relative MINE/YOURS/EMPTY frame (99.5% by the final layer vs 74-75% absolute) [nanda-etal-2023] The world model was linear all along under the basis the model itself uses, and the original nonlinear result was a labeling artifact [nanda-etal-2023] Adding the linear probe directions steers the model as effectively as Li et al.'s more complex gradient-based editing (0.02 vs 0.11 errors for erasing a tile) [nanda-etal-2023] Further linear sub-findings include an EMPTY direction, a causally-relevant FLIPPED direction, and monotonic layer-by-layer refinement of the board representation [nanda-etal-2023]

models: OthelloGPT · method: Linear probing, Mlp probing, Activation Steering (Addition)

SmolLM

Output Vector Editing for Memorization Mitigation in Large Language Models (2026)measured

Editing MLP output-vector directions suppresses memorized sequences

Details

Hakimi et al. identify MLP neurons implicated in verbatim memorization via logit-lens attribution, then edit each neuron's output vector to add a distractor direction while preserving its other superposed functions [hakimi-etal-2026-output-vector-editing-for-memorization] Four edit modes jointly suppress up to 96.5% of memorized continuations in ensemble, a 2.7x larger effect than zero-ablating the same neurons [hakimi-etal-2026-output-vector-editing-for-memorization] So the direction of the edit, not just the neuron's presence, drives the effect [hakimi-etal-2026-output-vector-editing-for-memorization] About 14% of memorized sequences resist MLP-only editing; ablating top attention heads recovers 60-64% of these, indicating a mechanism split across MLP and attention, and the method transfers to SmolLM-360M, OLMo-1B, and Llama2-7B [hakimi-etal-2026-output-vector-editing-for-memorization]

models: SmolLM-360M · method: Logit Lens

DiffRhythm

Learning Interpretable Features in Audio Latent Spaces via Sparse Autoencoders (2025)measured

SAE probe directions decode a pitch>loudness>timbre hierarchy and steer audio

Details

Paek et al. train SAEs on four pretrained audio autoencoders/codecs and fit linear probes from SAE features to pitch, loudness, and timbre [paek-etal-2025-learning-interpretable-features-audio-latent-spaces-sparse-autoencoders] Across all four latent spaces decodability follows the same order: pitch most separable (0.75-0.87), loudness intermediate, timbre hardest (0.17-0.46) [paek-etal-2025-learning-interpretable-features-audio-latent-spaces-sparse-autoencoders] Reusing each probe's weight row directly as a control vector causally isolates changes in the targeted property while leaving others largely intact [paek-etal-2025-learning-interpretable-features-audio-latent-spaces-sparse-autoencoders] Tracing DiffRhythm's rectified-flow generation reveals a coarse-to-fine emergence order: pitch converges first, then timbre, with loudness least resolved [paek-etal-2025-learning-interpretable-features-audio-latent-spaces-sparse-autoencoders]

models: DiffRhythm · method: Sparse Autoencoders (SAE), Linear probing, Causal interventions (steering), Activation Steering (Addition)

WavTokenizer

Learning Interpretable Features in Audio Latent Spaces via Sparse Autoencoders (2025)measured

SAE probe directions decode a pitch>loudness>timbre hierarchy and steer audio

Details

Paek et al. train SAEs on four pretrained audio autoencoders/codecs and fit linear probes from SAE features to pitch, loudness, and timbre [paek-etal-2025-learning-interpretable-features-audio-latent-spaces-sparse-autoencoders] Across all four latent spaces decodability follows the same order: pitch most separable (0.75-0.87), loudness intermediate, timbre hardest (0.17-0.46) [paek-etal-2025-learning-interpretable-features-audio-latent-spaces-sparse-autoencoders] Reusing each probe's weight row directly as a control vector causally isolates changes in the targeted property while leaving others largely intact [paek-etal-2025-learning-interpretable-features-audio-latent-spaces-sparse-autoencoders] Tracing DiffRhythm's rectified-flow generation reveals a coarse-to-fine emergence order: pitch converges first, then timbre, with loudness least resolved [paek-etal-2025-learning-interpretable-features-audio-latent-spaces-sparse-autoencoders]

models: WavTokenizer · method: Sparse Autoencoders (SAE), Linear probing, Causal interventions (steering), Activation Steering (Addition)

Chronos

On the Internal Semantics of Time-Series Foundation Models (2025)measured

Time-series foundation models linearly recover generative parameters by depth

Details

Pandey et al. fit per-layer linear probes on frozen Chronos and MOMENT to regress synthetic-series generative parameters [pandey-etal-2025-internal-semantics-time-series-foundation-models] Early layers capture local time-domain patterns (AR(1), trend, level shifts), deeper layers specialize in dispersion and change-points, and spectral/warping concepts stay hardest throughout [pandey-etal-2025-internal-semantics-time-series-foundation-models] Probe accuracy improves when a parameter varies smoothly along the UMAP manifold, and Chronos yields more linearly recoverable representations than MOMENT [pandey-etal-2025-internal-semantics-time-series-foundation-models] In compositional settings probe performance degrades and embedding arithmetic shows interference; no causal intervention is performed [pandey-etal-2025-internal-semantics-time-series-foundation-models]

models: Chronos · method: Linear probing
Exploring Representations and Interventions in Time Series Foundation Models (2024)measured

Time-series foundation models linearly encode concepts and steer, including ECG

Details

Wilinski et al. find block-like redundancy usable for pruning and linearly-represented concepts (periodicity, trends) in Chronos, MOMENT, and Moirai-1.1-R Large [wilinski-etal-2024-exploring-representations] A steering matrix is computed as the difference-in-medians of activation between concept classes, with the paper noting the mean-based variant works equivalently [wilinski-etal-2024-exploring-representations] Adding it steers outputs toward concept-informed predictions on synthetic and forecasting tasks [wilinski-etal-2024-exploring-representations] On a real ECG5000 dataset, MOMENT-plus-SVM samples initially classified 100% as normal all swap output class after steering, a causal classification flip [wilinski-etal-2024-exploring-representations]

models: Chronos · method: Difference-in-means direction, Linear probing, Activation Steering (Addition)

MOMENT

On the Internal Semantics of Time-Series Foundation Models (2025)measured

Time-series foundation models linearly recover generative parameters by depth

Details

Pandey et al. fit per-layer linear probes on frozen Chronos and MOMENT to regress synthetic-series generative parameters [pandey-etal-2025-internal-semantics-time-series-foundation-models] Early layers capture local time-domain patterns (AR(1), trend, level shifts), deeper layers specialize in dispersion and change-points, and spectral/warping concepts stay hardest throughout [pandey-etal-2025-internal-semantics-time-series-foundation-models] Probe accuracy improves when a parameter varies smoothly along the UMAP manifold, and Chronos yields more linearly recoverable representations than MOMENT [pandey-etal-2025-internal-semantics-time-series-foundation-models] In compositional settings probe performance degrades and embedding arithmetic shows interference; no causal intervention is performed [pandey-etal-2025-internal-semantics-time-series-foundation-models]

models: MOMENT · method: Linear probing
Exploring Representations and Interventions in Time Series Foundation Models (2024)measured

Time-series foundation models linearly encode concepts and steer, including ECG

Details

Wilinski et al. find block-like redundancy usable for pruning and linearly-represented concepts (periodicity, trends) in Chronos, MOMENT, and Moirai-1.1-R Large [wilinski-etal-2024-exploring-representations] A steering matrix is computed as the difference-in-medians of activation between concept classes, with the paper noting the mean-based variant works equivalently [wilinski-etal-2024-exploring-representations] Adding it steers outputs toward concept-informed predictions on synthetic and forecasting tasks [wilinski-etal-2024-exploring-representations] On a real ECG5000 dataset, MOMENT-plus-SVM samples initially classified 100% as normal all swap output class after steering, a causal classification flip [wilinski-etal-2024-exploring-representations]

models: MOMENT · method: Difference-in-means direction, Linear probing, Activation Steering (Addition)

ChatGLM

Probing then Editing Response Personality of Large Language Models (2025)measured

Personality traits are linearly encoded with a betweenness arrangement

Details

Ju et al. train layer-wise linear probes on 11 open LLMs (including InternLM-7B and InternLM2-7B, base and instruct) to classify which of three personality instructions a response is conditioned on, scored by V-information [ju-etal-2025-personality-editing] V-information rises through layers 5-12 then plateaus, with instruction-tuned models 1-3 points higher than their base counterparts [ju-etal-2025-personality-editing] A t-SNE of Llama-3.1-8B-Instruct shows Neuroticism and Agreeableness distant with Extraversion situated between them, an informal betweenness arrangement [ju-etal-2025-personality-editing] This geometry explains why converting either trait into Extraversion is easier than converting directly between Neuroticism and Agreeableness [ju-etal-2025-personality-editing]

models: ChatGLM3-6B · method: Linear probing

InternLM

Probing then Editing Response Personality of Large Language Models (2025)measured

Personality traits are linearly encoded with a betweenness arrangement

Details

Ju et al. train layer-wise linear probes on 11 open LLMs (including InternLM-7B and InternLM2-7B, base and instruct) to classify which of three personality instructions a response is conditioned on, scored by V-information [ju-etal-2025-personality-editing] V-information rises through layers 5-12 then plateaus, with instruction-tuned models 1-3 points higher than their base counterparts [ju-etal-2025-personality-editing] A t-SNE of Llama-3.1-8B-Instruct shows Neuroticism and Agreeableness distant with Extraversion situated between them, an informal betweenness arrangement [ju-etal-2025-personality-editing] This geometry explains why converting either trait into Extraversion is easier than converting directly between Neuroticism and Agreeableness [ju-etal-2025-personality-editing]

models: InternLM-7B, InternLM2-7B · method: Linear probing

ProSoRo (Proprioceptive Soft Robot)

Anchoring Morphological Representations Unlocks Latent Proprioception in Soft Robots (2025)measured

Cross-modally correlated soft-robot VAE axes are individually manipulable primitives

Details

Han et al. train a multi-modal VAE encoding a soft robot's motion, force, and shape measurements into a shared 32-dimensional latent code [han-etal-2025-anchoring-morphological-representations-prosoro] Six components stand out with high cross-modal correlation, marking coordinate axes shared by all three modalities' encoders rather than modality-specific noise [han-etal-2025-anchoring-morphological-representations-prosoro] Traversing each of these six axes in isolation drives a distinct, geometrically coherent physical deformation mode, a causal intervention on an extracted direction [han-etal-2025-anchoring-morphological-representations-prosoro] Clustering the full latent-code trajectory recovers four separable clusters matching four ground-truth physical interaction phases [han-etal-2025-anchoring-morphological-representations-prosoro]

models: ProSoRo Multi-modal Variational Autoencoder (motion/force/shape latent proprioception) · method: Causal interventions (steering)

DCGAN (Deep Convolutional GAN)

Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks (2016)measured

DCGAN latent vector arithmetic yields a smiling man from face-attribute offsets

Details

Radford et al. train a DCGAN on aligned faces and LSUN bedrooms and apply Mikolov-style vector-offset arithmetic directly in the generator's latent Z space [radford-etal-2016-dcgan] Averaging "smiling woman" Z vectors, subtracting averaged "neutral woman," and adding averaged "neutral man" yields a Z the generator renders as a smiling man [radford-etal-2016-dcgan] Similar arithmetic is demonstrated for other attributes (e.g. eyeglasses), a direct causal latent-space manipulation [radford-etal-2016-dcgan] Results are shown qualitatively via generated image grids, with no quantitative effect-size metric reported [radford-etal-2016-dcgan]

models: DCGAN (trained on aligned & cropped celebrity faces) · method: Vector-offset analogy test, Causal interventions (steering)

Byte-level multiplicative LSTM (Radford, Józefowicz & Sutskever 2017)

Learning to Generate Reviews and Discovering Sentiment (2017)measured

A single sentiment neuron in a byte-level mLSTM linearly encodes and steers sentiment

Details

Radford et al. train a 4,096-unit byte-level multiplicative LSTM unsupervised as a next-byte model on ~82M Amazon reviews [radford-etal-2017-sentiment-neuron] A logistic-regression probe reaches 91.8% on binary SST, and a single hidden unit's activation alone reaches comparable accuracy [radford-etal-2017-sentiment-neuron] Clamping/overwriting the sentiment neuron during generation flips the sentiment of the model's own generated review text [radford-etal-2017-sentiment-neuron] Later replication found ablating the neuron only marginally affects downstream classification, questioning how exclusively sentiment is localized to one unit [radford-etal-2017-sentiment-neuron]

models: Byte mLSTM (4096-unit, trained on ~82M Amazon reviews) · method: Linear probing, Causal interventions (steering)

GLM-4

Simulated Adoption: Decoupling Magnitude and Direction in LLM In-Context Conflict Resolution (2026)measured

In-context conflict shows architecture-dependent dilution but universal orthogonal interference

Details

Zhang & Lin inject counterfactual contexts into MMLU questions and decompose the residual-stream update into a radial (norm-ratio) and an angular (cosine to the correct-answer direction) component [zhang-lin-2026-simulated-adoption-decoupling-magnitude-and-direction-in-llm-in-context-conflict-resolution] Radial "Manifold Dilution" is architecture-dependent: only Llama-3.1-8B shows real dilution (gamma=0.978) while Qwen3-4B and GLM-4-9B show none despite equal logit collapse [zhang-lin-2026-simulated-adoption-decoupling-magnitude-and-direction-in-llm-in-context-conflict-resolution] Angular "Orthogonal Interference" holds universally: interference-to-correct-answer cosine clusters near zero across all three models, not antiparallel suppression [zhang-lin-2026-simulated-adoption-decoupling-magnitude-and-direction-in-llm-in-context-conflict-resolution] Regressing angular deviation against logit drop gives R^2=0.90 (Qwen) and 0.87 (GLM) [zhang-lin-2026-simulated-adoption-decoupling-magnitude-and-direction-in-llm-in-context-conflict-resolution]

models: GLM-4-9B · method: Geometric analysis

NeuralDaredevil

Refusal in Language Models Is Mediated by a Single Direction (2024), Representation Engineering: A Top-Down Approach to AI Transparency (2023), Programming Refusal with Conditional Activation Steering (2024), Refusal Direction is Universal Across Safety-Aligned Languages (2025)measured

Refusal is mediated by a single direction across models

Details

Arditi et al. show across 13 chat models (5 families, 1.8B-72B) that a single diff-in-means direction between harmful and harmless instructions both erases refusal when ablated and induces it when added, with no offset term [arditi-etal-2024] Zou et al. independently confirm via a LAT harmfulness vector in Vicuna-13B that stays a >90% classifier under jailbreaks and adversarial suffixes [arditi-etal-2024] Lee et al. replicate across nine further chat models and add conditional steering that gates the refusal vector on a condition vector for selective refusal [arditi-etal-2024] Yung et al. average per-model refusal vectors into a universal refusal direction transferring to jailbreak detection ~10% better than single-model baselines [arditi-etal-2024] The account is contested by Marshall et al. (an affine offset is needed) and Wollschlager et al. (a cone of directions), and Yin et al. localize attack-suppressed versus robust heads [arditi-etal-2024]

models: NeuralDaredevil-8B-abliterated · method: Difference-in-means direction, Causal interventions (steering), Activation Steering (Addition), Linear Artificial Tomography (LAT), PCA, Conditional (gated) activation steering

Hermes

Refusal in Language Models Is Mediated by a Single Direction (2024), Representation Engineering: A Top-Down Approach to AI Transparency (2023), Programming Refusal with Conditional Activation Steering (2024), Refusal Direction is Universal Across Safety-Aligned Languages (2025)measured

Refusal is mediated by a single direction across models

Details

Arditi et al. show across 13 chat models (5 families, 1.8B-72B) that a single diff-in-means direction between harmful and harmless instructions both erases refusal when ablated and induces it when added, with no offset term [arditi-etal-2024] Zou et al. independently confirm via a LAT harmfulness vector in Vicuna-13B that stays a >90% classifier under jailbreaks and adversarial suffixes [arditi-etal-2024] Lee et al. replicate across nine further chat models and add conditional steering that gates the refusal vector on a condition vector for selective refusal [arditi-etal-2024] Yung et al. average per-model refusal vectors into a universal refusal direction transferring to jailbreak detection ~10% better than single-model baselines [arditi-etal-2024] The account is contested by Marshall et al. (an affine offset is needed) and Wollschlager et al. (a cone of directions), and Yin et al. localize attack-suppressed versus robust heads [arditi-etal-2024]

models: Hermes-2-Pro-Llama-3-8B · method: Difference-in-means direction, Causal interventions (steering), Activation Steering (Addition), Linear Artificial Tomography (LAT), PCA, Conditional (gated) activation steering

H2O-Danube3

Refusal in Language Models Is Mediated by a Single Direction (2024), Representation Engineering: A Top-Down Approach to AI Transparency (2023), Programming Refusal with Conditional Activation Steering (2024), Refusal Direction is Universal Across Safety-Aligned Languages (2025)measured

Refusal is mediated by a single direction across models

Details

Arditi et al. show across 13 chat models (5 families, 1.8B-72B) that a single diff-in-means direction between harmful and harmless instructions both erases refusal when ablated and induces it when added, with no offset term [arditi-etal-2024] Zou et al. independently confirm via a LAT harmfulness vector in Vicuna-13B that stays a >90% classifier under jailbreaks and adversarial suffixes [arditi-etal-2024] Lee et al. replicate across nine further chat models and add conditional steering that gates the refusal vector on a condition vector for selective refusal [arditi-etal-2024] Yung et al. average per-model refusal vectors into a universal refusal direction transferring to jailbreak detection ~10% better than single-model baselines [arditi-etal-2024] The account is contested by Marshall et al. (an affine offset is needed) and Wollschlager et al. (a cone of directions), and Yin et al. localize attack-suppressed versus robust heads [arditi-etal-2024]

models: H2O-Danube3-4B-Chat · method: Difference-in-means direction, Causal interventions (steering), Activation Steering (Addition), Linear Artificial Tomography (LAT), PCA, Conditional (gated) activation steering

MusicGen

Steering Autoregressive Music Generation with Recursive Feature Machines (2025)measured

RFM/AGOP directions from frozen MusicGen enable real-time multi-attribute steering

Details

Zhao et al. adapt Recursive Feature Machines / Average Gradient Outer Product to train lightweight layer-wise probes on frozen MusicGen-Large hidden states using the SynTheory music-theory dataset [zhao-etal-2025-steering-autoregressive-music-generation-with-rfm] The AGOP-derived concept directions capture fine-grained musical attributes such as notes and chords [zhao-etal-2025-steering-autoregressive-music-generation-with-rfm] Injecting these directions at inference enables real-time steering without per-step optimization, including time-varying schedules and simultaneous multi-attribute control [zhao-etal-2025-steering-autoregressive-music-generation-with-rfm]

models: MusicGen-Large · method: RFM/AGOP Steering Direction Extraction, Causal interventions (steering)
Discovering and Steering Interpretable Concepts in Large Generative Music Models (2026)measured

SAE decoder directions in MusicGen are causally steerable; listeners prefer steered audio

Details

Singh, Cherep & Maes train k-sparse autoencoders on residual-stream activations from five layers each of MusicGen-Large and MusicGen-Small on the ~160k-clip MusicSet corpus [singh-cherep-maes-2026-discovering-and-steering-interpretable-concepts-in-large-generative-music-models] Adding a scaled SAE decoder direction to the residual stream causally steers generation, with 15-35% of tested features improving CLAP alignment [singh-cherep-maes-2026-discovering-and-steering-interpretable-concepts-in-large-generative-music-models] A controlled human listening study (10 participants, 100 trials) finds listeners picked the SAE-steered audio 66/100 versus 17/17 for random-direction and baseline controls (chi^2=48.02, p<.0001) [singh-cherep-maes-2026-discovering-and-steering-interpretable-concepts-in-large-generative-music-models] The steering is thus a perceptibly validated, geometry-tied causal effect in a real pretrained generative audio model [singh-cherep-maes-2026-discovering-and-steering-interpretable-concepts-in-large-generative-music-models]

models: MusicGen-Large, MusicGen-Small · method: Sparse Autoencoders (SAE)

Custom Recurrent RL Policies

Steering Multirobot Behavior via Closed-Loop Affine Activation Editing (2026)measured

SAE latents in a frozen multirobot policy are steerable via closed-loop affine edits

Details

Das et al. train a sparse autoencoder over a frozen multi-quadrotor navigation policy's activations to identify behavior-relevant latent features [das-etal-2026-steering-multirobot-behavior-via-closed-loop-affine-activation-editing] A lightweight RL steering policy applies state-dependent affine edits (scale and shift, not just addition) to selected SAE latents at every inference step [das-etal-2026-steering-multirobot-behavior-via-closed-loop-affine-activation-editing] Applied purely at the activation level without changing the frozen policy's weights, the edits steer velocity profiles and coordinate formation-preserving behavior [das-etal-2026-steering-multirobot-behavior-via-closed-loop-affine-activation-editing] They also induce a novel emergent behavior (reduced camera-surveillance exposure) absent from the unedited policy, establishing the latent directions as causally sufficient [das-etal-2026-steering-multirobot-behavior-via-closed-loop-affine-activation-editing]

models: Custom frozen multi-quadrotor navigation policy · method: Sparse Autoencoders (SAE), Closed-Loop Affine Activation Editing (CLAE)

Aya

Large Language Models Share Representations of Latent Grammatical Concepts Across Typologically Diverse Languages (2025)measured

Shared SAE morphosyntactic directions are causally necessary and sufficient across languages

Details

Brinkmann et al. train a Gated SAE on layer-16 residual activations of Llama-3-8B and Aya-23-8B and use attribution patching to find each language/concept's top causal features for grammatical number, gender, and tense [brinkmann-etal-2025-crosslingual-grammatical-concepts] Cross-lingual top-feature overlap reaches up to 50% (one feature is top-influential for grammatical gender across all 15 inflecting languages), with mean cross-concept overlap 13.9% [brinkmann-etal-2025-crosslingual-grammatical-concepts] Ablating only the massively-multilingual features drops classifier performance to 64%, so most of the causal effect concentrates in a small multilingual core [brinkmann-etal-2025-crosslingual-grammatical-concepts] Clamping a single multilingual feature during translation flips the intervened concept's probe label while leaving others unaffected, showing necessity and sufficiency [brinkmann-etal-2025-crosslingual-grammatical-concepts]

models: Aya-23-8B · method: Sparse Autoencoders (SAE), Causal interventions (steering), Activation Steering (Addition)

Vision Transformer (ViT)

From Edges to Depth: Probing the Spatial Hierarchy in Vision Transformers (2026)measured

A single linear ViT direction encodes depth and causally drives depth estimation

Details

Sanghavi fits linear probes on frozen ViT-B/16 (google/vit-base-patch16-224-in21k, plus a random-weight control) layer activations, finding depth best decoded at layer 8 (MAE=0.0875) [sanghavi-2026-from-edges-to-depth-vit-spatial-hierarchy] Ablating the probe-identified direction increases depth-estimation error by 49-165%, while ablating a random direction of the same rank changes error by under 1% [sanghavi-2026-from-edges-to-depth-vit-spatial-hierarchy] Depth is thus concentrated in a specific linear direction, not diffusely distributed [sanghavi-2026-from-edges-to-depth-vit-spatial-hierarchy] Targeted single-direction activation patching retains 76% of its causal effect at a 9-layer gap before decaying at longer range [sanghavi-2026-from-edges-to-depth-vit-spatial-hierarchy]

models: ViT-B/16 (ImageNet-21k, supervised) · method: Linear probing, Activation patching, Causal interventions (steering)

Poolside Laguna

Latent Programming Horizons in Coding Agents (2026)measured

Coding-agent residual streams decode program properties 25 edit-steps ahead

Details

Silva et al. train linear (logistic-regression) probes on residual-stream activations of real Qwen3.6-35B-A3B and Laguna-XS.2 running as coding agents [silva-tu-monperrus-2026-latent-programming-horizons] The probes decode well-formedness/correctness/regression properties of the program several edit-steps before the edit that produces them, from the current hidden state [silva-tu-monperrus-2026-latent-programming-horizons] Decoding AUC reaches up to ~0.83 and stays above a shuffled-label chance baseline out to roughly 25 steps ahead of materialization [silva-tu-monperrus-2026-latent-programming-horizons] The effect transfers across benchmarks without probe retraining and is purely observational, with no activation-level causal intervention [silva-tu-monperrus-2026-latent-programming-horizons]

models: Laguna-XS.2 · method: Linear probing

Counter-Language Transformer

Emergent Stack Representations in Modeling Counter Languages Using Transformers (2025)measured

Stack depth is linearly decoded in tiny counter-language transformers

Details

Tiwari et al. train a tiny 1-layer transformer per formal language on Dyck-1 and Shuffle-2/4/6 counter languages, reframing next-token prediction as multi-label valid-token prediction [tiwari-etal-2025] Linear probes on the final layer decode each stack's current depth with high accuracy, while a Hewitt-Liang control-task check stays near chance [tiwari-etal-2025] Accuracy is systematically higher for Shuffle languages than Dyck-1 and higher for larger k, attributed to each individual stack updating less often as k grows [tiwari-etal-2025] The paper stops at the linear-direction claim and explicitly disclaims any causal characterization of the stacks [tiwari-etal-2025]

models: Counter-Language Transformer (1 layer, 4 heads, causal-masked encoder) · method: Linear probing, Mlp probing

Ministral

Actionable Activation Directions for Detecting and Mitigating Emergent Misalignment Across Language Model Families (2026)measured

A diff-in-means misalignment direction transfers cross-architecture but non-specifically

Details

Syed QLoRA-finetunes four models identically on insecure code and extracts a diff-in-means misalignment direction, finding 99.6% within-model separability versus 50% for a secure-code control [syed-2026-actionable-activation-directions-emergent-misalignment] A ridge-regression map fit between any two models' activation spaces transfers the direction across all twelve model-family pairs with above-chance accuracy (Gemma-to-Llama 87%, Gemma-to-Ministral 90%) [syed-2026-actionable-activation-directions-emergent-misalignment] Steering a target model with the transferred direction suppresses code-related spillover (delta 13-46 points) but fails a specificity control, unlike the fully specific within-model effect (delta 21-51 points) [syed-2026-actionable-activation-directions-emergent-misalignment] This reveals an asymmetric donor/receiver topology, with Gemma and Qwen acting as donors and Llama as a receiver [syed-2026-actionable-activation-directions-emergent-misalignment]

models: Ministral-3-3B-Instruct · method: Difference-in-means direction, Cross-model direction transfer via ridge regression

Mini-ICL toy transformers (custom, Yan, Yang & Zhong 2026)

Task Vector Geometry Underlies Dual Modes of Task Inference in Transformers (2026)measured

Task-vector subspace implements Bayesian retrieval; OOD generalization is near-orthogonal

Details

Yan et al. train small RoPE transformers on synthetic latent-task mixtures and formalize task vectors as task-conditional-minus-global mean hidden states [yan-yang-zhong-2026-task-vector-geometry-dual-modes] Finite-context hidden states are well approximated as a simplex convex combination of task vectors whose coefficients closely track the exact Bayesian posterior as context accumulates [yan-yang-zhong-2026-task-vector-geometry-dual-modes] Substituting the task-vector mixture component with a target simplex point steers outputs to the theoretical mixture (KL 0.11 to 0.03 in E1) [yan-yang-zhong-2026-task-vector-geometry-dual-modes] Out-of-distribution generalization occupies a second, near-orthogonal subspace emerging only at high task diversity, confirmed by double-dissociation ablation [yan-yang-zhong-2026-task-vector-geometry-dual-modes] A real Qwen2.5-7B projection qualitatively echoes the geometry, with in-distribution vertex convergence versus out-of-distribution orthogonality [yan-yang-zhong-2026-task-vector-geometry-dual-modes]

models: Mini-ICL synthetic RoPE transformer (6L/2H/d128 for dice and Markov-chain tasks; 16L for linear regression) · method: Geometric analysis, Causal interventions (steering)

Synthetic GPT-2-style ICL transformers (custom, Yang, Lin, Lee, Papailiopoulos & Nowak 2025)

Task Vectors in In-Context Learning: Emergence, Formation, and Benefit (2025)measured

Natural task vectors emerge weakly in deep models; a TVP loss forces a layer to carry the task

Details

Yang et al. train small GPT-2-style transformers from scratch on synthetic ICL tasks and define a task vector behaviorally by whether its injection recovers ICL performance [yang-etal-2025-task-vectors-emergence-formation-benefit] On a shallow 3-layer model the natural task vector is causally effective (linear-regression MSE ~0.25 vs ~1.2 random) with layer-swap specificity [yang-etal-2025-task-vectors-emergence-formation-benefit] In deeper 8-layer models task information becomes distributed across layers and task-vector prompting performance is nearly random [yang-etal-2025-task-vectors-emergence-formation-benefit] An auxiliary task-vector-prompting loss trains a prescribed layer to serve as an injectable task vector, matching full ICL performance and improving out-of-distribution robustness [yang-etal-2025-task-vectors-emergence-formation-benefit]

models: Synthetic GPT-2-style ICL transformer (3-8 layers, trained from scratch on regression/token-offset/GINC/RegBench tasks) · method: ICL task-vector extraction via dummy-query patching, Causal interventions (steering)

GoogLeNet (Inception v1)

Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV) (2018)measured

TCAV concept activation vectors are causally significant linear directions

Details

Kim et al. define a Concept Activation Vector as the vector orthogonal to a linear classifier separating concept-example activations from random counterexamples in a layer [kim-etal-2018-tcav] Sensitivity is quantified via a directional derivative of the class logit along the CAV, aggregated into a TCAV score gated by 500 retrainings and a Bonferroni-corrected two-sided t-test [kim-etal-2018-tcav] On GoogLeNet and Inception V3, TCAV recovers intuitively correct sensitivities (striped scoring high for zebra, red for fire engine) [kim-etal-2018-tcav] It also reveals unintended biases the networks were never explicitly trained on, such as a "female" concept scoring high for the apron class [kim-etal-2018-tcav]

models: GoogLeNet (ImageNet-trained) · method: Linear probing, TCAV directional-derivative sensitivity testing

Inception

Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV) (2018)measured

TCAV concept activation vectors are causally significant linear directions

Details

Kim et al. define a Concept Activation Vector as the vector orthogonal to a linear classifier separating concept-example activations from random counterexamples in a layer [kim-etal-2018-tcav] Sensitivity is quantified via a directional derivative of the class logit along the CAV, aggregated into a TCAV score gated by 500 retrainings and a Bonferroni-corrected two-sided t-test [kim-etal-2018-tcav] On GoogLeNet and Inception V3, TCAV recovers intuitively correct sensitivities (striped scoring high for zebra, red for fire engine) [kim-etal-2018-tcav] It also reveals unintended biases the networks were never explicitly trained on, such as a "female" concept scoring high for the apron class [kim-etal-2018-tcav]

models: Inception V3 (ImageNet-trained) · method: Linear probing, TCAV directional-derivative sensitivity testing

Weather/Climate Foundation Model

Mechanistic Interpretability Tool for AI Weather Models (2026)measured

PCA on a real pretrained weather model's latent mesh nodes reveals a dipole direction tracking synoptic wave troughs

Details

Tempest, Beylich & Craig (2026) run PCA on 512-dimensional latent feature vectors extracted at mesh nodes after each of real pretrained GraphCast's 16 processor steps (1-degree configuration, ERA5 reanalysis input) [tempest-etal-2026-mechanistic-interpretability-tool-for-ai-weather-models] The first principal component forms a pronounced alternating dipole pattern over Northern mid-latitudes tracking synoptic-scale wave troughs (negative west / positive east of each trough); a separate leading direction correlates with specific-humidity gradients (e.g. Sahel moisture front) [tempest-etal-2026-mechanistic-interpretability-tool-for-ai-weather-models] Purely observational -- qualitative visual correspondence across two case studies and a small sample of forecast times, explicitly framed by the authors as preliminary; no causal intervention performed [tempest-etal-2026-mechanistic-interpretability-tool-for-ai-weather-models]

models: GraphCast (weather foundation model) · method: PCA

OPT

Eliciting Latent Predictions from Transformers with the Tuned Lens (2023)measured

The Tuned Lens replaces the logit lens with a trained per-layer affine correction

Details

Belrose et al. replace the logit lens's direct reuse of the final unembedding on intermediate states with a trained per-layer affine translator, distilled via KL divergence to the model's own final output [belrose-etal-2023-tuned-lens] This removes two documented logit-lens failure modes: a systematic 4-5 bit-per-layer vocabulary bias and unreliability from hidden-state covariance drifting across layers [belrose-etal-2023-tuned-lens] A causal-basis extraction finds the lens's most-sensitive directions strongly match the model's real causal directions (Spearman rho=0.89), with none influential to the lens but not the model [belrose-etal-2023-tuned-lens] Tested across GPT-2, GPT-Neo, GPT-NeoX-20B, OPT, BLOOM, and the full Pythia suite, it gives lower perplexity than the logit lens, transfers to a fine-tune (Vicuna-13B from LLaMA-13B), and detects prompt injection at near-perfect AUROC [belrose-etal-2023-tuned-lens]

models: OPT-125M, OPT-1.3B, OPT-6.7B · method: Tuned Lens

DINOv3

Compositional Generalization Requires Linear, Orthogonal Representations in Vision Embedding Models (2026)measured

Compositional generalization needs linear, additive, orthogonal per-concept factor vectors

Details

Uselis, Dittadi & Oh recover per-concept difference vectors from paired stimuli across three datasets in many pretrained encoders (CLIP, OpenCLIP, MetaCLIP, MetaCLIP2, SigLIP, SigLIP2, DINOv1/v2/v3) [uselis-dittadi-oh-2026-compositional-generalization-requires-linear-orthogonal-representations-in-vision-embedding-models] A whitened, probe-span-projected reconstruction of embeddings as a linear sum of recovered factors reaches R^2=0.42-0.63 versus 0.12-0.42 for a randomly-initialized-encoder baseline [uselis-dittadi-oh-2026-compositional-generalization-requires-linear-orthogonal-representations-in-vision-embedding-models] Cross-concept factor vectors are confirmed mutually orthogonal via cosine similarity and effective-rank analysis [uselis-dittadi-oh-2026-compositional-generalization-requires-linear-orthogonal-representations-in-vision-embedding-models] The degree of linear factorization correlates with each model's own compositional-generalization accuracy across variants [uselis-dittadi-oh-2026-compositional-generalization-requires-linear-orthogonal-representations-in-vision-embedding-models]

models: DINOv3 ViT-B/16 · method: Additive per-concept factor recovery and orthogonality scoring

Apertus

Valence-Arousal Subspace in LLMs: Circular Emotion Geometry and Multi-Behavioral Control (2026), Where Do Models Find Happiness? Emotion Vectors in Open-Source LLMs (2026)measured

A valence-arousal circumplex subspace causally steers emotion and refusal

Details

Sun et al. recover two near-orthogonal valence/arousal axes inside Llama-3.1-8B-Instruct (replicated in Qwen3-8B/14B) by PCA-projecting contrastive emotion-steering vectors and ridge-regressing against human ratings [sun-etal-2026-valence-arousal] Projecting the emotion vectors onto this plane traces a circle: a circularity statistic reaches 2.76-4.08 with fitted radii ~0.37-0.39, analogous to Russell's circumplex [sun-etal-2026-valence-arousal] The valence axis recovers self-reported valence at r=0.97 and the NRC-VAD lexicon at r=0.71, with cross-model valence agreement r=0.95 [sun-etal-2026-valence-arousal] Adding valence/arousal directions at specific circle angles produces dose-dependent, angle-specific shifts in generated-text affect (e.g. 0deg: delta-valence +0.75; 180deg: -0.73) [sun-etal-2026-valence-arousal] The same arousal axis causally controls refusal (20%->86% on OKTest) and sycophancy, with random-direction controls within 2-3 points of baseline [sun-etal-2026-valence-arousal] Logit-clamping and top-neuron ablation along the direction crash refusal while preserving MATH-500/IFEval, and an independent refusal direction is near-orthogonal (86.5deg) to the VA plane [sun-etal-2026-valence-arousal] Van der Ben et al. independently replicate the valence/arousal PCA structure (PC1-valence r=0.72-0.83, PC2-arousal r=0.21-0.45) in Apertus-8B and Gemma-4-E4B-it without computing circularity or steering [vanderben-etal-2026-emotion-vectors-open-source-llms] They add that cross-architecture layer-depth trajectories diverge sharply (a 3-phase plateau in Apertus vs a smooth gradient in Gemma via linear CKA) [vanderben-etal-2026-emotion-vectors-open-source-llms]

models: Apertus-8B-Instruct-2509 · method: PCA, Linear probing, Activation Steering (Addition)

AIMv2

Interpreting the Linear Structure of Vision-Language Model Embedding Spaces (2025)measured

VLM SAE concepts are single-modality yet gap-orthogonal

Details

- BatchTopK sparse autoencoders trained on ~600k COCO vision-language embeddings (dictionary 4096) put 99% of the energy in 512 concepts, the other ~3500 sharing 1%. [papadimitriou-etal-2025-vlm-linear-structure] - The energy-dominant concepts are nearly single-modality in activation, yet many of their directions are nearly orthogonal to the modality-gap subspace (accuracy ~0.5 as modality classifiers). [papadimitriou-etal-2025-vlm-linear-structure] - An "SAE projection effect" reconciles this: sparse thresholding interacts with the per-modality input distributions, so a direction can be gap-orthogonal yet fire for one modality; top-512 concepts are seed-stable (0.92) vs 0.16 for low-energy ones. [papadimitriou-etal-2025-vlm-linear-structure] - Observational, on CLIP, SigLIP, SigLIP2 and AIMv2 (the paper states no model variants). [papadimitriou-etal-2025-vlm-linear-structure]

models: AIMv2 · method: BatchTopK Sparse Autoencoders, Geometric analysis

SiT (Scalable Interpolant Transformers)

General and Efficient Steering of Diffusion Models (2026)measured

A single RFM/AGOP direction per concept steers diffusion at a fraction of guidance cost

Details

Wang et al. train a Recursive Feature Machine on flattened U-Net/DiT block activations from forward-noised labeled images, taking the sign-corrected top eigenvectors of its AGOP matrix as one reusable per-concept steering direction [wang-etal-2026-general-and-efficient-steering-of-diffusion-models-a-single-rfm-agop-direction-per-concept-steers-unconditional-and-latent-diffusion-models-at-a-fraction-of-classifier-guidance-cost] Adding this direction during sampling steers unconditional DDPM/ADM U-Nets (CIFAR-10, ImageNet-256, CelebA-HQ), the transformer latent-diffusion SiT-XL/2, and Stable Diffusion 1.5 [wang-etal-2026-general-and-efficient-steering-of-diffusion-models-a-single-rfm-agop-direction-per-concept-steers-unconditional-and-latent-diffusion-models-at-a-fraction-of-classifier-guidance-cost] On CIFAR-10, NA-RFM reaches 96.6% guidance accuracy versus 77.1% and 86.0% baselines, with FID 41.4 versus 73.9 and 41.9 [wang-etal-2026-general-and-efficient-steering-of-diffusion-models-a-single-rfm-agop-direction-per-concept-steers-unconditional-and-latent-diffusion-models-at-a-fraction-of-classifier-guidance-cost] It runs one fixed low-rank direction across the sampling trajectory at a 16x sampling speedup over a training-free-guidance baseline [wang-etal-2026-general-and-efficient-steering-of-diffusion-models-a-single-rfm-agop-direction-per-concept-steers-unconditional-and-latent-diffusion-models-at-a-fraction-of-classifier-guidance-cost]

models: SiT-XL/2 · method: RFM/AGOP Steering Direction Extraction

Moirai

Exploring Representations and Interventions in Time Series Foundation Models (2024)measured

Time-series foundation models linearly encode concepts and steer, including ECG

Details

Wilinski et al. find block-like redundancy usable for pruning and linearly-represented concepts (periodicity, trends) in Chronos, MOMENT, and Moirai-1.1-R Large [wilinski-etal-2024-exploring-representations] A steering matrix is computed as the difference-in-medians of activation between concept classes, with the paper noting the mean-based variant works equivalently [wilinski-etal-2024-exploring-representations] Adding it steers outputs toward concept-informed predictions on synthetic and forecasting tasks [wilinski-etal-2024-exploring-representations] On a real ECG5000 dataset, MOMENT-plus-SVM samples initially classified 100% as normal all swap output class after steering, a causal classification flip [wilinski-etal-2024-exploring-representations]

models: Moirai-1.1-R Large · method: Difference-in-means direction, Linear probing, Activation Steering (Addition)

Sudoku Transformer

Transformers Linearly Represent Highly Structured World Models (2026)measured

A Sudoku transformer builds substructure-level linear constraint directions

Details

Kniazev & Fijalkow train an 8-layer transformer on Sudoku solving traces and probe its residual stream at the clues-end token [kniazev-fijalkow-2026] Per-cell digit probes plateau at 80% (imperfect linear separability), but per-cell candidate probes reach 1.0 exact-match in mid-layers [kniazev-fijalkow-2026] Candidate probe directions align by shared substructure (cosine ~0.33 for one shared row/column/box, ~0.65 for two), consistent with each being a linear combination of substructure-level directions [kniazev-fijalkow-2026] Substructure "digit present in row/column/box" probes reach perfect accuracy, and patching only their direction components flips the top-1 prediction in 99% of cases [kniazev-fijalkow-2026] Mid-layer heads specialize by substructure, and a final-MLP set of monosemantic "naked-single" neurons commits the answer [kniazev-fijalkow-2026]

models: Sudoku Transformer (8 layers, 8 heads, d_model=576) · method: Linear probing, Activation patching, Causal interventions (steering), Direct Logit Attribution (DLA), Logit Lens

Euclid Q1 Galaxy Morphology Models (Zoobot ConvNeXt-Nano + self-supervised ViT-S MAE)

Re-envisioning Euclid Galaxy Morphology: Identifying and Interpreting Features with Sparse Autoencoders (2025)measured

SAEs trained on a real Euclid galaxy MAE surface monosemantic morphology directions outperforming raw PCA

Details

Wu & Walmsley (2026) train a Matryoshka SAE (batch top-k=64) on a real self-supervised ViT-S MAE (30.1M params, trained on 3M Euclid Q1 galaxy images) and on a real Zoobot ConvNeXt-Nano classifier's embeddings [wu-walmsley-2026-euclid-galaxy-morphology-saes] Top-64 SAE features correlate with Galaxy Zoo human morphology labels at mean-max Spearman r=0.523+/-0.123 (self-supervised) and 0.296+/-0.129 (supervised), versus only 0.434+/-0.174 and 0.176+/-0.138 for a raw-PCA baseline of the same rank; SAE features also stay coherent through k=64 while PCA degrades beyond ~5 components [wu-walmsley-2026-euclid-galaxy-morphology-saes] SAEs additionally surface features outside the Galaxy Zoo taxonomy (dust lanes, blue companions to ellipticals); purely correlational, no causal steering performed [wu-walmsley-2026-euclid-galaxy-morphology-saes]

models: Self-supervised ViT-S MAE (30.1M params, trained on 3M Euclid Q1 galaxy images, 90% masking) · method: Sparse Autoencoders (SAE)

StyleNeRF

NaviNeRF: NeRF-based 3D Representation Disentanglement by Latent Semantic Navigation (2023)measured

Self-supervised NaviNeRF finds dimension-restricted semantic directions in StyleNeRF W+

Details

Xie et al. train a self-supervised Navigator that discovers interpretable semantic directions in the W+ latent space of a real StyleNeRF 3D-aware generator [xie-etal-2023-navinerf-latent-semantic-navigation] Restricting shifts to the fine-grained 9th-18th dimensions of the 18-vector W+ style code outperforms shifting all 18 dimensions for disentangled attribute control [xie-etal-2023-navinerf-latent-semantic-navigation] Using the full W+ space (18 per-layer style vectors) outperforms the single-vector W space for fine-grained disentanglement [xie-etal-2023-navinerf-latent-semantic-navigation] NaviNeRF's FID/KID is competitive with its StyleNeRF backbone and far ahead of pi-GAN and GIRAFFE, extending 2D-latent direction discovery to a 3D generative space [xie-etal-2023-navinerf-latent-semantic-navigation]

models: StyleNeRF (FFHQ, 256x256) · method: Causal interventions (steering)

IRIS (VQ-VAE + Transformer world model)

What Do World Models Learn in RL? Probing Latent Representations in Learned Environment Simulators (2026)measured

Game-state variables are near-linearly decodable with architecture-dependent layer profiles

Details

Zhang applies linear and MLP probing to frozen hidden states of IRIS (VQ-VAE + transformer) and DIAMOND (diffusion UNet) world models trained on Atari, finding the two probe types agree closely (delta R^2 <=0.06) [zhang-2026-what-do-world-models-learn-in-rl] IRIS decodes paddle position and score at R^2>0.99 at every layer (a flat profile), while DIAMOND concentrates decodable abstract state at its UNet bottleneck layer [zhang-2026-what-do-world-models-learn-in-rl] Shifting IRIS layer-5 hidden states along the probe-derived direction produces monotonic, strongly correlated output changes (r>=0.96), confirming functional use rather than mere decodability [zhang-2026-what-do-world-models-learn-in-rl] Caveat: a solo-author preprint with no confirmed peer review and unusually clean results (R^2>0.99, r>=0.96) warranting independent replication [zhang-2026-what-do-world-models-learn-in-rl]

models: IRIS (Atari 100k, Breakout/Pong) · method: Linear probing, Causal interventions (steering)

DIAMOND (diffusion UNet world model)

What Do World Models Learn in RL? Probing Latent Representations in Learned Environment Simulators (2026)measured

Game-state variables are near-linearly decodable with architecture-dependent layer profiles

Details

Zhang applies linear and MLP probing to frozen hidden states of IRIS (VQ-VAE + transformer) and DIAMOND (diffusion UNet) world models trained on Atari, finding the two probe types agree closely (delta R^2 <=0.06) [zhang-2026-what-do-world-models-learn-in-rl] IRIS decodes paddle position and score at R^2>0.99 at every layer (a flat profile), while DIAMOND concentrates decodable abstract state at its UNet bottleneck layer [zhang-2026-what-do-world-models-learn-in-rl] Shifting IRIS layer-5 hidden states along the probe-derived direction produces monotonic, strongly correlated output changes (r>=0.96), confirming functional use rather than mere decodability [zhang-2026-what-do-world-models-learn-in-rl] Caveat: a solo-author preprint with no confirmed peer review and unusually clean results (R^2>0.99, r>=0.96) warranting independent replication [zhang-2026-what-do-world-models-learn-in-rl]

models: DIAMOND (Atari 100k, Breakout/Pong) · method: Linear probing, Causal interventions (steering)