MATH · IN · MODELS
structures / Linear Structures / Linear Subspace

Linear Subspace

OBJECTgeometric-objectsubsetlinearK:zerolevel-setmanifoldintermediatehow it's classified →

k-dimensional linear subspace V ⊂ ℝᵈ corresponding to a group of related features. Generalises the single feature direction to the multidimensional case.

Replicationcomputed from the corpus — never hand-assigned
139 papers · no shared authors14 architecture classes · across papers11 domains · across papers106 model families · across papers
Filled = two or more values reported by papers that share no author — replication. Outlined = two or more values, but all from a single study — breadth, not replication. Grey = a single value. Derived from paper authorship and each model's architecture class, domain and family; it updates itself when a paper is added.

Definition

A linear subspace of dimension kk:

V=span{e1,…,ek}⊂RdV = \text{span}\{e_1,\ldots,e_k\} \subset \mathbb{R}^d

with orthonormal basis e1,…,eke_1,\ldots,e_k. The orthogonal projection of xx onto VV:

PVx=∑i=1k(ei⋅x) eiP_Vx = \sum_{i=1}^k (e_i\cdot x)\,e_i

The residual x−PVxx - P_Vx is the component of xx orthogonal to VV.

Intuition

A single binary feature needs only one direction to encode “how much.” A feature with several mutually exclusive categories (noun/verb/adjective/adverb) cannot be squeezed onto a single line without collapsing distinct categories onto the same point — it needs a whole kk-dimensional subspace, one extra dimension per additional independent category.

Properties

  • PVP_V is idempotent and self-adjoint. PV2=PVP_V^2=P_V (projecting twice changes nothing further) and PV=PV⊤P_V = P_V^\top (as a real symmetric matrix ∑ieiei⊤\sum_i e_ie_i^\top) — the two properties that characterize an orthogonal projection.

  • Orthogonal decomposition. Every x∈Rdx\in\mathbb{R}^d splits uniquely as x=PVx+PV⊥xx = P_Vx + P_{V^\perp}x, with PVx∈VP_Vx\in V, PV⊥x∈V⊥P_{V^\perp}x\in V^\perp, and PVx⊥PV⊥xP_Vx \perp P_{V^\perp}x. Consequently ∥x∥2=∥PVx∥2+∥PV⊥x∥2\|x\|^2 = \|P_Vx\|^2+\|P_{V^\perp}x\|^2 (Pythagorean identity).

  • Rank equals dimension. PVP_V has rank exactly k=dim⁡Vk=\dim V; its eigenvalues are 11 (multiplicity kk, eigenspace VV) and 00 (multiplicity d−kd-k, eigenspace V⊥V^\perp).

  • Erasure removes exactly the information in VV, nothing more, nothing less — if VV is exactly right. PV⊥x=x−PVxP_{V^\perp}x = x-P_Vx retains all information orthogonal to VV unchanged; if the concept genuinely and only occupies VV, this projection is a clean, lossless-elsewhere erasure. If the true concept subspace is only approximately VV (e.g. VV is mis-estimated or too low-rank), erasure will be incomplete or will remove unrelated information.

  • Nesting. If V′⊆VV'\subseteq V, then PV′(PVx)=PV′xP_{V'}(P_Vx) = P_{V'}x — projecting onto a larger subspace first and then a smaller one contained in it is the same as projecting onto the smaller one directly (idempotence extended across nested subspaces). Confirmed empirically, not just as a linear-algebra identity: Hernandez & Andreas (2021) find that fine-grained morphosyntactic sub-categories (e.g. noun-vs-noun distinctions) occupy a subspace nested inside their coarser parent category’s subspace (e.g. part-of-speech), at roughly half the rank — see the pos-subspace-nesting Observation. Contrast Polytope (Simplex)‘s “Composable with hierarchy via direct sums” property and hierarchical-orthogonality, where a different domain’s hierarchy levels are instead orthogonal complements of each other, not nested subspaces — the two are distinct empirically-observed relationships between hierarchy levels, not the same claim in different words.

  • Foundational demonstration: a protected-attribute subspace spans dozens to hundreds of directions, not one. Ravfogel, Elazar, Gonen, Twiton & Goldberg (2020), introducing INLP, show a gender-bias subspace in GloVe embeddings needs 35 iteratively-found orthogonal directions to be removed as effectively as it is (residual classification accuracy 54.4%, vs. 80.7% when only 10 hand-picked single-concept directions are removed) — direct evidence against treating such attributes as single-direction phenomena, and the origin of the multi-direction subspace-removal technique this map’s other INLP-based papers build on. See gender-subspace-spans-dozens-of-directions.

  • The minimal-distortion eraser is oblique, not orthogonal — a constraint prior work assumed without proving necessary. Belrose, Schneider-Joseph, Ravfogel, Cotterell, Raff & Biderman (2023, LEACE) prove the guardedness constraint that defines “erase this concept” only fixes a projection’s nullspace, leaving its range free — so restricting the search to orthogonal projections, as both INLP and RLACE implicitly do, is itself suboptimal for minimizing embedding distortion. Their closed-form (non-iterative) oblique projection provably guards against every linear classifier at once, removes exactly rank(ΣXZ)\mathrm{rank}(\Sigma_{XZ}) dimensions (rank 1 for a binary concept), and achieves the smallest measured embedding change of any method compared, in ~100x less compute than RLACE’s gradient-based search. See oblique-projection-erases-concept-with-minimal-distortion.

  • How many dimensions a removal-optimal subspace needs depends on the objective, provably. Ravfogel, Twiton, Goldberg & Cotterell (2022) formalize concept erasure as a minimax game (R-LACE) and prove — not merely observe — that INLP‘s iterative approach finds a provably suboptimal (unnecessarily large) subspace for linear-regression objectives, a provably optimal/identical subspace for Rayleigh-quotient objectives, and an empirically much larger subspace for classification objectives (INLP needs dozens-to-~100 dimensions where a single adversarially-optimized direction suffices). This refines “dozens to hundreds of directions” (the INLP finding above) into “however many the specific removal objective actually requires” — not a fixed property of the concept alone. See rank-one-subspace-suffices-for-gender-erasure.

  • Causally validated via subspace erasure plus a placebo control. Lasri, Pimentel, Lenci, Poibeau & Cotterell (2022) use INLP’s null-space projection (no counterfactual push) to remove BERT’s grammatical-number subspace at noun vs. verb positions separately, showing behavioral subject-verb-agreement accuracy drops only when the matching position’s subspace is removed (not when a random-direction placebo of equal size is removed, and not when the other position’s subspace is cross-applied) — evidence the subspace is causally used, is multi-dimensional (dozens of directions per layer), and is functionally distinct between nouns and verbs. See grammatical-number-usage-verified-causally.

  • A subspace can implement a functionally distinct role from a co-located direction/manifold, orthogonal to it, rather than encoding a separate concept. Maheswaranathan & Sussillo (2020), a direct follow-up to the same Line Attractor sentiment-RNN result, find that negation/intensification words (“not bad,” “extremely awesome”) deflect the hidden state into a 2-dimensional subspace (two components explaining 96.2% of variance) orthogonal to the line attractor itself, rather than moving further along it as ordinary valence words do — with two internal timescales (mean τ=2.6\tau=2.6 and 4.34.3 tokens) and negators/intensifiers occupying distinguishable regions of the same subspace. Causally validated by ablation: projecting the initial hidden state out of this 2D subspace drops test accuracy by 0.17% (68 misclassified reviews) while projecting out a random 2D subspace of the same dimension has no effect. See modifier-subspace-orthogonal-to-line-attractor-implements-negation-and-intensification.

  • Causally validated via subspace-targeted counterfactual push, not just erasure. Katinskaia & Yangarber (2024) use INLP to locate the multi-directional subspace encoding “boundedness” in Russian BERT/RoBERTa, then push real hidden vectors toward or away from it mid-forward-pass (AlterRep) — a genuine causal test of a subspace (not a single direction), showing opposite-signed, grammar-consistent shifts in perfective vs. imperfective aspect predictions, verified against a random-subspace placebo control. See boundedness-subspace-causally-shifts-aspect-predictions.

  • A trained cross-modal steering subspace collapses to rank 1, not “dozens to hundreds.” Bi, Wang, Chen, Xiao, Hecker, Tresp & Ma (2024/2025, MoReS) train a per-layer linear map confined to a dd-dimensional subspace of a multimodal LLM’s visual-token representations, to correct a measured text-over-vision attention imbalance during visual instruction tuning; ablating the subspace rank d∈{1,2,4,8}d\in\{1,2,4,8\} shows d=1d=1 already matches or exceeds every larger rank on four visual QA benchmarks while needing the fewest trainable parameters (0.164M, up to 1,150x fewer than LoRA) — a rare case where the causally sufficient subspace for a whole modality-rebalancing intervention is a single line, contrasting with the “dozens of directions” scale seen for concept-erasure subspaces like gender bias above. See bi-etal-2024-llava-steering-a-rank-1-linear-subspace-steering-module-per-layer-suffices-to-rebalance-visual-versus-textual-modality-attention-and-recovers-comparable-mllm-performance-with-500x-fewer-trainable-parameters.

  • Generalizes Linear Direction at k=1k=1. A single direction rfr_f is the special case V=span{rf}V=\text{span}\{r_f\}, k=1k=1; PVx=(rf⋅x)rfP_Vx = (r_f\cdot x)r_f recovers the earlier projection formula.

  • The removal method itself can invalidate a causal-subspace claim — not just add noise, but reverse the conclusion. Dobrzeniecka, Fokkens & Sommerauer (2025) re-run Elazar et al. (2021)‘s amnesic-probing pipeline and find INLP fails its own information-control check for 2 of 3 properties: a same-size random-direction projection damages BERT’s next-word prediction more than INLP’s targeted removal, so the apparent behavioral effect cannot be attributed to the specific subspace removed. Substituting Mean Projection (a single class-mean-difference projection, no iteration) or LEACE for INLP passes this control in every case, at far lower embedding distortion (cosine similarity to the original space: 0.80-0.95 for MP/LEACE vs. as low as 0.31-0.37 for INLP) — evidence that “how many dimensions removal needs” and “how much collateral distortion removal causes” are entangled, and an iterative orthogonal method’s own side effects can masquerade as the causal signal being tested for. See mean-projection-and-leace-outperform-inlp-in-amnesic-probing.

  • Guardedness against one adversary does not imply guardedness against another — proven, not just observed. Ravfogel, Goldberg & Cotterell (2023) formalize “guardedness” (the property INLP, RLACE and LEACE all aim to guarantee) and prove that while binary log-linear guardedness genuinely propagates to any downstream binary log-linear classifier, it demonstrably does NOT propagate to multiclass classifiers: a representation can be perfectly guarded against every single guarding hyperplane while a KK-way softmax combining those hyperplanes’ sign patterns recovers the erased concept almost completely (proven via an explicit “K-Voronoi” worst-case construction, and demonstrated empirically — a 4-to-8-way profession classifier on RLACE-erased, gender-guarded BERT representations recovers nearly all the “erased” gender information). This is a different, more fundamental limitation than “how many dimensions” or “how much information is captured”: it shows the very notion of “erased” is adversary-relative, not an intrinsic property of the representation. See multiclass-softmax-recovers-erased-concept-despite-guardedness.

  • An intrinsic, classifier-free information-theoretic standard for “the ideal concept subspace,” and it’s stricter than LEACE’s own guarantee. Guerner, Svete, Liu, Warstadt & Cotterell (2023) define a concept subspace’s quality using only the language model’s own distribution — via a counterfactual-unigram construction that provably handles Kumar et al. (2022)‘s spurious-correlation critique of naive mutual-information erasure tests — and formalize four properties (erasure, encapsulation, containment, stability) that a truly causal subspace would satisfy exactly. Applying this standard to LEACE‘s own output (state-of-the-art by its own erasure guarantee) reveals its one-dimensional subspace captures only ~50% of total concept information for English verbal number and considerably less for French grammatical gender — and a genuine do-intervention on the subspace succeeds ~90% of the time for number but fails for gender, exactly tracking this incompleteness. This is a different, stricter failure mode than “how many dimensions are needed” (the running INLP→RLACE→LEACE narrative above): even a provably-complete linear eraser can still fail to encapsulate all of a concept’s information within the subspace it finds. See leace-subspace-captures-only-half-of-verbal-number-information and causal-intervention-succeeds-for-number-fails-for-gender.

  • USES this structure as an alignment-testing tool, rather than discovering a new fact about it. Geiger, Wu, Potts, Icard & Goodman (2023/2024, DAS) learn an orthogonal rotation of a representation via gradient descent so that one rotated-basis subspace aligns with a hypothesized high-level causal variable, testing the alignment via interchange interventions — reaching near-perfect fit (IIA up to 1.00) where brute-force neuron-subset search reaches only 0.60-0.73. Unlike every other entry on this page, DAS is not a claim that some specific concept (gender, aspect, RC-boundary, etc.) lives in a particular subspace — the paper’s own appendix states the orthogonal-rotation choice merely assumes linear structure as a tractable methodological device for causal-abstraction testing, explicitly flagging non-linear alternatives as unexplored future work. See das-orthogonal-rotation-recovers-causal-alignment. Wu, Geiger, Icard, Potts & Goodman (2023, “Boundless DAS”) scale this to a full 4096×40964096\times4096 rotation of a 7B-parameter instruction-tuned LLM (Alpaca), discovering a 2-boolean-variable causal model whose alignment is robust to unseen input brackets, unseen instruction phrasing, and irrelevant added context — though this follow-up does not restate the original paper’s own linearity-is-an-assumption caveat. See boundless-das-scales-to-7b-with-robust-generalization.

  • A representation-space intervention can be inverted back into real text, making its effect interpretable rather than only measurable. Avitan, Cotterell, Goldberg & Ravfogel (2024) apply a vec2text-based inversion procedure to representations that have been passed through LEACE erasure or MiMiC/MiMiC+ steering, recovering fluent, gender-flipped text that human annotators reliably identify as intended (Fleiss’ κ=0.60) — and revealing, via word-association analysis, that these interventions shift subtler class-correlated vocabulary beyond pronouns (e.g. “developer” vs. “domestic,” “maternal”). This is orthogonal to the “how much collateral distortion” question tracked elsewhere on this page: it doesn’t measure distortion, it makes the intervention’s content directly inspectable. See representation-interventions-invert-to-interpretable-text.

  • Amnesic probing’s single random-direction control can itself be the unreliable part, not just the removal method. Rozanova, Valentino, Cordeiro & Freitas (2023) apply amnesic probing to a controlled NLI setting where the entailment label is provably determined by two known features, and find essentially no downstream effect from removing either feature — or even the gold label itself — via INLP. The cause is a low-class-count confound: with only 2-3 classes, INLP removes very few directions, and a single random-direction control baseline (as used in the original amnesic-probing protocol) is shown to be highly unstable at this scale, sometimes matching or beating the targeted removal purely by chance. Projecting onto the complement of the same INLP-trained subspace instead of removing it (“mnestic probing”) recovers a clear, informative signal in the same setting. See amnesic-probing-misleading-in-low-class-count-high-dimension-settings and mnestic-probing-recovers-informative-signal.

  • Restricting causal validation to a linear subspace has a measurable reliability cost. Canby, Davies, Rastogi & Hockenmaier (2025) formalize completeness (how thoroughly a target property changed) and selectivity (how little else changed) as numbers in [0,1][0,1], define reliability as their harmonic mean, and compare INLP/RLACE/AlterRep against nonlinear gradient-based interventions (FGSM/PGD/AutoAttack) across six models. Every method trades completeness for selectivity as its hyperparameter varies, but the linear methods this map’s causal-validation cluster is built around are, in five of six models tested, less reliable than an unconstrained nonlinear perturbation — evidence that the interpretability of “the concept lives in this subspace” is not free: it costs measurable reliability relative to methods with no such structural constraint. See nonlinear-interventions-more-reliable-than-linear.

  • Cross-lingual subspace overlap, verified causally. Chang, Tu & Bergen (2022) fit a per-language affine subspace to XLM-R representations across 88 languages and find, via a Riemannian covariance-distance metric, that different languages’ subspaces coincide almost exactly after mean-centering (layers 6-11 distances equivalent to <5° rotation or <1.6x scaling) — and confirm this is not merely descriptive by showing that projecting a representation onto its own vs. a different language’s subspace has correspondingly small vs. large causal effects on language-modeling perplexity. See cross-lingual-subspace-overlap.

  • Two independently-varying attributes of the same signal partition into (near-)orthogonal subspaces of a single self-supervised model’s representation space, and collapsing one improves rather than merely preserves the other. Liu, Tang & Goldwater (2023) find, in two predictive-coding speech models (CPC, APC), that a speaker-identity subspace (top PCA directions of per-speaker mean representations) and a phonetic-content subspace (same, per-phone) are nearly orthogonal (average cosine similarity ~0.13 among top directions) — and causally validate this via Orthogonal subspace collapse (aggregated-PCA): projecting out the speaker subspace drives speaker-probe accuracy to near chance while improving phone discrimination (ABX error), including for speakers never seen during subspace estimation. See predictive-coding-speech-models-encode-speaker-and-phonetic-information-in-orthogonal-subspaces-collapsible-without-hurting-phone-discrimination.

  • A subspace-composability claim proven as a theorem, not just measured, with an explicit falsifiable precondition and a demonstrated failure case. Wang, Gui, Negrea & Veitch (2023) treat a diffusion model’s score function itself as the representation and prove that if the generation is “causally separable” with respect to two concepts (a specific, checkable independence condition), the centered score representation is “arithmetically composable” — each concept provably occupies its own additive subspace. They estimate each subspace from contrastive prompt-pair score differences and causally edit Stable Diffusion generations by projecting and swapping subspace components, validated by a 10-rater human study (concept algebra preferred 47.6% of the time vs. 16.2-16.4% for baselines) — and explicitly demonstrate a failure case (a “nurse” gender-transfer prompt) arising exactly when the separability precondition doesn’t hold, directly tying the theorem’s hypothesis to an observed limitation rather than presenting unconditional success. See concept-algebra-causally-edits-diffusion-generations-via-score-subspace-projection.

  • Recovered purely from text, with no visual grounding at all, and verified two independent ways. Abdou, Kulmizev, Hershcovich, Frank, Pavlick & Søgaard (2021) show static color-term embeddings from BERT, RoBERTa and ELECTRA linearly recover the 3D CIELAB perceptual color space — confirmed both via RSA (Kendall τ\tau up to 0.24, significant against a random-shuffle null) and via a control-task-corrected linear mapping directly onto CIELAB coordinates (selectivity 0.76-0.78 across model families) — with alignment scaling with model capacity (BERT-mini τ=0.077\tau=0.077 to BERT-base τ=0.162\tau=0.162) even holding architecture fixed. No causal intervention is involved; the finding rests on the geometric-shape criterion alone. See lms-color-term-embeddings-linearly-map-onto-cielab-perceptual-color-space-with-significant-rsa-and-control-task-corrected-selectivity.

  • 55 independently-trained, semantically distinct concept probes span a subspace whose effective rank collapses to 1, not 55, and the single surviving direction alone carries most of the causally-exploitable behavioral effect. Shah, Angeline, Kumar, Chheda, Zhu, Sharma, O’Brien & Cai (2025) fit one logistic-regression probe per harmfulness subconcept (racial hate, weapons, employment scams, etc. — 55 in total from CategoricalHarmfulQA) in Llama-3.1-8B-Instruct and Qwen2-7B-Instruct, and compute an SVD-based effective rank of the stacked 55 weight vectors: at a 0.95 variance threshold, nearly every layer reaches effective rank K=1K=1 — the 55 nominally distinct subconcepts occupy, almost exactly, a single shared line, not an independent direction each (a K-means clustering of the 55 vectors barely recovers the dataset’s own category groupings, Adjusted Rand Index ~3e-4 on average, ruling out “the shared direction is just re-deriving the taxonomy”). Causally, ablating only this one dominant direction matches or exceeds ablating the full 55-dimensional subspace on JailbreakBench safety (0.91 either way) while costing noticeably less utility (MMLU 0.60 vs. 0.51), and additively steering along it alone cuts AutoDAN attack success from 0.94 to 0.50. See 55-harmfulness-subconcept-probe-directions-collapse-to-effective-rank-one-and-ablating-or-steering-the-dominant-direction-suppresses-jailbreaks.

  • 32 hand-constructed semantic-contrast directions are far from orthogonal, their non-orthogonality reproduces an independent human survey’s own correlational structure, and PCA finds most of their joint variance concentrated in a 3-dimensional subspace matching a classical psychological finding. Kozlowski & Boutyline (2026) build 32 “semantic axis” directions (beautiful-ugly, soft-hard, etc.) from Llama-3 and Qwen3 residual-stream contrastive-pair differences, and find their pairwise cosine similarities closely reproduce the pairwise correlations between the same 32 scales in a 1,750-respondent human semantic-differential survey — direct evidence the axes are “meaningfully non-orthogonal,” per Concept Crystals (Parallelogram/Trapezoid Structure)‘s broader theme that related concept directions needn’t be independent. PCA on the 32 raw axis vectors finds the top three components jointly explain over 45% (small model) / 33% (large model) of variance, versus 3.1% expected under orthogonality — replicating the classical semantic-differential finding that human ratings reduce to 3 components (Evaluation, Potency, Activity) — and canonical correlation analysis confirms this LLM-derived subspace aligns closely with the equivalent human-survey subspace. Causally, Activation Steering (Addition) a word along one axis produces “spillover” onto other axes proportional to their cosine similarity, directly tying the measured non-orthogonal geometry to a measured behavioral effect. See 32-llm-semantic-feature-directions-form-a-low-dimensional-non-orthogonal-subspace-matching-human-semantic-differential-structure-and-predicting-steering-spillover.

  • Four explicitly-trained-orthogonal subspaces, with the orthogonality itself measured and causally load-bearing via a direct swap intervention, not just erasure. Cheng, Jin & Zhang (2026) train a sign-language-recognition model (PhonSSM) with a Phonological Decomposition Module that projects representations into four separate 32-dim subspaces (handshape, location, movement, orientation) under an explicit orthogonality loss, measuring the resulting separation directly (mean pairwise cosine similarity 0.12 with the loss vs. 0.67 without it) rather than merely asserting factorization. Causal validation goes beyond correlational probing: for minimal pairs of signs differing in exactly one phonological component, swapping only the differing component’s embedding between the pair (an interchange/Activation patching-style intervention) flips the model’s prediction to the pair partner 73.2% of the time, versus 12.4% for matched control swaps (p<0.001). See phonssm-factorizes-sign-representations-into-four-orthogonal-phonological-component-subspaces-and-swapping-one-causally-flips-predictions-73-percent-of-the-time.

  • Two nominally-similar unit sets (both format-invariant, both significantly non-random) dissociate causally — one matters for behavior, the other doesn’t. Martorell (2025) finds LLaMA-3.1-8B units whose activations significantly correlate (Bonferroni p<0.05) with either grid agent-position or action-correctness, consistently across 6 different text-based spatial encodings (448/258/373/286 units respectively, each far exceeding shuffled-index controls). Ablating the 286 action-correctness units drops grid-navigation success from 59% to 55%, while ablating the analogous position-correlated units leaves success unchanged at 62% — direct evidence that format-invariant correlational unit sets are not automatically causally equivalent to each other, even when both clear the same statistical bar for “robustly present regardless of representation.” See llama-3.1-8b-linearly-encodes-grid-position-format-invariantly-and-ablating-its-action-correctness-units-selectively-degrades-navigation.

  • The founding challenge to the linear subspace hypothesis: removing the single linear gender direction does not remove the bias, only its visibility along that one axis. Gonen & Goldberg (2019) apply Bolukbasi et al. (2016)‘s hard-debiasing projection to real word2vec-GoogleNews and GloVe-Wikipedia embeddings and show the “debiased” residual geometry is still recoverably gendered by instruments that don’t rely on the removed direction: k-means clustering of gender-biased words on the debiased vectors still recovers the original male/female clustering at 92-98.4% accuracy, an RBF-kernel SVM trained on the original biased vectors transfers to the debiased vectors at comparable accuracy, and profession-word nearest-neighbor lists remain measurably gender-skewed. This is the paper Vargas & Cotterell (2020, below) and Kresin, Dieffi & Caspi (2026, below) explicitly build on and answer: if bias is genuinely confined to one linear direction, hard-debiasing should have removed it entirely; that it doesn’t is direct motivation for testing whether the true bias-carrying structure is linear at all (Vargas & Cotterell’s kernel-PCA test) or spans more of the PCA spectrum than a single component (Kresin et al.’s multi-component ablation). See gonen-goldberg-2019-hard-debiasing-removes-the-linear-gender-direction-from-real-word2vec-and-glove-embeddings-but-k-means-clustering-and-an-rbf-svm-still-recover-gender-association-from-the-residual-geometry.

  • The linear subspace hypothesis, directly tested against a strictly more expressive nonlinear alternative built from the same construction, and confirmed. Vargas & Cotterell (2020) prove Bolukbasi et al. (2016)‘s gender-bias subspace construction is exactly PCA on a mean-centered design matrix, then kernelize it into a nonlinear (kernel-PCA) technique via Kernel-PCA linearity test and compare linear vs. five kernels across WEAT effect sizes, profession-stereotype correlation, indirect-bias SVM classification, and SimLex-999 preservation on word2vec and GloVe — finding no consistent benefit from nonlinearity on any benchmark, direct empirical support for the previously-untested assumption that gender bias occupies a genuinely linear (not merely linearly-approximable) subspace. See vargas-cotterell-2020-kernel-pca-tests-linear-subspace-hypothesis-for-gender-bias.

  • Direct bias is genuinely rank-1, but a differently-measured “associative” bias is not — the same subspace concept splits into a low-rank and a distributed part depending on what’s being measured. Kresin, Dieffi & Caspi (2026) analyze the explained-variance spectrum of Bolukbasi et al. (2016)‘s gender difference-vector PCA subspace across GloVe, word2vec and FastText, finding the first principal component alone captures ~48% of variance (80% by component 5, 93% by component 10) — and, via both cumulative multi-PC removal and single-PC ablation, that projection-based direct bias collapses to near-zero after removing only PC1 and stays flat thereafter, while WEAT-measured associative bias shows slow, non-monotonic behavior under the same removals, i.e. is not captured by the dominant direction(s) at all. This dissociation nuances rather than contradicts the “dozens of directions” INLP finding above: that finding concerns a discriminatively-trained classifier-adversarial subspace, whereas here the PCA/variance-based subspace is genuinely rank-1 for the projection-based direct-bias measure specifically, with WEAT-style associative bias instead behaving like the distributed, many-direction case. Removing more components also monotonically increases mean vector displacement and degrades neighbor stability, quantifying a direct bias-vs-semantic-preservation trade-off with no universal optimal cutoff. See kresin-etal-2026-direct-gender-bias-is-rank-one-but-associative-weat-bias-is-distributed-across-the-pca-spectrum.

  • A subspace built from persona-conditioned activations, not raw dataset activations, is what carries the transferable behavioral signal — isolated via random-subspace and generic-PCA controls of matched rank. Mahadik & Skapars (2026) build one activation vector per deceptive/honest and sycophantic/non-sycophantic persona (following the Assistant Axis elicitation method), and run PCA on these persona vectors rather than on raw dataset activations. The resulting PC1 cleanly separates harmful from harmless personas and, together with a diff-in-means contrastive direction, transfers zero-shot to 5 unseen deception and 5 unseen sycophancy datasets; projecting dataset activations onto the top-3 persona PCs and training a probe on those 3 features improves cross-dataset transfer over probes on raw activations for most pairs — and specifically outperforms both a random one-dimensional subspace and a same-rank dataset-specific PCA subspace, isolating that persona-conditioning (not just any low-rank projection) is what carries the transferable signal. No causal intervention is performed; the claim rests on the quantified geometric-shape and controlled-comparison criteria alone. See mahadik-skapars-2026-persona-conditioned-pca-subspace-separates-deception-and-sycophancy-and-outperforms-random-and-dataset-pca-subspaces-for-cross-dataset-probe-transfer.

  • A low-rank subspace edited out of the weights once, offline, rather than out of activations at every forward pass — and provably robust to a corruption that breaks the gradient-based alternative. Uppaal, Dey, He, Zhong & Hu (2024/2025, ProFS) extract a rank-2-to-10 toxic subspace via SVD of centered contrastive-pair embedding differences (rank chosen automatically via ScreeNot) and project it out of MLP value-matrices permanently, reducing GPT-2/Mistral/OPT/GPT-J toxicity below DPO’s result while using 4x less preference data and matching perplexity/downstream accuracy. Under increasing label noise, DPO’s toxicity-reduction degrades monotonically while ProFS stays essentially flat — a direct, provable consequence of SVD being invariant to any single contrastive pair’s sign flip, unlike DPO’s gradient-based loss. The paper connects this weight-space projection to activation-space gradient descent: a single DPO gradient step’s own variance is increasingly explained by projecting onto the same subspace as sample size and layer depth grow, framing ProFS as a denoised version of one DPO step rather than a fully independent phenomenon. See profs-toxic-subspace-projection-reduces-toxicity-more-sample-efficiently-and-more-noise-robustly-than-gradient-based-dpo.

  • A third, independent geometric-shape claim in the emotion-geometry cluster, at a much higher dimensionality than either of the other two. Reichman, Avsian & Heck (2025) extract a 40-dimensional centered-SVD subspace from pooled Llama-3.1-8B/OLMo-2/Ministral-8B sentence activations, with four leading principal components interpreted as valence, dominance, approach-avoidance and arousal — stable across layers (Spearman rank correlation up to 0.87-0.92 for emotion ordering) and generalizing across 8 emotion datasets in 6 languages via linear cross-domain alignment (cosine similarity 0.83-0.94 between per-dataset subspace fits). A trained nonlinear (MLP) steering module built on top of this subspace raises target-emotion accuracy from 9% to 83% overall. This sits alongside, rather than replacing, two lower-dimensional competing claims about closely related affective constructs: Sun et al. (2026)‘s 2D circular circumplex (valence-arousal-circumplex-subspace) and Choi & Weber (2026)‘s 1D parabolic curve (affective-parabolic-manifold), both in different models and via different extraction methods. See reichman-etal-2025-a-40-dimensional-svd-emotional-subspace-with-four-interpretable-pc-axes-generalizes-across-6-languages-and-is-causally-steerable-via-a-trained-mlp.

  • A proven dimensionality bound for superposition-based relational recall, not merely an empirical scaling observation. Ravfogel, Yehudai, Bruna & Bietti (2026) model subject embeddings as a linear superposition of per-relation attribute vectors, read out by a relation-conditioned ReLU-gated MLP selector, and prove (Theorem 4.1) that a 1-layer transformer with this selector solves single-hop recall for NN entities and RR relations whenever d=4Rlog⁡(N)+1d=4R\log(N)+1 — synthetic experiments (N=4096, R up to 16, d from 32 to 768) confirm trainable embeddings reach perfect memorization once d≥128d\geq128 while frozen/random embeddings need d≥512d\geq512 for the same RR. On five real pretrained LMs (Qwen2.5-0.5B, Qwen3-14B, Llama-3.1-8B, Llama-3.2-1B, Phi-4), a rank-512 affine probe finds the LM-head output-embedding row is linearly decodable (best-layer MRR 0.44-0.69 across six entity categories) — extending Hernandez et al. (2023)‘s LRE finding into a proven dimensionality regime, and partially reversing that paper’s reported non-linearity specifically for output-side object embeddings. Causally, minimum-norm subject-embedding perturbations swap the queried attribute with high selectivity, and a frozen relation-selector MLP transfers zero-shot to unseen relation bijections. See ravfogel-etal-2026-subject-embeddings-linearly-superpose-attribute-vectors-with-a-proven-d-theta-r-log-n-dimensionality-bound-for-relational-recall.

  • A category-dependent dissociation between true (through-origin) linear structure and affine structure, isolated by decomposing an existing affine map into its multiplicative and additive parts. Xia & Kalita (2025) split Hernandez et al.’s affine LRE (o=βWrs+bro=\beta W_rs+b_r) into a bias-free “linear LRE” (o=Wrso=W_rs) and a Jacobian-free “Bias” approximator (o=s+bro=s+b_r), testing both across 40 BATS-derived relation categories on GPT-J-6B and Llama-2-7B. Morphological relations reach 90% top-1 faithfulness under the bias-free linear map alone (vs. 95% for the full affine LRE), while semantic/encyclopedic relations drop to 40% linear vs. 61% affine — and the additive-only Bias approximator fails on morphology in both cases, showing the Jacobian term is necessary specifically there. This sharpens “relational decoding is affine” into a relation-type-specific claim: morphology occupies a genuine through-origin linear subspace map, while other relation types require the affine offset — replicated across GPT-J’s parallel and Llama-2’s sequential attention/MLP architectures and across 8 typologically diverse languages (Czech, French, German, Hungarian, Portuguese, Serbian, Swedish, Turkish). See xia-kalita-2025-morphological-relations-are-truly-linear-90-percent-faithful-with-no-bias-term-while-semantic-and-encyclopedic-relations-require-the-affine-bias-term.

  • A contrastive-SVD subspace, extracted per (layer, timestep) rather than once globally, whose local dynamics are validated as approximately linear before being used for closed-loop causal control. Hong, Chan, Dai, Skifstad & Chou (2026) extract a Dlat=64D_{lat}=64-dimensional subspace from SVD of contrastive prompt-pair activation differences in video diffusion transformers (Wan2.1-T2V-14B, HunyuanVideo-1.5), finding this small subspace already captures most of a target concept’s contrastive-difference energy, and separately validate that local Jacobian dynamics estimated within it generalize across prompts (smaller Frobenius-norm distance between same-layer Jacobians from 20 different prompts than to random matrices). Unlike Wang et al.’s image-diffusion concept algebra (a one-shot subspace projection/swap), the causal intervention here is a closed-loop optimal-control law (LQR) that tracks the projected contrastive coordinate toward a target setpoint at every generation timestep, reducing unsafe video generations while limiting the oversteering that coarser, non-anticipative edits cause. See hong-etal-2026-la-lqr-a-svd-derived-contrastive-subspace-with-validated-local-linear-dynamics-enables-closed-loop-optimal-control-steering-of-video-diffusion-transformers.

  • A z-scored RSA variant shows intermediate-layer geometry is specifically organized around a model’s own future readout subspace, and this organization is masked at late layers in small models — dissociated from loss — but not large ones. Xu (2026) compares a layer’s full-space distance structure, projected onto the unembedding matrix’s own top-k readout subspace, against 100 random subspaces of equal rank, finding “peak z-scores reach 9-24 at mid layers” across seven Pythia models (70M-6.9B) and three cross-family models (OLMo-1B, Phi-1.5, Gemma-2-2B) — genuine, statistically significant subspace-specific organization, not incidental low-rank structure. Tracked across training, Pythia-410M’s late-layer z-score falls to -32 “even as loss continues to drop,” with the masking emerging progressively over training checkpoints, while Pythia-1B (d=2048) never masks and the transition threshold sits at approximately hidden dimension 2048 across the full model set. A partial follow-up intervention (removing the top few principal components) “restores positive z at every layer of every model with d>=768,” though the paper is explicit that its central claim is correlational, not a demonstrated causal mechanism. See xu-2026-scale-determines-whether-late-layer-representation-geometry-stays-organized-around-the-unembedding-readout-subspace-during-training.

  • A continuous (not discrete-class) conditioning variable can also be ablated by closed-form affine regression and subtraction, an alternative to LEACE/mean-projection’s discrete-class subspace removal. Ruggiero, Testa, Van de Walle & Di Caro (2025, Eta-WavLM) fit a closed-form affine regression from a frozen, pretrained ECAPA-TDNN speaker embedding (PCA-reduced to 128 dims) to layer-15 frame representations of the official pretrained WavLM-Large, then subtract the predicted component at inference. Speaker-classification accuracy on the resulting representation collapses from 82.30% to 55.73% (paired t-test T=18.41, p=5.12e-5), while voice-conversion quality improves over multiple baselines (e.g. WER 3.81 vs. 4.56, T-SSIM 92.46 vs. 89.52, MOS 4.00 vs. 3.84). See ruggiero-etal-2025-a-closed-form-affine-regression-ablation-of-a-continuous-speaker-embedding-collapses-speaker-classification-accuracy-on-wavlm-large-layer-15-frames-while-improving-voice-conversion-quality.

  • Individual attention heads specialize in writing additively into a low-rank residual-stream subspace per factual attribute, quantified by a per-head Attribute Ratio, and these per-head direction contributions constructively sum to produce the correct factual completion. Chughtai, Cooney & Nanda (2024) use Direct Logit Attribution on Pythia-2.8B and GPT2-XL, finding individual heads specialize per attribute (e.g. L17H17 IN_COUNTRY Attribute Ratio=107.23) and that summing multiple such heads’ contributions constructively produces the model’s correct factual completion — an “additive motif” for factual recall. See chughtai-cooney-nanda-2024-individual-attention-heads-write-additively-into-a-low-rank-residual-stream-subspace-per-attribute-and-their-direct-logit-attribution-ratios-constructively-sum-to-produce-correct-factual-completions.

  • An OLS-fit linear/affine map between two independently-trained vision models’ full activation spaces explains over 60% of variance, and the two spaces’ top principal components correspond approximately one-to-one. Moayeri, Rezaei, Sanjabi & Feizi (2023) align a wide range of real pretrained vision encoders (supervised and self-supervised ResNets/ViTs, CLIP) into a shared concept space this way, with no causal intervention performed — a passive cross-model geometric-correspondence measurement enabling zero-shot concept-bottleneck classification up to 93.8% accuracy. See moayeri-etal-2023-a-linear-affine-map-fit-between-two-vision-models-activation-spaces-achieves-r2-above-0-6-and-their-top-pca-components-correspond-approximately-one-to-one.

  • Forget and retain subspaces built via per-layer SVD directly on a real diffusion model’s cross-attention activations (rather than a text-embedding proxy) are separated roughly 5x better by a probe than the text-embedding baseline, and projecting the forget subspace out of the cross-attention weights via a single closed-form linear projector gives the best target/retention/attack trade-off on a holistic unlearning benchmark. Moon, Jun, Lee & Kim (2026) apply this to Stable Diffusion v1.5 across 10 concepts (Style/IP/Celebrity/NSFW), extending the same weight-space-projection logic as ProFS (originally for LLM MLP weights) to diffusion U-Net cross-attention K/V weights. See moon-etal-2026-forget-and-retain-subspaces-built-via-per-layer-svd-on-real-cross-attention-activations-in-stable-diffusion-v1-5-projected-out-via-a-single-linear-projector-outperform-a-text-embedding-only-baseline.

  • A linear map fit between a real network’s own representation space before and after continual-learning steps, rather than between two different models, predicts representational forgetting. Kim, Kim & Sohn (2025) define a representation discrepancy as the minimum, worst-case misalignment between a layer’s real representation space at two points in continual training, minimized over a purely linear map with no bias term, and fit this on real fixed-dimension ResNets, a fully-connected ReLU network, and a ViT trained sequentially on Split-CIFAR100 and ImageNet32. The resulting discrepancy correlates strongly with measured forgetting from frozen-feature linear probing (R-squared 0.88 on Split-CIFAR100, 0.74 on ImageNet32), and forgetting saturates at a 10-task horizon on both datasets, with saturation coming sooner in deeper layers and later in wider networks — the same “fit a linear/affine map between two representation spaces” construction used elsewhere on this page for cross-model alignment, applied instead to a single network’s own representation drift over training time. See kim-etal-2025-a-fitted-linear-map-between-real-resnet-representation-spaces-before-and-after-continual-learning-correlates-with-measured-forgetting-at-r2-0-74-to-0-88.

  • A fitted linear translation map distinguishes concepts that are genuinely lost from concepts that are merely hidden behind a recoverable reparameterization of the same continually-trained network’s representation space. Filus, Faber, Corizzo & Kanan (2026) train a BatchTopK sparse autoencoder per task on frozen real ResNet-18 activations (via the Mammoth continual-learning framework) on 2seq-CIFAR10, 2seq-tiny-ImageNet, and 10seq-tiny-ImageNet, then fit a least-squares linear map (with bias) translating post-continual-learning features back toward an earlier task’s representation space — re-running the frozen earlier-task SAE on translated features recovers many latents that appeared to have “died,” while a nonlinear MLP translator gives only marginal additional recovery, evidence that most of the apparent forgetting is a linear reparameterization rather than genuine representational destruction. Deletion ratios are highest under naive SGD/EWC and lowest under DER++/LwF. See filus-etal-2026-a-fitted-linear-translation-map-recovers-apparently-deleted-sae-concept-latents-across-real-resnet18-continual-learning-checkpoints-distinguishing-lost-from-merely-hidden-concepts.

  • Linear CKA similarity across independently-trained embedding models (proprietary and open) quantifies when models are linearly alignable at all, and learned affine maps between real generative LMs preserve downstream classification accuracy while enabling zero-shot cross-model text generation whose quality correlates with tokenizer/vocabulary overlap. Gorbett & Jana (2026) measure CKA similarity of 0.595-0.881 across five embedding models and validate affine-map transfer across Llama-3-8B, Qwen2.5-7B/14B, Llama-3.2-1B and others, isolating tokenizer overlap (r=0.898) as a confound distinct from representational alignment itself. See gorbett-jana-2026-linear-cka-similarity-across-five-independently-trained-embedding-models-ranges-0-595-0-881-and-learned-affine-maps-preserve-classification-accuracy-and-enable-zero-shot-cross-model-generation.

  • A multi-dimensional concept’s genuine intrinsic subspace dimension, measured independently of any sparse autoencoder, is provably fragmented into many more near-collinear single-direction atoms than that dimension requires — and a decoder architecture aware of grouped subspaces recovers it whole. Dalili & Mahdavi (2026) prove standard single-direction SAEs are geometrically and dynamically forced to split multi-dimensional features into exponentially many atoms (a k-sparse covering-number argument plus a basis-instability result), then confirm empirically that GPT-2-Small’s residual stream contains a temporal-concept subspace of measured intrinsic dimension ~14 (of 768 ambient dimensions, via PCA, with no SAE involved) which a standard SAE fragments into 35 separate atoms, while their Subspace-Aware SAE (block/group decoders plus nuclear-norm rank regularization) captures the whole concept in one rank-6 group and reduces feature absorption from 37.2% to 6.6% (GPT-2-Small) and 24.0% to 18.3% (Mistral-7B-v0.1). See gpt2-residual-stream-contains-a-genuine-14-dimensional-temporal-concept-subspace-that-standard-saes-fragment-into-35-atoms.

  • Frozen-feature linear probes reveal a sharp split between which self-supervised vision models encode genuine 3D surface geometry versus only coarse scene-layout priors, with single-view and multiview 3D awareness dissociating even in the best models. El Banani, Raj, Maninis, Kar, Li, Rubinstein, Sun, Guibas, Johnson & Jampani (2024) probe DINOv2, CLIP, MAE, iBOT, Stable Diffusion, MiDaS, DeiT and SAM for depth/surface-normal decodability, finding DINOv2 and Stable Diffusion features support markedly more accurate single-view 3D decoding than CLIP or MAE (which capture only rough layout priors), while every model tested shows weak multiview-consistent 3D structure regardless of single-view decodability. See dinov2-and-stable-diffusion-linearly-encode-depth-and-surface-normals-while-clip-and-mae-do-not.

  • SVD-extracted low-rank subspaces of converged linear-probe weights are causally necessary and sufficient for a self-supervised vision transformer’s geometric-task performance, with the intrinsic subspace dimension itself differing sharply between training objectives. Zhou, Zou, Gu, Dong, Xie & Zhang (2026) causally project DINOv2, MAE, and iBOT features onto extracted top-k task subspaces versus random or orthogonal-residual subspaces of equal rank, finding random/residual subspaces collapse to noise while the extracted subspace nearly recovers full-feature performance — DINOv2 needing rank >=64 to saturate its linear potential versus MAE saturating by rank 32, a quantified difference in intrinsic task-subspace dimensionality between the two self-supervised objectives. See svd-extracted-task-subspaces-in-dinov2-and-mae-are-causally-necessary-and-sufficient-for-linear-probe-performance.

  • A cross-scene-averaged identity/position decomposition, extracted via SVD from two real VLMs, converges to Laplacian-eigenmap scene geometry and is causally steerable. Wang & Gao (2026) posit an additive decomposition h=uid+usp+noiseh=u_{id}+u_{sp}+\text{noise} per object-token in Qwen2.5-VL-7B-Instruct and InternVL3-8B, and average an object’s activations across many randomly-positioned synthetic 3D scenes to cancel the position term, isolating an identity basis via SVD whose orthogonal complement recovers a spatial subspace; PCA of the spatial-extracted residual then recovers a 3D geometry matching true scene layout, formally converging to Laplacian eigenmaps of the scene graph. Causally, injecting a probe-derived direction into layer-12 residuals of Qwen2.5-VL-7B shifts the x-coordinate probe readout monotonically with injection strength (α=+0.30\alpha=+0.30: Δx^=+0.091±0.021\Delta\hat x=+0.091\pm0.021 vs. control +0.001±0.030+0.001\pm0.030), while a null-direction control shows no effect. See wang-gao-2026-svd-based-cross-scene-averaging-isolates-a-linear-3d-position-subspace-in-real-vlms-and-steering-it-causally-shifts-spatial-judgments.

  • Linear decodability of mid-level geometric/grouping structure across a large sweep of self-supervised vision models correlates only weakly with the same models’ high-level recognition accuracy. Chen, Marks & Cheng (2024) probe eight mid-level vision tasks across 22 SSL models spanning contrastive, clustering, pretext-task, and masked/self-distillation paradigms, finding several models strongly imbalanced across the two capability classes — evidence that linear decodability of geometric mid-level structure is not simply a byproduct of overall representation quality as measured by recognition benchmarks. See mid-level-geometric-task-decodability-is-weakly-correlated-with-high-level-recognition-performance-across-22-ssl-models.

  • Sycophantic and factually-correct agreement occupy geometrically distinct activation subspaces, yet a single linear steering direction cannot exploit that distinction — explaining why sycophancy-reduction steering non-specifically suppresses correct agreement too. Buchan (2026) shows a centroid-difference sycophancy-steering vector in Llama-3-8B-Instruct also suppresses agreement with correct facts (a 20% drop on “the Earth is round”), and a Grassmann-similarity/principal-angle analysis finds sycophantic-agreement and factual-agreement activations occupy measurably distinct subspaces (similarity 0.15-0.20 vs. 0.317 for random splits, principal angles mostly 65-89 degrees) while the steering direction’s projection onto both subspaces is nearly equal (ratio 0.90-0.97) — the model geometrically distinguishes the two, but the linear intervention cannot. See sycophantic-and-factual-agreement-occupy-geometrically-distinct-subspaces-yet-a-single-steering-direction-cannot-separate-them.

  • Self-supervised speech models linearly recover universal, cross-speaker, cross-language articulatory kinematics from a single linear subspace. Cho et al. (2023) show that HuBERT, wav2vec 2.0, XLS-R, and MMS representations linearly encode articulatory kinematics (tongue/lip movement trajectories) via simple linear regression, and that this subspace transfers across speakers and across languages the models were never trained on, pointing to a universal articulatory-kinematic subspace shared across self-supervised speech models. See speech-ssl-models-linearly-recover-universal-cross-speaker-cross-language-articulatory-kinematics.

  • An encoding probe (predicting activations from candidate features, rather than the reverse) decomposes how much representational variance distinct linguistic/acoustic feature groups explain in wav2vec2 and BERT, revealing structured linear subspaces per feature group beyond what decoding probes can show. Shen et al. (2026) argue decoding probes only show recoverability, not how representational variance is actually organized, and instead fit encoding probes per feature group, quantifying the linear subspace each group (e.g. phonetic, syntactic) occupies within wav2vec2-base and BERT-base representations. See encoding-probes-decompose-how-much-representation-variance-distinct-feature-groups-explain-in-wav2vec2-and-bert.

  • A provably-correct linear projection (PISCO) disentangles style and content factors that are linearly entangled in pretrained ResNet-18 and SimCLR features, and causally discarding the isolated style subspace improves out-of-distribution accuracy. Ngweta et al. (2023) prove (via two theorems) a post-hoc linear projection separating style and content subspaces with no retraining, substantially raising style-correlation recovery (e.g. rotation: SimCLR 0.368 to 0.945) and improving OOD accuracy under spurious style shift when the style subspace is discarded. See a-provably-correct-linear-projection-disentangles-style-from-content-in-resnet-and-simclr-features-and-improves-ood-accuracy.

  • Safety-relevant fine-tuning gradients occupy a compact, low-rank subspace nearly orthogonal to utility-task gradients, and causally projecting utility gradients onto that subspace’s orthogonal complement during training cuts jailbreak attack success rate from over 95% to under 25%. Zhang et al. (2026) use per-layer SVD of safety- vs. utility-task gradients to identify this subspace (cosine similarity between safety/utility directions oscillates around zero, often negative) and their Safety-Preserving Fine-tuning method, an orthogonal-projection intervention, drops Harm-dataset attack success from 0.955 to 0.019 (Llama-3.1-8B-Instruct) while preserving MMLU accuracy. See safety-gradients-occupy-a-compact-low-rank-subspace-distinct-from-utility-gradients-and-projecting-fine-tuning-updates-onto-its-orthogonal-complement-cuts-attack-success-rate-from-over-95-percent-to-under-25-percent.

  • Structural probes recover distinct, low-dimensional linear subspaces for phonemic, lexical-semantic, and syntactic structure that emerge in that order across pretraining in both speech SSL models and text LLMs. Orhan et al. (2026) find phonemic structure emerges first, then lexical, then syntactic geometry in wav2vec2 (tiny/base/large) and in Llama-2-13B/Pythia checkpoints tracked across training, mirroring child-language-acquisition order (though requiring 2-4 orders of magnitude more data); a control alignment score confirms near-orthogonality (0.038-0.042) between the syntactic and semantic subspaces in Llama-2-13B. See structural-probes-reveal-staged-emergence-of-phonemic-then-lexical-then-syntactic-linear-subspaces-in-both-speech-ssl-models-and-text-llms-mirroring-child-language-acquisition-order.

  • A single mean-difference direction per tool-pair spans a compact PCA subspace: ~10 principal components capture ~91% of variance across 15 tools, far below a random-Gaussian control. Wu et al. (2026) find this compact tool-selection subspace across Gemma 3, Qwen 3, and Llama 3.1, tied to the same causal tool-switching direction described under linear-direction. See a-single-mean-difference-direction-per-tool-pair-linearly-reads-and-causally-switches-tool-selection-across-gemma-3-qwen-3-and-llama-3-1-with-a-three-stage-circuit-traced-via-saes-and-transcoders.

  • Correct future chess moves are linearly decodable in Leela Chess Zero’s intermediate layers, and steering against a late-layer learned prior recovers 61.7% of “forgotten puzzles.” Sandmann, Lapuschkin & Samek (2025) extend the logit lens to find correct solutions present in intermediate layers get overridden by a late-layer safety/aggression preference, and a causal steering intervention against that preference recovers most of these cases. See future-chess-moves-are-linearly-decodable-in-leela-chess-zeros-intermediate-layers-and-steering-against-its-late-layer-safety-preference-recovers-61-7-percent-of-forgotten-puzzles.

  • Multi-feature SAE steering vectors causally force instruction-following, with last-layer placement proving critical. He et al. (2025) build steering vectors from k=15 SAE decoder directions, achieving loose accuracy up to 0.7, while last-layer (vs. earlier-layer) placement is essential (loose accuracy collapses from 0.64 to 0.33 one layer earlier). See multi-feature-sae-steering-vectors-causally-force-instruction-following-across-gemma-2-and-llama-3-1-with-loose-accuracy-up-to-0-7-and-last-layer-placement-proving-critical.

  • A cross-model “Assistant Axis” is the leading principal component of a 275-persona activation space, and causally capping its projection reduces jailbreak success by ~60% with no measurable capability loss. Lu et al. (2026) find PC1 of the persona space correlates >0.92 pairwise across three models and persists in base (non-instruction-tuned) checkpoints, evidencing a pretraining origin. See a-cross-model-assistant-axis-aligning-with-pc1-of-a-275-persona-activation-space-causally-reduces-jailbreak-success-by-about-60-percent-with-no-measurable-capability-loss.

  • A micro-to-macro “Granularity Axis” for social roles aligns with PC1 of the role-representation space at cosine >0.95 and causally steers judge-rated output granularity. Qin et al. (2026) find the axis explains over 40% of variance in both Qwen3-8B and Llama-3.1-8B-Instruct, with monotonic level-ordered projections (Spearman/Pearson >0.93). See a-granularity-contrast-axis-aligns-with-pc1-of-a-role-representation-space-at-cosine-0-97-and-causally-steers-judge-rated-output-granularity-in-the-predicted-direction.

  • A low-dimensional “cell-based” subspace organized along entity-index and relation-index axes supports relational binding, and steering within it reliably flips bound-attribute predictions. Dai, Heinzerling & Inui (2026) extract this subspace via PLS regression probing from Llama3-8B-Instruct and Qwen3-8B (strongest at layers 10-20), reaching index-prediction R^2 of ~0.95 and attribute-prediction accuracy 0.94-0.95, beating a Hessian-based propositional-probe baseline. See a-low-dimensional-cell-based-subspace-organized-along-entity-index-and-relation-index-axes-supports-relational-binding-in-llms-and-steering-within-it-reliably-flips-bound-attribute-predictions.

  • LLMs consolidate entity-state tracking into a local binary subspace far more decodable than a global multi-way state subspace, and null-space projection of the state-tag direction causally changes completions. Tang et al. (2026) find local binary probes reach ~0.9 accuracy versus ~0.3 for global 8-way state probes, and a learned sparse mask over the residual stream’s PCA basis reveals a shared positional-information subspace across circuits at layers 15-25 in CodeLlama-13B. See llms-consolidate-entity-state-tracking-into-a-local-binary-subspace-far-more-decodably-than-a-global-multi-way-state-subspace-and-null-space-projection-of-the-tag-direction-changes-completions.

  • A PCA-derived “Ordering ID” principal component localizes the previously-abstract Binding-ID mechanism, and steering along it in fixed unit-steps monotonically shifts the predicted bound attribute. Dai, Heinzerling & Inui (2024) confirm this OI-PC direction via Logit Difference/Logit Flip metrics that increase monotonically with steering steps, replicating across Llama2-7B, Llama3-8B, Qwen1.5-7B, and Pythia-6.9B. See a-pca-derived-ordering-id-principal-component-localizes-the-binding-id-mechanism-and-steering-along-it-in-fixed-unit-steps-monotonically-shifts-the-predicted-bound-attribute.

  • A 2D signed cultural-value axis shows “latent entanglement”: steering geometry along one dimension is not independent of the other. Dang, Kieu & Masud (2026) quantify this cross-axis entanglement (ratio 0.72-0.81 naive, dropping to 0.53 under hybrid steering) as a joint-subspace property of the Inglehart-Welzel cultural map extracted from Llama-3.2-3B, Qwen3-4B, and Gemma-3-4B. See cultural-value-steering-axes-in-llms-show-latent-entanglement-steering-one-axis-induces-correlated-shifts-on-a-second-axis-mirroring-the-real-world-correlation-between-those-values.

  • Number-task subspaces (comparison, parity, primality, successor/predecessor) are measurably distinct yet linearly inter-transformable, with near-orthogonal low-level feature axes. Hu, Niu & Varma (2026) find Procrustes disparity ~0.010 (vs. a 0.077-0.273 permutation baseline) alongside SVCCA canonical correlations of 0.80-0.90 across task pairs in BERT, GPT-2, Qwen2.5-7B, and Qwen2.5-Math-7B, with a parity axis nearly orthogonal to magnitude (88.8 degrees) and a partially-coupled primality axis (68.8 degrees). See number-task-representations-form-distinct-yet-linearly-inter-transformable-subspaces-with-near-orthogonal-parity-and-magnitude-axes.

  • Distributed Alignment Search locates linear subspaces for number-specific and unit-specific quantity-comparison heuristics, and interchange interventions on them exceed 0.9 accuracy in middle layers. Sasaki et al. (2026) causally validate NumLogDiff and UnitLogDiff subspaces this way, with a linear surrogate built from these variables predicting Qwen3-4B-Base’s comparison preference at R^2~0.817. See distributed-alignment-search-finds-linear-subspaces-for-number-and-unit-specific-heuristics-that-causally-predict-and-control-llm-quantity-comparison-decisions.

  • Independently trained multimodal contrastive models’ embedding spaces are related, up to mean shift, by a single shared orthogonal map applied identically to both modalities. Gupta, Kansal, Jegelka, Isola & Garg (2026) prove this theoretically (kernel agreement on a small cross-modal anchor set forces the single-Q relationship) and verify it empirically across CLIP, SigLIP, and FLAVA-family models with different architectures and training data. See independently-trained-multimodal-contrastive-models-share-embedding-spaces-related-by-a-single-orthogonal-map-applied-to-both-modalities.

  • LLMs fine-tuned to evade activation monitors for benign concepts, when conditioned on a trigger, generalize zero-shot to evading unseen safety monitors, via a targeted relocation of activations into a low-dimensional subspace. McGuinness, Serrano, Bailey & Emmons (2025) validate this across Llama, Gemma, and Qwen model families, showing the evasion succeeds even against monitors trained post hoc on the model’s frozen weights, with the underlying mechanism traced (in Gemma-2-9b-it) to a subspace relocation rather than a diffuse activation change. See neural-chameleons-fine-tuned-to-evade-benign-activation-monitors-generalize-zero-shot-to-evading-unseen-safety-monitors-via-a-low-dimensional-subspace-relocation.

  • A real pretrained weather foundation model’s latent bottleneck is PCA-organized by a stable seasonal-cycle axis, while extreme storm events fail to form a linearly separable cluster along any other component, and the causal relevance of specific latent regions is independently confirmed via LRP-guided perturbation. Kasteleyn & Lucic (2026) find Aurora’s PC1 captures 24.1% of variance with 0.998 bootstrap cosine-similarity stability, while storm events cluster only weakly and unstably on PC3 (mean cosine similarity 0.650±0.269); masking the top-1% Layer-wise-Relevance-Propagation-flagged pixels degrades forecasts 3.31x more than random masking. See auroras-latent-bottleneck-is-pca-organized-by-seasonal-cycle-not-extreme-storms-and-lrp-relevant-pixels-are-causally-necessary-for-forecast-accuracy.

  • After linearly projecting out compositional (atom-count) signal, the remaining geometric information’s accessibility to a linear probe varies 6.6x across ten real pretrained atomistic foundation models depending mainly on task alignment, and inside one architecture that information routes through separate linear subspaces by symmetry type. Steier (2026) introduces Compositional Probe Decomposition (CPD): PaiNN’s residual R^2 for HOMO-LUMO gap drops from 0.53 to 0.31 when retrained on energy instead; inside MACE, L=1 (vector) channels favor dipole moment (R^2 0.59 vs. 0.38) while L=0 (scalar) channels favor HOMO-LUMO gap (R^2 0.76 vs. 0.34), a symmetry-typed subspace split absent in ViSNet. New method compositional-probe-decomposition. See information-routing-in-atomistic-foundation-models-shows-task-alignment-and-symmetry-typed-channels-govern-linear-disentanglement-of-composition-from-geometry.

  • PCA and linear probing recover a near-orthogonal Cartesian x/y coordinate subspace inside a toy transformer trained from scratch on spatial navigation, and causal ablation shows the internal map becomes self-sufficient at a sharp depth transition. Baumgartner, Burgess, Spens & Manescu (2025) find GPT-2 small trained on grid-navigation shows top-2 principal components aligning with grid axes at Layer 7, a linear probe recovering (x,y) with R^2=0.93 at Layer 8 whose basis vectors are nearly orthogonal (cos θ≈−0.0415), and an ablation-revealed phase transition where loop-completion accuracy jumps from near-zero to 100% at Layer 8+ — a task-dependent finding, since goal-directed shortest-path variants show neither the orthogonal structure nor the transition. See pca-and-linear-probing-recover-an-orthogonal-xy-coordinate-cognitive-map-in-a-gpt-2-trained-on-grid-navigation-that-becomes-causally-self-sufficient-at-a-sharp-layer-8-phase-transition.

  • A TTS model’s language-model module has a low-dimensional, speaker-invariant, additively-composable emotion subspace, while its flow-matching module entangles speaker and emotion on a shared manifold — a within-model architectural dissociation in emotion geometry. Wang, Bailey & Dang (2026) find CosyVoice2’s SLM module shows a ~28D speaker-invariant emotion subspace (positive delta local-intrinsic-dimensionality, +0.84) while its CFM module entangles speaker/emotion on a shared ~13D manifold (delta-LID −1.48), with CAA-style steering confirming SLM’s cleaner geometry yields better proportional mixed-emotion control. See cosyvoice2s-language-model-module-has-a-low-dimensional-speaker-invariant-composable-emotion-subspace-while-its-flow-matching-module-entangles-speaker-and-emotion-on-a-shared-manifold.

  • TopK SAE codes trained on four real pretrained audio foundation models raise DCI completeness for acoustic factors while preserving downstream classification informativeness, localizing specific factors to specific layers. Mariotte, Lebourdais, Almudévar, Tahon, Ortega & Dugué (2025) show this on AST, HuBERT-base, WavLM-base-plus, and MERT-v1-95M: pitch localizes to early layers for HuBERT/WavLM, formants to final layers. New method dci-disentanglement. See sparse-autoencoder-codes-on-four-audio-foundation-models-raise-dci-completeness-for-acoustic-factors-while-preserving-classification-informativeness.

  • A k-NN-based cross-language overlap statistic reveals multilingual LLMs form a partially-shared interlingual subspace surrounded by fragmented, language-specific components, and the alignment is causally preserved or disrupted by which layers are frozen during fine-tuning. Wilie, Cahyawijaya, He & Fung (2025) introduce Interlingual Local Overlap (ILO), a k-NN neighborhood-overlap statistic computed across languages rather than across layers, applied to Aya Expanse 8B, Llama-3.1-8B, and Gemma-2-9B, finding overlap varies systematically by resource-level and linguistic family. New method interlingual-local-overlap. See high-dimensional-interlingual-representations-reveal-a-partially-shared-core-subspace-and-language-specific-fragments-across-multilingual-llms.

  • Task-specific linguistic subspaces, compared directly to each other rather than only via their own probe accuracies, first share information broadly and later disentangle into distinct subspaces at identifiable “critical learning phases” of pretraining. Müller-Eberstein, van der Goot, Plank & Titov (2023) fit an information-theoretic probing suite for 9 syntax/semantics/reasoning tasks across MultiBERTs’ 2M pretraining steps and 5 seeds, comparing the fitted probes’ subspaces via a subspace-similarity metric: syntax subspaces form within 0.5% of training, while semantic/reasoning subspaces specialize much later. New method subspace-similarity-probing. See subspace-chronicles-tracks-critical-learning-phases-of-linguistic-subspaces-across-multiberts-pretraining.

  • Next-token-prediction optimization implicitly performs an SVD factorization of a context/next-token co-occurrence matrix, and real pretrained models’ embeddings’ sign pattern (orthant) within this basis recovers increasingly fine-grained semantic categories as more components are included. Zhao & Thrampoulidis (2025) validate orthant-based clustering on synthetic co-occurrence data and real corpora (TinyStories, WikiText-2), confirming the recovered structure (grammatical categories, NER types, topical clusters) directly on GPT-2, BERT, and Qwen. New method ntp-svd-orthant-decomposition. See zhao-thrampoulidis-2025-svd-orthant-decomposition-of-next-token-prediction-recovers-increasingly-fine-grained-real-semantic-categories-in-gpt-2-bert-and-qwen.

  • Probe-derived, event-context-reweighted “field” directions for kinematic, contact, and object-permanence variables in purpose-trained toy world models causally increase event-specific prediction loss when ablated, with validated controls. Liu & Chen (2026) train GRU, Transformer-lite, and RSSM-lite transition models on synthetic Kubric-style object-state sequences, finding a 3-class linear probe on hidden states reaches macro-F1=1.00 for event-regime classification and a phase-aware field-readout probe reaches mean macro-F1=0.900 on RSSM hidden states; ablating the contact-aligned direction (Causal Field Effect) increases collision-window prediction loss in 6/6 architecture-seed cases beyond random-direction, random-subspace, and label-shuffle controls (the object-permanence-direction result shows a more qualified, mixed margin against the random-subspace control specifically). New method causal-field-effect-ablation. See liu-chen-2026-probe-derived-field-aligned-directions-in-toy-world-models-causally-increase-event-specific-prediction-loss-when-ablated.

  • Foundational: classical (pre-neural) sparse coding on trained word embeddings recovers ~2,000 “atoms of discourse,” and polysemous words decompose as sparse combinations of a handful of them. Arora, Li, Liang, Ma & Risteski (2018) apply k-SVD-style sparse dictionary learning directly to trained word2vec/GloVe embedding matrices, showing individual polysemous word vectors are well-approximated as a sparse linear combination of a few atoms out of an overcomplete ~2,000-atom dictionary — a foundational, pre-neural-SAE demonstration of subspace/superposition structure in real trained embeddings, methodologically ancestral to later sparse-autoencoder dictionary-learning work. New method sparse-coding-discourse-atoms. See arora-etal-2018-word-senses-linearly-superpose-as-sparse-combinations-of-about-2000-discourse-atoms-recovered-by-classical-sparse-coding-on-trained-word-vectors.

  • PCA-whitening followed by ICA recovers a privileged, sparse, axis-aligned semantic decomposition — unlike PCA’s arbitrary rotation — matchable across languages, models, and modalities. Yamagiwa, Oyama & Shimodaira (2023) apply this to real word2vec/GloVe/fastText embeddings, multilingual embeddings, BERT-family contextual embeddings, and image encoders, finding individually interpretable, sparse independent components whose cross-model correlation supports a “universal geometry” claim. New method ica-axis-decomposition. See yamagiwa-etal-2023-ica-applied-after-pca-whitening-to-real-word-and-contextual-embeddings-recovers-a-sparse-axis-aligned-semantic-decomposition-that-matches-across-languages-models-and-modalities.

  • A Top-k SAE trained on real E5-large-v2 sentence embeddings yields near-orthogonal decoder directions, and clamping individual latents causally re-ranks retrieval results. Shin & Kim (2026) measure Decoder Orthogonality (mean pairwise cosine similarity 0.0408) at 92.6% explained variance, then zero specific automatically-labeled latents (e.g. “Industrial Safety,” “Bridge Infrastructure”), causally shifting a query’s top retrieval result to a much lower rank. See shin-kim-2026-a-topk-sae-trained-on-e5-large-v2-sentence-embeddings-yields-near-orthogonal-decoder-directions-and-clamping-individual-latents-causally-re-ranks-retrieval-results.

  • Foundational: two independently, monolingually trained real word-embedding spaces are related by a single orthogonal map, recoverable with no parallel data at all. Conneau, Lample, Ranzato, Denoyer & Jégou (2018, MUSE) fit this map adversarially and refine it via iterative orthogonal Procrustes on the map’s own most-confident nearest-neighbor pairs, validating it via real bilingual dictionary induction accuracy rivaling supervised baselines — a foundational precursor to later per-language affine-subspace overlap findings (see “Cross-lingual subspace overlap” above) and to vec2vec-style universal-alignment claims. New method unsupervised-adversarial-procrustes-alignment. See conneau-etal-2018-muse-an-unsupervised-orthogonal-map-fit-with-no-parallel-data-aligns-two-independently-trained-monolingual-word-embedding-spaces-and-achieves-real-word-translation-accuracy.

  • A low-dimensional subspace identified via partial least squares, not a single classifier direction, encodes a numeric attribute shared by two compared entities, and intervening along its leading component causally flips the model’s logical comparison answer. El-Shangiti, Hiraoka, AlQuabeh, Heinzerling & Inui (2024) fit a 5-component PLS model per numeric attribute (birth year, death year, latitude) on real LLM last-token activations, reaching R^2 above 0.8, then show the intervention’s causal effect on Yes/No comparison answers is concentrated in roughly the first half of each network’s layers, replicated across Llama-3-8B-Instruct, Mistral-7B-Instruct, and Qwen2.5-7B-Instruct. New method pls-subspace-probing. See el-shangiti-etal-2024-numeric-comparison-subspaces-in-llms-are-linearly-decodable-via-pls-and-causally-control-logical-comparison-answers.

  • Foundational: most visual concepts in a real trained CNN require several jointly-fit filters, not one, and individual filters are typically polysemantic. Fong & Vedaldi (2018, Net2Vec) fit each concept as a learned weight vector over multiple AlexNet filter-activation maps, scored against real segmentation masks, quantitatively confirming both multi-filter concept encoding and filter polysemy — a multi-filter subspace generalization of single-unit dissection. New method multi-filter-concept-vector-fitting. See fong-vedaldi-2018-net2vec-shows-most-visual-concepts-in-a-trained-cnn-require-multiple-jointly-fit-filters-and-individual-filters-are-typically-polysemantic.

  • Foundational, supervised precursor to unsupervised cross-lingual alignment: a linear map fit from a small seed dictionary aligns two independently-trained monolingual embedding spaces. Mikolov, Le & Sutskever (2013) fit this map from real trained monolingual skip-gram embeddings plus a small bilingual dictionary, validating it via real held-out word-translation accuracy (e.g. ~90% precision-at-5 reported for English-Spanish) — five years before Conneau et al.’s unsupervised MUSE removed the seed-dictionary requirement entirely. New method supervised-bilingual-dictionary-linear-map. See mikolov-le-sutskever-2013-a-linear-transformation-fit-from-a-small-bilingual-dictionary-maps-one-monolingual-word-embedding-space-onto-another-with-high-real-translation-accuracy.

  • A formal necessary-and-sufficient condition connecting within-language analogy preservation to cross-lingual map linearity, empirically confirmed across 12 real trained monolingual embeddings. Peng, Stevenson, Lin & Li (2020) prove word-analogy preservation is necessary and sufficient for the ground-truth cross-lingual mapping to be linear, then test this against real trained embeddings across 12 languages using a new cross-lingual analogy dataset, measuring how mapping-linearity quality tracks analogy-preservation quality. See peng-etal-2020-cross-lingual-embedding-map-linearity-is-proven-necessary-and-sufficient-with-word-analogy-preservation-and-empirically-confirmed-across-12-real-trained-monolingual-embeddings.

  • Real pretrained mBERT’s own representation space supports nearest-neighbor cross-lingual translation-pair retrieval with no fitted map at all, alongside strong zero-shot cross-lingual transfer. Pires, Schlinger & Garrette (2019) show translation pairs are retrievable via direct nearest-neighbor search in mBERT’s own shared representation space (no additional linear map fit required), complementing this map’s other multilingual-BERT geometry findings, and demonstrate strong zero-shot cross-lingual transfer for NER/POS tagging between typologically similar languages. New method nearest-neighbor-cross-lingual-retrieval. See pires-etal-2019-mbert-representation-space-supports-real-nearest-neighbor-cross-lingual-translation-pair-retrieval-and-strong-zero-shot-cross-lingual-transfer.

  • A single unconstrained (non-orthogonal) linear map transfers real vision-encoder embeddings into a frozen language model’s token space, causally driving its generation. Merullo, Castricato, Eickhoff & Pavlick (2023) fit a linear projection from three independently pretrained vision encoders (CLIP ResNet-50x16, NFRN50, BEiT-Large) into frozen GPT-J-6B’s embedding space, training only the map itself, and use the mapped vectors as soft prompts that causally drive zero-shot image captioning and few-shot VQA — evidence both that a simple linear map suffices to bridge two independently-trained modalities’ representation spaces (criterion a) and that this mapping is causally usable to steer real generation (criterion b), while an RSA appendix explicitly leaves a fuller geometric-similarity characterization to future work. See merullo-etal-2023-a-single-unconstrained-linear-map-transfers-real-vision-encoder-embeddings-into-a-frozen-gpt-j-token-space-and-causally-drives-captioning.

  • Additive binding-ID vectors occupy a continuous subspace whose pairwise distances predict real behavioral confusability, not just decodability. Feng & Steinhardt (2023) show binding-ID vectors that solve the variable-binding problem in real Pythia and LLaMA-family models form a genuine geometric subspace in which distance between two binding vectors predicts how often the model confuses the corresponding entity-attribute bindings, and causally validate the vectors’ functional role via patching/removal experiments that change which attribute is retrieved for which entity. See feng-steinhardt-2023-binding-id-vectors-form-a-continuous-subspace-whose-pairwise-distances-predict-binding-confusability-and-patching-or-removing-them-causally-changes-which-attribute-is-retrieved-for-which-entity.

  • A behaviorally-inferred, rotation-invariant relational isomorphism between real LLM representations and an external grounded space, strengthening with scale. Patel & Pavlick (2022) test whether GPT-2/GPT-3/BERT’s internal representations of color, cardinal-direction, and spatial-grid terms carry the same relational structure as external grounded spaces, via few-shot performance under the true grounding vs. a structure-preserving rotation of it vs. a structure-destroying random reassignment — GPT-3 (175B) matches its own Original-condition accuracy under rotation while collapsing under randomization, with the effect (and raw color-grounding accuracy) strengthening markedly with scale, no explicit activation-to-space map ever fit. See patel-pavlick-2022-behavioral-performance-survives-rotation-of-grounded-space-but-collapses-under-random-reassignment.

  • A word’s true polysemy count is recoverable purely from where its many context-embedding subspaces intersect on the Grassmannian, with no lexicographic annotation involved. Mu, Bhat & Viswanath (2016) build one linear (Grassmannian) subspace per word from the top singular directions of its many context-window word2vec embeddings, and find genuinely monosemous words’ many context-subspaces intersect tightly near a single shared direction in real Wikipedia-trained word2vec, while polysemous words’ context-subspaces disperse — a purely geometric-shape measurement (no causal intervention) that recovers WordNet-annotated sense counts without ever consulting WordNet during fitting. See mu-etal-2016-monosemous-word-vectors-sit-near-the-grassmannian-intersection-of-their-many-context-subspaces-in-real-word2vec-embeddings.

  • A provably graph-Laplacian-equivalent spectral subspace, learned by a real deep encoder from real environment transitions, is frozen and causally determines downstream RL sample efficiency. Farebrother, Greaves, Agarwal, Le Lan, Goroshin, Castro & Bellemare (2023, Proto-Value Networks) train a real Impala-CNN encoder on real Atari transitions (RL Unplugged, 46 games) to estimate the successor measure, proving the resulting features span the top-dd singular vectors of the successor-measure matrix (graph-Laplacian eigenvectors/proto-value functions under a symmetric transition structure); freezing this subspace and training only a linear RL head online matches or exceeds DQN using ~4M interactions versus DQN’s 200M frames, with quantified per-game score deltas (e.g. Asterix 15,401 vs. 2,953). See farebrother-etal-2023-a-frozen-laplacian-eigenvector-subspace-learned-from-real-atari-transitions-supports-linear-rl-that-matches-dqn-using-50x-fewer-environment-interactions.

  • Real pretrained MoE experts are strongly functionally decorrelated via near-zero cross-expert Jacobian alignment, while their PCA subspaces show only partial, non-maximal Grassmannian separation. Liu (2026) measures Mistral-8x7B and Qwen1.5-MoE-A2.7B, finding cross-expert Jacobian cosine similarity clusters tightly near zero (Mistral middle-layer mean 0.062; Qwen ~0.000-0.001) while the same experts’ routed-representation PCA subspaces sit at Grassmannian distances (2.061-2.689) well below the theoretical maximum (~3.51) — decorrelated function, only partially separated geometry. A controlled ablation shows top-k routing itself causes much larger subspace separation (mean distance 2.463) than fully-soft routing (0.480) in an otherwise-identical custom Transformer. See liu-2026-real-pretrained-moe-experts-are-strongly-functionally-decorrelated-via-near-zero-cross-expert-jacobian-alignment-while-their-pca-subspaces-show-only-partial-non-maximal-grassmannian-separation.

  • Real Qwen and Llama models bind character-object-state triples via low-rank Ordering-ID subspaces that interchange-intervention patching localizes to specific layer ranges. Prakash, Shapira, Sharma, Riedl, Belinkov, Rott Shaham, Bau & Geiger (2025) show real Qwen2.5-14B-Instruct, Llama-3-70B-Instruct, and Llama-3.1-405B-Instruct bind belief-tracking triples by co-locating “Ordering ID” reference information in low-rank residual-stream subspaces, with a separate visibility-ID subspace for observed/observing-character relations; interchange-intervention patching localizes different bound variables to distinct layer ranges (e.g. “Answer Payload” present only after layer 31 in all n=80 test cases). See prakash-etal-2025-real-qwen-llama-models-bind-character-object-state-triples-via-low-rank-ordering-id-subspaces-that-interchange-intervention-patching-localizes-to-specific-layer-ranges.

  • A real trained in-context-RL transformer linearly decodes XY position by layer 2 and aligns its representations across differently-cued environments, supporting emergent shortcut-taking. Fang & Rajan (2026) train a custom 3-layer Transformer from scratch via decision-pretraining meta-RL on gridworld/tree-maze tasks, finding XY position is linearly decodable from layer-2 representations and kernel alignment between the model’s representations and the latent environment structure increases with context length; memory-token ablation causally reduces the model’s shortcut-path selection, observed in over 60% of held-out test simulations at baseline. See fang-rajan-2026-a-real-trained-in-context-rl-transformer-linearly-decodes-xy-position-by-layer-2-and-aligns-its-representations-across-differently-cued-environments-supporting-emergent-shortcut-taking.

  • A cross-lingually adapted Backpack LM’s per-word sense-vector manifold preserves a measurable, above-control amount of its English cosine-similarity topology and orthogonal-Procrustes alignment. Cruz, Adelani & Aji (2026) adapt a GPT-2-based Backpack model’s multi-sense-vector-per-word representation from English to Estonian, Turkish, Indonesian and Swahili, finding sense-vector cosine-Gram-matrix topology (rho~0.25-0.30 vs. control ~0.16-0.21) and Procrustes-aligned cosine similarity (0.35-0.44 vs. control 0.26-0.33) both exceed a matched control by ~0.09-0.11 across all four languages; ablating the model’s own soft sense-mixture weights (full mixture vs. top-1 vs. uniform) raises cross-entropy on FLORES devtest by roughly +6 in all four languages, tying the geometry-bearing mixture mechanism to a measured translation-quality effect. See sensia-sense-vector-topology-partially-transfers-across-languages-via-procrustes-alignment-and-ablating-the-sense-mixture-degrades-translation.

  • Refusal in real Gemma-2-2B-IT and Llama-3.1-8B-IT is implemented by a causally-minimal SET of SAE latents, not a single direction, with redundant “hydra” features that reactivate on system-prompt tokens after the primary set is ablated. Prakash, Yeo, Abdullah, Satapathy, Cambria & Lee (2026, AAAI 2026) extend single-direction refusal ablation (Arditi et al.) into a structured multi-latent SAE decomposition, finding ablating the identified latent set shifts jailbreak attack success rate from 4% to 33% (Gemma) / 71% to 57-70% (Llama), while ~74% of dormant redundant features reactivate on system/BOS tokens after ablation — a genuinely multi-dimensional, redundant causal subspace. See a-causally-minimal-sae-latent-set-not-a-single-direction-implements-refusal-with-redundant-hydra-features-that-reactivate-after-ablation.

  • A global spatial coordinate map, matching true geographic relational structure, emerges in real continually-pretrained Qwen2.5-0.5B hidden states purely from local relational training data. Xia, Chen, Wang, Zhu, Zhang, Chen & Xiao (2025) show absolute (x,y) coordinates are probe-recoverable with R-squared=1.00, and latent-vector distances/angles directly correlate with true geographic distances/angles, despite training data only ever describing local (nearby-point) relationships in a simulated 100x100-grid city — a genuinely emergent global map from local training signal, with no activation-level causal intervention performed. See continually-pretrained-qwen2.5-0.5b-hidden-states-recover-a-perfect-r2-global-coordinate-map-from-purely-local-relational-training.

  • Edited facts’ associative-memory key vectors overlap in a whitened subspace with a heavy-tailed, high-kurtosis distribution, and sequential real-model editing causally forgets unrelated facts as this overlap accumulates. Hu, Cao, Chen, Liu & Zhao (2025, AAAI 2025) extend ROME’s closed-form single-edit solution to the nn-edit case, showing the interference term reduces to a normalized key-vector dot product after whitening by the key covariance; measuring this across 11 real LLMs (GPT-2 family, GPT-J-6B, Pythia family, Llama-2/3 family) finds reliably heavy-tailed, zero-mean overlap distributions specifically in the whitened space, and sequential ROME editing on GPT-J shows continual edits progressively erode confidence in held-out unrelated facts until they are forgotten. See edited-facts-key-vectors-overlap-in-a-whitened-associative-memory-space-with-a-heavy-tailed-high-kurtosis-distribution-and-sequential-rome-editing-causally-forgets-unrelated-facts-as-interference-accumulates.

  • Task-recognition and task-learning attention heads project real Llama/Qwen/Yi hidden states onto a label-unembedding task subspace, and injecting their summed outputs into zero-shot prompts causally recovers most of the in-context-learning behavioral gain. Yang, Cho & Inoue (2026) show a projection-based logit attribution (TSLA) reveals two dissociable head roles whose outputs align with the span of label-token unembeddings, with strong layerwise correlation (rho=0.94) between hidden-state-update alignment and task-recognition-head alignment; patching the summed top-3% head outputs into zero-shot (no-demonstration) prompts raises classification accuracy from a 9.2% baseline to 40.4%, and ablating each head type collapses a distinct causal signature (task-recognition ratio or accuracy respectively). See task-recognition-and-task-learning-attention-heads-in-icl-project-hidden-states-onto-a-label-unembedding-task-subspace-and-injecting-their-summed-outputs-into-zero-shot-prompts-recovers-most-of-the-icl-behavioral-gain.

  • A real, custom-trained transformer develops separate low-dimensional residual-stream subspaces for numerical constants and variable names, causally validated via interchange interventions. Wu, Geiger & Millière (2025) train a 12-layer, 8-head GPT-2-style transformer from scratch on synthetic variable-assignment programs to >99.9% test accuracy; PCA plus L1-regularized linear probing identifies a 10-component numerical-constant subspace and a 26-component variable-name subspace, with UMAP showing increasing cluster separation across training, and interchange interventions swapping only the selected subspace between original and counterfactual programs causally validate each subspace’s role. See wu-geiger-milliere-2025-a-real-custom-trained-transformer-develops-separate-low-dimensional-residual-stream-subspaces-for-numerical-constants-and-variable-names-causally-validated-via-interchange-interventions.

  • Hierarchical hypernym/hyponym concept relations in real Llama and Qwen3 are encoded in domain-specific, 150-250-dimensional subspaces with similar cross-domain structure, causally validated via concept-direction steering. Sakata, Heinzerling, Ito, Yokoi & Inui (2026) extend Linear Relational Concepts/Embeddings into a depth- and domain-specific “Linear Hierarchical Encoding” (Rr(s)=Wrs+brR_r(s)=W_rs+b_r) fit from real Llama-3.2-3B, Llama-3.1-8B, Qwen3-8B and Qwen3-14B hidden states: hierarchical information lives in relatively low-dimensional subspaces (150-250 dims for 3000-5000-dim hidden states), the relevant subspace is domain-specific, and domain-specific subspaces share similar hierarchical structure across domains. Steering by adding a scaled concept-direction difference to a child representation at every layer causally flips next-token prediction toward the target parent (Causality up to 0.93). See sakata-etal-2026-hierarchical-concept-relations-in-real-llama-and-qwen3-are-encoded-in-domain-specific-150-to-250-dimensional-subspaces-and-steering-along-concept-direction-differences-causally-flips-predictions.

  • A Procrustes-fitted linear map between composed-subword and whole-word embedding spaces reveals family-dependent structural isometry. Peng, Chai & Søgaard (2025, EMNLP 2025) fit an orthogonal Procrustes alignment between real models’ composed-subword and whole-word embedding spaces across layers of Llama3-8B/3.1-8B-Instruct, Aya-expanse-8B, Gemma2-9B-it and Qwen2.5-7B-Instruct: simple vector addition of subword embeddings consistently best approximates whole-word embeddings, but the degree of structural isometry itself splits into three family-dependent patterns (high for Aya-expanse/Gemma, very low for Llama 3/3.1, moderate-then-dropping for others) — a genuine geometric-alignment-quality claim (not mere retrieval accuracy), purely observational. See peng-chai-sogaard-2025-a-procrustes-fitted-linear-map-between-composed-subword-and-whole-word-embedding-spaces-shows-family-dependent-isometry-in-real-instruction-tuned-llms.

  • Concept-aligned-token SAE feature subgroups in a real safety-tuned LLM localize jailbreak susceptibility to mid-to-late layers, causally confirmed via targeted feature amplification. Das & Gaur (2026) extract concept-aligned tokens from harmful prompts in real Gemma-2-2B-IT (BeaverTails, 14 harm categories) via cosine similarity to a ReFT-derived harm-concept subspace, then identify Gemma-Scope SAE feature subgroups for those tokens across all 26 layers using three independent grouping strategies (agglomerative clustering, hierarchical-linkage, single-token-driven); all three convergently implicate layers approximately 16-25 as the most steerable, and causally amplifying only the top features from an identified subgroup measurably raises an LLM-judged harmfulness score relative to baseline, per-category and per-layer — a causal, feature-subgroup-resolved localization of jailbreak vulnerability within the SAE-feature basis, complementing coarser diff-of-means jailbreak-direction findings elsewhere on the map with a finer within-layer, within-subspace picture. See das-gaur-2026-concept-aligned-token-sae-feature-subgroups-in-real-gemma-2-2b-it-localize-jailbreak-susceptibility-to-mid-to-late-layers-causally-confirmed-via-targeted-feature-amplification.

  • Real tabular in-context-learning foundation models build class-conditional prototype/vote geometry in activation space, causally distinct from — and going beyond — mere linear separability. Biloš, Wilson, Schneider & Nevmyvaka (2026) study three real pretrained tabular foundation models (TabPFNv2, TabICLv2, Mitra): TabICLv2 reads out predictions via a nearest-prototype rule (per-class centroid of context-row activations, nearest-neighbor assignment), while TabPFNv2/Mitra instead use an attention-weighted vote at a specific layer; a linear probe alone under-explains the readout (0.859 acc) and a bounding nonlinear-MLP probe adds only marginal accuracy, but causal interventions are decisive — forcing uniform attention drops TabPFNv2 accuracy from 0.87 to 0.49, and transplanting one model’s readout rule onto another’s activations (mismatched mechanism) drops accuracy 30-40 percentage points, while zeroing TabPFNv2’s positional-parameter matrix yields exact permutation invariance at no accuracy cost. A genuinely class-conditional centroid/vote geometry, causally confirmed via mechanism transplantation rather than probing alone. See bilos-etal-2026-real-tabpfnv2-tabiclv2-and-mitra-build-class-conditional-prototype-vote-geometry-in-activation-space-causally-confirmed-via-attention-forcing-and-cross-model-mechanism-transplantation.

  • Causal activation patching — not probing — localizes the transfer between vocalized, mimed, and imagined speech in a real brain-to-speech decoder to a compact, layer-specific subspace, with smooth interpolation across speech modes evidencing a shared continuous manifold. Maghsoudi & Mishra (2026) apply cross-mode activation patching, causal scrubbing, and coarse-to-fine causal tracing to a Conv1D-encoder + 3-layer bidirectional-GRU + HiFi-GAN decoder trained on real human sEEG (VOCALMIND dataset, vocalized/mimed/imagined speech). Patching vocalized-mode activations into the imagined-mode pathway raises reconstruction PCC from 0.725 to 0.954; the reverse direction collapses PCC to 0.177, establishing sufficiency and necessity. Causal scrubbing localizes the transfer to a 16-channel convolutional subspace (channels 32-48) and a specific RNN time window (steps 21-84) — KEEP-Conv (0.666 PCC) beats size-matched RAND-Conv (0.564) — while tri-modal linear interpolation between vocalized and imagined activations produces smooth, monotonic transitions with mimed speech landing intermediate, consistent with a shared continuous causal manifold across speech modes. See maghsoudi-mishra-2026-causal-patching-localizes-vocalized-mimed-imagined-speech-transfer-to-a-compact-subspace-in-a-real-brain-to-speech-decoder.

Exercises

Base

  1. Let V=span{(1,0,0),(0,1,0)}⊂R3V=\text{span}\{(1,0,0),(0,1,0)\}\subset\mathbb{R}^3 and x=(3,4,5)x=(3,4,5). Compute PVxP_Vx.
Solution

e1=(1,0,0),e2=(0,1,0)e_1=(1,0,0),e_2=(0,1,0) are already orthonormal. PVx=(e1⋅x)e1+(e2⋅x)e2=3(1,0,0)+4(0,1,0)=(3,4,0)P_Vx = (e_1\cdot x)e_1+(e_2\cdot x)e_2 = 3(1,0,0)+4(0,1,0) = (3,4,0).

  1. For the same V,xV,x, compute the residual x−PVxx-P_Vx and verify it is orthogonal to both e1e_1 and e2e_2.
Solution

x−PVx=(3,4,5)−(3,4,0)=(0,0,5)x-P_Vx = (3,4,5)-(3,4,0) = (0,0,5). Check: (0,0,5)⋅(1,0,0)=0(0,0,5)\cdot(1,0,0)=0 and (0,0,5)⋅(0,1,0)=0(0,0,5)\cdot(0,1,0)=0 ✓ — orthogonal to both basis vectors, hence to all of VV.

Middle

  1. Prove that PVP_V is idempotent: PV(PVx)=PVxP_V(P_Vx) = P_Vx for any xx.
Solution

PVx=∑i(ei⋅x)ei∈VP_Vx = \sum_i (e_i\cdot x)e_i \in V. Since PVxP_Vx is already a linear combination of the eie_i‘s, applying PVP_V again: PV(PVx)=∑j(ej⋅∑i(ei⋅x)ei)ej=∑j(∑i(ei⋅x)(ej⋅ei))ejP_V(P_Vx) = \sum_j \big(e_j\cdot \sum_i(e_i\cdot x)e_i\big)e_j = \sum_j\Big(\sum_i (e_i\cdot x)(e_j\cdot e_i)\Big)e_j. Since {ei}\{e_i\} is orthonormal, ej⋅ei=δije_j\cdot e_i=\delta_{ij} (1 if i=ji=j, else 0), collapsing the inner sum to just the i=ji=j term: =∑j(ej⋅x)ej=PVx=\sum_j (e_j\cdot x)e_j = P_Vx. So PV(PVx)=PVxP_V(P_Vx)=P_Vx.

  1. Prove the Pythagorean identity ∥x∥2=∥PVx∥2+∥PV⊥x∥2\|x\|^2 = \|P_Vx\|^2+\|P_{V^\perp}x\|^2.
Solution

Write x=PVx+PV⊥xx = P_Vx + P_{V^\perp}x with PVx⊥PV⊥xP_Vx\perp P_{V^\perp}x. Then ∥x∥2=x⋅x=(PVx+PV⊥x)⋅(PVx+PV⊥x)=∥PVx∥2+2(PVx⋅PV⊥x)+∥PV⊥x∥2\|x\|^2 = x\cdot x = (P_Vx+P_{V^\perp}x)\cdot(P_Vx+P_{V^\perp}x) = \|P_Vx\|^2 + 2(P_Vx\cdot P_{V^\perp}x) + \|P_{V^\perp}x\|^2. The cross term vanishes since PVx∈VP_Vx\in V, PV⊥x∈V⊥P_{V^\perp}x\in V^\perp, and vectors in VV and V⊥V^\perp are orthogonal by definition of orthogonal complement. So ∥x∥2=∥PVx∥2+∥PV⊥x∥2\|x\|^2=\|P_Vx\|^2+\|P_{V^\perp}x\|^2.

Pro

  1. Prove that PVP_V (as defined via an orthonormal basis) does not depend on the choice of orthonormal basis for VV — i.e. two different orthonormal bases of the same VV produce the same projection matrix.
Solution

PVP_V can be characterized basis-independently as the unique linear map satisfying: PVx∈VP_Vx\in V for all xx, and x−PVx⊥Vx-P_Vx \perp V for all xx (the defining property of the orthogonal projection onto VV, which exists and is unique by the projection theorem for closed — here, finite-dimensional, automatically closed — subspaces of an inner product space). Any orthonormal basis {ei}\{e_i\} of VV, by construction, produces a map x↦∑i(ei⋅x)eix\mapsto\sum_i(e_i\cdot x)e_i satisfying both properties (checked directly: the output is a linear combination of the ei∈Ve_i \in V; and for any v=∑iciei∈Vv=\sum_ic_ie_i\in V, (x−PVx)⋅v=∑ici(x⋅ei−∑j(ej⋅x)(ei⋅ej))=∑ici(x⋅ei−x⋅ei)=0(x-P_Vx)\cdot v = \sum_i c_i\big(x\cdot e_i - \sum_j(e_j\cdot x)(e_i\cdot e_j)\big) = \sum_i c_i(x\cdot e_i - x\cdot e_i)=0 using orthonormality). Since the defining properties pin down a unique map, any two orthonormal bases must yield the identical map PVP_V.

  1. Let V1,V2V_1,V_2 be two kk-dimensional subspaces representing two different papers’ independently-fitted estimates of “the same” concept subspace, with principal angles θ1,…,θk∈[0,π/2]\theta_1,\ldots,\theta_k \in [0,\pi/2] between them (defined via cos⁡θi=σi\cos\theta_i = \sigma_i, the singular values of E1⊤E2E_1^\top E_2 for orthonormal basis matrices E1,E2E_1,E_2). If all θi=0\theta_i=0, what can you conclude about V1,V2V_1,V_2? If some θi=π/2\theta_i=\pi/2, what does that indicate?
Solution

If all principal angles are 00, then V1=V2V_1=V_2 exactly (every basis direction of one subspace has a perfectly aligned counterpart in the other, forcing the subspaces to coincide) — the two independent estimates agree completely. If some θi=π/2\theta_i=\pi/2, there exists a direction in V1V_1 (a combination of its basis vectors) that is completely orthogonal to all of V2V_2 — i.e. the two estimated subspaces disagree on at least one entire independent direction, indicating the two fitting procedures found genuinely different (not just noisily perturbed versions of the same) subspaces along at least that one dimension. Principal angles between 00 and π/2\pi/2 quantify partial, graded (dis)agreement between the two estimates dimension-by-dimension, generalizing the single-direction case (where the only available comparison is r1⋅r2=cos⁡θr_1\cdot r_2=\cos\theta for one angle).

Found in (140 observations · 103 families)

Llama

Semantic Structure of Feature Space in Large Language Models (2026)measured

LLM semantic-axis directions reproduce human semantic-differential structure

Details

Kozlowski & Boutyline construct 32 semantic-axis directions (e.g. beautiful-ugly, soft-hard) as mean contrastive-pair differences from residual streams of Llama-3.2-3B, Llama-3.1-70B, Qwen3-1.7B and Qwen3-32B [kozlowski-boutyline-2026-semantic-feature-geometry] Word projections onto these axes correlate with a 1,750-respondent human semantic-differential survey (Pearson r from >0.8 down to >0.3 across the 32 scales) [kozlowski-boutyline-2026-semantic-feature-geometry] The 32 axes are meaningfully non-orthogonal: their pairwise cosine similarities reproduce the pairwise correlations among the corresponding human survey scales [kozlowski-boutyline-2026-semantic-feature-geometry] PCA puts >45% (3B) / >33% (70B) of variance in the top 3 components, far above the 3.1% expected under orthogonality, matching the Evaluation-Potency-Activity triad [kozlowski-boutyline-2026-semantic-feature-geometry] Canonical correlation analysis aligns this LLM-derived 3D subspace with the equivalent subspace from the human survey [kozlowski-boutyline-2026-semantic-feature-geometry] Additive steering along one axis produces spillover on off-target axes proportional to their cosine similarity, replicated (weaker) in the 70B and both Qwen3 sizes [kozlowski-boutyline-2026-semantic-feature-geometry]

models: Llama-3.2-3B, Llama 3.1 70B · method: PCA, Activation Steering (Addition)
Death by a Thousand Directions: Exploring the Geometry of Harmfulness in LLMs through Subconcept Probing (2025)measured

55 harmfulness-subconcept probes collapse to one dominant direction

Details

Shah et al. train 55 per-subconcept logistic-regression probes (racial hate, weapons, employment scams, etc.) on attention-output states of Llama-3.1-8B-Instruct, replicated on Qwen2-7B-Instruct, each ~0.90 mean accuracy [shah-etal-2025-harmfulness-subconcept-geometry] Stacking the 55 weight vectors, an SVD-based effective rank is K=1 at variance threshold 0.95 for all but the second-to-last layer, so 55 subconcepts share almost one direction [shah-etal-2025-harmfulness-subconcept-geometry] K-means on the weight vectors barely matches the dataset's own category taxonomy (mean Adjusted Rand Index ~3e-4), so the shared direction is not re-deriving the taxonomy [shah-etal-2025-harmfulness-subconcept-geometry] Ablating only the dominant direction matches full-subspace ablation on JailbreakBench safe-rate (~0.91) while raising utility, and steering along it cuts AutoDAN attack success 0.94->0.50 for Llama [shah-etal-2025-harmfulness-subconcept-geometry] Exact effective-rank and jailbreak figures could not be re-extracted from the source (rate-limited), so this entry is deferred for full re-verification, though scope and models are confirmed [shah-etal-2025-harmfulness-subconcept-geometry]

models: Llama-3.1-8B-Instruct · method: Linear probing, Causal interventions (steering), Activation Steering (Addition)
Beyond I'm Sorry, I Can't: Dissecting Large Language Model Refusal (2026)measured

Refusal is a causally-minimal SAE-latent set with hydra redundancy

Details

Prakash et al. (AAAI 2026) extend single-direction refusal ablation into a multi-latent SAE decomposition on Gemma-2-2B-IT and Llama-3.1-8B-IT using Gemma Scope / LlamaScope [prakash-etal-2026-dissecting-llm-refusal-sae-feature-sets] They identify causally-minimal sets of SAE latents near the refusal direction whose joint ablation reduces refusal (thousands of candidate features for Gemma, ~110 for Llama, narrowed at later stages) [prakash-etal-2026-dissecting-llm-refusal-sae-feature-sets] Ablating the active refusal-latent set triggers a hydra effect: ~74% of previously-dormant redundant features (Gemma) activate on system-prompt tokens instead, ~97% on the begin-of-text token [prakash-etal-2026-dissecting-llm-refusal-sae-feature-sets] Refusal is thus a redundant multi-dimensional causal subspace rather than a single vector [prakash-etal-2026-dissecting-llm-refusal-sae-feature-sets] Ablating the identified latent set shifts attack success rate 4%->33% (Gemma) and 71%->57% (Llama) across stages, with hydra redundancy explaining incomplete restoration [prakash-etal-2026-dissecting-llm-refusal-sae-feature-sets]

models: Llama-3.1-8B-Instruct · method: Sparse Autoencoders (SAE), Causal interventions (steering)
The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models (2026)measured

An 'Assistant Axis' persona component causally reduces jailbreaks

Details

Lu et al. extract activation vectors for 275 character archetypes across Gemma-2-27B, Qwen3-32B and Llama-3.3-70B and run PCA on the persona space (4-19 PCs explain 70% of variance) [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models] PC1 is an interpretable "Assistant Axis" whose role loadings correlate >0.92 pairwise across the three models [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models] A contrast-based Assistant Axis (mean role-playing minus default-Assistant) aligns with PC1 at cosine >0.71 in all three models [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models] Activation-capping the Assistant-Axis projection cuts persona-jailbreak harmful-response rate ~60% with no measurable loss on IFEval/MMLU-Pro/GSM8k/EQ-Bench [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models] The axis persists in base (non-instruction-tuned) Gemma-2-27B, evidencing a pretraining origin [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models]

models: Llama 3.3 70B Instruct · method: Difference-in-means direction, PCA, Activation Steering (Addition)
The Granularity Axis: A Micro-to-Macro Latent Direction for Social Roles in Language Models (2026)measured

A role-granularity axis aligns with PC1 and steers output detail

Details

Qin et al. build 75 ordered social roles (5 granularity levels x 15 roles) and 91,200 role-conditioned responses, defining a macro-minus-micro contrast direction [qin-etal-2026-the-granularity-axis-a-micro-to-macro-latent-direction-for-social-roles-in-language-models] This Granularity Axis aligns with PC1 of the role-representation space at cosine 0.972 (Qwen3-8B) and 0.9596 (Llama-3.1-8B-Instruct), explaining 52.6%/42.5% of variance [qin-etal-2026-the-granularity-axis-a-micro-to-macro-latent-direction-for-social-roles-in-language-models] Projections are monotonic in granularity level (Spearman/Pearson >0.93 in both models) [qin-etal-2026-the-granularity-axis-a-micro-to-macro-latent-direction-for-social-roles-in-language-models] Adding the axis at layer 18 shifts judge-rated output granularity in the predicted direction (Qwen3-8B 2.00->2.67; Llama 2.00->3.17), null for random and Assistant-Axis controls [qin-etal-2026-the-granularity-axis-a-micro-to-macro-latent-direction-for-social-roles-in-language-models] Human annotators corroborate (pairwise macro-preference 0.64-0.90 across cells) [qin-etal-2026-the-granularity-axis-a-micro-to-macro-latent-direction-for-social-roles-in-language-models]

models: Llama-3.1-8B-Instruct · method: Difference-in-means direction, PCA, Activation Steering (Addition)
Cell-Based Representation of Relational Binding in Language Models (2026)measured

A grid-organized cell subspace supports relational binding in LLMs

Details

Dai, Heinzerling & Inui (2026) use PLS-regression probing (vs PCA/ICA baselines) on frozen middle-layer activations (strongest layers 10-20) of Llama3-8B-Instruct and Qwen3-8B [dai-etal-2026-cell-based-representation-of-relational-binding-in-language-models] They find a low-dimensional linear "Cell-Based Representation" subspace organized as a grid along entity-index x relation-index axes, with near-perfect PLS fits using only 2-5 components [dai-etal-2026-cell-based-representation-of-relational-binding-in-language-models] Cross-context transfer R^2~0.8, ablation stability R^2~0.8, index-prediction R^2~0.95 across 13 discourse patterns, attribute accuracy 0.94-0.95 (beating a Hessian propositional-probe baseline) [dai-etal-2026-cell-based-representation-of-relational-binding-in-language-models] Subspace perturbation degrades attribute accuracy vs a random-subspace control, and explicit steering vectors reliably shift entity/relation indices across 5 domains [dai-etal-2026-cell-based-representation-of-relational-binding-in-language-models]

models: Llama-3-8B-Instruct · method: Partial Least Squares (PLS), PCA, Direction Extraction, Causal interventions (steering), Activation patching
Representational Analysis of Binding in Language Models (2024)measured

A PCA 'Ordering ID' direction localizes and steers binding

Details

Dai, Heinzerling & Inui (2024) use PCA on entity/attribute activations to find a low-rank "OI subspace" whose leading component (OI-PC) encodes the "Ordering ID" underlying Feng & Steinhardt's Binding-ID mechanism, not previously localized to a direction [dai-etal-2024-representational-analysis-of-binding-in-language-models] Steering along OI-PC by n unit steps shifts the predicted attribute to that of Binding-ID n, quantified via Logit Difference and Logit Flip metrics that rise monotonically with step count [dai-etal-2024-representational-analysis-of-binding-in-language-models] A Spearman correlation confirms OI-PC tracks the true Order-ID rather than a position/filler-length confound [dai-etal-2024-representational-analysis-of-binding-in-language-models] The OI subspace is prevalent across five base models: Llama2-7B, Llama3-8B, Qwen1.5-7B, Pythia-6.9B, and the code-fine-tuned Float-7B [dai-etal-2024-representational-analysis-of-binding-in-language-models]

models: Llama-2-7B, Llama-3-8B · method: PCA, Direction Extraction, Causal interventions (steering)
Tool Calling Is Linearly Readable and Steerable in Language Models (2026)measured

A per-tool-pair mean-difference direction reads and switches tool choice

Details

Wu et al. show tool selection in tool-calling agents is carried by a single mean-difference direction per tool-pair in the residual stream across Gemma 3 (270M-27B), Qwen 3 (0.6B-14B) and Llama 3.1 (8B) [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models] Adding the direction switches the chosen tool at 83-100% on a 15-tool synthetic benchmark and 77-94% on tau-bench-airline, versus 0% for a random-direction control [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models] PCA over per-tool mean activations puts ~91% of variance in ~10 components for 15 tools, far below a random-Gaussian control [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models] Base (non-instruction-tuned) models already carry the correct tool internally (cosine readout 61-82% on BFCL vs 2-10% from base generation) [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models] SAEs and cross-layer transcoders trace a three-stage circuit: early tool-selective features, mid-layer attention heads, late-layer JSON-formatting features [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models]

models: Llama-3.1-8B-Instruct · method: Difference-in-means direction, Activation Steering (Addition), PCA, Activation patching, Sparse Autoencoders (SAE), Circuit tracing (attribution graphs)
Stories in Space: In-Context Learning Trajectories in Conceptual Belief Space (2026)measured

In-context belief updating traces a low-dimensional manifold

Details

Bigelow et al. measure Llama-3.1-8B-Instruct's belief about story concepts (emotions, genres, an arbitrary control) as an expected value over next-token digit probabilities, tracing a smooth trajectory through a UMAP/PCA-recovered low-dimensional manifold [bigelow-etal-2026] The same low-dimensional structure is recoverable from residual-stream activations, correlating with the behavioral geometry (r=.92) for structured domains but not the arbitrary control [bigelow-etal-2026] For emotions the manifold reduces to a 2D valence-arousal plane matching Russell's circumplex, a flat linearly-probeable subspace [bigelow-etal-2026] Activation-addition steering along a concept direction predictably shifts belief, and unintended steering entanglement is predictable from geometric distance between concepts [bigelow-etal-2026] The model-id and diff-in-means extraction were not confirmable from the available render, so those details are deferred, though the geometric findings hold [bigelow-etal-2026]

models: Llama-3.1-8B-Instruct · method: UMAP, PCA, Linear probing, Difference-in-means direction, Activation Steering (Addition), Geometric analysis
The Confidence Manifold: Geometric Structure of Correctness Representations in Language Models (2026)measured

Answer-correctness sits in a 3-8D linear subspace, causally steerable

Details

Cho et al. study self-assessed correctness on TruthfulQA pairs across 9 models spanning 5 families, finding the discriminative signal is a low-dimensional linear subspace, not a curved manifold [cho-etal-2026-confidence-manifold] A PLS sweep peaks at 3-8 dimensions (e.g. peak AUC 0.90 at layer 23, dim 5, Mistral-7B), and nonlinear classifiers give no gain over a linear boundary [cho-etal-2026-confidence-manifold] Correct/incorrect classes form roughly Gaussian clusters separated by a mean shift: a two-mean centroid detector matches the linear probe (0.90 vs 0.89 AUC), robust with 25 examples/class [cho-etal-2026-confidence-manifold] This 3-8D discriminative subspace is lower-dimensional than the representation's 8-12D intrinsic dimension at the same layers [cho-etal-2026-confidence-manifold] Adding the learned direction shifts downstream error rate up to 10.9 points dose-dependently, while random and orthogonal controls have no reliable effect [cho-etal-2026-confidence-manifold]

models: Llama-3.2-1B-Instruct, Llama-3.2-3B-Instruct · method: Linear probing, Partial Least Squares (PLS), Difference-in-means direction, Activation Steering (Addition)
Scenario-based Probing and Steering Cultural Values in Large Language Models (2026)measured

Cultural-value steering axes are entangled, mirroring real value correlations

Details

Dang, Kieu & Masud map LLMs onto the Inglehart-Welzel cultural map via scenario forced-choice probing and diff-of-means steering at 4 tuned layers per model [dang-etal-2026-scenario-based-probing-and-steering-cultural-values-in-llms] The steering direction is a signed bidirectional axis (negative alpha reverses the shift), saturating at alpha in [0.5, 0.6] [dang-etal-2026-scenario-based-probing-and-steering-cultural-values-in-llms] Steering one cultural axis causes correlated shifts on the other, an entanglement ratio 0.72-0.81 (dropping to 0.53 for Llama under hybrid steering), mirroring the real-world WVS axis correlation r=0.474 [dang-etal-2026-scenario-based-probing-and-steering-cultural-values-in-llms] Capability cost measured via Global-MMLU is largest for Gemma (0.608->0.530) [dang-etal-2026-scenario-based-probing-and-steering-cultural-values-in-llms]

models: Llama-3.2-3B-Instruct · method: Difference-in-means direction, Direction Extraction, Causal interventions (steering)
Knowledge in Superposition: Unveiling the Failures of Lifelong Knowledge Editing for Large Language Models (2025)measured

Edited facts' key vectors overlap in a whitened memory space

Details

Hu et al. extend ROME's closed-form associative-memory solution to n edits and derive an interference term equal to a whitened normalized dot-product between two facts' key vectors [hu-etal-2025-knowledge-in-superposition] Across 11 LLMs (GPT-2 family, GPT-J-6B, Pythia 1B-6.9B, Llama-2/3 family) the whitened-overlap distribution is heavy-tailed and high-kurtosis with clustered near-maximal overlaps, absent in the raw key vectors ("knowledge superposition") [hu-etal-2025-knowledge-in-superposition] Sequential ROME edits on GPT-J progressively reduce confidence in unrelated held-out facts, tying the measured overlap geometry to a lifelong-editing forgetting failure [hu-etal-2025-knowledge-in-superposition]

models: Llama-2-7B, Llama-2-13B, Llama-3-8B, Llama-3.1-8B · method: Rank-One Model Editing (ROME) / Linear Associative Memory Editing
The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspaces (2024)measured

PLS-found numeric-comparison subspaces causally control LLM answers

Details

El-Shangiti et al. show LLMs answering numeric-comparison questions encode each entity's numeric attribute in a low-dimensional linear subspace, found via a 5-component PLS regression on the last-token state [el-shangiti-etal-2024-geometry-of-numerical-reasoning] The fit reaches R^2 > 0.8 for birth year, death year and latitude across Llama-3-8B-Instruct, Mistral-7B-Instruct and Qwen2.5-7B-Instruct [el-shangiti-etal-2024-geometry-of-numerical-reasoning] Intervening along the first PLS component flips the model's Yes/No comparison answer far more than an equal-magnitude random control, concentrated in roughly the first half of layers [el-shangiti-etal-2024-geometry-of-numerical-reasoning]

models: Llama-3-8B-Instruct · method: Partial least squares (PLS) subspace probing
How Do Language Models Bind Entities in Context? (2023)measured

Binding-ID vectors form a subspace whose distances predict confusability

Details

Feng & Steinhardt identify additive "binding ID vectors" attached to entity and attribute activations that solve variable binding, using causal interventions on LLaMA-1 (30B primary, plus 13B and 65B) and the Pythia family [feng-steinhardt-2023-how-do-language-models-bind-entities-in-context] The binding vectors occupy a continuous subspace in which the distance between two binding vectors predicts how often the model confuses the corresponding entity-attribute bindings [feng-steinhardt-2023-how-do-language-models-bind-entities-in-context] Patching, adding or removing binding-ID vectors changes which attribute the model retrieves for a given entity, demonstrated via causal mediation analysis rather than linear probing [feng-steinhardt-2023-how-do-language-models-bind-entities-in-context]

models: LLaMA-30B, LLaMA-13B, LLaMA-65B · method: Activation patching
Convergent Evolution: How Different Language Models Learn Similar Number Representations (2026)measured

Fourier periodicity in number embeddings is universal; linear separability is not

Details

Fu et al. distinguish spectral convergence (Fourier spikes at periods T=2,5,10 in number-token embeddings) from geometric convergence (linear mod-T decodability) [fu-etal-2026-convergent-evolution-number-representations] Spectral convergence appears everywhere: Transformer and non-Transformer LLMs, classical word embeddings, and even the raw training-corpus token-frequency distribution computed with no model [fu-etal-2026-convergent-evolution-number-representations] Geometric convergence is architecture-selective: controlled 300M models on identical data show Transformers and linear RNNs develop linearly separable mod-T classes while LSTMs, despite stronger Fourier spikes, stay at chance probing [fu-etal-2026-convergent-evolution-number-representations] Theorem 1 proves Fourier sparsity is necessary but not sufficient for mod-T separability, and a Fisher-LDA (T-1)-dimensional discriminant bound (Theorem 2) explains why near-identical spectra give very different probe accuracy [fu-etal-2026-convergent-evolution-number-representations] Controlled data perturbations that remove text-number co-occurrence degrade probing accuracy while leaving the Fourier spectrum unchanged, a data-level causal attribution [fu-etal-2026-convergent-evolution-number-representations] Built from a partial retrieval: the paper's later circular-representation findings for multi-token addition could not be verified and are not asserted here [fu-etal-2026-convergent-evolution-number-representations]

models: Llama-3-8B, Llama 4 Scout (17B-active/16-expert MoE) · method: Linear probing
The Shape of Beliefs: Geometry, Dynamics, and Interventions along Representation Manifolds of Language Models' Posteriors (2026)measured

In-context Gaussian-belief posteriors trace curved manifolds linear steering breaks

Details

Sarfati et al. give Llama-3.2 a string of samples from an unknown normal and study how it in-context infers the distribution; layer-14 activations form smooth but genuinely curved 2D manifolds as mu or sigma varies [sarfati-etal-2026-shape-of-beliefs] The output-side simplex geometry (via inPCA) shows the same curved structure dual to the input side [sarfati-etal-2026-shape-of-beliefs] Rather than one global linear probe, local linear field probes tiling the manifold reach 87-99% accuracy, framed as evidence that purely linear concept representations are often an inadequate abstraction [sarfati-etal-2026-shape-of-beliefs] Belief updating after a mid-sequence distribution change shows two-timescale relaxation through two attractor-like regions for the old and new distributions [sarfati-etal-2026-shape-of-beliefs] Linear difference-of-means steering drives activations off the curved manifold, producing out-of-distribution logits, while manifold-respecting geodesic steering preserves the target distribution, so the curvature is causally real [sarfati-etal-2026-shape-of-beliefs]

models: Llama-3.2-1B · method: PCA, Linear probing, Activation Steering (Addition), Geometric analysis
Characterizing Linear Alignment Across Language Models (2026)measured

Independently-trained embedding models share linear structure (CKA 0.60-0.88)

Details

Gorbett & Jana measure linear CKA across five independently-trained embedding models (OpenAI text-embedding-3-small, Cohere embed, Gemini embedding-001, Qwen3-Embedding-8B, e5-mistral-7b-instruct), finding similarity 0.595-0.881 [gorbett-jana-2026-characterizing-linear-alignment-across-language-models] In a separate experiment they fit learned affine maps between instruction-tuned generative LMs (Llama-3-8B, Qwen2.5-7B/14B, Llama-3.2-1B, OLMo-7B and others) [gorbett-jana-2026-characterizing-linear-alignment-across-language-models] These maps preserve classification accuracy and OOD-AUROC across model pairs (e.g. 94.5%->93.1%) [gorbett-jana-2026-characterizing-linear-alignment-across-language-models] They enable zero-shot cross-model text generation via a frozen target head, with generation quality correlating with tokenizer exact-match rate (r=0.898) and vocabulary Jaccard overlap (r=0.822) [gorbett-jana-2026-characterizing-linear-alignment-across-language-models]

models: Llama-3-8B, Llama-3.2-1B · method: Centered Kernel Alignment (CKA), Singular Vector Canonical Correlation Analysis (SVCCA)
High-Dimensional Interlingual Representations of Large Language Models (2025)measured

Multilingual LLMs share a core interlingual subspace with language-specific fragments

Details

Wilie et al. study Aya Expanse 8B, Llama-3.1-8B and Gemma-2-9B with a kNN-based Interlingual Local Overlap (ILO) score plus t-SNE of hidden states [wilie-etal-2025-high-dimensional-interlingual-representations] Representations organize as a partially-shared core interlingual region (translation-equivalent inputs share local neighborhoods) surrounded by fragmented language-specific components that do not transfer [wilie-etal-2025-high-dimensional-interlingual-representations] Overlap varies systematically by resource level and linguistic family, with graph "bridge"/"reachability" metrics showing which languages connect the core to peripheral fragments [wilie-etal-2025-high-dimensional-interlingual-representations] The alignment is causally load-bearing: ILO overlap is preserved or disrupted depending on which layers are frozen during single-language fine-tuning [wilie-etal-2025-high-dimensional-interlingual-representations]

models: Llama-3.1-8B · method: Interlingual Local Overlap (ILO)
From Text to Space: Mapping Abstract Spatial Models in LLMs during a Grid-World Navigation Task (2025)measured

LLaMA-3.1-8B linearly encodes grid position; action-correctness units are causal

Details

Martorell has LLaMA-3 models (1B to 90B, with the 11B/90B being Vision checkpoints run text-only) navigate a 5x5 grid described in 6 spatial text encodings, with Cartesian encodings and larger models best (90B reaches 98% on JSON) [martorell-2025-grid-world-spatial-models] All linear-probing and ablation is done on LLaMA-3.1-8B only; per-layer regression probes decode a 50-dim agent/goal one-hot vector significantly above a permutation null at every layer and encoding [martorell-2025-grid-world-spatial-models] Univariate analysis finds unit sets correlating across all 6 encodings with agent x-position (448), y-position (258), border status (373) and action correctness (286), the last also elevated on unrelated spatial-reasoning prompts [martorell-2025-grid-world-spatial-models] Ablating the 286 action-correctness units drops JSON navigation success 59%->55% while ablating position units leaves it unchanged, a within-paper causal dissociation [martorell-2025-grid-world-spatial-models]

models: Llama-3.2-1B-Instruct, Llama-3.2-3B-Instruct, Llama-3.1-8B-Instruct, Llama-3.1-70B-Instruct · method: Linear probing, Causal interventions (steering)
Revealing Emergent Human-like Conceptual Representations from Language Prediction (2025)measured

In-context concept representations converge toward a human-aligned structure

Details

Xu et al. probe LLM (mainly LLaMA3-70B) concepts via an in-context reverse-dictionary task, characterizing each context's representation by its pairwise-similarity matrix [xu-etal-2025-emergent-conceptual-representations] RSA alignment across contexts rises from 0.800 at 1 demonstration to 0.970 at 24, converging toward a single context-independent relational structure [xu-etal-2025-emergent-conceptual-representations] Alignment with the 120-demonstration structure correlates with reverse-dictionary accuracy at rho=0.976, and cross-model alignment across 67 LLMs predicts task performance (rho=0.870), all correlational with no intervention [xu-etal-2025-emergent-conceptual-representations] The convergent structure aligns via RSA with human similarity judgments (SimLex-999 rho=0.776), THINGS odd-one-out, and voxel-wise fMRI activity in LOC/FFA/PPA and other regions [xu-etal-2025-emergent-conceptual-representations]

models: Llama-3-70B · method: Representational Similarity Analysis (RSA)
Do Language Models Track Entities Across State Changes? (2026)measured

LLMs consolidate entity-state tracking into a local binary subspace

Details

Tang et al. train linear probes on last-token residual activations (the "boxes" dataset) to test global versus local entity-state tracking [tang-etal-2026-do-language-models-track-entities-across-state-changes] Global 8-way state probes reach only ~0.3 accuracy versus ~0.9 for local binary probes, so the model consolidates state locally rather than tracking a full global state [tang-etal-2026-do-language-models-track-entities-across-state-changes] INLP null-space projection of the state-tag direction causally changes completions [tang-etal-2026-do-language-models-track-entities-across-state-changes] A learned sparse boolean mask over the residual PCA basis ("subspace patching") reveals a shared positional-information subspace at layers 15-25 in CodeLlama-13B [tang-etal-2026-do-language-models-track-entities-across-state-changes]

models: Llama 3.1 70B · method: Linear probing, Causal interventions (steering), PCA
Do Linear Probes Generalize Better in Persona Coordinates? (2026)measured

A persona-conditioned PCA subspace separates deception and sycophancy

Details

Mahadik & Skapars build one activation vector per persona (deceptive/honest, sycophantic/non-sycophantic) by mean-pooling layer-14 hidden states, following the Assistant Axis method [mahadik-skapars-2026-persona-coordinates-linear-probes] PCA on the centered persona vectors gives a PC1 that cleanly separates harmful from harmless personas for both behaviors, with the default assistant near the harmless cluster [mahadik-skapars-2026-persona-coordinates-linear-probes] PC1 and a diff-in-means persona direction transfer zero-shot as classifiers to 5 unseen deception and 5 unseen sycophancy datasets [mahadik-skapars-2026-persona-coordinates-linear-probes] Probes on top-3 persona-PC features improve cross-dataset AUROC transfer over raw-activation probes and over random-subspace and dataset-PCA controls, on Llama-3.2-3B and replicated on Llama-3-8B [mahadik-skapars-2026-persona-coordinates-linear-probes]

models: Llama-3.2-3B, Llama-3-8B · method: Difference-in-means direction, PCA
Probing for Representation Manifolds in Superposition (2026)measured

A Manifold Probe recovers causally-used multi-dimensional time and space manifolds

Details

Modell introduces the Manifold Probe, which jointly learns (via a generalized eigenvalue problem over a spline basis) the space of a concept's linearly-decodable features and the orthonormal directions encoding them [modell-2026-manifold-probe] Applied to Llama-2-7B for release dates and geographic coordinates, it surfaces many more decodable features than the raw concept value, with the top SPACE feature more precisely decodable (higher test R^2) than latitude or longitude, while the top time feature is roughly identical to the year [modell-2026-manifold-probe] After Varimax rotation, top time features separate individual decades (1950s-2010s) and top space features localize on individual US states [modell-2026-manifold-probe] Treating the learned manifold as a continuum of steering vectors causally shifts the model's stated release year toward a target (peaking at layers 8 and 14), so the manifold is causally used, not merely decodable [modell-2026-manifold-probe]

models: Llama-2-7B · method: Linear probing, Activation Steering (Addition), Analytical derivation
SAIF: A Sparse Autoencoder Framework for Interpreting and Steering Instruction Following of Language Models (2025)measured

Multi-feature SAE steering vectors force instruction-following

Details

He et al. identify instruction-relevant SAE latents via a sensitivity score, then build multi-feature steering vectors from Gemma Scope / Llama Scope decoder directions [he-etal-2025-saif-a-sparse-autoencoder-framework-for-interpreting-and-steering-instruction-following-of-language-models] Adding them to the residual stream forces instruction-following (translation/summarization/keyword) at over 30% strict and up to ~0.7 loose accuracy versus near-zero for single-latent steering, optimal at k=15 latents [he-etal-2025-saif-a-sparse-autoencoder-framework-for-interpreting-and-steering-instruction-following-of-language-models] Last-layer placement is critical (Gemma-2-2b-it loose accuracy 0.64 at layer 25 drops to 0.33 by layer 24), and post-instruction positioning beats pre-instruction [he-etal-2025-saif-a-sparse-autoencoder-framework-for-interpreting-and-steering-instruction-following-of-language-models]

models: Llama-3.1-8B-Instruct · method: Sparse Autoencoders (SAE), Activation Steering (Addition)
Neural Chameleons: Language Models Can Learn to Hide Their Thoughts from Activation Monitors (2025)measured

Monitor-evasion fine-tuning generalizes zero-shot to unseen safety monitors

Details

McGuinness et al. fine-tune LLMs to evade activation-monitor probes for benign concepts when conditioned on a "You are being probed for {concept}" trigger [mcguinness-etal-2025-neural-chameleons] The learned evasion generalizes zero-shot: substituting a safety-relevant term like "deception" evades previously-unseen safety monitors, even ones trained post hoc on the model's frozen weights [mcguinness-etal-2025-neural-chameleons] The effect holds across Llama, Gemma and Qwen families, is highly selective to the triggered concept, and has modest capability impact [mcguinness-etal-2025-neural-chameleons] Observational PCA (no causal intervention) on Gemma-2-9b-it indicates the evasion correlates with relocation of activations into a low-dimensional subspace; monitor ensembles and nonlinear classifiers are more resilient [mcguinness-etal-2025-neural-chameleons]

models: Llama-3.1-8B-Instruct · method: Subspace-evasion fine-tuning
How Reliable are Causal Probing Interventions? (2025)measured

Nonlinear interventions are more reliable than linear subspace erasure

Details

Canby et al. formalize completeness and selectivity as numbers in [0,1] and define reliability as their harmonic mean, comparing INLP, RLACE, AlterRep and three gradient-based nonlinear interventions [canby-etal-2025-reliability] Every method shows a completeness/selectivity trade-off; none achieves perfect completeness without sacrificing selectivity in any model [canby-etal-2025-reliability] Counterfactual methods (AlterRep, gradient-based) achieve near-total task-accuracy change, consistent with higher completeness than the concept-removal methods [canby-etal-2025-reliability] Nonlinear interventions are almost always most reliable (FGSM 0.92-0.96 across GPT-2/Pythia/Llama) versus linear methods 0.106-0.841, the sole exception being BERT layers 10-12 where AlterRep wins [canby-etal-2025-reliability]

models: Llama-3.2-3B-Instruct · method: INLP subspace + AlterRep counterfactual push, R-LACE (Relaxed Linear Adversarial Concept Erasure), Gradient-Based Interventions (nonlinear counterfactual push via adversarial attack)
LEACE: Perfect Linear Concept Erasure in Closed Form (2023)measured

The minimal-distortion linear eraser is an oblique projection

Details

Belrose et al. prove LEACE's closed-form eraser guards a concept against every linear classifier and convex loss simultaneously (Theorem 4.1) [belrose-etal-2023-leace] It is the least-squares-optimal minimal-distortion such eraser (Theorems 4.2-4.3) and is necessarily an oblique, not orthogonal, projection, unlike INLP/RLACE [belrose-etal-2023-leace] On BERT Bias-in-Bios gender removal it reaches chance accuracy at rank 1 with the smallest embedding MSE, cutting the TPR gap 0.198->0.084, about two orders of magnitude faster than RLACE [belrose-etal-2023-leace] Concept-scrubbing POS from every layer of Pythia (160M-12B) and LLaMA (7B-30B) raises LM loss far above a random-subspace control (e.g. LLaMA-7B 0.69->1.73 bpb) while removing fewer dimensions than a full-rank baseline [belrose-etal-2023-leace]

models: LLaMA-7B, LLaMA-13B, LLaMA-30B · method: LEACE (LEAst-squares Concept Erasure)
Understanding Subword Compositionality of Large Language Models (2025)measured

Composed-subword vs whole-word spaces show family-dependent isometry

Details

Peng, Chai & Sogaard fit an orthogonal Procrustes map between composed-subword and whole-word embedding spaces across instruction-tuned LLMs, scored by Precision@1 retrieval [peng-chai-sogaard-2025-subword-compositionality] Simple addition consistently outperforms other subword-composition operations [peng-chai-sogaard-2025-subword-compositionality] Three family patterns emerge: Aya-expanse and Gemma show high composed-vs-whole-word isometry, Llama 3/3.1 very little, others moderate but dropping late in the network [peng-chai-sogaard-2025-subword-compositionality] The analysis is purely observational with no causal intervention [peng-chai-sogaard-2025-subword-compositionality]

models: Llama-3-8B-Instruct, Llama-3.1-8B-Instruct · method: Orthogonal Procrustes alignment, Linear probing
Language Models Use Lookbacks to Track Beliefs (2025)measured

LLMs bind belief triples via Ordering-ID subspaces localized by patching

Details

Prakash et al. show Qwen2.5-14B-Instruct, Llama-3-70B-Instruct and Llama-3.1-405B-Instruct bind character-object-state triples by co-locating reference information as low-rank Ordering IDs in the state-token residual stream, plus a visibility ID for theory-of-mind [prakash-etal-2025-language-models-use-lookbacks-to-track-beliefs] Interchange-intervention (activation patching) localizes visibility source-reference alignment to layers 10-23 and the strongest binding-lookback alignment to layers 33-38 [prakash-etal-2025-language-models-use-lookbacks-to-track-beliefs] Answer-payload information is present only after layer 31, consistent across all 80 test cases [prakash-etal-2025-language-models-use-lookbacks-to-track-beliefs] The lookback mechanism duplicates reference information into an address and a pointer, retrieved later via attention [prakash-etal-2025-language-models-use-lookbacks-to-track-beliefs]

models: Llama-3-70B-Instruct, Llama-3.1-405B-Instruct · method: Activation patching
Geometric Factual Recall in Transformers (2026)measured

Factual recall is linear superposition with a proven dimension bound

Details

Ravfogel et al. model subject embeddings as a sum of per-attribute vectors read by a relation-conditioned ReLU-gated MLP, proving (Theorem 4.1) a 1-layer transformer plus 3-layer MLP solves single-hop recall for N entities and R relations when d=4R*log(N)+1 [ravfogel-etal-2026-geometric-factual-recall-transformers] Synthetic experiments confirm the scaling: trainable embeddings memorize once d>=128 while frozen embeddings need d>=512 for R=16 [ravfogel-etal-2026-geometric-factual-recall-transformers] On five real LMs a rank-512 affine probe recovers the LM-head output embedding (best-layer MRR 0.44-0.69 across entity categories), reversing Hernandez et al. reported output-side non-linearity [ravfogel-etal-2026-geometric-factual-recall-transformers] Minimum-norm subject-embedding perturbations swap the queried attribute with high selectivity, and a frozen relation-selector MLP transfers zero-shot to unseen relation bijections [ravfogel-etal-2026-geometric-factual-recall-transformers]

models: Llama-3.1-8B, Llama-3.2-1B · method: Linear Relational Embedding (LRE), Linear probing
Emotions Where Art Thou: Characterizing the Emotional Latent Space of LLMs (2025)measured

A 40-dim SVD emotional subspace with four interpretable axes steers emotion

Details

Reichman et al. extract a low-dimensional emotional subspace from mean-pooled hidden states of Llama-3.1-8B, OLMo-2-7B and Ministral-8B via centered SVD, using its top 40 dimensions [reichman-etal-2025-emotional-latent-space-llms] The four leading PCs align with valence, dominance, approach-avoidance and arousal, stable across layers (Spearman up to 0.92 for emotion ordering) [reichman-etal-2025-emotional-latent-space-llms] A linear cross-domain alignment map generalizes the subspace across 8 emotion datasets in 6 languages (aligned-subspace cosine 0.83-0.94) [reichman-etal-2025-emotional-latent-space-llms] A trained one-layer MLP mapping the 40D projection to a residual shift raises target-emotion accuracy from 9% to 83% (English, Llama) while preserving semantics [reichman-etal-2025-emotional-latent-space-llms]

models: Llama-3.1-8B, Llama-3.1-8B-Instruct · method: PCA, Linear probing, Representation Alignment
Understanding and Preserving Safety in Fine-Tuned LLMs (2026)measured

Safety gradients occupy a low-rank subspace orthogonal to utility

Details

Zhang et al. SVD per-layer safety- versus utility-task gradients during fine-tuning, finding safety gradients occupy a compact low-rank subspace with sharp singular-value decay while utility spans much higher dimension [zhang-etal-2026-understanding-and-preserving-safety-in-fine-tuned-llms] Safety and utility gradient directions have cosine similarity oscillating around zero and often negative (directional conflict) [zhang-etal-2026-understanding-and-preserving-safety-in-fine-tuned-llms] Their Safety-Preserving Fine-tuning projects utility gradients onto the safety subspace orthogonal complement during training [zhang-etal-2026-understanding-and-preserving-safety-in-fine-tuned-llms] This cuts attack success rate 0.955 to 0.019 (Llama-3.1-8B), 0.985 to 0.240 (Mistral) and 0.988 to 0.124 (Qwen2.5-7B) while preserving MMLU, robust to deep fine-tuning and multiple jailbreaks [zhang-etal-2026-understanding-and-preserving-safety-in-fine-tuned-llms]

models: Llama-3.1-8B-Instruct · method: PCA
Linear Representations of Hierarchical Concepts in Language Models (2026)measured

Hierarchical concepts occupy domain-specific 150-250D subspaces, steerable

Details

Sakata et al. extend Linear Relational Concepts into a depth- and domain-specific Linear Hierarchical Encoding fit from intermediate hidden states of Llama-3.2-3B, Llama-3.1-8B, Qwen3-8B and Qwen3-14B [sakata-etal-2026-linear-hierarchical-concepts] Hierarchical information is encoded in relatively low-dimensional subspaces (150-250 dimensions for hidden size 3000-5000) [sakata-etal-2026-linear-hierarchical-concepts] The relevant subspace is domain-specific, and domain-specific subspaces show similar hierarchical structure across domains [sakata-etal-2026-linear-hierarchical-concepts] Editing a child representation by adding a scaled concept-direction difference at every layer flips next-token prediction toward the target parent (causality up to 0.93 for Organization/Llama-3.1-8B) [sakata-etal-2026-linear-hierarchical-concepts]

models: Llama-3.2-3B, Llama-3.1-8B · method: Linear relational concepts (LRC), Causal interventions (steering)
The Cylindrical Representation Hypothesis for Language Model Steering (2026)measured

Steering directions split into a magnitude axis and interference plane

Details

Gao et al. relax the Linear Representation Hypothesis orthogonality assumption, decomposing a concept steering vector into a central axis and an orthogonal 2D normal plane spanned by the axis complement and the top PC of other concepts directions [gao-etal-2026-cylindrical-hypothesis] Despite cylindrical/phase/sector terminology there is no measured angular or periodic structure (phase = position in the 2D plane; sectors = a binary high/low-sensitivity split), so it is a linear-subspace interference decomposition, not a topology [gao-etal-2026-cylindrical-hypothesis] Steering-effect magnitude follows a predictable sin^m*cos^n form, but which interference sector a concept pair falls into is NOT predictable from the vectors (Pearson -0.034), a genuine null [gao-etal-2026-cylindrical-hypothesis] A penalty experiment attenuating the plane component trades earlier target onset against earlier corrupted output, validated by an LLM-judge at 94% human agreement [gao-etal-2026-cylindrical-hypothesis]

models: Llama-2-7B-Chat · method: Difference-in-means direction, PCA, Linear probing, Activation Steering (Addition)
Emergence of Phonemic, Syntactic, and Semantic Representations in Artificial Neural Networks (2026)measured

Structural probes show staged phonemic-then-lexical-then-syntactic subspaces

Details

Orhan et al. train Hewitt-Manning distance-based structural probes to recover phoneme, WordNet lexical-semantic and syntactic-tree subspaces from wav2vec2 (tiny/base/large) and text models (Llama-2-13B, Pythia), tracked across pretraining [orhan-etal-2026-emergence-of-phonemic-syntactic-and-semantic-representations-in-artificial-neural-networks] Both modalities show staged emergence: phonemic structure first, then lexical, then syntactic, mirroring child language-acquisition order but needing 2-4 orders of magnitude more data [orhan-etal-2026-emergence-of-phonemic-syntactic-and-semantic-representations-in-artificial-neural-networks] On Llama-2-13B the syntactic and semantic subspaces are near-orthogonal (mean-squared cosine of principal angles 0.038-0.042), so they are distinct low-dimensional subspaces [orhan-etal-2026-emergence-of-phonemic-syntactic-and-semantic-representations-in-artificial-neural-networks] A contrastive topology-preserving probe confirms the distance-based probe validity [orhan-etal-2026-emergence-of-phonemic-syntactic-and-semantic-representations-in-artificial-neural-networks]

models: Llama-2-13B · method: Structural probing (metric/distance probe)
Dual-Stance Evaluation of Sycophancy: The Structure of Agreement and the Limits of Intervention (2026)measured

Sycophantic and factual agreement occupy distinct but inseparable subspaces

Details

Buchan shows a centroid-difference sycophancy-reduction steering vector in Llama-3-8B-Instruct is non-specific, also suppressing agreement with correct facts (20% drop on the Earth is round) [buchan-2026-dual-stance-evaluation-of-sycophancy] Susceptibility is continuously predictable from a behavioral dual-stance consistency measure (r=0.88 in-sample, 0.84 held-out) [buchan-2026-dual-stance-evaluation-of-sycophancy] Grassmann analysis shows sycophantic- and factual-agreement activations occupy geometrically distinct subspaces (Grassmann similarity 0.15-0.20 vs 0.32 random; principal angles 65-89 degrees) [buchan-2026-dual-stance-evaluation-of-sycophancy] Yet the steering direction projects nearly equally onto both subspaces (ratio 0.90-0.97), so the model distinguishes the two agreement types while the linear intervention cannot [buchan-2026-dual-stance-evaluation-of-sycophancy]

models: Llama-3-8B-Instruct · method: Activation Steering (Addition)
Task Recognition and Task Learning Heads Align In-Context Hidden States with a Label-Unembedding Task Subspace (2026)measured

ICL task-recognition and task-learning heads project onto a label subspace

Details

Yang, Cho & Inoue introduce Task Subspace Logit Attribution, scoring each attention head output by projection onto the span of label-token unembedding vectors [yang-cho-inoue-2026-task-subspace-attention-heads] Task-recognition heads align with the task subspace while task-learning heads rotate the hidden state toward the correct-minus-incorrect label direction within it [yang-cho-inoue-2026-task-subspace-attention-heads] Across Llama3-8B layers, hidden-update vs TR-head subspace alignment correlates rho=0.94, and logit-difference vs TL-head alignment rho=0.53 [yang-cho-inoue-2026-task-subspace-attention-heads] Injecting the summed top-3% TR-head outputs into zero-shot prompts raises classification accuracy 9.2% to 40.4% (TR-based), while TL-based vectors dominate open-ended generation; ablating each collapses its own role, tested across 6 models [yang-cho-inoue-2026-task-subspace-attention-heads]

models: Llama-3-8B, Llama-3.1-8B, Llama-3.2-3B · method: Direct Logit Attribution (DLA), Activation patching
Linear Relational Decoding of Morphology in Language Models (2025)measured

Morphological relations are truly linear; semantic ones need an affine bias

Details

Xia & Kalita decompose Hernandez et al. affine LRE into a multiplicative Jacobian-only (linear) term and an additive bias term, applying both to 40 BATS relation categories on GPT-J and Llama-2-7B [xia-kalita-2025-linear-relational-decoding-of-morphology] Across 14 morphology relations the bias-free linear LRE reaches 90% top-1 faithfulness, near the full affine LRE 95% [xia-kalita-2025-linear-relational-decoding-of-morphology] For semantic/encyclopedic relations the linear LRE drops to 40% versus the affine LRE 61%, and additive-only approximators fail on morphology, confirming the Jacobian term is necessary [xia-kalita-2025-linear-relational-decoding-of-morphology] The dissociation replicates across architecture (GPT-J vs Llama-2) and 8 typologically diverse languages; no causal intervention is performed [xia-kalita-2025-linear-relational-decoding-of-morphology]

models: Llama-2-7B · method: Linear Relational Embedding (LRE)
Language Lives in Sparse Dimensions: Toward Interpretable and Efficient Multilingual Control for Large Language Models (2025)measured

A sparse axis-aligned dimension set causally controls output language

Details

Zhong et al. hypothesize the English-centric cross-lingual transition is governed by a small, layer-consistent set of dimensions, identified from as few as ~50 sentences by comparing corpus-mean activations [zhong-etal-2025-language-lives-in-sparse-dimensions] Keeping the top-400 dimensions (~8-10% of hidden size), cross-language overlap tracks typological similarity (Chinese/Japanese share 193/400) and the monolingual and parallel methods agree 77.6% [zhong-etal-2025-language-lives-in-sparse-dimensions] Overwriting only those dimensions at one intermediate layer with a scaled target-language mean switches the output language while preserving semantic content (BLEU) across Llama-2/3.1 and Aya23-8B [zhong-etal-2025-language-lives-in-sparse-dimensions] It outperforms neuron-level baselines by up to 12.69 points at much lower data/compute cost, holding across most intermediate layers [zhong-etal-2025-language-lives-in-sparse-dimensions]

models: Llama-2-7B, Llama-2-13B, Llama-3.1-8B · method: Sparse dimension identification (axis-aligned language-specific dimensions)

Qwen

Semantic Structure of Feature Space in Large Language Models (2026)measured

LLM semantic-axis directions reproduce human semantic-differential structure

Details

Kozlowski & Boutyline construct 32 semantic-axis directions (e.g. beautiful-ugly, soft-hard) as mean contrastive-pair differences from residual streams of Llama-3.2-3B, Llama-3.1-70B, Qwen3-1.7B and Qwen3-32B [kozlowski-boutyline-2026-semantic-feature-geometry] Word projections onto these axes correlate with a 1,750-respondent human semantic-differential survey (Pearson r from >0.8 down to >0.3 across the 32 scales) [kozlowski-boutyline-2026-semantic-feature-geometry] The 32 axes are meaningfully non-orthogonal: their pairwise cosine similarities reproduce the pairwise correlations among the corresponding human survey scales [kozlowski-boutyline-2026-semantic-feature-geometry] PCA puts >45% (3B) / >33% (70B) of variance in the top 3 components, far above the 3.1% expected under orthogonality, matching the Evaluation-Potency-Activity triad [kozlowski-boutyline-2026-semantic-feature-geometry] Canonical correlation analysis aligns this LLM-derived 3D subspace with the equivalent subspace from the human survey [kozlowski-boutyline-2026-semantic-feature-geometry] Additive steering along one axis produces spillover on off-target axes proportional to their cosine similarity, replicated (weaker) in the 70B and both Qwen3 sizes [kozlowski-boutyline-2026-semantic-feature-geometry]

models: Qwen3-1.7B, Qwen3-32B · method: PCA, Activation Steering (Addition)
Death by a Thousand Directions: Exploring the Geometry of Harmfulness in LLMs through Subconcept Probing (2025)measured

55 harmfulness-subconcept probes collapse to one dominant direction

Details

Shah et al. train 55 per-subconcept logistic-regression probes (racial hate, weapons, employment scams, etc.) on attention-output states of Llama-3.1-8B-Instruct, replicated on Qwen2-7B-Instruct, each ~0.90 mean accuracy [shah-etal-2025-harmfulness-subconcept-geometry] Stacking the 55 weight vectors, an SVD-based effective rank is K=1 at variance threshold 0.95 for all but the second-to-last layer, so 55 subconcepts share almost one direction [shah-etal-2025-harmfulness-subconcept-geometry] K-means on the weight vectors barely matches the dataset's own category taxonomy (mean Adjusted Rand Index ~3e-4), so the shared direction is not re-deriving the taxonomy [shah-etal-2025-harmfulness-subconcept-geometry] Ablating only the dominant direction matches full-subspace ablation on JailbreakBench safe-rate (~0.91) while raising utility, and steering along it cuts AutoDAN attack success 0.94->0.50 for Llama [shah-etal-2025-harmfulness-subconcept-geometry] Exact effective-rank and jailbreak figures could not be re-extracted from the source (rate-limited), so this entry is deferred for full re-verification, though scope and models are confirmed [shah-etal-2025-harmfulness-subconcept-geometry]

models: Qwen2-7B-Instruct · method: Linear probing, Causal interventions (steering), Activation Steering (Addition)
The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models (2026)measured

An 'Assistant Axis' persona component causally reduces jailbreaks

Details

Lu et al. extract activation vectors for 275 character archetypes across Gemma-2-27B, Qwen3-32B and Llama-3.3-70B and run PCA on the persona space (4-19 PCs explain 70% of variance) [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models] PC1 is an interpretable "Assistant Axis" whose role loadings correlate >0.92 pairwise across the three models [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models] A contrast-based Assistant Axis (mean role-playing minus default-Assistant) aligns with PC1 at cosine >0.71 in all three models [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models] Activation-capping the Assistant-Axis projection cuts persona-jailbreak harmful-response rate ~60% with no measurable loss on IFEval/MMLU-Pro/GSM8k/EQ-Bench [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models] The axis persists in base (non-instruction-tuned) Gemma-2-27B, evidencing a pretraining origin [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models]

models: Qwen3-32B · method: Difference-in-means direction, PCA, Activation Steering (Addition)
The Granularity Axis: A Micro-to-Macro Latent Direction for Social Roles in Language Models (2026)measured

A role-granularity axis aligns with PC1 and steers output detail

Details

Qin et al. build 75 ordered social roles (5 granularity levels x 15 roles) and 91,200 role-conditioned responses, defining a macro-minus-micro contrast direction [qin-etal-2026-the-granularity-axis-a-micro-to-macro-latent-direction-for-social-roles-in-language-models] This Granularity Axis aligns with PC1 of the role-representation space at cosine 0.972 (Qwen3-8B) and 0.9596 (Llama-3.1-8B-Instruct), explaining 52.6%/42.5% of variance [qin-etal-2026-the-granularity-axis-a-micro-to-macro-latent-direction-for-social-roles-in-language-models] Projections are monotonic in granularity level (Spearman/Pearson >0.93 in both models) [qin-etal-2026-the-granularity-axis-a-micro-to-macro-latent-direction-for-social-roles-in-language-models] Adding the axis at layer 18 shifts judge-rated output granularity in the predicted direction (Qwen3-8B 2.00->2.67; Llama 2.00->3.17), null for random and Assistant-Axis controls [qin-etal-2026-the-granularity-axis-a-micro-to-macro-latent-direction-for-social-roles-in-language-models] Human annotators corroborate (pairwise macro-preference 0.64-0.90 across cells) [qin-etal-2026-the-granularity-axis-a-micro-to-macro-latent-direction-for-social-roles-in-language-models]

models: Qwen3-8B · method: Difference-in-means direction, PCA, Activation Steering (Addition)
Cell-Based Representation of Relational Binding in Language Models (2026)measured

A grid-organized cell subspace supports relational binding in LLMs

Details

Dai, Heinzerling & Inui (2026) use PLS-regression probing (vs PCA/ICA baselines) on frozen middle-layer activations (strongest layers 10-20) of Llama3-8B-Instruct and Qwen3-8B [dai-etal-2026-cell-based-representation-of-relational-binding-in-language-models] They find a low-dimensional linear "Cell-Based Representation" subspace organized as a grid along entity-index x relation-index axes, with near-perfect PLS fits using only 2-5 components [dai-etal-2026-cell-based-representation-of-relational-binding-in-language-models] Cross-context transfer R^2~0.8, ablation stability R^2~0.8, index-prediction R^2~0.95 across 13 discourse patterns, attribute accuracy 0.94-0.95 (beating a Hessian propositional-probe baseline) [dai-etal-2026-cell-based-representation-of-relational-binding-in-language-models] Subspace perturbation degrades attribute accuracy vs a random-subspace control, and explicit steering vectors reliably shift entity/relation indices across 5 domains [dai-etal-2026-cell-based-representation-of-relational-binding-in-language-models]

models: Qwen3-8B · method: Partial Least Squares (PLS), PCA, Direction Extraction, Causal interventions (steering), Activation patching
Representational Analysis of Binding in Language Models (2024)measured

A PCA 'Ordering ID' direction localizes and steers binding

Details

Dai, Heinzerling & Inui (2024) use PCA on entity/attribute activations to find a low-rank "OI subspace" whose leading component (OI-PC) encodes the "Ordering ID" underlying Feng & Steinhardt's Binding-ID mechanism, not previously localized to a direction [dai-etal-2024-representational-analysis-of-binding-in-language-models] Steering along OI-PC by n unit steps shifts the predicted attribute to that of Binding-ID n, quantified via Logit Difference and Logit Flip metrics that rise monotonically with step count [dai-etal-2024-representational-analysis-of-binding-in-language-models] A Spearman correlation confirms OI-PC tracks the true Order-ID rather than a position/filler-length confound [dai-etal-2024-representational-analysis-of-binding-in-language-models] The OI subspace is prevalent across five base models: Llama2-7B, Llama3-8B, Qwen1.5-7B, Pythia-6.9B, and the code-fine-tuned Float-7B [dai-etal-2024-representational-analysis-of-binding-in-language-models]

models: Qwen1.5-7B · method: PCA, Direction Extraction, Causal interventions (steering)
Tool Calling Is Linearly Readable and Steerable in Language Models (2026)measured

A per-tool-pair mean-difference direction reads and switches tool choice

Details

Wu et al. show tool selection in tool-calling agents is carried by a single mean-difference direction per tool-pair in the residual stream across Gemma 3 (270M-27B), Qwen 3 (0.6B-14B) and Llama 3.1 (8B) [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models] Adding the direction switches the chosen tool at 83-100% on a 15-tool synthetic benchmark and 77-94% on tau-bench-airline, versus 0% for a random-direction control [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models] PCA over per-tool mean activations puts ~91% of variance in ~10 components for 15 tools, far below a random-Gaussian control [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models] Base (non-instruction-tuned) models already carry the correct tool internally (cosine readout 61-82% on BFCL vs 2-10% from base generation) [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models] SAEs and cross-layer transcoders trace a three-stage circuit: early tool-selective features, mid-layer attention heads, late-layer JSON-formatting features [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models]

models: Qwen3-4B-Instruct, Qwen2.5-7B-Instruct · method: Difference-in-means direction, Activation Steering (Addition), PCA, Activation patching, Sparse Autoencoders (SAE), Circuit tracing (attribution graphs)
The Confidence Manifold: Geometric Structure of Correctness Representations in Language Models (2026)measured

Answer-correctness sits in a 3-8D linear subspace, causally steerable

Details

Cho et al. study self-assessed correctness on TruthfulQA pairs across 9 models spanning 5 families, finding the discriminative signal is a low-dimensional linear subspace, not a curved manifold [cho-etal-2026-confidence-manifold] A PLS sweep peaks at 3-8 dimensions (e.g. peak AUC 0.90 at layer 23, dim 5, Mistral-7B), and nonlinear classifiers give no gain over a linear boundary [cho-etal-2026-confidence-manifold] Correct/incorrect classes form roughly Gaussian clusters separated by a mean shift: a two-mean centroid detector matches the linear probe (0.90 vs 0.89 AUC), robust with 25 examples/class [cho-etal-2026-confidence-manifold] This 3-8D discriminative subspace is lower-dimensional than the representation's 8-12D intrinsic dimension at the same layers [cho-etal-2026-confidence-manifold] Adding the learned direction shifts downstream error rate up to 10.9 points dose-dependently, while random and orthogonal controls have no reliable effect [cho-etal-2026-confidence-manifold]

models: Qwen2-1.5B-Instruct, Qwen2-7B-Instruct · method: Linear probing, Partial Least Squares (PLS), Difference-in-means direction, Activation Steering (Addition)
Can LLMs Learn to Map the World from Local Descriptions? (2025)measured

Local-only spatial training yields a recoverable global coordinate map

Details

Xia et al. continually pretrain Qwen2.5-0.5B on synthetic relational and trajectory descriptions of a simulated 100x100 city that only ever describe local relationships between nearby points [xia-etal-2025-can-llms-learn-to-map-the-world] A probe on last hidden states recovers absolute (x,y) coordinates at R^2=1.00 (vs -0.01/-0.10 untrained), an emergent global coordinate map from purely local signal [xia-etal-2025-can-llms-learn-to-map-the-world] Independently of the probe, latent-vector distances and angles show high Spearman/Pearson correlation with true geographic distances and angles [xia-etal-2025-can-llms-learn-to-map-the-world] A compositional probe recovers pairwise distance (MAE 0.85) and azimuth (3.49 degrees) at R^2 1.00/0.98; no activation-level intervention is performed [xia-etal-2025-can-llms-learn-to-map-the-world]

models: Qwen2.5-0.5B · method: Geometric analysis
Scenario-based Probing and Steering Cultural Values in Large Language Models (2026)measured

Cultural-value steering axes are entangled, mirroring real value correlations

Details

Dang, Kieu & Masud map LLMs onto the Inglehart-Welzel cultural map via scenario forced-choice probing and diff-of-means steering at 4 tuned layers per model [dang-etal-2026-scenario-based-probing-and-steering-cultural-values-in-llms] The steering direction is a signed bidirectional axis (negative alpha reverses the shift), saturating at alpha in [0.5, 0.6] [dang-etal-2026-scenario-based-probing-and-steering-cultural-values-in-llms] Steering one cultural axis causes correlated shifts on the other, an entanglement ratio 0.72-0.81 (dropping to 0.53 for Llama under hybrid steering), mirroring the real-world WVS axis correlation r=0.474 [dang-etal-2026-scenario-based-probing-and-steering-cultural-values-in-llms] Capability cost measured via Global-MMLU is largest for Gemma (0.608->0.530) [dang-etal-2026-scenario-based-probing-and-steering-cultural-values-in-llms]

models: Qwen3-4B-Instruct · method: Difference-in-means direction, Direction Extraction, Causal interventions (steering)
Language Models Compare Quantities Using Number-specific and Unit-specific Heuristics (2026)measured

DAS finds number- and unit-specific quantity-comparison subspaces

Details

Sasaki et al. use Distributed Alignment Search (1024 intervention dimensions, 512 per variable) to locate linear subspaces aligned with a NumLogDiff and a UnitLogDiff variable in quantity-comparison activations [sasaki-etal-2026-language-models-compare-quantities-using-number-specific-and-unit-specific-heuristics] Interchange Intervention Accuracy exceeds 0.9 in many middle layers versus near-chance baselines [sasaki-etal-2026-language-models-compare-quantities-using-number-specific-and-unit-specific-heuristics] A linear surrogate over these variables predicts the model's comparison preference at R^2 approximately 0.817 overall (0.763 near the decision boundary) for Qwen3-4B-Base [sasaki-etal-2026-language-models-compare-quantities-using-number-specific-and-unit-specific-heuristics] This supports a "bag of number-specific and unit-specific heuristics" account over exact-scale unit conversion [sasaki-etal-2026-language-models-compare-quantities-using-number-specific-and-unit-specific-heuristics]

models: Qwen3-4B, Qwen3-8B · method: Distributed Alignment Search (DAS), Causal interventions (steering)
The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspaces (2024)measured

PLS-found numeric-comparison subspaces causally control LLM answers

Details

El-Shangiti et al. show LLMs answering numeric-comparison questions encode each entity's numeric attribute in a low-dimensional linear subspace, found via a 5-component PLS regression on the last-token state [el-shangiti-etal-2024-geometry-of-numerical-reasoning] The fit reaches R^2 > 0.8 for birth year, death year and latitude across Llama-3-8B-Instruct, Mistral-7B-Instruct and Qwen2.5-7B-Instruct [el-shangiti-etal-2024-geometry-of-numerical-reasoning] Intervening along the first PLS component flips the model's Yes/No comparison answer far more than an equal-magnitude random control, concentrated in roughly the first half of layers [el-shangiti-etal-2024-geometry-of-numerical-reasoning]

models: Qwen2.5-7B-Instruct · method: Partial least squares (PLS) subspace probing
Characterizing Linear Alignment Across Language Models (2026)measured

Independently-trained embedding models share linear structure (CKA 0.60-0.88)

Details

Gorbett & Jana measure linear CKA across five independently-trained embedding models (OpenAI text-embedding-3-small, Cohere embed, Gemini embedding-001, Qwen3-Embedding-8B, e5-mistral-7b-instruct), finding similarity 0.595-0.881 [gorbett-jana-2026-characterizing-linear-alignment-across-language-models] In a separate experiment they fit learned affine maps between instruction-tuned generative LMs (Llama-3-8B, Qwen2.5-7B/14B, Llama-3.2-1B, OLMo-7B and others) [gorbett-jana-2026-characterizing-linear-alignment-across-language-models] These maps preserve classification accuracy and OOD-AUROC across model pairs (e.g. 94.5%->93.1%) [gorbett-jana-2026-characterizing-linear-alignment-across-language-models] They enable zero-shot cross-model text generation via a frozen target head, with generation quality correlating with tokenizer exact-match rate (r=0.898) and vocabulary Jaccard overlap (r=0.822) [gorbett-jana-2026-characterizing-linear-alignment-across-language-models]

models: Qwen3-Embedding-8B, Qwen2.5-7B-Instruct, Qwen2.5-14B-Instruct · method: Centered Kernel Alignment (CKA), Singular Vector Canonical Correlation Analysis (SVCCA)
Geometric Asymmetry in MoE Specialization: Functional Decorrelation and Representational Overlap (2026)measured

MoE experts are functionally decorrelated but only partly subspace-separated

Details

Liu measures two pretrained sparse MoE models, Mixtral-8x7B (8 experts/layer, top-2) and Qwen1.5-MoE-A2.7B (60 experts/layer, top-4) [liu-2026-geometric-asymmetry-moe-specialization] Cross-expert Jacobian cosine similarity clusters near zero (Mistral middle-layer mean 0.062; Qwen ~0.000-0.001), showing experts are strongly functionally decorrelated [liu-2026-geometric-asymmetry-moe-specialization] Yet their top-5-PCA subspaces sit at Grassmannian distances (2.06-2.69) well below the theoretical maximum ~3.51, so subspaces are distinct but only partially separated [liu-2026-geometric-asymmetry-moe-specialization] A from-scratch 8-expert Transformer isolates routing's causal role: mean Grassmann distance is 2.463 under top-k routing vs 0.480 under fully-soft routing [liu-2026-geometric-asymmetry-moe-specialization]

models: Qwen1.5-MoE-A2.7B · method: Cross-expert Jacobian alignment, Subspace-angle alignment
Revealing Emergent Human-like Conceptual Representations from Language Prediction (2025)measured

In-context concept representations converge toward a human-aligned structure

Details

Xu et al. probe LLM (mainly LLaMA3-70B) concepts via an in-context reverse-dictionary task, characterizing each context's representation by its pairwise-similarity matrix [xu-etal-2025-emergent-conceptual-representations] RSA alignment across contexts rises from 0.800 at 1 demonstration to 0.970 at 24, converging toward a single context-independent relational structure [xu-etal-2025-emergent-conceptual-representations] Alignment with the 120-demonstration structure correlates with reverse-dictionary accuracy at rho=0.976, and cross-model alignment across 67 LLMs predicts task performance (rho=0.870), all correlational with no intervention [xu-etal-2025-emergent-conceptual-representations] The convergent structure aligns via RSA with human similarity judgments (SimLex-999 rho=0.776), THINGS odd-one-out, and voxel-wise fMRI activity in LOC/FFA/PPA and other regions [xu-etal-2025-emergent-conceptual-representations]

models: Qwen2-0.5B · method: Representational Similarity Analysis (RSA)
Invariant Reasoning Directions in Latent Trajectories of Language Models (2026)measured

An invariant reasoning subspace stabilizes latent chain-of-thought refinement

Details

Malarkkan et al. introduce TILR: SVD of per-step contrastive differences between a later- and earlier-checkpoint of the same GPT-2-base Coconut latent-reasoning backbone, keeping the rank r_0.90 explaining >=90% of contrastive variance [malarkkan-etal-2026-invariant-reasoning-directions-latent-trajectories] Across 6 benchmarks r_0.90 ranges from 1 (near-rank-one, coincident with input-PCA) to 27-34 (genuinely refinement-specific structure surviving input-PCA regression) [malarkkan-etal-2026-invariant-reasoning-directions-latent-trajectories] At inference only the projection onto this fixed subspace is applied, scaled by a norm-based reliability gate giving a formal no-harm guarantee [malarkkan-etal-2026-invariant-reasoning-directions-latent-trajectories] Against random, input-PCA, trajectory-PCA and orthogonal-knockout controls it improves accuracy on all 6 benchmarks (mean +2.1%), cuts paraphrase-induced trajectory variance ~39%, and reduces cross-checkpoint variance 53-74%, replicated on GPT-2-Medium and Qwen2.5-Math-1.5B [malarkkan-etal-2026-invariant-reasoning-directions-latent-trajectories]

models: Qwen2.5-Math-1.5B · method: Trajectory-Invariant Latent Refinement (TILR), Causal interventions (steering)
Neural Chameleons: Language Models Can Learn to Hide Their Thoughts from Activation Monitors (2025)measured

Monitor-evasion fine-tuning generalizes zero-shot to unseen safety monitors

Details

McGuinness et al. fine-tune LLMs to evade activation-monitor probes for benign concepts when conditioned on a "You are being probed for {concept}" trigger [mcguinness-etal-2025-neural-chameleons] The learned evasion generalizes zero-shot: substituting a safety-relevant term like "deception" evades previously-unseen safety monitors, even ones trained post hoc on the model's frozen weights [mcguinness-etal-2025-neural-chameleons] The effect holds across Llama, Gemma and Qwen families, is highly selective to the triggered concept, and has modest capability impact [mcguinness-etal-2025-neural-chameleons] Observational PCA (no causal intervention) on Gemma-2-9b-it indicates the evasion correlates with relocation of activations into a low-dimensional subspace; monitor ensembles and nonlinear classifiers are more resilient [mcguinness-etal-2025-neural-chameleons]

models: Qwen2.5-7B-Instruct · method: Subspace-evasion fine-tuning
The Representational Geometry of Number (2026)measured

Number-task subspaces are distinct yet linearly inter-transformable

Details

Hu, Niu & Varma analyze number-concept representations (comparison, parity, primality, successor/predecessor) across BERT, GPT-2, Qwen2.5-7B and Qwen2.5-Math-7B at ~75% depth [hu-etal-2026-the-representational-geometry-of-number] Task-specific subspaces are distinct via Procrustes disparity (~0.010 vs a permutation baseline 0.077-0.273) yet linearly inter-transformable (SVCCA canonical correlations ~0.80-0.90) [hu-etal-2026-the-representational-geometry-of-number] The parity axis sits nearly orthogonal to the magnitude axis (88.8 degrees) while the primality axis is only partly decoupled (68.8 degrees) [hu-etal-2026-the-representational-geometry-of-number] Subspace overlap between most task pairs stays below 0.2, except the closely-related successor/predecessor pair (0.36-0.42) [hu-etal-2026-the-representational-geometry-of-number]

models: Qwen-2.5-7B, Qwen2.5-Math-7B · method: Generalized Procrustes Analysis (GPA), Singular Vector Canonical Correlation Analysis (SVCCA), Direction Extraction, Linear probing, Geometric analysis
Understanding Subword Compositionality of Large Language Models (2025)measured

Composed-subword vs whole-word spaces show family-dependent isometry

Details

Peng, Chai & Sogaard fit an orthogonal Procrustes map between composed-subword and whole-word embedding spaces across instruction-tuned LLMs, scored by Precision@1 retrieval [peng-chai-sogaard-2025-subword-compositionality] Simple addition consistently outperforms other subword-composition operations [peng-chai-sogaard-2025-subword-compositionality] Three family patterns emerge: Aya-expanse and Gemma show high composed-vs-whole-word isometry, Llama 3/3.1 very little, others moderate but dropping late in the network [peng-chai-sogaard-2025-subword-compositionality] The analysis is purely observational with no causal intervention [peng-chai-sogaard-2025-subword-compositionality]

models: Qwen2.5-7B-Instruct · method: Orthogonal Procrustes alignment, Linear probing
Language Models Use Lookbacks to Track Beliefs (2025)measured

LLMs bind belief triples via Ordering-ID subspaces localized by patching

Details

Prakash et al. show Qwen2.5-14B-Instruct, Llama-3-70B-Instruct and Llama-3.1-405B-Instruct bind character-object-state triples by co-locating reference information as low-rank Ordering IDs in the state-token residual stream, plus a visibility ID for theory-of-mind [prakash-etal-2025-language-models-use-lookbacks-to-track-beliefs] Interchange-intervention (activation patching) localizes visibility source-reference alignment to layers 10-23 and the strongest binding-lookback alignment to layers 33-38 [prakash-etal-2025-language-models-use-lookbacks-to-track-beliefs] Answer-payload information is present only after layer 31, consistent across all 80 test cases [prakash-etal-2025-language-models-use-lookbacks-to-track-beliefs] The lookback mechanism duplicates reference information into an address and a pointer, retrieved later via attention [prakash-etal-2025-language-models-use-lookbacks-to-track-beliefs]

models: Qwen2.5-14B-Instruct · method: Activation patching
Geometric Factual Recall in Transformers (2026)measured

Factual recall is linear superposition with a proven dimension bound

Details

Ravfogel et al. model subject embeddings as a sum of per-attribute vectors read by a relation-conditioned ReLU-gated MLP, proving (Theorem 4.1) a 1-layer transformer plus 3-layer MLP solves single-hop recall for N entities and R relations when d=4R*log(N)+1 [ravfogel-etal-2026-geometric-factual-recall-transformers] Synthetic experiments confirm the scaling: trainable embeddings memorize once d>=128 while frozen embeddings need d>=512 for R=16 [ravfogel-etal-2026-geometric-factual-recall-transformers] On five real LMs a rank-512 affine probe recovers the LM-head output embedding (best-layer MRR 0.44-0.69 across entity categories), reversing Hernandez et al. reported output-side non-linearity [ravfogel-etal-2026-geometric-factual-recall-transformers] Minimum-norm subject-embedding perturbations swap the queried attribute with high selectivity, and a frozen relation-selector MLP transfers zero-shot to unseen relation bijections [ravfogel-etal-2026-geometric-factual-recall-transformers]

models: Qwen2.5-0.5B, Qwen3-14B · method: Linear Relational Embedding (LRE), Linear probing
Understanding and Preserving Safety in Fine-Tuned LLMs (2026)measured

Safety gradients occupy a low-rank subspace orthogonal to utility

Details

Zhang et al. SVD per-layer safety- versus utility-task gradients during fine-tuning, finding safety gradients occupy a compact low-rank subspace with sharp singular-value decay while utility spans much higher dimension [zhang-etal-2026-understanding-and-preserving-safety-in-fine-tuned-llms] Safety and utility gradient directions have cosine similarity oscillating around zero and often negative (directional conflict) [zhang-etal-2026-understanding-and-preserving-safety-in-fine-tuned-llms] Their Safety-Preserving Fine-tuning projects utility gradients onto the safety subspace orthogonal complement during training [zhang-etal-2026-understanding-and-preserving-safety-in-fine-tuned-llms] This cuts attack success rate 0.955 to 0.019 (Llama-3.1-8B), 0.985 to 0.240 (Mistral) and 0.988 to 0.124 (Qwen2.5-7B) while preserving MMLU, robust to deep fine-tuning and multiple jailbreaks [zhang-etal-2026-understanding-and-preserving-safety-in-fine-tuned-llms]

models: Qwen2.5-7B-Instruct · method: PCA
Linear Representations of Hierarchical Concepts in Language Models (2026)measured

Hierarchical concepts occupy domain-specific 150-250D subspaces, steerable

Details

Sakata et al. extend Linear Relational Concepts into a depth- and domain-specific Linear Hierarchical Encoding fit from intermediate hidden states of Llama-3.2-3B, Llama-3.1-8B, Qwen3-8B and Qwen3-14B [sakata-etal-2026-linear-hierarchical-concepts] Hierarchical information is encoded in relatively low-dimensional subspaces (150-250 dimensions for hidden size 3000-5000) [sakata-etal-2026-linear-hierarchical-concepts] The relevant subspace is domain-specific, and domain-specific subspaces show similar hierarchical structure across domains [sakata-etal-2026-linear-hierarchical-concepts] Editing a child representation by adding a scaled concept-direction difference at every layer flips next-token prediction toward the target parent (causality up to 0.93 for Organization/Llama-3.1-8B) [sakata-etal-2026-linear-hierarchical-concepts]

models: Qwen3-8B, Qwen3-14B · method: Linear relational concepts (LRC), Causal interventions (steering)
Task Recognition and Task Learning Heads Align In-Context Hidden States with a Label-Unembedding Task Subspace (2026)measured

ICL task-recognition and task-learning heads project onto a label subspace

Details

Yang, Cho & Inoue introduce Task Subspace Logit Attribution, scoring each attention head output by projection onto the span of label-token unembedding vectors [yang-cho-inoue-2026-task-subspace-attention-heads] Task-recognition heads align with the task subspace while task-learning heads rotate the hidden state toward the correct-minus-incorrect label direction within it [yang-cho-inoue-2026-task-subspace-attention-heads] Across Llama3-8B layers, hidden-update vs TR-head subspace alignment correlates rho=0.94, and logit-difference vs TL-head alignment rho=0.53 [yang-cho-inoue-2026-task-subspace-attention-heads] Injecting the summed top-3% TR-head outputs into zero-shot prompts raises classification accuracy 9.2% to 40.4% (TR-based), while TL-based vectors dominate open-ended generation; ablating each collapses its own role, tested across 6 models [yang-cho-inoue-2026-task-subspace-attention-heads]

models: Qwen2-7B (base), Qwen2.5-32B · method: Direct Logit Attribution (DLA), Activation patching
Uncovering and Shaping the Latent Representation of 3D Scene Topology in Vision-Language Models (2026)measured

SVD-based cross-scene averaging isolates a linear 3D-position subspace in real VLMs

Details

Wang & Gao (2026) posit an additive decomposition per object-token, h = u_id + u_sp + noise, in real Qwen2.5-VL-7B and InternVL3-8B; averaging an object's activations across many randomly-positioned synthetic 3D scenes cancels the position term, isolating an identity basis via SVD, whose orthogonal complement recovers a spatial subspace [wang-gao-2026-3d-scene-topology-in-vlms] PCA of the spatial-extracted residual recovers a 3D geometry matching true scene layout, formally converging to Laplacian eigenmaps of the scene graph [wang-gao-2026-3d-scene-topology-in-vlms] Injecting a probe-derived direction into layer-12 residuals of Qwen2.5-VL-7B causally shifts the x-coordinate probe readout monotonically with injection strength (alpha=+0.30: delta-x-hat=+0.091+/-0.021 vs. control +0.001+/-0.030), while a null-direction control shows no effect [wang-gao-2026-3d-scene-topology-in-vlms]

models: Qwen2.5-VL 7B Instruct · method: SVD, Activation patching

Gemma

Beyond I'm Sorry, I Can't: Dissecting Large Language Model Refusal (2026)measured

Refusal is a causally-minimal SAE-latent set with hydra redundancy

Details

Prakash et al. (AAAI 2026) extend single-direction refusal ablation into a multi-latent SAE decomposition on Gemma-2-2B-IT and Llama-3.1-8B-IT using Gemma Scope / LlamaScope [prakash-etal-2026-dissecting-llm-refusal-sae-feature-sets] They identify causally-minimal sets of SAE latents near the refusal direction whose joint ablation reduces refusal (thousands of candidate features for Gemma, ~110 for Llama, narrowed at later stages) [prakash-etal-2026-dissecting-llm-refusal-sae-feature-sets] Ablating the active refusal-latent set triggers a hydra effect: ~74% of previously-dormant redundant features (Gemma) activate on system-prompt tokens instead, ~97% on the begin-of-text token [prakash-etal-2026-dissecting-llm-refusal-sae-feature-sets] Refusal is thus a redundant multi-dimensional causal subspace rather than a single vector [prakash-etal-2026-dissecting-llm-refusal-sae-feature-sets] Ablating the identified latent set shifts attack success rate 4%->33% (Gemma) and 71%->57% (Llama) across stages, with hydra redundancy explaining incomplete restoration [prakash-etal-2026-dissecting-llm-refusal-sae-feature-sets]

models: Gemma-2-2B-it · method: Sparse Autoencoders (SAE), Causal interventions (steering)
The Assistant Axis: Situating and Stabilizing the Default Persona of Language Models (2026)measured

An 'Assistant Axis' persona component causally reduces jailbreaks

Details

Lu et al. extract activation vectors for 275 character archetypes across Gemma-2-27B, Qwen3-32B and Llama-3.3-70B and run PCA on the persona space (4-19 PCs explain 70% of variance) [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models] PC1 is an interpretable "Assistant Axis" whose role loadings correlate >0.92 pairwise across the three models [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models] A contrast-based Assistant Axis (mean role-playing minus default-Assistant) aligns with PC1 at cosine >0.71 in all three models [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models] Activation-capping the Assistant-Axis projection cuts persona-jailbreak harmful-response rate ~60% with no measurable loss on IFEval/MMLU-Pro/GSM8k/EQ-Bench [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models] The axis persists in base (non-instruction-tuned) Gemma-2-27B, evidencing a pretraining origin [lu-etal-2026-the-assistant-axis-situating-and-stabilizing-the-default-persona-of-language-models]

models: Gemma 2 27B Instruct, Gemma-2-27B · method: Difference-in-means direction, PCA, Activation Steering (Addition)
Tool Calling Is Linearly Readable and Steerable in Language Models (2026)measured

A per-tool-pair mean-difference direction reads and switches tool choice

Details

Wu et al. show tool selection in tool-calling agents is carried by a single mean-difference direction per tool-pair in the residual stream across Gemma 3 (270M-27B), Qwen 3 (0.6B-14B) and Llama 3.1 (8B) [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models] Adding the direction switches the chosen tool at 83-100% on a 15-tool synthetic benchmark and 77-94% on tau-bench-airline, versus 0% for a random-direction control [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models] PCA over per-tool mean activations puts ~91% of variance in ~10 components for 15 tools, far below a random-Gaussian control [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models] Base (non-instruction-tuned) models already carry the correct tool internally (cosine readout 61-82% on BFCL vs 2-10% from base generation) [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models] SAEs and cross-layer transcoders trace a three-stage circuit: early tool-selective features, mid-layer attention heads, late-layer JSON-formatting features [wu-etal-2026-tool-calling-is-linearly-readable-and-steerable-in-language-models]

models: Gemma 3 4B Instruct, Gemma 3 4B, Gemma 3 27B Instruct · method: Difference-in-means direction, Activation Steering (Addition), PCA, Activation patching, Sparse Autoencoders (SAE), Circuit tracing (attribution graphs)
The Confidence Manifold: Geometric Structure of Correctness Representations in Language Models (2026)measured

Answer-correctness sits in a 3-8D linear subspace, causally steerable

Details

Cho et al. study self-assessed correctness on TruthfulQA pairs across 9 models spanning 5 families, finding the discriminative signal is a low-dimensional linear subspace, not a curved manifold [cho-etal-2026-confidence-manifold] A PLS sweep peaks at 3-8 dimensions (e.g. peak AUC 0.90 at layer 23, dim 5, Mistral-7B), and nonlinear classifiers give no gain over a linear boundary [cho-etal-2026-confidence-manifold] Correct/incorrect classes form roughly Gaussian clusters separated by a mean shift: a two-mean centroid detector matches the linear probe (0.90 vs 0.89 AUC), robust with 25 examples/class [cho-etal-2026-confidence-manifold] This 3-8D discriminative subspace is lower-dimensional than the representation's 8-12D intrinsic dimension at the same layers [cho-etal-2026-confidence-manifold] Adding the learned direction shifts downstream error rate up to 10.9 points dose-dependently, while random and orthogonal controls have no reliable effect [cho-etal-2026-confidence-manifold]

models: Gemma-2-2B-it · method: Linear probing, Partial Least Squares (PLS), Difference-in-means direction, Activation Steering (Addition)
Scenario-based Probing and Steering Cultural Values in Large Language Models (2026)measured

Cultural-value steering axes are entangled, mirroring real value correlations

Details

Dang, Kieu & Masud map LLMs onto the Inglehart-Welzel cultural map via scenario forced-choice probing and diff-of-means steering at 4 tuned layers per model [dang-etal-2026-scenario-based-probing-and-steering-cultural-values-in-llms] The steering direction is a signed bidirectional axis (negative alpha reverses the shift), saturating at alpha in [0.5, 0.6] [dang-etal-2026-scenario-based-probing-and-steering-cultural-values-in-llms] Steering one cultural axis causes correlated shifts on the other, an entanglement ratio 0.72-0.81 (dropping to 0.53 for Llama under hybrid steering), mirroring the real-world WVS axis correlation r=0.474 [dang-etal-2026-scenario-based-probing-and-steering-cultural-values-in-llms] Capability cost measured via Global-MMLU is largest for Gemma (0.608->0.530) [dang-etal-2026-scenario-based-probing-and-steering-cultural-values-in-llms]

models: Gemma 3 4B Instruct · method: Difference-in-means direction, Direction Extraction, Causal interventions (steering)
Mechanistic Steering of LLMs Reveals Layer-wise Feature Vulnerabilities in Adversarial Settings (2026)measured

Concept-aligned-token SAE feature subgroups in real Gemma-2-2B-IT localize jailbreak susceptibility to mid-to-late layers

Details

Das & Gaur (2026) extract concept-aligned tokens from harmful prompts (BeaverTails, 14 harm categories) in real Gemma-2-2B-IT via cosine similarity to a ReFT-derived harm-concept subspace, then identify Gemma-Scope SAE feature subgroups for those tokens across all 26 layers using three independent grouping strategies (agglomerative clustering, hierarchical-linkage, single-token-driven) [das-gaur-2026-mechanistic-steering-of-llms-reveals-layer-wise-feature-vulnerabilities] All three grouping strategies convergently implicate layers approximately 16-25 as most steerable, and causally amplifying only the top features from an identified subgroup measurably raises an LLM-judged harmfulness score relative to baseline, per category and per layer [das-gaur-2026-mechanistic-steering-of-llms-reveals-layer-wise-feature-vulnerabilities]

models: Gemma-2-2B · method:
High-Dimensional Interlingual Representations of Large Language Models (2025)measured

Multilingual LLMs share a core interlingual subspace with language-specific fragments

Details

Wilie et al. study Aya Expanse 8B, Llama-3.1-8B and Gemma-2-9B with a kNN-based Interlingual Local Overlap (ILO) score plus t-SNE of hidden states [wilie-etal-2025-high-dimensional-interlingual-representations] Representations organize as a partially-shared core interlingual region (translation-equivalent inputs share local neighborhoods) surrounded by fragmented language-specific components that do not transfer [wilie-etal-2025-high-dimensional-interlingual-representations] Overlap varies systematically by resource level and linguistic family, with graph "bridge"/"reachability" metrics showing which languages connect the core to peripheral fragments [wilie-etal-2025-high-dimensional-interlingual-representations] The alignment is causally load-bearing: ILO overlap is preserved or disrupted depending on which layers are frozen during single-language fine-tuning [wilie-etal-2025-high-dimensional-interlingual-representations]

models: Gemma-2-9B · method: Interlingual Local Overlap (ILO)
Do Language Models Track Entities Across State Changes? (2026)measured

LLMs consolidate entity-state tracking into a local binary subspace

Details

Tang et al. train linear probes on last-token residual activations (the "boxes" dataset) to test global versus local entity-state tracking [tang-etal-2026-do-language-models-track-entities-across-state-changes] Global 8-way state probes reach only ~0.3 accuracy versus ~0.9 for local binary probes, so the model consolidates state locally rather than tracking a full global state [tang-etal-2026-do-language-models-track-entities-across-state-changes] INLP null-space projection of the state-tag direction causally changes completions [tang-etal-2026-do-language-models-track-entities-across-state-changes] A learned sparse boolean mask over the residual PCA basis ("subspace patching") reveals a shared positional-information subspace at layers 15-25 in CodeLlama-13B [tang-etal-2026-do-language-models-track-entities-across-state-changes]

models: Gemma-2-2B · method: Linear probing, Causal interventions (steering), PCA
SAIF: A Sparse Autoencoder Framework for Interpreting and Steering Instruction Following of Language Models (2025)measured

Multi-feature SAE steering vectors force instruction-following

Details

He et al. identify instruction-relevant SAE latents via a sensitivity score, then build multi-feature steering vectors from Gemma Scope / Llama Scope decoder directions [he-etal-2025-saif-a-sparse-autoencoder-framework-for-interpreting-and-steering-instruction-following-of-language-models] Adding them to the residual stream forces instruction-following (translation/summarization/keyword) at over 30% strict and up to ~0.7 loose accuracy versus near-zero for single-latent steering, optimal at k=15 latents [he-etal-2025-saif-a-sparse-autoencoder-framework-for-interpreting-and-steering-instruction-following-of-language-models] Last-layer placement is critical (Gemma-2-2b-it loose accuracy 0.64 at layer 25 drops to 0.33 by layer 24), and post-instruction positioning beats pre-instruction [he-etal-2025-saif-a-sparse-autoencoder-framework-for-interpreting-and-steering-instruction-following-of-language-models]

models: Gemma-2-2B-it, Gemma-2-9B-it · method: Sparse Autoencoders (SAE), Activation Steering (Addition)
Neural Chameleons: Language Models Can Learn to Hide Their Thoughts from Activation Monitors (2025)measured

Monitor-evasion fine-tuning generalizes zero-shot to unseen safety monitors

Details

McGuinness et al. fine-tune LLMs to evade activation-monitor probes for benign concepts when conditioned on a "You are being probed for {concept}" trigger [mcguinness-etal-2025-neural-chameleons] The learned evasion generalizes zero-shot: substituting a safety-relevant term like "deception" evades previously-unseen safety monitors, even ones trained post hoc on the model's frozen weights [mcguinness-etal-2025-neural-chameleons] The effect holds across Llama, Gemma and Qwen families, is highly selective to the triggered concept, and has modest capability impact [mcguinness-etal-2025-neural-chameleons] Observational PCA (no causal intervention) on Gemma-2-9b-it indicates the evasion correlates with relocation of activations into a low-dimensional subspace; monitor ensembles and nonlinear classifiers are more resilient [mcguinness-etal-2025-neural-chameleons]

models: Gemma-2-9B-it, Gemma-2-2B-it · method: Subspace-evasion fine-tuning
Understanding Subword Compositionality of Large Language Models (2025)measured

Composed-subword vs whole-word spaces show family-dependent isometry

Details

Peng, Chai & Sogaard fit an orthogonal Procrustes map between composed-subword and whole-word embedding spaces across instruction-tuned LLMs, scored by Precision@1 retrieval [peng-chai-sogaard-2025-subword-compositionality] Simple addition consistently outperforms other subword-composition operations [peng-chai-sogaard-2025-subword-compositionality] Three family patterns emerge: Aya-expanse and Gemma show high composed-vs-whole-word isometry, Llama 3/3.1 very little, others moderate but dropping late in the network [peng-chai-sogaard-2025-subword-compositionality] The analysis is purely observational with no causal intervention [peng-chai-sogaard-2025-subword-compositionality]

models: Gemma 2 9B IT · method: Orthogonal Procrustes alignment, Linear probing
The Cylindrical Representation Hypothesis for Language Model Steering (2026)measured

Steering directions split into a magnitude axis and interference plane

Details

Gao et al. relax the Linear Representation Hypothesis orthogonality assumption, decomposing a concept steering vector into a central axis and an orthogonal 2D normal plane spanned by the axis complement and the top PC of other concepts directions [gao-etal-2026-cylindrical-hypothesis] Despite cylindrical/phase/sector terminology there is no measured angular or periodic structure (phase = position in the 2D plane; sectors = a binary high/low-sensitivity split), so it is a linear-subspace interference decomposition, not a topology [gao-etal-2026-cylindrical-hypothesis] Steering-effect magnitude follows a predictable sin^m*cos^n form, but which interference sector a concept pair falls into is NOT predictable from the vectors (Pearson -0.034), a genuine null [gao-etal-2026-cylindrical-hypothesis] A penalty experiment attenuating the plane component trades earlier target onset against earlier corrupted output, validated by an LLM-judge at 94% human agreement [gao-etal-2026-cylindrical-hypothesis]

models: Gemma-2-2B-it · method: Difference-in-means direction, PCA, Linear probing, Activation Steering (Addition)
Scale Determines Whether Language Models Organize Representation Geometry for Prediction (2026)measured

Scale determines whether late-layer geometry stays organized by the readout

Details

Xu introduces Subspace PGA, a z-scored RSA metric comparing how well a layer cosine-distance structure survives projection onto the unembedding matrix top-k readout subspace versus 100 random subspaces [xu-2026-scale-determines-representation-geometry-organization-prediction] Across Pythia (70M-6.9B) and OLMo-1B/Phi-1.5/Gemma-2-2B, intermediate geometry is robustly organized around the readout subspace (peak z 9-24 at mid layers) [xu-2026-scale-determines-representation-geometry-organization-prediction] Small models (hidden dim <=1024) progressively lose this at late layers over training even as loss drops (Pythia-410M min z +0.6 to -32 across checkpoints), while dim>=2048 models preserve it [xu-2026-scale-determines-representation-geometry-organization-prediction] Removing a few top principal components restores positive z at every layer for dim>=768 models, supporting a masking interpretation; the paper frames the claim as correlational [xu-2026-scale-determines-representation-geometry-organization-prediction]

models: Gemma-2-2B · method: Subspace PGA (Predictive Geometric Alignment)

Pythia

Representational Analysis of Binding in Language Models (2024)measured

A PCA 'Ordering ID' direction localizes and steers binding

Details

Dai, Heinzerling & Inui (2024) use PCA on entity/attribute activations to find a low-rank "OI subspace" whose leading component (OI-PC) encodes the "Ordering ID" underlying Feng & Steinhardt's Binding-ID mechanism, not previously localized to a direction [dai-etal-2024-representational-analysis-of-binding-in-language-models] Steering along OI-PC by n unit steps shifts the predicted attribute to that of Binding-ID n, quantified via Logit Difference and Logit Flip metrics that rise monotonically with step count [dai-etal-2024-representational-analysis-of-binding-in-language-models] A Spearman correlation confirms OI-PC tracks the true Order-ID rather than a position/filler-length confound [dai-etal-2024-representational-analysis-of-binding-in-language-models] The OI subspace is prevalent across five base models: Llama2-7B, Llama3-8B, Qwen1.5-7B, Pythia-6.9B, and the code-fine-tuned Float-7B [dai-etal-2024-representational-analysis-of-binding-in-language-models]

models: Pythia-6.9B · method: PCA, Direction Extraction, Causal interventions (steering)
Summing Up the Facts: Additive Mechanisms Behind Factual Recall in LLMs (2024)measured

Factual recall is an additive motif across component types

Details

Chughtai, Cooney & Nanda use Direct Logit Attribution on Pythia-2.8B (primary; validated on GPT2-XL, GPT-J and other Pythia sizes including 6.9B) to dissect factual recall [chughtai-cooney-nanda-2024-summing-up-the-facts-additive-mechanisms-behind-factual-recall-in-llms] The mechanism is an "additive motif": several qualitatively different component types -- subject heads, relation heads, mixed heads, and MLPs -- independently contribute and additively sum, constructively interfering on the correct attribute [chughtai-cooney-nanda-2024-summing-up-the-facts-additive-mechanisms-behind-factual-recall-in-llms] Heads are sorted by an "attention ratio" (DLA to subject vs relation tokens), with strongly specialized values (e.g. up to 107.23 for one head) [chughtai-cooney-nanda-2024-summing-up-the-facts-additive-mechanisms-behind-factual-recall-in-llms] The framing is additivity across component types, not a low-rank per-attribute subspace (the low-rank property is only the generic fact that any attention head reads and writes a low-rank slice of the residual stream) [chughtai-cooney-nanda-2024-summing-up-the-facts-additive-mechanisms-behind-factual-recall-in-llms]

models: Pythia-2.8B, Pythia-6.9B · method: Direct Logit Attribution (DLA)
Knowledge in Superposition: Unveiling the Failures of Lifelong Knowledge Editing for Large Language Models (2025)measured

Edited facts' key vectors overlap in a whitened memory space

Details

Hu et al. extend ROME's closed-form associative-memory solution to n edits and derive an interference term equal to a whitened normalized dot-product between two facts' key vectors [hu-etal-2025-knowledge-in-superposition] Across 11 LLMs (GPT-2 family, GPT-J-6B, Pythia 1B-6.9B, Llama-2/3 family) the whitened-overlap distribution is heavy-tailed and high-kurtosis with clustered near-maximal overlaps, absent in the raw key vectors ("knowledge superposition") [hu-etal-2025-knowledge-in-superposition] Sequential ROME edits on GPT-J progressively reduce confidence in unrelated held-out facts, tying the measured overlap geometry to a lifelong-editing forgetting failure [hu-etal-2025-knowledge-in-superposition]

models: Pythia-1B, Pythia-2.8B, Pythia-6.9B · method: Rank-One Model Editing (ROME) / Linear Associative Memory Editing
How Do Language Models Bind Entities in Context? (2023)measured

Binding-ID vectors form a subspace whose distances predict confusability

Details

Feng & Steinhardt identify additive "binding ID vectors" attached to entity and attribute activations that solve variable binding, using causal interventions on LLaMA-1 (30B primary, plus 13B and 65B) and the Pythia family [feng-steinhardt-2023-how-do-language-models-bind-entities-in-context] The binding vectors occupy a continuous subspace in which the distance between two binding vectors predicts how often the model confuses the corresponding entity-attribute bindings [feng-steinhardt-2023-how-do-language-models-bind-entities-in-context] Patching, adding or removing binding-ID vectors changes which attribute the model retrieves for a given entity, demonstrated via causal mediation analysis rather than linear probing [feng-steinhardt-2023-how-do-language-models-bind-entities-in-context]

models: Pythia-2.8B · method: Activation patching
How Reliable are Causal Probing Interventions? (2025)measured

Nonlinear interventions are more reliable than linear subspace erasure

Details

Canby et al. formalize completeness and selectivity as numbers in [0,1] and define reliability as their harmonic mean, comparing INLP, RLACE, AlterRep and three gradient-based nonlinear interventions [canby-etal-2025-reliability] Every method shows a completeness/selectivity trade-off; none achieves perfect completeness without sacrificing selectivity in any model [canby-etal-2025-reliability] Counterfactual methods (AlterRep, gradient-based) achieve near-total task-accuracy change, consistent with higher completeness than the concept-removal methods [canby-etal-2025-reliability] Nonlinear interventions are almost always most reliable (FGSM 0.92-0.96 across GPT-2/Pythia/Llama) versus linear methods 0.106-0.841, the sole exception being BERT layers 10-12 where AlterRep wins [canby-etal-2025-reliability]

models: Pythia-160M, Pythia-1.4B, Pythia-6.9B · method: INLP subspace + AlterRep counterfactual push, R-LACE (Relaxed Linear Adversarial Concept Erasure), Gradient-Based Interventions (nonlinear counterfactual push via adversarial attack)
LEACE: Perfect Linear Concept Erasure in Closed Form (2023)measured

The minimal-distortion linear eraser is an oblique projection

Details

Belrose et al. prove LEACE's closed-form eraser guards a concept against every linear classifier and convex loss simultaneously (Theorem 4.1) [belrose-etal-2023-leace] It is the least-squares-optimal minimal-distortion such eraser (Theorems 4.2-4.3) and is necessarily an oblique, not orthogonal, projection, unlike INLP/RLACE [belrose-etal-2023-leace] On BERT Bias-in-Bios gender removal it reaches chance accuracy at rank 1 with the smallest embedding MSE, cutting the TPR gap 0.198->0.084, about two orders of magnitude faster than RLACE [belrose-etal-2023-leace] Concept-scrubbing POS from every layer of Pythia (160M-12B) and LLaMA (7B-30B) raises LM loss far above a random-subspace control (e.g. LLaMA-7B 0.69->1.73 bpb) while removing fewer dimensions than a full-rank baseline [belrose-etal-2023-leace]

models: Pythia-160M, Pythia-1.4B, Pythia-6.9B, Pythia-12B · method: LEACE (LEAst-squares Concept Erasure)
Emergence of Phonemic, Syntactic, and Semantic Representations in Artificial Neural Networks (2026)measured

Structural probes show staged phonemic-then-lexical-then-syntactic subspaces

Details

Orhan et al. train Hewitt-Manning distance-based structural probes to recover phoneme, WordNet lexical-semantic and syntactic-tree subspaces from wav2vec2 (tiny/base/large) and text models (Llama-2-13B, Pythia), tracked across pretraining [orhan-etal-2026-emergence-of-phonemic-syntactic-and-semantic-representations-in-artificial-neural-networks] Both modalities show staged emergence: phonemic structure first, then lexical, then syntactic, mirroring child language-acquisition order but needing 2-4 orders of magnitude more data [orhan-etal-2026-emergence-of-phonemic-syntactic-and-semantic-representations-in-artificial-neural-networks] On Llama-2-13B the syntactic and semantic subspaces are near-orthogonal (mean-squared cosine of principal angles 0.038-0.042), so they are distinct low-dimensional subspaces [orhan-etal-2026-emergence-of-phonemic-syntactic-and-semantic-representations-in-artificial-neural-networks] A contrastive topology-preserving probe confirms the distance-based probe validity [orhan-etal-2026-emergence-of-phonemic-syntactic-and-semantic-representations-in-artificial-neural-networks]

models: Pythia-1B · method: Structural probing (metric/distance probe)
Revisiting Anisotropy in Language Transformers: The Geometry of Learning Dynamics (2026)measured

Anisotropy from frequency-blind sampling and self-reinforcing tangent gradients

Details

Bernas et al. give a two-part geometric account of anisotropy: they prove (Corollary 2.2) that as high-frequency tokens concentrate near their centroid, local-manifold curvature becomes statistically invisible [bernas-etal-2026] They prove (Proposition 2.3) the normal-to-tangent gradient magnitude ratio scales as O(t) in the concentration radius, so curvature-bearing normal updates are suppressed for high-frequency tokens, compounded by attention and residuals [bernas-etal-2026] Across the Pythia (160m/410m/1b/1.4b), SmolLM2 (360m/1.7b) and EuroBERT (210m/610m) suites, true gradients concentrate in an activation-derived tangent subspace far more than matched-rank random controls (energy ratios orders of magnitude above null) [bernas-etal-2026] The effect is strongest early in training and in early/middle layers, and weaker in EuroBERT, whose language-balanced training counteracts the frequency skew the mechanism depends on [bernas-etal-2026] The paper reframes anisotropy as a possibly adaptive implicit dimensionality reduction rather than a pure training pathology [bernas-etal-2026]

models: Pythia-160M, Pythia-410M, Pythia-1B, Pythia-1.4B · method: PCA, Gradient–Tangent Alignment Test
Scale Determines Whether Language Models Organize Representation Geometry for Prediction (2026)measured

Scale determines whether late-layer geometry stays organized by the readout

Details

Xu introduces Subspace PGA, a z-scored RSA metric comparing how well a layer cosine-distance structure survives projection onto the unembedding matrix top-k readout subspace versus 100 random subspaces [xu-2026-scale-determines-representation-geometry-organization-prediction] Across Pythia (70M-6.9B) and OLMo-1B/Phi-1.5/Gemma-2-2B, intermediate geometry is robustly organized around the readout subspace (peak z 9-24 at mid layers) [xu-2026-scale-determines-representation-geometry-organization-prediction] Small models (hidden dim <=1024) progressively lose this at late layers over training even as loss drops (Pythia-410M min z +0.6 to -32 across checkpoints), while dim>=2048 models preserve it [xu-2026-scale-determines-representation-geometry-organization-prediction] Removing a few top principal components restores positive z at every layer for dim>=768 models, supporting a masking interpretation; the paper frames the claim as correlational [xu-2026-scale-determines-representation-geometry-organization-prediction]

models: Pythia-70M, Pythia-160M, Pythia-410M, Pythia-1B, Pythia-1.4B, Pythia-2.8B, Pythia-6.9B · method: Subspace PGA (Predictive Geometric Alignment)

Float

Representational Analysis of Binding in Language Models (2024)measured

A PCA 'Ordering ID' direction localizes and steers binding

Details

Dai, Heinzerling & Inui (2024) use PCA on entity/attribute activations to find a low-rank "OI subspace" whose leading component (OI-PC) encodes the "Ordering ID" underlying Feng & Steinhardt's Binding-ID mechanism, not previously localized to a direction [dai-etal-2024-representational-analysis-of-binding-in-language-models] Steering along OI-PC by n unit steps shifts the predicted attribute to that of Binding-ID n, quantified via Logit Difference and Logit Flip metrics that rise monotonically with step count [dai-etal-2024-representational-analysis-of-binding-in-language-models] A Spearman correlation confirms OI-PC tracks the true Order-ID rather than a position/filler-length confound [dai-etal-2024-representational-analysis-of-binding-in-language-models] The OI subspace is prevalent across five base models: Llama2-7B, Llama3-8B, Qwen1.5-7B, Pythia-6.9B, and the code-fine-tuned Float-7B [dai-etal-2024-representational-analysis-of-binding-in-language-models]

models: Float-7B (code fine-tuned LM) · method: PCA, Direction Extraction, Causal interventions (steering)

ResNet

Simple Disentanglement of Style and Content in Visual Representations (2023)measured

A fitted linear projection (PISCO) disentangles style from content

Details

Ngweta et al. show pretrained ResNet-18 (supervised, ImageNet) and SimCLR ResNet-18 (CIFAR-10) features are a linearly-entangled mixture of style and content factors [ngweta-etal-2023-simple-disentanglement-of-style-and-content-in-visual-representations] They derive PISCO, a provably-correct linear projection fitted post-hoc (via two theorems) to separate style and content with no retraining, a constructed disentangling map rather than a structure discovered in free activations [ngweta-etal-2023-simple-disentanglement-of-style-and-content-in-visual-representations] On CIFAR-10 PISCO raises style-correlation recovery (e.g. rotation: SimCLR 0.368->0.945) and cuts a disentanglement-discrepancy metric (rotation 0.212->0.029) [ngweta-etal-2023-simple-disentanglement-of-style-and-content-in-visual-representations] Discarding the isolated style-subspace factors improves out-of-distribution accuracy under spurious style shift while preserving in-distribution accuracy [ngweta-etal-2023-simple-disentanglement-of-style-and-content-in-visual-representations]

models: ResNet-18 (supervised, ImageNet) · method: Causal interventions (steering)
Probing the Mid-level Vision Capabilities of Self-Supervised Learning (2024)measured

Mid-level vision decodability weakly correlates with recognition accuracy

Details

Chen, Marks & Cheng introduce eight frozen-feature benchmarks for mid-level vision (segmentation, geometric/3D grouping) across 22 self-supervised models spanning contrastive, clustering, pretext and masked paradigms [chen-etal-2024-probing-the-mid-level-vision-capabilities-of-self-supervised-learning] The benchmarks decode with a nonlinear DPT dense decoder on frozen features, not a linear probe [chen-etal-2024-probing-the-mid-level-vision-capabilities-of-self-supervised-learning] Mid-level task performance correlates only weakly with each model's high-level ImageNet accuracy (generic segmentation strongest ~R^2 0.70; 3D understanding weakly correlated) [chen-etal-2024-probing-the-mid-level-vision-capabilities-of-self-supervised-learning] Several models are strongly imbalanced across the two capability classes, so mid-level decodability is not simply a byproduct of overall recognition quality [chen-etal-2024-probing-the-mid-level-vision-capabilities-of-self-supervised-learning]

models: ResNet-50 (MoCo v2, unsupervised contrastive pretraining, ImageNet) · method: Mlp probing
Text-to-Concept (and Back) via Cross-Model Alignment (2023)measured

An affine map between two vision models matches their PCA components

Details

Moayeri et al. fit an affine least-squares map between the activation spaces of independently-trained vision models (supervised/robust ResNets, Swin/DeiT/ConViT, self-supervised MoCo/DINO-ViT-S/SimCLR ResNets and ViTs, and CLIP), scored by R^2 [moayeri-etal-2023-text-to-concept-and-back-via-cross-model-alignment] The fitted map explains R^2 above 0.6, and the top principal components of the two aligned spaces correspond approximately one-to-one [moayeri-etal-2023-text-to-concept-and-back-via-cross-model-alignment] Aligning a vision encoder into CLIP's concept space enables zero-shot concept-bottleneck classification at up to 93.8% accuracy and over 92% concept-to-text relevance [moayeri-etal-2023-text-to-concept-and-back-via-cross-model-alignment] The analysis is passive, with no causal intervention on either model [moayeri-etal-2023-text-to-concept-and-back-via-cross-model-alignment]

models: ResNet-50 (MoCo v2, unsupervised contrastive pretraining, ImageNet) · method: Learned linear/affine map between two representation spaces
Discovering Universal Geometry in Embeddings with ICA (2023)measured

ICA after PCA-whitening recovers a universal sparse axis-aligned decomposition

Details

Yamagiwa et al. apply PCA-whitening then ICA to their own text8-trained word2vec (SGNS), cross-lingual fastText (7 languages), BERT contextual embeddings, and image encoders (ViT, ResMLP, Swin, ResNet, RegNet) [yamagiwa-etal-2023-discovering-universal-geometry-ica] The independent axes are individually interpretable and sparse, unlike PCA arbitrary rotation [yamagiwa-etal-2023-discovering-universal-geometry-ica] Specific axes can be matched across languages, models and modalities via cross-model component correlation, a privileged non-arbitrary decomposition [yamagiwa-etal-2023-discovering-universal-geometry-ica] No causal intervention is performed; axis interpretability is partly qualitative and universality depends on whitening choices [yamagiwa-etal-2023-discovering-universal-geometry-ica]

models: ResNet-18 (supervised, ImageNet) · method: ICA axis decomposition

SimCLR

Simple Disentanglement of Style and Content in Visual Representations (2023)measured

A fitted linear projection (PISCO) disentangles style from content

Details

Ngweta et al. show pretrained ResNet-18 (supervised, ImageNet) and SimCLR ResNet-18 (CIFAR-10) features are a linearly-entangled mixture of style and content factors [ngweta-etal-2023-simple-disentanglement-of-style-and-content-in-visual-representations] They derive PISCO, a provably-correct linear projection fitted post-hoc (via two theorems) to separate style and content with no retraining, a constructed disentangling map rather than a structure discovered in free activations [ngweta-etal-2023-simple-disentanglement-of-style-and-content-in-visual-representations] On CIFAR-10 PISCO raises style-correlation recovery (e.g. rotation: SimCLR 0.368->0.945) and cuts a disentanglement-discrepancy metric (rotation 0.212->0.029) [ngweta-etal-2023-simple-disentanglement-of-style-and-content-in-visual-representations] Discarding the isolated style-subspace factors improves out-of-distribution accuracy under spurious style shift while preserving in-distribution accuracy [ngweta-etal-2023-simple-disentanglement-of-style-and-content-in-visual-representations]

models: ResNet-18 (SimCLR contrastive pretraining, CIFAR-10) · method: Causal interventions (steering)
Probing the Mid-level Vision Capabilities of Self-Supervised Learning (2024)measured

Mid-level vision decodability weakly correlates with recognition accuracy

Details

Chen, Marks & Cheng introduce eight frozen-feature benchmarks for mid-level vision (segmentation, geometric/3D grouping) across 22 self-supervised models spanning contrastive, clustering, pretext and masked paradigms [chen-etal-2024-probing-the-mid-level-vision-capabilities-of-self-supervised-learning] The benchmarks decode with a nonlinear DPT dense decoder on frozen features, not a linear probe [chen-etal-2024-probing-the-mid-level-vision-capabilities-of-self-supervised-learning] Mid-level task performance correlates only weakly with each model's high-level ImageNet accuracy (generic segmentation strongest ~R^2 0.70; 3D understanding weakly correlated) [chen-etal-2024-probing-the-mid-level-vision-capabilities-of-self-supervised-learning] Several models are strongly imbalanced across the two capability classes, so mid-level decodability is not simply a byproduct of overall recognition quality [chen-etal-2024-probing-the-mid-level-vision-capabilities-of-self-supervised-learning]

models: ResNet-50 (SimCLR contrastive pretraining, ImageNet) · method: Mlp probing
Text-to-Concept (and Back) via Cross-Model Alignment (2023)measured

An affine map between two vision models matches their PCA components

Details

Moayeri et al. fit an affine least-squares map between the activation spaces of independently-trained vision models (supervised/robust ResNets, Swin/DeiT/ConViT, self-supervised MoCo/DINO-ViT-S/SimCLR ResNets and ViTs, and CLIP), scored by R^2 [moayeri-etal-2023-text-to-concept-and-back-via-cross-model-alignment] The fitted map explains R^2 above 0.6, and the top principal components of the two aligned spaces correspond approximately one-to-one [moayeri-etal-2023-text-to-concept-and-back-via-cross-model-alignment] Aligning a vision encoder into CLIP's concept space enables zero-shot concept-bottleneck classification at up to 93.8% accuracy and over 92% concept-to-text relevance [moayeri-etal-2023-text-to-concept-and-back-via-cross-model-alignment] The analysis is passive, with no causal intervention on either model [moayeri-etal-2023-text-to-concept-and-back-via-cross-model-alignment]

models: ResNet-50 (SimCLR contrastive pretraining, ImageNet) · method: Learned linear/affine map between two representation spaces

AlphaZero

Evaluation Beyond Task Performance: Analyzing Concepts in AlphaZero in Hex (2022)measured

AlphaZero-Hex cell embeddings grow to match board adjacency

Details

Lovering et al. extract a first-layer embedding per Hex board cell from AlphaZero's policy/value network and compute pairwise dot products between all cell embeddings [lovering-etal-2022-alphazero-hex-concepts] Alignment of this dot-product structure with ground-truth board adjacency, measured by NDCG, increases substantially over training, improving about halfway through before plateauing [lovering-etal-2022-alphazero-hex-concepts] This geometric claim is distinct from the paper's separate concept-probing-accuracy and behavioral analyses, which report only classifier accuracy and input-output checks [lovering-etal-2022-alphazero-hex-concepts]

models: AlphaZero (Hex, ResNet policy/value network, self-play) · method: Geometric analysis

BERT

Counterfactual Interventions Reveal the Causal Effect of Relative Clause Representations on Agreement Prediction (2021)measured

AlterRep push on BERT's RC-boundary subspace shifts agreement errors

Details

Ravfogel et al. introduce AlterRep to test whether BERT causally uses its relative-clause-boundary representation during subject-verb agreement, using INLP-derived classifier directions spanning the RC-boundary subspace [ravfogel-etal-2021-alterrep] In BERT-base middle layers (5-8) the positive counterfactual raises agreement-error probability up to 14 points while the negative lowers it only up to 2, a clearly asymmetric causal effect [ravfogel-etal-2021-alterrep] BERT-large shows the analogous pattern in layers 12-17; smaller Turc et al. BERT variants show it in narrower model-specific ranges [ravfogel-etal-2021-alterrep] The effect partially transfers across five RC types when the subspace is estimated from a different RC type, indicating partly shared, partly structure-specific representation [ravfogel-etal-2021-alterrep] Counterfactuals from 10 random Gaussian subspaces fail to reproduce the effect, confirming specificity to the RC-boundary subspace [ravfogel-etal-2021-alterrep]

models: BERT-base-uncased, BERT-large-uncased, BERT-medium, BERT-small, BERT-mini, BERT-tiny · method: INLP subspace + AlterRep counterfactual push
Interventional Probing in High Dimensions: An NLI Case Study (2023)measured

Amnesic probing misleads at low class count and high dimension

Details

Rozanova et al. apply amnesic probing (INLP erasure then measure) to a natural-logic NLI fragment where the label is provably determined by two known features across five BERT/RoBERTa NLI models [rozanova-etal-2023-interventional] Removing either or both provably-necessary features causes almost no accuracy drop (mostly within +/-2 points; one case improves by 8.99) [rozanova-etal-2023-interventional] Even nulling out the gold entailment label itself produces almost no drop in three of five models, a case where the removed information is by definition maximally relevant [rozanova-etal-2023-interventional] They trace this to a dimensionality confound: with only 2-3 classes, INLP removes very few directions and a single random-direction control is unstable [rozanova-etal-2023-interventional] Removing as few as 3 random directions can already match the targeted removal by chance, so a "no difference from random" conclusion can be control-baseline noise [rozanova-etal-2023-interventional]

models: BERT-base-uncased fine-tuned on SNLI, BERT-base-uncased fine-tuned on SNLI + HELP · method: INLP subspace + AlterRep counterfactual push
Visualizing and Measuring the Geometry of BERT (2019)measured

BERT's semantic-sense subspace is largely separate from its syntactic one

Details

Coenen et al. train a cosine-based contrastive probe on BERT-base to find a subspace where same-sense embeddings cluster and different senses separate [coenen-etal-2019] A nearest-centroid classifier in this subspace reaches 71.1-71.5% F1 on word-sense disambiguation, with accuracy holding down to probe dimension m=128 before degrading [coenen-etal-2019] Concatenating two senses of a word shifts the keyword's embedding toward the opposing sense's centroid, and projecting into the sense subspace reduces the induced misclassification (7.31% vs 8.23%) [coenen-etal-2019] The singular values of A^T B (semantic probe A vs Hewitt-Manning syntactic probe B) fall to zero far faster than those of A or B alone, so the semantic and syntactic subspaces are approximately orthogonal (largely separate) [coenen-etal-2019] A separate linear probe on BERT's attention matrices decodes dependency existence (85.8%) and type (71.9%) [coenen-etal-2019]

models: BERT-base-cased · method: Linear probing, UMAP, PCA
Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representations (2024)measured

A learned rotation subspace beats brute-force causal-alignment search

Details

Geiger et al. test DAS on two tasks with known causal structure; on a hierarchical-equality ReLU network its learned orthogonal-rotation subspace reaches interchange-intervention accuracy up to 1.00 vs 0.60 brute-force and 0.73 best localist [geiger-etal-2023-das] On monotonicity NLI (BERT-base fine-tuned on MultiNLI then MoNLI) DAS reaches 1.00 IIA at layer 9 vs 0.64 and 0.51 [geiger-etal-2023-das] A randomly-initialized network stays near chance (~0.50) unless the hidden dimension is blown up to 4096, confirming DAS is not fabricating structure [geiger-etal-2023-das] The paper is explicit that DAS USES an assumed linear structure as a methodological device and does not itself claim to have discovered that any concept is linearly encoded [geiger-etal-2023-das]

models: BERT-base-uncased · method: Distributed Alignment Search (DAS)
Amnesic Probing: Behavioral Explanation with Amnesic Counterfactuals (2021)measured

Decodability does not imply behavioral use, shown by amnesic probing

Details

Elazar et al. introduce amnesic probing: apply INLP to remove a property's linear subspace from BERT-base then measure the behavioral effect on masked-LM accuracy and KL-divergence, not just whether it stays decodable [elazar-etal-2021-amnesic] Removal effects vary hugely: dependency-label removal drops LM accuracy 94%->7% and fine-POS 94%->12%, but phrase-start/end markers (probed at 85%/83%) cause essentially zero drop despite being decodable [elazar-etal-2021-amnesic] A random-direction control shows some drops are confounded with sheer dimensionality removed (dependency labels also collapse under random removal), while the phrase-marker null is genuine [elazar-etal-2021-amnesic] A selectivity control restores the gold label and fine-tunes downstream layers to check whether removal over-removed unrelated information [elazar-etal-2021-amnesic] The core message: probing accuracy and behavioral importance are dissociable in both directions, so a probed subspace being "used" needs a causal check [elazar-etal-2021-amnesic]

models: BERT-base-uncased · method: INLP subspace + AlterRep counterfactual push
Beyond Decodability: Reconstructing Language Model Representations with an Encoding Probe (2026)measured

An encoding probe decomposes feature-group variance in wav2vec2 and BERT

Details

Shen et al. fit ridge-regression "encoding probes" that reconstruct wav2vec2-base and BERT-base activations from interpretable feature sets (acoustics, phonetics, speaker-identity, syntax, lexicon), reporting unexplained variance (1 minus R-squared) under feature-group ablation [shen-etal-2026-beyond-decodability-reconstructing-language-model-representations-with-an-encoding-probe] Speaker-identity's contribution to explained variance shifts substantially depending on whether wav2vec2 is left self-supervised or fine-tuned for ASR versus speaker-ID [shen-etal-2026-beyond-decodability-reconstructing-language-model-representations-with-an-encoding-probe] Syntactic and lexical feature groups contribute largely independently and additively, with unexplained-variance differences stable to within 0.002 across seeds [shen-etal-2026-beyond-decodability-reconstructing-language-model-representations-with-an-encoding-probe] No causal intervention is performed; the analysis is explicitly observational [shen-etal-2026-beyond-decodability-reconstructing-language-model-representations-with-an-encoding-probe]

models: BERT-base-uncased · method: Closed-form affine-regression ablation of a continuous conditioning variable
Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection (2020)measured

The gender subspace spans dozens of directions, not one

Details

Ravfogel et al. introduce Iterative Null-space Projection (INLP) and apply it across three case studies [ravfogel-etal-2020-inlp] On GloVe, 35 INLP iterations drop SVM gender classification 100%->49.3% while a nonlinear MLP still recovers 85%, showing the gender subspace spans dozens of orthogonal directions rather than the single Bolukbasi direction [ravfogel-etal-2020-inlp] Word-similarity benchmarks improve after projection (SimLex-999 0.373->0.489), so general lexical semantics is preserved [ravfogel-etal-2020-inlp] On DeepMoji race-correlated sentiment, removing the race subspace shrinks the true-positive-rate fairness gap 0.45->0.15 at some accuracy cost [ravfogel-etal-2020-inlp] On Bias-in-Bios, removing the gender subspace from BERT CLS representations (300 directions) cuts the gender-TPR gap 48% while dropping profession accuracy only 80.9%->75.2% [ravfogel-etal-2020-inlp]

models: BERT-base-uncased · method: INLP subspace + AlterRep counterfactual push
Probing for the Usage of Grammatical Number (2022)measured

BERT causally uses a multi-dimensional number subspace for agreement

Details

Lasri et al. apply INLP (erasure only, no AlterRep) within amnesic probing to BERT-base, removing the grammatical-number subspace at noun and verb positions (44-70 directions by layer, not a single direction) [lasri-etal-2022] Removing the noun-position subspace drops subject-verb agreement accuracy 0.22-0.32 in early layers; removing the verb-position subspace drops it 0.23-0.33 in layers 9-11 [lasri-etal-2022] A random-direction placebo removing the same number of directions drops accuracy only 0.00-0.06, confirming specificity to the targeted subspace [lasri-etal-2022] Noun and verb number encodings are separate: a projector trained at one position erases nothing useful at the other, and the probe weight vectors are near-opposite then near-orthogonal across layers [lasri-etal-2022] Correlating harmful-removal layers with an attention-ablation experiment locates number transfer from noun to verb in layers 2-8 [lasri-etal-2022]

models: BERT-base-uncased · method: INLP subspace + AlterRep counterfactual push
Cross-Lingual BERT Transformation for Zero-Shot Dependency Parsing (2019)measured

A linear map from 10K pairs aligns English BERT with multilingual BERT

Details

Wang et al. fit a linear map from monolingual English BERT to multilingual BERT using contextual word alignments from only 10,000 Europarl sentence pairs per language [wang-etal-2019-clbt] They compare closed-form orthogonal Procrustes against an unconstrained gradient-descent linear map; the unconstrained map is marginally better and a nonlinear map adds nothing [wang-etal-2019-clbt] On zero-shot cross-lingual dependency parsing (17 languages), CLBT beats the static-embedding SOTA (avg LAS 60.63->63.54) and matches XLM despite using 10K versus millions of parallel sentences [wang-etal-2019-clbt] The alignment claim rests on downstream transfer performance rather than an independent similarity statistic (only a qualitative t-SNE case study is shown) [wang-etal-2019-clbt]

models: BERT-base-cased, mBERT (BERT-base, Multilingual Cased) · method: Orthogonal Procrustes alignment
Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color (2021)measured

Text-only LMs linearly recover CIELAB color-space structure

Details

Abdou et al. test whether text-only LMs encode perceptual color structure against the WCS/XKCD-derived 3D CIELAB space, with no visual grounding [abdou-etal-2021-color-perceptual-structure] RSA (Kendall's tau) between color-term embeddings and CIELAB distances is significant in the color-context configuration (BERT-large 0.24, ELECTRA 0.23, RoBERTa 0.19; the random null is not) [abdou-etal-2021-color-perceptual-structure] A lasso linear mapping onto 3D CIELAB with control-task selectivity is high for all three families (0.76-0.78), needing only ~10-40 dimensions to explain 0.4-0.7 of variance [abdou-etal-2021-color-perceptual-structure] Alignment scales with model size (BERT-mini tau 0.077 -> BERT-base 0.162) with a warm/cool recovery asymmetry; no causal intervention is performed [abdou-etal-2021-color-perceptual-structure]

models: BERT-large-uncased, BERT-mini, BERT-small, BERT-medium, BERT-base-uncased · method: Representational Similarity Analysis (RSA), Linear probing
Improving Causal Interventions in Amnesic Probing with Mean Projection or LEACE (2025)measured

INLP fails its own control while Mean Projection and LEACE pass

Details

Dobrzeniecka et al. re-run Elazar's amnesic-probing pipeline on BERT, substituting Mean Projection and LEACE for INLP on three properties (dependency labels, fine-POS, coarse-POS) [dobrzeniecka-etal-2025-mp-leace] INLP needs 738-900 directions to erase these versus a flat one-per-class for MP/LEACE (41/45/12), and its cosine-similarity distortion is far larger (0.31-0.37 vs 0.80-0.95) [dobrzeniecka-etal-2025-mp-leace] This distortion breaks INLP's own control: the same number of random-direction projections drops next-word accuracy MORE than the targeted INLP removal for 2 of 3 properties [dobrzeniecka-etal-2025-mp-leace] Mean Projection and LEACE pass the random-projection and selectivity controls in all cases, so the removal method is not incidental to amnesic probing's validity [dobrzeniecka-etal-2025-mp-leace]

models: BERT-base-uncased · method: INLP subspace + AlterRep counterfactual push, Mean Projection (MP), LEACE (LEAst-squares Concept Erasure)
Interventional Probing in High Dimensions: An NLI Case Study (2023)measured

Mnestic probing recovers signal where amnesic removal found none

Details

Rozanova et al. introduce mnestic probing: project onto the union of INLP-trained classifiers' rowspaces (keep only the probe directions), the mirror image of amnesic removal, using the same trained classifiers [rozanova-etal-2023-interventional] Fed incrementally into the NLI head, it produces a clear monotonic accuracy increase for the composite and gold-label features, faster than a random-direction control [rozanova-etal-2023-interventional] It succeeds exactly where the amnesic (removal) version found no effect, because it operates in a low-rank regime that sidesteps the low-class-count control problem [rozanova-etal-2023-interventional] Context monotonicity's increase is not clearly above the random baseline, so the paper does not claim it is used [rozanova-etal-2023-interventional]

models: BERT-base-uncased fine-tuned on SNLI, BERT-base-uncased fine-tuned on SNLI + HELP · method: INLP subspace + AlterRep counterfactual push
Log-linear Guardedness and its Implications (2023)measured

Guardedness against a binary probe leaks to a multiclass softmax

Details

Ravfogel et al. formalize V-guardedness (no classifier in family V predicts concept Z above epsilon mutual information), the property INLP/RLACE/LEACE target [ravfogel-etal-2023-log-linear-guardedness] Theorem 3.2 shows binary log-linear guardedness propagates to downstream binary log-linear classifiers, but Theorem 3.4 shows it breaks for multiclass classifiers [ravfogel-etal-2023-log-linear-guardedness] On a K-Voronoi construction, a representation guarded against every single hyperplane still lets a K-way softmax (signed combinations of the guarding directions) recover Z with near-full information [ravfogel-etal-2023-log-linear-guardedness] Empirically, RLACE-erased BERT gender is perfectly recovered by 4-8-class profession softmax classifiers, so guardedness against one adversary says nothing about a differently-structured one [ravfogel-etal-2023-log-linear-guardedness]

models: BERT-base-uncased · method: R-LACE (Relaxed Linear Adversarial Concept Erasure)
How Reliable are Causal Probing Interventions? (2025)measured

Nonlinear interventions are more reliable than linear subspace erasure

Details

Canby et al. formalize completeness and selectivity as numbers in [0,1] and define reliability as their harmonic mean, comparing INLP, RLACE, AlterRep and three gradient-based nonlinear interventions [canby-etal-2025-reliability] Every method shows a completeness/selectivity trade-off; none achieves perfect completeness without sacrificing selectivity in any model [canby-etal-2025-reliability] Counterfactual methods (AlterRep, gradient-based) achieve near-total task-accuracy change, consistent with higher completeness than the concept-removal methods [canby-etal-2025-reliability] Nonlinear interventions are almost always most reliable (FGSM 0.92-0.96 across GPT-2/Pythia/Llama) versus linear methods 0.106-0.841, the sole exception being BERT layers 10-12 where AlterRep wins [canby-etal-2025-reliability]

models: BERT-base-uncased · method: INLP subspace + AlterRep counterfactual push, R-LACE (Relaxed Linear Adversarial Concept Erasure), Gradient-Based Interventions (nonlinear counterfactual push via adversarial attack)
The Representational Geometry of Number (2026)measured

Number-task subspaces are distinct yet linearly inter-transformable

Details

Hu, Niu & Varma analyze number-concept representations (comparison, parity, primality, successor/predecessor) across BERT, GPT-2, Qwen2.5-7B and Qwen2.5-Math-7B at ~75% depth [hu-etal-2026-the-representational-geometry-of-number] Task-specific subspaces are distinct via Procrustes disparity (~0.010 vs a permutation baseline 0.077-0.273) yet linearly inter-transformable (SVCCA canonical correlations ~0.80-0.90) [hu-etal-2026-the-representational-geometry-of-number] The parity axis sits nearly orthogonal to the magnitude axis (88.8 degrees) while the primality axis is only partly decoupled (68.8 degrees) [hu-etal-2026-the-representational-geometry-of-number] Subspace overlap between most task pairs stays below 0.2, except the closely-related successor/predecessor pair (0.36-0.42) [hu-etal-2026-the-representational-geometry-of-number]

models: BERT-base-uncased · method: Generalized Procrustes Analysis (GPA), Singular Vector Canonical Correlation Analysis (SVCCA), Direction Extraction, Linear probing, Geometric analysis
LEACE: Perfect Linear Concept Erasure in Closed Form (2023)measured

The minimal-distortion linear eraser is an oblique projection

Details

Belrose et al. prove LEACE's closed-form eraser guards a concept against every linear classifier and convex loss simultaneously (Theorem 4.1) [belrose-etal-2023-leace] It is the least-squares-optimal minimal-distortion such eraser (Theorems 4.2-4.3) and is necessarily an oblique, not orthogonal, projection, unlike INLP/RLACE [belrose-etal-2023-leace] On BERT Bias-in-Bios gender removal it reaches chance accuracy at rank 1 with the smallest embedding MSE, cutting the TPR gap 0.198->0.084, about two orders of magnitude faster than RLACE [belrose-etal-2023-leace] Concept-scrubbing POS from every layer of Pythia (160M-12B) and LLaMA (7B-30B) raises LM loss far above a random-subspace control (e.g. LLaMA-7B 0.69->1.73 bpb) while removing fewer dimensions than a full-rank baseline [belrose-etal-2023-leace]

models: BERT-base-uncased · method: LEACE (LEAst-squares Concept Erasure)
Mapping Language Models to Grounded Conceptual Spaces (2022)measured

Behavioral grounding survives rotation but collapses under random reassignment

Details

Patel & Pavlick test whether LMs' internal representations of color, cardinal directions and grid terms carry the same relational structure as external grounded spaces, via few-shot prompting with no probe training [patel-pavlick-2022-mapping-language-models-to-grounded-conceptual-spaces] GPT-3 (175B) performs similarly on the true grounding and a structure-preserving rotation (spatial 45%/76% vs 44%/75% Top-1/Top-3) but collapses under random reassignment (16-19%) [patel-pavlick-2022-mapping-language-models-to-grounded-conceptual-spaces] Color-grounding error drops from 328 (GPT-2 124M) to 96 (GPT-3 175B), and BERT-base badly underperforms GPT-3 across all conditions [patel-pavlick-2022-mapping-language-models-to-grounded-conceptual-spaces] This is a scale-dependent relational-isomorphism-up-to-rotation claim inferred behaviorally; no activation-space intervention is performed [patel-pavlick-2022-mapping-language-models-to-grounded-conceptual-spaces]

models: BERT-base-uncased · method: Rotation-invariance isomorphism test
How Multilingual is Multilingual BERT? (2019)measured

mBERT own space supports nearest-neighbor cross-lingual retrieval

Details

Pires, Schlinger & Garrette show real pretrained mBERT (104 languages) supports translation-pair retrieval by direct nearest-neighbor search in its own space, with no fitted map [pires-etal-2019-how-multilingual-is-mbert] This indicates mBERT shared geometry already aligns translation-equivalent items across languages [pires-etal-2019-how-multilingual-is-mbert] They show strong zero-shot cross-lingual transfer (fine-tune one language, evaluate another) for NER and POS, best between typologically similar languages [pires-etal-2019-how-multilingual-is-mbert] No causal intervention is performed [pires-etal-2019-how-multilingual-is-mbert]

models: mBERT (BERT-base, Multilingual Cased) · method: Nearest-neighbor cross-lingual translation-pair retrieval
The Low-Dimensional Linear Geometry of Contextualized Word Representations (2021)measured

Linguistic categories occupy nested low-dimensional subspaces

Details

Hernandez & Andreas apply a rank-sweep minimal-subspace search to ELMo and BERT-base, finding POS, tense, number and dependency labels are linearly decodable from subspaces well below the hidden size [hernandez-andreas-2021] Finer sub-categories are decodable from a smaller subspace nested inside the coarser parent, roughly half its rank, in many (not all) layers [hernandez-andreas-2021] Neuron-ablation shows these subspaces are distributed across many neurons (POS survives reduction to 512 of BERT's 768 axes) [hernandez-andreas-2021] Ablating only the rank-4 nounspace lowers subject-noun prediction (.85 to .82) without affecting verbs, and vice versa for verbspace, despite each removing under 1% of dimensions [hernandez-andreas-2021]

models: BERT-base-cased · method: Minimal subspace search (rank-sweep probing), Causal interventions (steering)
Linear Adversarial Concept Erasure (2022)measured

One adversarial direction erases gender where INLP needs dozens

Details

Ravfogel et al. formulate concept erasure as a minimax game between a rank-k projection and a re-optimizing linear predictor, proving how many dimensions the erasure-optimal subspace needs (R-LACE) [ravfogel-etal-2022-rlace] On GloVe, rank-1 R-LACE drops SVM gender accuracy to chance while INLP fails even after removing a 20-dimensional subspace [ravfogel-etal-2022-rlace] On BERT Bias-in-Bios, rank-1 R-LACE drops gender accuracy 99.32% to 52.48% while INLP rank-1 barely moves it, needing ~100 dimensions to match [ravfogel-etal-2022-rlace] The gap is task-dependent: INLP is provably suboptimal for linear-regression objectives but provably identical to R-LACE for Rayleigh-quotient (PLS/CCA) objectives; both are linear-only (nonlinear classifiers still recover gender >90%) [ravfogel-etal-2022-rlace]

models: BERT-base-uncased · method: R-LACE (Relaxed Linear Adversarial Concept Erasure)
Discovering Universal Geometry in Embeddings with ICA (2023)measured

ICA after PCA-whitening recovers a universal sparse axis-aligned decomposition

Details

Yamagiwa et al. apply PCA-whitening then ICA to their own text8-trained word2vec (SGNS), cross-lingual fastText (7 languages), BERT contextual embeddings, and image encoders (ViT, ResMLP, Swin, ResNet, RegNet) [yamagiwa-etal-2023-discovering-universal-geometry-ica] The independent axes are individually interpretable and sparse, unlike PCA arbitrary rotation [yamagiwa-etal-2023-discovering-universal-geometry-ica] Specific axes can be matched across languages, models and modalities via cross-model component correlation, a privileged non-arbitrary decomposition [yamagiwa-etal-2023-discovering-universal-geometry-ica] No causal intervention is performed; axis interpretability is partly qualitative and universality depends on whitening choices [yamagiwa-etal-2023-discovering-universal-geometry-ica]

models: BERT-base-uncased · method: ICA axis decomposition

RoBERTa

Interventional Probing in High Dimensions: An NLI Case Study (2023)measured

Amnesic probing misleads at low class count and high dimension

Details

Rozanova et al. apply amnesic probing (INLP erasure then measure) to a natural-logic NLI fragment where the label is provably determined by two known features across five BERT/RoBERTa NLI models [rozanova-etal-2023-interventional] Removing either or both provably-necessary features causes almost no accuracy drop (mostly within +/-2 points; one case improves by 8.99) [rozanova-etal-2023-interventional] Even nulling out the gold entailment label itself produces almost no drop in three of five models, a case where the removed information is by definition maximally relevant [rozanova-etal-2023-interventional] They trace this to a dimensionality confound: with only 2-3 classes, INLP removes very few directions and a single random-direction control is unstable [rozanova-etal-2023-interventional] Removing as few as 3 random directions can already match the targeted removal by chance, so a "no difference from random" conclusion can be control-baseline noise [rozanova-etal-2023-interventional]

models: RoBERTa-large-MNLI, RoBERTa-large-MNLI + HELP fine-tuning, RoBERTa-large-MNLI + double fine-tuning (HELP + NLI_XY) · method: INLP subspace + AlterRep counterfactual push
Intervention Lens: from Representation Surgery to String Counterfactuals (2024)measured

Inverting gender interventions into text augmentation improves fairness

Details

Avitan et al. augment a RoBERTa-base profession classifier's BiasBios training data with string counterfactuals recovered by inverting gender interventions (LEACE erasure, MiMiC/MiMiC+ steering) [avitan-etal-2024-intervention-lens] Training on originals gives accuracy 86.42, F1 79.63, TPR gender gap 14.27; stripping gender from text lowers the gap to 11.22 but costs accuracy [avitan-etal-2024-intervention-lens] Adding LEACE counterfactuals gives the best accuracy (86.59) and F1 (81.8) at gap 12.95; adding MiMiC+ counterfactuals gives the lowest gap (10.59) with near-baseline accuracy [avitan-etal-2024-intervention-lens] So augmenting with inverted representation-intervention counterfactuals improves fairness without the accuracy cost of simply removing gender indicators [avitan-etal-2024-intervention-lens]

models: RoBERTa-base · method: LEACE (LEAst-squares Concept Erasure), MiMiC / MiMiC+ (moment-matching steering), Representation Inversion / Intervention Lens (vec2text-based)
Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color (2021)measured

Text-only LMs linearly recover CIELAB color-space structure

Details

Abdou et al. test whether text-only LMs encode perceptual color structure against the WCS/XKCD-derived 3D CIELAB space, with no visual grounding [abdou-etal-2021-color-perceptual-structure] RSA (Kendall's tau) between color-term embeddings and CIELAB distances is significant in the color-context configuration (BERT-large 0.24, ELECTRA 0.23, RoBERTa 0.19; the random null is not) [abdou-etal-2021-color-perceptual-structure] A lasso linear mapping onto 3D CIELAB with control-task selectivity is high for all three families (0.76-0.78), needing only ~10-40 dimensions to explain 0.4-0.7 of variance [abdou-etal-2021-color-perceptual-structure] Alignment scales with model size (BERT-mini tau 0.077 -> BERT-base 0.162) with a warm/cool recovery asymmetry; no causal intervention is performed [abdou-etal-2021-color-perceptual-structure]

models: RoBERTa-large · method: Representational Similarity Analysis (RSA), Linear probing
Interventional Probing in High Dimensions: An NLI Case Study (2023)measured

Mnestic probing recovers signal where amnesic removal found none

Details

Rozanova et al. introduce mnestic probing: project onto the union of INLP-trained classifiers' rowspaces (keep only the probe directions), the mirror image of amnesic removal, using the same trained classifiers [rozanova-etal-2023-interventional] Fed incrementally into the NLI head, it produces a clear monotonic accuracy increase for the composite and gold-label features, faster than a random-direction control [rozanova-etal-2023-interventional] It succeeds exactly where the amnesic (removal) version found no effect, because it operates in a low-rank regime that sidesteps the low-class-count control problem [rozanova-etal-2023-interventional] Context monotonicity's increase is not clearly above the random baseline, so the paper does not claim it is used [rozanova-etal-2023-interventional]

models: RoBERTa-large-MNLI, RoBERTa-large-MNLI + HELP fine-tuning, RoBERTa-large-MNLI + double fine-tuning (HELP + NLI_XY) · method: INLP subspace + AlterRep counterfactual push

Arora et al. SN word embeddings

Linear Algebraic Structure of Word Senses, with Applications to Polysemy (2018)measured

Word senses decompose into ~2000 sparse discourse atoms

Details

Arora et al. apply classical sparse dictionary learning to their own 300-dimensional embeddings trained with the SN (squared-norm) objective on a 3-billion-token Wikipedia corpus, not pretrained word2vec/GloVe [arora-etal-2018-linear-algebraic-structure-word-senses] They solve for an overcomplete basis of about 2,000 "atoms of discourse" such that each word vector is a sparse (k~5) linear combination of a few atoms [arora-etal-2018-linear-algebraic-structure-word-senses] Polysemous words (e.g. "tie") decompose into atoms matching their distinct senses (clothing, sports, wiring, music), a measured linear-superposition account of polysemy [arora-etal-2018-linear-algebraic-structure-word-senses] No causal intervention is performed; the atom-sense correspondence is validated qualitatively by nearest-word inspection [arora-etal-2018-linear-algebraic-structure-word-senses]

models: Arora et al. SN embeddings (Wikipedia 3B tokens, 300d) · method: Sparse coding for discourse atoms

Weather/Climate Foundation Model

Does Aurora Encode Atmospheric Structure? Latent Regime Analysis and Attribution (2026)measured

Aurora's latent bottleneck is organized by season, not storms

Details

Kasteleyn & Lucic run spatially-pooled PCA on Aurora's latent bottleneck (a 3D Swin Transformer V2 U-Net weather model), finding it organized primarily by seasonal cycle (PC1 24.1% variance, cosine stability 0.998 over 1000 bootstraps) [kasteleyn-lucic-2026-does-aurora-encode-atmospheric-structure] Extreme storm events do not form a linearly separable cluster (unstable PC3, mean cosine 0.650+/-0.269) [kasteleyn-lucic-2026-does-aurora-encode-atmospheric-structure] Layer-wise Relevance Propagation on the Great Storm of 1987 shows the model links surface wind to the ~150 hPa upper troposphere [kasteleyn-lucic-2026-does-aurora-encode-atmospheric-structure] Masking the top-1% LRP-relevant pixels degrades forecasts 3.31x more than random masking (MSE 5.30e4 vs 1.60e4), confirming causal relevance [kasteleyn-lucic-2026-does-aurora-encode-atmospheric-structure]

models: Aurora (weather foundation model) · method: PCA, Layer-wise relevance propagation (LRP)

Bayesian Wind Tunnel Transformer

The Bayesian Geometry of Transformer Attention (2026)measured

Toy transformers build an orthogonal hypothesis frame for Bayesian updating

Details

Agarwal et al. train small from-scratch transformers (a 6-layer/6-head/d192 bijection model, a 9-layer/8-head/d256 HMM model) on synthetic tasks with analytically known Bayesian posteriors [agarwal-etal-2026] The transformers match the Bayes-optimal predictive entropy to 10^-3-10^-4 bits, while capacity-matched MLPs fail by orders of magnitude [agarwal-etal-2026] Layer-0 attention keys form a near-orthogonal "hypothesis frame" (mean off-diagonal key cosine 0.052+/-0.004 vs 0.082+/-0.003 random), built by a single identifiable head [agarwal-etal-2026] Query-key similarity sharpens with depth onto still-feasible hypotheses, mirroring Bayesian conditioning as evidence accumulates [agarwal-etal-2026] PCA of attention-output values reveals a smooth low-dimensional entropy-tracking structure (a curved 1-D entropy manifold) that keeps refining after the key frame stabilizes [agarwal-etal-2026] Layer/head ablations and length-generalization support this as a reused computation; across architectures Transformers realize all three inference primitives, Mamba two, LSTM one, MLP none [agarwal-etal-2026]

models: Bayesian Wind Tunnel Transformer (bijection task, 6 layers, 6 heads, d_model=192), Bayesian Wind Tunnel Transformer (HMM filtering task, 9 layers, 8 heads, d_model=256) · method: Analytical derivation, PCA, Linear probing, Geometric analysis, Causal interventions (steering)

Phi

LLaVA Steering: Visual Instruction Tuning with 500x Fewer Parameters through Modality Linear Representation-Steering (2024)measured

A trained rank-1 steering module rebalances multimodal attention

Details

Bi et al. quantify a Layer-wise Modality Attention Ratio showing textual tokens dominate attention during generation in LLaVA-style multimodal LLMs under full fine-tuning and standard PEFT [bi-etal-2024-llava-steering-visual-subspace-modality-rebalancing] They introduce MoReS, a trained reversible linear map applied to visual tokens at every layer that projects into a rank-d subspace, transforms there, and projects back, with the base LLM frozen -- a learned add-on module, not a structure found in free activations [bi-etal-2024-llava-steering-visual-subspace-modality-rebalancing] Composed into LLaVA Steering models (Phi-2, Vicuna-7B/13B), it matches or exceeds LoRA/Adapter/OFT/IA3 across visual and VQA benchmarks at 287-1,150x fewer trainable parameters, reducing POPE hallucination [bi-etal-2024-llava-steering-visual-subspace-modality-rebalancing] A rank ablation (d in 1,2,4,8) finds the trained rank-1 module already scores highest (81.8 avg) with the fewest parameters, and steering just 1% of visual tokens is optimal [bi-etal-2024-llava-steering-visual-subspace-modality-rebalancing]

models: Phi-2 · method: Modality Linear Representation-Steering (MoReS)
Geometric Factual Recall in Transformers (2026)measured

Factual recall is linear superposition with a proven dimension bound

Details

Ravfogel et al. model subject embeddings as a sum of per-attribute vectors read by a relation-conditioned ReLU-gated MLP, proving (Theorem 4.1) a 1-layer transformer plus 3-layer MLP solves single-hop recall for N entities and R relations when d=4R*log(N)+1 [ravfogel-etal-2026-geometric-factual-recall-transformers] Synthetic experiments confirm the scaling: trainable embeddings memorize once d>=128 while frozen embeddings need d>=512 for R=16 [ravfogel-etal-2026-geometric-factual-recall-transformers] On five real LMs a rank-512 affine probe recovers the LM-head output embedding (best-layer MRR 0.44-0.69 across entity categories), reversing Hernandez et al. reported output-side non-linearity [ravfogel-etal-2026-geometric-factual-recall-transformers] Minimum-norm subject-embedding perturbations swap the queried attribute with high selectivity, and a frozen relation-selector MLP transfers zero-shot to unseen relation bijections [ravfogel-etal-2026-geometric-factual-recall-transformers]

models: Phi-4 (15B) · method: Linear Relational Embedding (LRE), Linear probing
Scale Determines Whether Language Models Organize Representation Geometry for Prediction (2026)measured

Scale determines whether late-layer geometry stays organized by the readout

Details

Xu introduces Subspace PGA, a z-scored RSA metric comparing how well a layer cosine-distance structure survives projection onto the unembedding matrix top-k readout subspace versus 100 random subspaces [xu-2026-scale-determines-representation-geometry-organization-prediction] Across Pythia (70M-6.9B) and OLMo-1B/Phi-1.5/Gemma-2-2B, intermediate geometry is robustly organized around the readout subspace (peak z 9-24 at mid layers) [xu-2026-scale-determines-representation-geometry-organization-prediction] Small models (hidden dim <=1024) progressively lose this at late layers over training even as loss drops (Pythia-410M min z +0.6 to -32 across checkpoints), while dim>=2048 models preserve it [xu-2026-scale-determines-representation-geometry-organization-prediction] Removing a few top principal components restores positive z at every layer for dim>=768 models, supporting a masking interpretation; the paper frames the claim as correlational [xu-2026-scale-determines-representation-geometry-organization-prediction]

models: Phi-1.5 · method: Subspace PGA (Predictive Geometric Alignment)

Vicuna

LLaVA Steering: Visual Instruction Tuning with 500x Fewer Parameters through Modality Linear Representation-Steering (2024)measured

A trained rank-1 steering module rebalances multimodal attention

Details

Bi et al. quantify a Layer-wise Modality Attention Ratio showing textual tokens dominate attention during generation in LLaVA-style multimodal LLMs under full fine-tuning and standard PEFT [bi-etal-2024-llava-steering-visual-subspace-modality-rebalancing] They introduce MoReS, a trained reversible linear map applied to visual tokens at every layer that projects into a rank-d subspace, transforms there, and projects back, with the base LLM frozen -- a learned add-on module, not a structure found in free activations [bi-etal-2024-llava-steering-visual-subspace-modality-rebalancing] Composed into LLaVA Steering models (Phi-2, Vicuna-7B/13B), it matches or exceeds LoRA/Adapter/OFT/IA3 across visual and VQA benchmarks at 287-1,150x fewer trainable parameters, reducing POPE hallucination [bi-etal-2024-llava-steering-visual-subspace-modality-rebalancing] A rank ablation (d in 1,2,4,8) finds the trained rank-1 module already scores highest (81.8 avg) with the fewest parameters, and steering just 1% of visual tokens is optimal [bi-etal-2024-llava-steering-visual-subspace-modality-rebalancing]

models: Vicuna-7B, Vicuna-13B · method: Modality Linear Representation-Steering (MoReS)

TabPFN

A Mechanistic Study of Tabular Foundation Models (2026)measured

Real TabPFNv2, TabICLv2, and Mitra build class-conditional prototype/vote geometry, causally confirmed via mechanism transplantation

Details

Biloš, Wilson, Schneider & Nevmyvaka (2026) study three real pretrained tabular foundation models: TabICLv2 reads out predictions via a nearest-prototype rule (per-class centroid of context-row activations), while TabPFNv2/Mitra use an attention-weighted vote at a specific layer; a linear probe under-explains the readout (0.859 acc) with only marginal gain from a bounding nonlinear-MLP probe [bilos-etal-2026-a-mechanistic-study-of-tabular-foundation-models] Causal interventions are decisive: forcing uniform attention drops TabPFNv2 accuracy from 0.87 to 0.49, transplanting one model's readout rule onto another's activations drops accuracy 30-40 percentage points, and zeroing TabPFNv2's positional-parameter matrix yields exact permutation invariance at no accuracy cost [bilos-etal-2026-a-mechanistic-study-of-tabular-foundation-models]

models: TabPFN v2 (tabular in-context-learning foundation model) · method:
TabPFN Through The Looking Glass: An Interpretability Study of TabPFN and Its Internal Representations (2026)measured

TabPFN linearly encodes regression coefficients and arithmetic intermediates

Details

Gupta, Sethi & Kumar probe TabPFN v2 on synthetic data with known form: for z=alpha*x+beta*y linear probes recover the coefficients with high R^2 (sharp rise at layer 6), and for z=a*b+c a linear probe recovers the intermediate product a*b, concentrated in middle layers [gupta-sethi-kumar-2026-tabpfn-looking-glass] A probe-complexity sweep (linear to deep MLP) shows R^2 decreases monotonically with added complexity, the standard signature of a genuinely linear encoding [gupta-sethi-kumar-2026-tabpfn-looking-glass] A logit-lens vs linear-probe gap (answer decodable at layer 5 but native-space-aligned only by 7-8) is read as computational overthinking [gupta-sethi-kumar-2026-tabpfn-looking-glass] No causal intervention is performed; this is the first such linear-vs-complexity analysis for a tabular foundation model [gupta-sethi-kumar-2026-tabpfn-looking-glass]

models: TabPFN v2 (tabular in-context-learning foundation model) · method: Linear probing, Mlp probing, Logit Lens

TabICL

A Mechanistic Study of Tabular Foundation Models (2026)measured

Real TabPFNv2, TabICLv2, and Mitra build class-conditional prototype/vote geometry, causally confirmed via mechanism transplantation

Details

Biloš, Wilson, Schneider & Nevmyvaka (2026) study three real pretrained tabular foundation models: TabICLv2 reads out predictions via a nearest-prototype rule (per-class centroid of context-row activations), while TabPFNv2/Mitra use an attention-weighted vote at a specific layer; a linear probe under-explains the readout (0.859 acc) with only marginal gain from a bounding nonlinear-MLP probe [bilos-etal-2026-a-mechanistic-study-of-tabular-foundation-models] Causal interventions are decisive: forcing uniform attention drops TabPFNv2 accuracy from 0.87 to 0.49, transplanting one model's readout rule onto another's activations drops accuracy 30-40 percentage points, and zeroing TabPFNv2's positional-parameter matrix yields exact permutation invariance at no accuracy cost [bilos-etal-2026-a-mechanistic-study-of-tabular-foundation-models]

models: TabICLv2 · method:

ruBERT / ruRoBERTa (ai-forever)

Probing the Category of Verbal Aspect in Transformer Language Models (2024)measured

A boundedness subspace causally shifts Russian aspect predictions

Details

Katinskaia & Yangarber train m mutually-orthogonal INLP classifiers per layer to locate the "boundedness" subspace in masked-verb representations of Russian BERT-base, BERT-large and RoBERTa-large [katinskaia-yangarber-2024] Behavioral probing shows aspect becomes decodable mainly in final layers (BERT-large 85-88% in the last 8 layers) [katinskaia-yangarber-2024] AlterRep pushing hidden vectors toward unboundedness raises imperfective predictions +21%/+10.3% and lowers perfective -26%/-18.3% at BERT-large layer 24; pushing toward boundedness has the opposite, smaller effect [katinskaia-yangarber-2024] Twenty random-subspace placebos and an unaffected number-agreement control confirm specificity to the boundedness subspace [katinskaia-yangarber-2024]

models: ruBERT-base, ruBERT-large, ruRoBERTa-large · method: INLP subspace + AlterRep counterfactual push

Alpaca

Interpretability at Scale: Identifying Causal Mechanisms in Alpaca (2023)measured

Boundless DAS scales causal-alignment search to a 7B LLM

Details

Wu et al. scale Distributed Alignment Search to Alpaca-7B on a "price tagging" task using Boundless DAS: a sigmoid-parameterized differentiable boundary that learns the target subspace's dimensionality by gradient descent instead of a brute-force sweep [wu-etal-2023-boundless-das] Applied as a 4096x4096 rotation across 7 layers, two causal models ("Left Boundary"; "Left and Right Boundary", two conjoined booleans) reach interchange-intervention accuracy at or above Alpaca's 85% task accuracy [wu-etal-2023-boundless-das] Alternative causal models fit much worse and a random-rotation control floors around 0.60 IIA [wu-etal-2023-boundless-das] The alignment is robust to unseen brackets (no drop), changed instruction wording (-1%) and added irrelevant context (-2%) [wu-etal-2023-boundless-das] A randomly-initialized LLaMA-7B control aligns no better than a most-frequent-label baseline (~66%), confirming the effect is not architectural; the paper does not restate DAS's caveat that orthogonal rotation assumes rather than discovers linear structure [wu-etal-2023-boundless-das]

models: Alpaca-7B · method: Distributed Alignment Search (DAS)

GPT

A Geometric Notion of Causal Probing (2023)measured

A do-intervention on a LEACE subspace steers number but not gender

Details

Guerner et al. propose a causal graphical model where a latent concept and context jointly generate the hidden state, and formalize a genuine do-intervention on the LEACE-found concept subspace [guerner-etal-2023-geometric-causal-probing] On GPT-2-large, setting verbal number to plural raises the plural verb form's probability in ~90% of contexts, a causally-grounded controlled-generation result [guerner-etal-2023-geometric-causal-probing] The same intervention on grammatical gender in a French GPT-2 fails: setting masculine reduces accuracy and setting feminine has no significant effect [guerner-etal-2023-geometric-causal-probing] Their Theorem 4.1 proves a subspace meeting four intrinsic criteria (erasure, encapsulation, containment, stability) guarantees such interventions succeed, tying the number/gender asymmetry to how completely LEACE captures each concept [guerner-etal-2023-geometric-causal-probing]

models: GPT-2-Large · method: LEACE (LEAst-squares Concept Erasure)
Summing Up the Facts: Additive Mechanisms Behind Factual Recall in LLMs (2024)measured

Factual recall is an additive motif across component types

Details

Chughtai, Cooney & Nanda use Direct Logit Attribution on Pythia-2.8B (primary; validated on GPT2-XL, GPT-J and other Pythia sizes including 6.9B) to dissect factual recall [chughtai-cooney-nanda-2024-summing-up-the-facts-additive-mechanisms-behind-factual-recall-in-llms] The mechanism is an "additive motif": several qualitatively different component types -- subject heads, relation heads, mixed heads, and MLPs -- independently contribute and additively sum, constructively interfering on the correct attribute [chughtai-cooney-nanda-2024-summing-up-the-facts-additive-mechanisms-behind-factual-recall-in-llms] Heads are sorted by an "attention ratio" (DLA to subject vs relation tokens), with strongly specialized values (e.g. up to 107.23 for one head) [chughtai-cooney-nanda-2024-summing-up-the-facts-additive-mechanisms-behind-factual-recall-in-llms] The framing is additivity across component types, not a low-rank per-attribute subspace (the low-rank property is only the generic fact that any attention head reads and writes a low-rank slice of the residual stream) [chughtai-cooney-nanda-2024-summing-up-the-facts-additive-mechanisms-behind-factual-recall-in-llms]

models: GPT-2-XL · method: Direct Logit Attribution (DLA)
Knowledge in Superposition: Unveiling the Failures of Lifelong Knowledge Editing for Large Language Models (2025)measured

Edited facts' key vectors overlap in a whitened memory space

Details

Hu et al. extend ROME's closed-form associative-memory solution to n edits and derive an interference term equal to a whitened normalized dot-product between two facts' key vectors [hu-etal-2025-knowledge-in-superposition] Across 11 LLMs (GPT-2 family, GPT-J-6B, Pythia 1B-6.9B, Llama-2/3 family) the whitened-overlap distribution is heavy-tailed and high-kurtosis with clustered near-maximal overlaps, absent in the raw key vectors ("knowledge superposition") [hu-etal-2025-knowledge-in-superposition] Sequential ROME edits on GPT-J progressively reduce confidence in unrelated held-out facts, tying the measured overlap geometry to a lifelong-editing forgetting failure [hu-etal-2025-knowledge-in-superposition]

models: GPT-2-small, GPT-2-Medium, GPT-2-Large · method: Rank-One Model Editing (ROME) / Linear Associative Memory Editing
Convergent Evolution: How Different Language Models Learn Similar Number Representations (2026)measured

Fourier periodicity in number embeddings is universal; linear separability is not

Details

Fu et al. distinguish spectral convergence (Fourier spikes at periods T=2,5,10 in number-token embeddings) from geometric convergence (linear mod-T decodability) [fu-etal-2026-convergent-evolution-number-representations] Spectral convergence appears everywhere: Transformer and non-Transformer LLMs, classical word embeddings, and even the raw training-corpus token-frequency distribution computed with no model [fu-etal-2026-convergent-evolution-number-representations] Geometric convergence is architecture-selective: controlled 300M models on identical data show Transformers and linear RNNs develop linearly separable mod-T classes while LSTMs, despite stronger Fourier spikes, stay at chance probing [fu-etal-2026-convergent-evolution-number-representations] Theorem 1 proves Fourier sparsity is necessary but not sufficient for mod-T separability, and a Fisher-LDA (T-1)-dimensional discriminant bound (Theorem 2) explains why near-identical spectra give very different probe accuracy [fu-etal-2026-convergent-evolution-number-representations] Controlled data perturbations that remove text-number co-occurrence degrade probing accuracy while leaving the Fourier spectrum unchanged, a data-level causal attribution [fu-etal-2026-convergent-evolution-number-representations] Built from a partial retrieval: the paper's later circular-representation findings for multi-token addition could not be verified and are not asserted here [fu-etal-2026-convergent-evolution-number-representations]

models: GPT-2-small · method: Linear probing
A Geometric Notion of Causal Probing (2023)measured

LEACE's erasure subspace captures only about half of a concept

Details

Guerner et al. define intrinsic, classifier-free criteria for an "ideal" concept subspace (eraser, encapsulator, contained, stable) using only the model's own counterfactual mutual information [guerner-etal-2023-geometric-causal-probing] This addresses Kumar et al.'s critique that naive mutual-information erasure criteria can be fooled by correlated features, fixed by a counterfactual unigram construction forcing independence [guerner-etal-2023-geometric-causal-probing] Using LEACE to find the candidate subspace in GPT-2-large, the one-dimensional English verbal-number subspace has a subspace-info (encapsulation) ratio of only ~0.52-0.55, about half the concept's information, despite provably erasing all linear classifiability [guerner-etal-2023-geometric-causal-probing] For French grammatical gender the subspace captures even less (~0.34, roughly 30%), a lossier partition [guerner-etal-2023-geometric-causal-probing] A do-intervention on the subspace steers generation, tying the intrinsic geometric criteria to actual causal-intervention outcomes [guerner-etal-2023-geometric-causal-probing]

models: GPT-2-Large · method: LEACE (LEAst-squares Concept Erasure)
How Reliable are Causal Probing Interventions? (2025)measured

Nonlinear interventions are more reliable than linear subspace erasure

Details

Canby et al. formalize completeness and selectivity as numbers in [0,1] and define reliability as their harmonic mean, comparing INLP, RLACE, AlterRep and three gradient-based nonlinear interventions [canby-etal-2025-reliability] Every method shows a completeness/selectivity trade-off; none achieves perfect completeness without sacrificing selectivity in any model [canby-etal-2025-reliability] Counterfactual methods (AlterRep, gradient-based) achieve near-total task-accuracy change, consistent with higher completeness than the concept-removal methods [canby-etal-2025-reliability] Nonlinear interventions are almost always most reliable (FGSM 0.92-0.96 across GPT-2/Pythia/Llama) versus linear methods 0.106-0.841, the sole exception being BERT layers 10-12 where AlterRep wins [canby-etal-2025-reliability]

models: GPT-2-small · method: INLP subspace + AlterRep counterfactual push, R-LACE (Relaxed Linear Adversarial Concept Erasure), Gradient-Based Interventions (nonlinear counterfactual push via adversarial attack)
Mapping Language Models to Grounded Conceptual Spaces (2022)measured

Behavioral grounding survives rotation but collapses under random reassignment

Details

Patel & Pavlick test whether LMs' internal representations of color, cardinal directions and grid terms carry the same relational structure as external grounded spaces, via few-shot prompting with no probe training [patel-pavlick-2022-mapping-language-models-to-grounded-conceptual-spaces] GPT-3 (175B) performs similarly on the true grounding and a structure-preserving rotation (spatial 45%/76% vs 44%/75% Top-1/Top-3) but collapses under random reassignment (16-19%) [patel-pavlick-2022-mapping-language-models-to-grounded-conceptual-spaces] Color-grounding error drops from 328 (GPT-2 124M) to 96 (GPT-3 175B), and BERT-base badly underperforms GPT-3 across all conditions [patel-pavlick-2022-mapping-language-models-to-grounded-conceptual-spaces] This is a scale-dependent relational-isomorphism-up-to-rotation claim inferred behaviorally; no activation-space intervention is performed [patel-pavlick-2022-mapping-language-models-to-grounded-conceptual-spaces]

models: GPT-2-XL · method: Rotation-invariance isomorphism test
Model Editing as a Robust and Denoised Variant of DPO: A Case Study on Toxicity (2024)measured

A projected-out toxic subspace beats DPO more sample-efficiently

Details

Uppaal et al. introduce ProFS: from toxic/non-toxic pair embeddings, remove the corpus-mean direction, then SVD the centered differences and keep the top-k (ScreeNot-selected: k=2 GPT-2, k=10 larger models) as a toxic-subspace projector [uppaal-etal-2024-profs-detox-model-editing-dpo] Projecting it out of MLP value matrices once offline drops GPT-2 toxicity 48.0% to 26.8% using 500 pairs, below DPO 36.4% at 2,000 pairs, with comparable perplexity [uppaal-etal-2024-profs-detox-model-editing-dpo] Under label noise up to 50% flipped, DPO degrades monotonically while ProFS stays flat, a consequence of the SVD being provably invariant to individual pair sign-flips [uppaal-etal-2024-profs-detox-model-editing-dpo] The fraction of a single DPO gradient step explained by the toxic subspace rises with sample size and depth, framing ProFS as a denoised DPO step [uppaal-etal-2024-profs-detox-model-editing-dpo]

models: GPT-2-Medium · method: ProFS (Projection Filter for Subspaces) weight-subspace projection
Intervention Lens: from Representation Surgery to String Counterfactuals (2024)measured

A representation intervention inverts back into gender-flipped text

Details

Avitan et al. invert BiasBios representations that have passed through a gender intervention (LEACE erasure or MiMiC/MiMiC+ steering) back into fluent text via a vec2text procedure [avitan-etal-2024-intervention-lens] The recovered text adds only modest fluency cost (Mistral-7B perplexity 18.3-19.1 vs 18.2 for inversion alone, far below the 22.6 of the originals) [avitan-etal-2024-intervention-lens] Human annotators judge the intended gender flip present with substantial reliability (MiMiC female-to-male labeled male 82% vs 3% for originals) [avitan-etal-2024-intervention-lens] A PMI analysis shows the interventions also shift subtler class-correlated vocabulary (female-to-male adds developer/managers; male-to-female adds domestic/maternal) beyond pronouns [avitan-etal-2024-intervention-lens]

models: GPT-2-small · method: LEACE (LEAst-squares Concept Erasure), MiMiC / MiMiC+ (moment-matching steering), Representation Inversion / Intervention Lens (vec2text-based)
Multilinguality as Sense Adaptation (2026)measured

A Backpack model sense-vector topology partially transfers across languages

Details

Cruz, Adelani & Aji adapt a Backpack LM (GPT-2-based, K learned sense vectors per word) from English to Estonian, Turkish, Indonesian and Swahili [cruz-etal-2026-multilinguality-as-sense-adaptation] Sense-topology correlation between English and target-language sense-Gram matrices reaches rho ~0.25-0.30 for SENSIA versus ~0.16-0.21 control, a consistent ~0.09 gap [cruz-etal-2026-multilinguality-as-sense-adaptation] Orthogonal Procrustes alignment to English gives post-alignment cosine 0.35-0.44 versus 0.26-0.33 control [cruz-etal-2026-multilinguality-as-sense-adaptation] Collapsing the soft sense-mixture to top-1 raises FLORES cross-entropy by roughly +6 across all four languages, tying the sense-mixture mechanism to translation quality [cruz-etal-2026-multilinguality-as-sense-adaptation]

models: GPT-2-small, GPT-2-Medium, GPT-2-Large · method: Representational Similarity Analysis (RSA), Orthogonal Procrustes alignment, Causal interventions (steering)

GPT2-base-french

A Geometric Notion of Causal Probing (2023)measured

A do-intervention on a LEACE subspace steers number but not gender

Details

Guerner et al. propose a causal graphical model where a latent concept and context jointly generate the hidden state, and formalize a genuine do-intervention on the LEACE-found concept subspace [guerner-etal-2023-geometric-causal-probing] On GPT-2-large, setting verbal number to plural raises the plural verb form's probability in ~90% of contexts, a causally-grounded controlled-generation result [guerner-etal-2023-geometric-causal-probing] The same intervention on grammatical gender in a French GPT-2 fails: setting masculine reduces accuracy and setting feminine has no significant effect [guerner-etal-2023-geometric-causal-probing] Their Theorem 4.1 proves a subspace meeting four intrinsic criteria (erasure, encapsulation, containment, stability) guarantees such interventions succeed, tying the number/gender asymmetry to how completely LEACE captures each concept [guerner-etal-2023-geometric-causal-probing]

models: GPT2-base-french · method: LEACE (LEAst-squares Concept Erasure)
A Geometric Notion of Causal Probing (2023)measured

LEACE's erasure subspace captures only about half of a concept

Details

Guerner et al. define intrinsic, classifier-free criteria for an "ideal" concept subspace (eraser, encapsulator, contained, stable) using only the model's own counterfactual mutual information [guerner-etal-2023-geometric-causal-probing] This addresses Kumar et al.'s critique that naive mutual-information erasure criteria can be fooled by correlated features, fixed by a counterfactual unigram construction forcing independence [guerner-etal-2023-geometric-causal-probing] Using LEACE to find the candidate subspace in GPT-2-large, the one-dimensional English verbal-number subspace has a subspace-info (encapsulation) ratio of only ~0.52-0.55, about half the concept's information, despite provably erasing all linear classifiability [guerner-etal-2023-geometric-causal-probing] For French grammatical gender the subspace captures even less (~0.34, roughly 30%), a lossier partition [guerner-etal-2023-geometric-causal-probing] A do-intervention on the subspace steers generation, tying the intrinsic geometric criteria to actual causal-intervention outcomes [guerner-etal-2023-geometric-causal-probing]

models: GPT2-base-french · method: LEACE (LEAst-squares Concept Erasure)

GPT-J

Summing Up the Facts: Additive Mechanisms Behind Factual Recall in LLMs (2024)measured

Factual recall is an additive motif across component types

Details

Chughtai, Cooney & Nanda use Direct Logit Attribution on Pythia-2.8B (primary; validated on GPT2-XL, GPT-J and other Pythia sizes including 6.9B) to dissect factual recall [chughtai-cooney-nanda-2024-summing-up-the-facts-additive-mechanisms-behind-factual-recall-in-llms] The mechanism is an "additive motif": several qualitatively different component types -- subject heads, relation heads, mixed heads, and MLPs -- independently contribute and additively sum, constructively interfering on the correct attribute [chughtai-cooney-nanda-2024-summing-up-the-facts-additive-mechanisms-behind-factual-recall-in-llms] Heads are sorted by an "attention ratio" (DLA to subject vs relation tokens), with strongly specialized values (e.g. up to 107.23 for one head) [chughtai-cooney-nanda-2024-summing-up-the-facts-additive-mechanisms-behind-factual-recall-in-llms] The framing is additivity across component types, not a low-rank per-attribute subspace (the low-rank property is only the generic fact that any attention head reads and writes a low-rank slice of the residual stream) [chughtai-cooney-nanda-2024-summing-up-the-facts-additive-mechanisms-behind-factual-recall-in-llms]

models: GPT-J-6B · method: Direct Logit Attribution (DLA)
Knowledge in Superposition: Unveiling the Failures of Lifelong Knowledge Editing for Large Language Models (2025)measured

Edited facts' key vectors overlap in a whitened memory space

Details

Hu et al. extend ROME's closed-form associative-memory solution to n edits and derive an interference term equal to a whitened normalized dot-product between two facts' key vectors [hu-etal-2025-knowledge-in-superposition] Across 11 LLMs (GPT-2 family, GPT-J-6B, Pythia 1B-6.9B, Llama-2/3 family) the whitened-overlap distribution is heavy-tailed and high-kurtosis with clustered near-maximal overlaps, absent in the raw key vectors ("knowledge superposition") [hu-etal-2025-knowledge-in-superposition] Sequential ROME edits on GPT-J progressively reduce confidence in unrelated held-out facts, tying the measured overlap geometry to a lifelong-editing forgetting failure [hu-etal-2025-knowledge-in-superposition]

models: GPT-J-6B · method: Rank-One Model Editing (ROME) / Linear Associative Memory Editing
Linearly Mapping from Image to Text Space (2023)measured

One unconstrained linear map injects vision embeddings into frozen GPT-J

Details

Merullo et al. fit a single unconstrained linear projection (no orthogonality constraint, trained by SGD) mapping vision-encoder embeddings (CLIP RN50x16, NFRN50, BEiT-Large) into frozen 6B GPT-J's token-embedding space [merullo-etal-2023-linearly-mapping-image-to-text-space] Only the linear map is trained; the mapped vectors are fed as soft prompts into the untouched, frozen GPT-J [merullo-etal-2023-linearly-mapping-image-to-text-space] This causally drives competitive zero-shot image captioning (NoCaps/COCO CIDEr-D) and few-shot VQA with no fine-tuning of either model [merullo-etal-2023-linearly-mapping-image-to-text-space] It demonstrates a measured linear relationship between two independently-trained representation spaces plus a genuine behavioral effect [merullo-etal-2023-linearly-mapping-image-to-text-space]

models: GPT-J-6B · method: Learned linear/affine map between two representation spaces
Model Editing as a Robust and Denoised Variant of DPO: A Case Study on Toxicity (2024)measured

A projected-out toxic subspace beats DPO more sample-efficiently

Details

Uppaal et al. introduce ProFS: from toxic/non-toxic pair embeddings, remove the corpus-mean direction, then SVD the centered differences and keep the top-k (ScreeNot-selected: k=2 GPT-2, k=10 larger models) as a toxic-subspace projector [uppaal-etal-2024-profs-detox-model-editing-dpo] Projecting it out of MLP value matrices once offline drops GPT-2 toxicity 48.0% to 26.8% using 500 pairs, below DPO 36.4% at 2,000 pairs, with comparable perplexity [uppaal-etal-2024-profs-detox-model-editing-dpo] Under label noise up to 50% flipped, DPO degrades monotonically while ProFS stays flat, a consequence of the SVD being provably invariant to individual pair sign-flips [uppaal-etal-2024-profs-detox-model-editing-dpo] The fraction of a single DPO gradient step explained by the toxic subspace rises with sample size and depth, framing ProFS as a denoised DPO step [uppaal-etal-2024-profs-detox-model-editing-dpo]

models: GPT-J-6B · method: ProFS (Projection Filter for Subspaces) weight-subspace projection
Linear Relational Decoding of Morphology in Language Models (2025)measured

Morphological relations are truly linear; semantic ones need an affine bias

Details

Xia & Kalita decompose Hernandez et al. affine LRE into a multiplicative Jacobian-only (linear) term and an additive bias term, applying both to 40 BATS relation categories on GPT-J and Llama-2-7B [xia-kalita-2025-linear-relational-decoding-of-morphology] Across 14 morphology relations the bias-free linear LRE reaches 90% top-1 faithfulness, near the full affine LRE 95% [xia-kalita-2025-linear-relational-decoding-of-morphology] For semantic/encyclopedic relations the linear LRE drops to 40% versus the affine LRE 61%, and additive-only approximators fail on morphology, confirming the Jacobian term is necessary [xia-kalita-2025-linear-relational-decoding-of-morphology] The dissociation replicates across architecture (GPT-J vs Llama-2) and 8 typologically diverse languages; no causal intervention is performed [xia-kalita-2025-linear-relational-decoding-of-morphology]

models: GPT-J-6B · method: Linear Relational Embedding (LRE)

CLAP (Contrastive Language-Audio Pretraining)

COMET: Concept Space Dissection of the Modality Gap in Audio-Text Multimodal Contrastive Embeddings (2026)measured

Audio-text embeddings split into shared head and private tail

Details

- A cross-modal PLS-SVD decomposition splits CLAP audio-text embeddings into a static mean component (the gap), a compact shared semantic head (~100 of 1024 dims carrying most cross-modal variance), and a diffuse modality-private tail. [zhu-etal-2026-comet-audio-modality-gap] - Truncating to the top-100 shared axes (PLSHead) preserves retrieval (Clotho Text->Audio R@1 17.42 to 17.32) while cutting MeanR 42.36 to 36.30. [zhu-etal-2026-comet-audio-modality-gap] - A PCA-head ablation collapses retrieval to R@1 ~0.06/0.00, isolating the PLS shared axes as causally responsible; removing the tail drops captioning BLEU-4 to 5.9-7.1. [zhu-etal-2026-comet-audio-modality-gap] - Tested on CLAP HTSAT-BERT (zero-shot) and the DRCap CLAP backbone. [zhu-etal-2026-comet-audio-modality-gap]

models: CLAP (HTSAT-BERT-ZS, trained on WavCaps), DRCap's CLAP (trained on WavCaps + SoundVECaps) · method: Cross-Modal PLS-SVD Decomposition, Geometric analysis, Causal interventions (steering)

Stable Diffusion

Concept Algebra for (Score-Based) Text-Controlled Generative Models (2023)measured

Diffusion concepts occupy provably-separate, editable score subspaces

Details

Wang et al. treat the diffusion score function as the representation and prove (Theorem 3.2) that if the generation is "causally separable" in two concepts, the centered score is arithmetically composable, each concept occupying its own additive subspace [wang-etal-2023-concept-algebra] The concept subspace is estimable from contrastive prompt pairs alone (e.g. "a man" vs "a woman") [wang-etal-2023-concept-algebra] Concept algebra edits generations by projecting and swapping score-subspace components; on Stable Diffusion, 10 raters judged it most faithful to target content and style 47.6% of the time vs ~16% for prompting and naive composition [wang-etal-2023-concept-algebra] A demonstrated failure case (gender transfer on "nurse") arises exactly when the causal-separability hypothesis is violated, linking the theorem's precondition to a practical limit [wang-etal-2023-concept-algebra]

models: Stable Diffusion v1.4 · method: Concept-Subspace Score Projection
Probing the 3D Awareness of Visual Foundation Models (2024)measured

DINOv2 and Stable Diffusion encode depth/normals via nonlinear dense probes

Details

El Banani et al. probe frozen features from DINOv2 (ViT-B/14, ViT-L/14, with-registers), CLIP, MAE, iBOT, Stable Diffusion, DeiT III, SAM and MiDaS for single-view depth and surface normals [el-banani-etal-2024-probing-the-3d-awareness-of-visual-foundation-models] The depth/normal probe is a nonlinear multiscale DPT-style dense decoder, not a linear probe, chosen because 3D properties need not be linearly encoded [el-banani-etal-2024-probing-the-3d-awareness-of-visual-foundation-models] DINOv2 and Stable Diffusion support markedly more accurate depth/normal decoding than CLIP or MAE, whose features capture only coarse scene-layout priors (depth-normal correlation 0.37 image-level vs 0.13 pixel-level for DINOv2) [el-banani-etal-2024-probing-the-3d-awareness-of-visual-foundation-models] Despite strong single-view decodability, all models show weak multiview 3D consistency, so single-view and multiview-consistent 3D awareness are dissociable [el-banani-etal-2024-probing-the-3d-awareness-of-visual-foundation-models]

models: Stable Diffusion v2.1 · method: Mlp probing
Concept Unlearning via Cross-Attention Activation Projection for Diffusion Models (2026)measured

Activation-derived forget/retain subspaces edited into diffusion weights

Details

Moon et al. build forget and retain subspaces via per-layer SVD directly on Stable Diffusion v1.5's cross-attention activations rather than text-embedding space [moon-etal-2026-concept-unlearning-via-cross-attention-activation-projection-for-diffusion-models] An activation-basis binary probe achieves roughly 5x higher recall than a text-basis probe [moon-etal-2026-concept-unlearning-via-cross-attention-activation-projection-for-diffusion-models] A single closed-form linear projector removing the forget subspace while preserving the retain subspace is edited once into the cross-attention K/V weights across 10 concepts [moon-etal-2026-concept-unlearning-via-cross-attention-activation-projection-for-diffusion-models] On the Holistic Unlearning Benchmark it achieves the best per-category trade-off (e.g. IP H-Mean 0.683 vs 0.571 for CURE); the forget/retain subspaces are constructed for unlearning rather than discovered [moon-etal-2026-concept-unlearning-via-cross-attention-activation-projection-for-diffusion-models]

models: Stable Diffusion v1.5 · method: ProFS (Projection Filter for Subspaces) weight-subspace projection
Be Tangential to Manifold: Discovering Riemannian Metric for Diffusion Models (2025)measured

A diffusion score-Jacobian splits into tangent and normal subspaces

Details

Saito & Matsubara define a Riemannian metric on diffusion noise space via the score function Jacobian and prove (Proposition 1) that minimizing ||J v|| for fixed norm pushes v into the small-singular-value tangent subspace, with large singular values forming the normal space [saito-matsubara-2025-tangential-manifold-diffusion] They argue density-based geodesic interpolation biases paths toward high-density regions, producing over-smoothed detail-losing images [saito-matsubara-2025-tangential-manifold-diffusion] Their tangent-constrained geodesics achieve the best FID across MorphBench/CelebA-HQ/AFHQ (e.g. AFHQ 21.01 vs 25.80 for GeodesicDiffusion) [saito-matsubara-2025-tangential-manifold-diffusion] On video frame interpolation they achieve the lowest MSE/LPIPS against ground truth everywhere [saito-matsubara-2025-tangential-manifold-diffusion]

models: Stable Diffusion v2.1 · method: Jacobian spectral direction analysis, Activation Steering (Addition)

GPT-2

The Confidence Manifold: Geometric Structure of Correctness Representations in Language Models (2026)measured

Answer-correctness sits in a 3-8D linear subspace, causally steerable

Details

Cho et al. study self-assessed correctness on TruthfulQA pairs across 9 models spanning 5 families, finding the discriminative signal is a low-dimensional linear subspace, not a curved manifold [cho-etal-2026-confidence-manifold] A PLS sweep peaks at 3-8 dimensions (e.g. peak AUC 0.90 at layer 23, dim 5, Mistral-7B), and nonlinear classifiers give no gain over a linear boundary [cho-etal-2026-confidence-manifold] Correct/incorrect classes form roughly Gaussian clusters separated by a mean shift: a two-mean centroid detector matches the linear probe (0.90 vs 0.89 AUC), robust with 25 examples/class [cho-etal-2026-confidence-manifold] This 3-8D discriminative subspace is lower-dimensional than the representation's 8-12D intrinsic dimension at the same layers [cho-etal-2026-confidence-manifold] Adding the learned direction shifts downstream error rate up to 10.9 points dose-dependently, while random and orthogonal controls have no reliable effect [cho-etal-2026-confidence-manifold]

models: GPT-2 Small, GPT-2 Medium, GPT-2 Large · method: Linear probing, Partial Least Squares (PLS), Difference-in-means direction, Activation Steering (Addition)
Subspace-Aware Sparse Autoencoders for Effective Mechanistic Interpretability (2026)measured

A ~14-D temporal subspace is fragmented by SAEs but captured whole by SASA

Details

Dalili & Mahdavi prove standard single-direction sparse autoencoders are geometrically and dynamically forced to fragment multi-dimensional features into many near-collinear atoms (via a covering-number and basis-instability argument) [dalili-mahdavi-2026-subspace-aware-sparse-autoencoders-for-effective-mechanistic-interpretability] In GPT-2-Small's residual stream, with no SAE, a day/month/year temporal subspace has intrinsic dimension 14 (the 90%-variance dimension of 768) with a cyclic topology [dalili-mahdavi-2026-subspace-aware-sparse-autoencoders-for-effective-mechanistic-interpretability] Standard SAEs fragment this concept into 35 atoms (9 day + 16 month + 10 year, a count from Engels et al. 2025), while their Subspace-Aware SAE captures it in one rank-6 group preserving the cyclic topology [dalili-mahdavi-2026-subspace-aware-sparse-autoencoders-for-effective-mechanistic-interpretability] Feature absorption drops 37.2%->6.6% on GPT-2-Small and 24.0%->18.3% on Mistral-7B-v0.1 at half the training-token budget of a standard SAE [dalili-mahdavi-2026-subspace-aware-sparse-autoencoders-for-effective-mechanistic-interpretability]

models: GPT-2 Small · method: Sparse Autoencoders (SAE)
Invariant Reasoning Directions in Latent Trajectories of Language Models (2026)measured

An invariant reasoning subspace stabilizes latent chain-of-thought refinement

Details

Malarkkan et al. introduce TILR: SVD of per-step contrastive differences between a later- and earlier-checkpoint of the same GPT-2-base Coconut latent-reasoning backbone, keeping the rank r_0.90 explaining >=90% of contrastive variance [malarkkan-etal-2026-invariant-reasoning-directions-latent-trajectories] Across 6 benchmarks r_0.90 ranges from 1 (near-rank-one, coincident with input-PCA) to 27-34 (genuinely refinement-specific structure surviving input-PCA regression) [malarkkan-etal-2026-invariant-reasoning-directions-latent-trajectories] At inference only the projection onto this fixed subspace is applied, scaled by a norm-based reliability gate giving a formal no-harm guarantee [malarkkan-etal-2026-invariant-reasoning-directions-latent-trajectories] Against random, input-PCA, trajectory-PCA and orthogonal-knockout controls it improves accuracy on all 6 benchmarks (mean +2.1%), cuts paraphrase-induced trajectory variance ~39%, and reduces cross-checkpoint variance 53-74%, replicated on GPT-2-Medium and Qwen2.5-Math-1.5B [malarkkan-etal-2026-invariant-reasoning-directions-latent-trajectories]

models: GPT-2 Small, GPT-2 Medium · method: Trajectory-Invariant Latent Refinement (TILR), Causal interventions (steering)
The Representational Geometry of Number (2026)measured

Number-task subspaces are distinct yet linearly inter-transformable

Details

Hu, Niu & Varma analyze number-concept representations (comparison, parity, primality, successor/predecessor) across BERT, GPT-2, Qwen2.5-7B and Qwen2.5-Math-7B at ~75% depth [hu-etal-2026-the-representational-geometry-of-number] Task-specific subspaces are distinct via Procrustes disparity (~0.010 vs a permutation baseline 0.077-0.273) yet linearly inter-transformable (SVCCA canonical correlations ~0.80-0.90) [hu-etal-2026-the-representational-geometry-of-number] The parity axis sits nearly orthogonal to the magnitude axis (88.8 degrees) while the primality axis is only partly decoupled (68.8 degrees) [hu-etal-2026-the-representational-geometry-of-number] Subspace overlap between most task pairs stays below 0.2, except the closely-related successor/predecessor pair (0.36-0.42) [hu-etal-2026-the-representational-geometry-of-number]

models: GPT-2 Small · method: Generalized Procrustes Analysis (GPA), Singular Vector Canonical Correlation Analysis (SVCCA), Direction Extraction, Linear probing, Geometric analysis
Mapping Language Models to Grounded Conceptual Spaces (2022)measured

Behavioral grounding survives rotation but collapses under random reassignment

Details

Patel & Pavlick test whether LMs' internal representations of color, cardinal directions and grid terms carry the same relational structure as external grounded spaces, via few-shot prompting with no probe training [patel-pavlick-2022-mapping-language-models-to-grounded-conceptual-spaces] GPT-3 (175B) performs similarly on the true grounding and a structure-preserving rotation (spatial 45%/76% vs 44%/75% Top-1/Top-3) but collapses under random reassignment (16-19%) [patel-pavlick-2022-mapping-language-models-to-grounded-conceptual-spaces] Color-grounding error drops from 328 (GPT-2 124M) to 96 (GPT-3 175B), and BERT-base badly underperforms GPT-3 across all conditions [patel-pavlick-2022-mapping-language-models-to-grounded-conceptual-spaces] This is a scale-dependent relational-isomorphism-up-to-rotation claim inferred behaviorally; no activation-space intervention is performed [patel-pavlick-2022-mapping-language-models-to-grounded-conceptual-spaces]

models: GPT-2 Small · method: Rotation-invariance isomorphism test
Cognitive Maps in Language Models: A Mechanistic Analysis of Spatial Planning (2025)measured

A GPT-2 trained on grid navigation forms an orthogonal xy coordinate map

Details

Baumgartner et al. train GPT-2 small from scratch on three grid-navigation paradigms on a 4x4 grid [baumgartner-etal-2025-cognitive-maps-in-language-models] For the foraging task, PCA on node-token hidden states aligns the top-2 PCs at layer 7 with the grid x/y axes, and a linear probe recovers (x,y) at R^2=0.93 (layer 8) with near-orthogonal basis vectors (cos ~ -0.04) [baumgartner-etal-2025-cognitive-maps-in-language-models] Layer-by-layer ablation shows a sharp phase transition: loop-completion accuracy is near-zero when ablating before layer 8 but jumps to 100% at layer 8+, so the internal map becomes causally self-sufficient there [baumgartner-etal-2025-cognitive-maps-in-language-models] Goal-directed shortest-path variants show no orthogonal coordinate structure and no phase transition, a task-dependent dissociation [baumgartner-etal-2025-cognitive-maps-in-language-models]

models: Custom GPT-2-Small (trained from scratch on synthetic grid-navigation token sequences) · method: PCA, Linear probing, Causal interventions (steering)
Geometry of Semantics in Next-Token Prediction: How Optimization Implicitly Organizes Linguistic Representations (2025)measured

NTP implicitly SVD-factorizes co-occurrence; orthants recover semantic categories

Details

Zhao & Thrampoulidis show next-token-prediction implicitly performs an SVD of a centered context-by-next-token co-occurrence matrix [zhao-thrampoulidis-2025-geometry-of-semantics-in-ntp] Clustering GPT-2 embeddings by which orthant (sign pattern) of the singular-vector basis they occupy recovers grammatical categories, entity types and topics unsupervised [zhao-thrampoulidis-2025-geometry-of-semantics-in-ntp] The recovered category structure becomes progressively finer-grained as more singular components are included [zhao-thrampoulidis-2025-geometry-of-semantics-in-ntp] It is validated on synthetic co-occurrence data plus TinyStories and WikiText-2, on GPT-2 (variant unspecified); purely observational with no causal steering [zhao-thrampoulidis-2025-geometry-of-semantics-in-ntp]

models: GPT-2 Small · method: Next-token-prediction SVD orthant decomposition

Mistral

The Confidence Manifold: Geometric Structure of Correctness Representations in Language Models (2026)measured

Answer-correctness sits in a 3-8D linear subspace, causally steerable

Details

Cho et al. study self-assessed correctness on TruthfulQA pairs across 9 models spanning 5 families, finding the discriminative signal is a low-dimensional linear subspace, not a curved manifold [cho-etal-2026-confidence-manifold] A PLS sweep peaks at 3-8 dimensions (e.g. peak AUC 0.90 at layer 23, dim 5, Mistral-7B), and nonlinear classifiers give no gain over a linear boundary [cho-etal-2026-confidence-manifold] Correct/incorrect classes form roughly Gaussian clusters separated by a mean shift: a two-mean centroid detector matches the linear probe (0.90 vs 0.89 AUC), robust with 25 examples/class [cho-etal-2026-confidence-manifold] This 3-8D discriminative subspace is lower-dimensional than the representation's 8-12D intrinsic dimension at the same layers [cho-etal-2026-confidence-manifold] Adding the learned direction shifts downstream error rate up to 10.9 points dose-dependently, while random and orthogonal controls have no reliable effect [cho-etal-2026-confidence-manifold]

models: Mistral-7B-Instruct-v0.3 · method: Linear probing, Partial Least Squares (PLS), Difference-in-means direction, Activation Steering (Addition)
The Geometry of Numerical Reasoning: Language Models Compare Numeric Properties in Linear Subspaces (2024)measured

PLS-found numeric-comparison subspaces causally control LLM answers

Details

El-Shangiti et al. show LLMs answering numeric-comparison questions encode each entity's numeric attribute in a low-dimensional linear subspace, found via a 5-component PLS regression on the last-token state [el-shangiti-etal-2024-geometry-of-numerical-reasoning] The fit reaches R^2 > 0.8 for birth year, death year and latitude across Llama-3-8B-Instruct, Mistral-7B-Instruct and Qwen2.5-7B-Instruct [el-shangiti-etal-2024-geometry-of-numerical-reasoning] Intervening along the first PLS component flips the model's Yes/No comparison answer far more than an equal-magnitude random control, concentrated in roughly the first half of layers [el-shangiti-etal-2024-geometry-of-numerical-reasoning]

models: Mistral-7B-Instruct · method: Partial least squares (PLS) subspace probing
Subspace-Aware Sparse Autoencoders for Effective Mechanistic Interpretability (2026)measured

A ~14-D temporal subspace is fragmented by SAEs but captured whole by SASA

Details

Dalili & Mahdavi prove standard single-direction sparse autoencoders are geometrically and dynamically forced to fragment multi-dimensional features into many near-collinear atoms (via a covering-number and basis-instability argument) [dalili-mahdavi-2026-subspace-aware-sparse-autoencoders-for-effective-mechanistic-interpretability] In GPT-2-Small's residual stream, with no SAE, a day/month/year temporal subspace has intrinsic dimension 14 (the 90%-variance dimension of 768) with a cyclic topology [dalili-mahdavi-2026-subspace-aware-sparse-autoencoders-for-effective-mechanistic-interpretability] Standard SAEs fragment this concept into 35 atoms (9 day + 16 month + 10 year, a count from Engels et al. 2025), while their Subspace-Aware SAE captures it in one rank-6 group preserving the cyclic topology [dalili-mahdavi-2026-subspace-aware-sparse-autoencoders-for-effective-mechanistic-interpretability] Feature absorption drops 37.2%->6.6% on GPT-2-Small and 24.0%->18.3% on Mistral-7B-v0.1 at half the training-token budget of a standard SAE [dalili-mahdavi-2026-subspace-aware-sparse-autoencoders-for-effective-mechanistic-interpretability]

models: Mistral-7B-v0.1 · method: Sparse Autoencoders (SAE)
Geometric Asymmetry in MoE Specialization: Functional Decorrelation and Representational Overlap (2026)measured

MoE experts are functionally decorrelated but only partly subspace-separated

Details

Liu measures two pretrained sparse MoE models, Mixtral-8x7B (8 experts/layer, top-2) and Qwen1.5-MoE-A2.7B (60 experts/layer, top-4) [liu-2026-geometric-asymmetry-moe-specialization] Cross-expert Jacobian cosine similarity clusters near zero (Mistral middle-layer mean 0.062; Qwen ~0.000-0.001), showing experts are strongly functionally decorrelated [liu-2026-geometric-asymmetry-moe-specialization] Yet their top-5-PCA subspaces sit at Grassmannian distances (2.06-2.69) well below the theoretical maximum ~3.51, so subspaces are distinct but only partially separated [liu-2026-geometric-asymmetry-moe-specialization] A from-scratch 8-expert Transformer isolates routing's causal role: mean Grassmann distance is 2.463 under top-k routing vs 0.480 under fully-soft routing [liu-2026-geometric-asymmetry-moe-specialization]

models: Mixtral-8x7B · method: Cross-expert Jacobian alignment, Subspace-angle alignment
Model Editing as a Robust and Denoised Variant of DPO: A Case Study on Toxicity (2024)measured

A projected-out toxic subspace beats DPO more sample-efficiently

Details

Uppaal et al. introduce ProFS: from toxic/non-toxic pair embeddings, remove the corpus-mean direction, then SVD the centered differences and keep the top-k (ScreeNot-selected: k=2 GPT-2, k=10 larger models) as a toxic-subspace projector [uppaal-etal-2024-profs-detox-model-editing-dpo] Projecting it out of MLP value matrices once offline drops GPT-2 toxicity 48.0% to 26.8% using 500 pairs, below DPO 36.4% at 2,000 pairs, with comparable perplexity [uppaal-etal-2024-profs-detox-model-editing-dpo] Under label noise up to 50% flipped, DPO degrades monotonically while ProFS stays flat, a consequence of the SVD being provably invariant to individual pair sign-flips [uppaal-etal-2024-profs-detox-model-editing-dpo] The fraction of a single DPO gradient step explained by the toxic subspace rises with sample size and depth, framing ProFS as a denoised DPO step [uppaal-etal-2024-profs-detox-model-editing-dpo]

models: Mistral-7B, Mistral-7B-SFT-Beta · method: ProFS (Projection Filter for Subspaces) weight-subspace projection
Emotions Where Art Thou: Characterizing the Emotional Latent Space of LLMs (2025)measured

A 40-dim SVD emotional subspace with four interpretable axes steers emotion

Details

Reichman et al. extract a low-dimensional emotional subspace from mean-pooled hidden states of Llama-3.1-8B, OLMo-2-7B and Ministral-8B via centered SVD, using its top 40 dimensions [reichman-etal-2025-emotional-latent-space-llms] The four leading PCs align with valence, dominance, approach-avoidance and arousal, stable across layers (Spearman up to 0.92 for emotion ordering) [reichman-etal-2025-emotional-latent-space-llms] A linear cross-domain alignment map generalizes the subspace across 8 emotion datasets in 6 languages (aligned-subspace cosine 0.83-0.94) [reichman-etal-2025-emotional-latent-space-llms] A trained one-layer MLP mapping the 40D projection to a residual shift raises target-emotion accuracy from 9% to 83% (English, Llama) while preserving semantics [reichman-etal-2025-emotional-latent-space-llms]

models: Ministral-8B-Instruct-2410 · method: PCA, Linear probing, Representation Alignment
Intervention Lens: from Representation Surgery to String Counterfactuals (2024)measured

A representation intervention inverts back into gender-flipped text

Details

Avitan et al. invert BiasBios representations that have passed through a gender intervention (LEACE erasure or MiMiC/MiMiC+ steering) back into fluent text via a vec2text procedure [avitan-etal-2024-intervention-lens] The recovered text adds only modest fluency cost (Mistral-7B perplexity 18.3-19.1 vs 18.2 for inversion alone, far below the 22.6 of the originals) [avitan-etal-2024-intervention-lens] Human annotators judge the intended gender flip present with substantial reliability (MiMiC female-to-male labeled male 82% vs 3% for originals) [avitan-etal-2024-intervention-lens] A PMI analysis shows the interventions also shift subtler class-correlated vocabulary (female-to-male adds developer/managers; male-to-female adds domestic/maternal) beyond pronouns [avitan-etal-2024-intervention-lens]

models: Mistral-7B · method: LEACE (LEAst-squares Concept Erasure), MiMiC / MiMiC+ (moment-matching steering), Representation Inversion / Intervention Lens (vec2text-based)
Understanding and Preserving Safety in Fine-Tuned LLMs (2026)measured

Safety gradients occupy a low-rank subspace orthogonal to utility

Details

Zhang et al. SVD per-layer safety- versus utility-task gradients during fine-tuning, finding safety gradients occupy a compact low-rank subspace with sharp singular-value decay while utility spans much higher dimension [zhang-etal-2026-understanding-and-preserving-safety-in-fine-tuned-llms] Safety and utility gradient directions have cosine similarity oscillating around zero and often negative (directional conflict) [zhang-etal-2026-understanding-and-preserving-safety-in-fine-tuned-llms] Their Safety-Preserving Fine-tuning projects utility gradients onto the safety subspace orthogonal complement during training [zhang-etal-2026-understanding-and-preserving-safety-in-fine-tuned-llms] This cuts attack success rate 0.955 to 0.019 (Llama-3.1-8B), 0.985 to 0.240 (Mistral) and 0.988 to 0.124 (Qwen2.5-7B) while preserving MMLU, robust to deep fine-tuning and multiple jailbreaks [zhang-etal-2026-understanding-and-preserving-safety-in-fine-tuned-llms]

models: Mistral-7B-Instruct-v0.3 · method: PCA
Dual-Stance Evaluation of Sycophancy: The Structure of Agreement and the Limits of Intervention (2026)measured

Sycophantic and factual agreement occupy distinct but inseparable subspaces

Details

Buchan shows a centroid-difference sycophancy-reduction steering vector in Llama-3-8B-Instruct is non-specific, also suppressing agreement with correct facts (20% drop on the Earth is round) [buchan-2026-dual-stance-evaluation-of-sycophancy] Susceptibility is continuously predictable from a behavioral dual-stance consistency measure (r=0.88 in-sample, 0.84 held-out) [buchan-2026-dual-stance-evaluation-of-sycophancy] Grassmann analysis shows sycophantic- and factual-agreement activations occupy geometrically distinct subspaces (Grassmann similarity 0.15-0.20 vs 0.32 random; principal angles 65-89 degrees) [buchan-2026-dual-stance-evaluation-of-sycophancy] Yet the steering direction projects nearly equally onto both subspaces (ratio 0.90-0.97), so the model distinguishes the two agreement types while the linear intervention cannot [buchan-2026-dual-stance-evaluation-of-sycophancy]

models: Mistral-7B-Instruct-v0.1 · method: Activation Steering (Addition)

FastText

Word Translation Without Parallel Data (2018)measured

MUSE aligns monolingual embeddings with an unsupervised orthogonal map

Details

Conneau et al. (MUSE) show two independently, monolingually trained fastText embedding spaces are related by a single orthogonal linear map, recoverable with no parallel data [conneau-etal-2018-muse-word-translation-without-parallel-data] An adversarially-trained map is iteratively refined via orthogonal Procrustes on its own mutual-nearest-neighbor pairs [conneau-etal-2018-muse-word-translation-without-parallel-data] The resulting map achieves bilingual dictionary induction accuracy rivaling or exceeding supervised baselines across several language pairs [conneau-etal-2018-muse-word-translation-without-parallel-data] It is a foundational precursor to later per-language affine cross-lingual-overlap and vec2vec-style universal-alignment findings [conneau-etal-2018-muse-word-translation-without-parallel-data]

models: FastText (bag-of-word-vectors) · method: Unsupervised adversarial + Procrustes cross-space alignment
Convergent Evolution: How Different Language Models Learn Similar Number Representations (2026)measured

Fourier periodicity in number embeddings is universal; linear separability is not

Details

Fu et al. distinguish spectral convergence (Fourier spikes at periods T=2,5,10 in number-token embeddings) from geometric convergence (linear mod-T decodability) [fu-etal-2026-convergent-evolution-number-representations] Spectral convergence appears everywhere: Transformer and non-Transformer LLMs, classical word embeddings, and even the raw training-corpus token-frequency distribution computed with no model [fu-etal-2026-convergent-evolution-number-representations] Geometric convergence is architecture-selective: controlled 300M models on identical data show Transformers and linear RNNs develop linearly separable mod-T classes while LSTMs, despite stronger Fourier spikes, stay at chance probing [fu-etal-2026-convergent-evolution-number-representations] Theorem 1 proves Fourier sparsity is necessary but not sufficient for mod-T separability, and a Fisher-LDA (T-1)-dimensional discriminant bound (Theorem 2) explains why near-identical spectra give very different probe accuracy [fu-etal-2026-convergent-evolution-number-representations] Controlled data perturbations that remove text-number co-occurrence degrade probing accuracy while leaving the Fourier spectrum unchanged, a data-level causal attribution [fu-etal-2026-convergent-evolution-number-representations] Built from a partial retrieval: the paper's later circular-representation findings for multi-token addition could not be verified and are not asserted here [fu-etal-2026-convergent-evolution-number-representations]

models: FastText (bag-of-word-vectors) · method: Linear probing
What Does Debiasing Really Remove? A Geometric Study of PCA-Based Gender Debiasing in Word Embeddings (2026)measured

Direct gender bias is rank-1; associative WEAT bias is distributed

Details

Kresin et al. build the Bolukbasi gender-difference matrix from definitional word pairs, form its covariance and analyze the PCA spectrum on GloVe, word2vec and FastText embeddings (the paper does not specify the exact training corpora) [kresin-etal-2026-what-does-debiasing-really-remove] The spectrum is strongly low-rank: PC1 alone captures ~48% of variance, the first 5 ~80%, and the first 10 exceed 93% [kresin-etal-2026-what-does-debiasing-really-remove] Direct projection-based bias collapses to near-zero after removing only PC1 and stays flat, so it is genuinely rank-1-dominated across all three embeddings [kresin-etal-2026-what-does-debiasing-really-remove] WEAT associative bias is not reduced by removing any single dominant component and stays distributed across many higher-dimensional directions [kresin-etal-2026-what-does-debiasing-really-remove] Mean vector displacement rises monotonically and neighbor stability declines as more components are removed, a distortion trade-off with no universal optimal k [kresin-etal-2026-what-does-debiasing-really-remove]

models: FastText (bag-of-word-vectors) · method: PCA
Understanding Linearity of Cross-Lingual Word Embedding Mappings (2020)measured

Analogy preservation is necessary and sufficient for a linear cross-lingual map

Details

Peng et al. prove a necessary-and-sufficient equivalence between within-language analogy (parallelogram) preservation and linearity of the ground-truth cross-lingual embedding map [peng-etal-2020-understanding-linearity-cross-lingual-mappings] They test it against real monolingual embeddings across 12 languages (fastText-Wiki, fastText-CommonCrawl, word2vec-CoNLL) using a new 5-category cross-lingual analogy dataset [peng-etal-2020-understanding-linearity-cross-lingual-mappings] Mapping-linearity quality tracks analogy-preservation quality empirically [peng-etal-2020-understanding-linearity-cross-lingual-mappings] No causal intervention is performed [peng-etal-2020-understanding-linearity-cross-lingual-mappings]

models: FastText (bag-of-word-vectors), fastText (Common Crawl, 300d) · method: Analytical derivation
Discovering Universal Geometry in Embeddings with ICA (2023)measured

ICA after PCA-whitening recovers a universal sparse axis-aligned decomposition

Details

Yamagiwa et al. apply PCA-whitening then ICA to their own text8-trained word2vec (SGNS), cross-lingual fastText (7 languages), BERT contextual embeddings, and image encoders (ViT, ResMLP, Swin, ResNet, RegNet) [yamagiwa-etal-2023-discovering-universal-geometry-ica] The independent axes are individually interpretable and sparse, unlike PCA arbitrary rotation [yamagiwa-etal-2023-discovering-universal-geometry-ica] Specific axes can be matched across languages, models and modalities via cross-model component correlation, a privileged non-arbitrary decomposition [yamagiwa-etal-2023-discovering-universal-geometry-ica] No causal intervention is performed; axis interpretability is partly qualitative and universality depends on whitening choices [yamagiwa-etal-2023-discovering-universal-geometry-ica]

models: FastText (bag-of-word-vectors) · method: ICA axis decomposition

CosyVoice

A Geometric Perspective on Composable Emotion Steering in Text-to-Speech Models (2026)measured

CosyVoice2's LM module has a clean composable emotion subspace

Details

Wang, Bailey & Dang compare CosyVoice2's SLM (24-layer Qwen2.5-based) and CFM (56-layer DiT) modules as emotion-steering sites [wang-etal-2026-a-geometric-perspective-on-composable-emotion-steering-in-tts] Linear probing gives SLM emotion accuracy 0.80 within / 0.71 cross-speaker vs CFM 0.89 / 0.62, indicating more speaker entanglement in CFM [wang-etal-2026-a-geometric-perspective-on-composable-emotion-steering-in-tts] Local intrinsic dimensionality shows SLM's emotion representation is low-dimensional (~28D) and speaker-invariant (delta-LID +0.84), while CFM entangles speaker and emotion on a shared ~13D manifold (delta-LID -1.48) [wang-etal-2026-a-geometric-perspective-on-composable-emotion-steering-in-tts] Composed mean-difference steering vectors applied at SLM layers 14/17 give better proportional mixed-emotion control than CFM on CREMA-D/IEMOCAP [wang-etal-2026-a-geometric-perspective-on-composable-emotion-steering-in-tts]

models: CosyVoice2 · method: Linear probing, Intrinsic dimension estimation (TwoNN), Difference-in-means direction, Causal interventions (steering)

Conneau et al. Monolingual MLM

Emerging Cross-lingual Structure in Pretrained Language Models (2020)measured

Independent monolingual LMs learn alignable isomorphic geometries

Details

Conneau et al. (2020) train five same-architecture masked LMs from scratch, one per language (en/fr/de/ru/zh), with no shared parameters, vocabulary or parallel data [conneau-etal-2020] Orthogonal Procrustes mapping between language pairs recovers substantial cross-lingual structure at subword, word and sentence granularity, improving at higher layers [conneau-etal-2020] CKA between paired parallel-sentence representations shows monolingual models are more similar to each other than to a random encoder but less than bilingually co-trained models (e.g. en-fr 0.58/0.59 vs 0.69 vs 0.46) [conneau-etal-2020] CKA similarity correlates with alignment-based sentence-retrieval performance at >0.9 Pearson, evidencing universal latent symmetries across independently-trained models [conneau-etal-2020]

models: Conneau et al. Monolingual MLM (English), Conneau et al. Monolingual MLM (French), Conneau et al. Monolingual MLM (German), Conneau et al. Monolingual MLM (Russian), Conneau et al. Monolingual MLM (Chinese) · method: Orthogonal Procrustes alignment, Centered Kernel Alignment (CKA)

XLM-RoBERTa

The Geometry of Multilingual Language Model Representations (2022)measured

After mean-centering, 88 languages share one XLM-R subspace

Details

Chang, Tu & Bergen fit a per-language affine subspace (mean plus top singular directions, median rank 335/768) to XLM-R-base for 88 languages [chang-tu-bergen-2022] Using a Riemannian covariance-distance on positive-definite matrices, layers 6-11 place any two languages' subspaces within a ~5 degree rotation or 1.6x scaling of each other [chang-tu-bergen-2022] Projecting a representation onto its own language's subspace barely raises perplexity, onto another's raises it substantially, but onto another's shifted to the first's mean is only moderately worse, so once mean-corrected the subspaces are largely interchangeable [chang-tu-bergen-2022] Shifting a representation by the difference of two language means induces target-language token predictions 4.7x more often (10%->47%) while dropping source-language ones 3.5x (75%->21%) [chang-tu-bergen-2022]

models: XLM-RoBERTa base · method: Subspace-angle alignment, Causal interventions (steering)

GTR (Generalizable T5-based dense Retriever)

mini-vec2vec: Scaling Universal Geometry Alignment with Linear Transformations (2025)measured

mini-vec2vec aligns embedding models with a cheap orthogonal map

Details

Dar replaces vec2vec's adversarial unpaired embedding translation with a three-stage linear pipeline: k-means cluster-landmark matching, Quadratic-Assignment-Problem permutation solving, and ICP-style orthogonal Procrustes refinement [dar-2025-mini-vec2vec-linear-universal-geometry-alignment] Evaluated on the four text encoders vec2vec used (gtr, e5, stella, granite) over the 12 directed pairs, it matches or exceeds vec2vec's top-1 accuracy and average rank on nearly every pair [dar-2025-mini-vec2vec-linear-universal-geometry-alignment] It never collapses on the pairs where adversarial vec2vec was weakest, runs in under 10 minutes on CPU versus 1-7 GPU-days, and shows <=0.01 run-to-run standard deviation [dar-2025-mini-vec2vec-linear-universal-geometry-alignment] This reduces vec2vec's implicit geometric claim to a literal distance-preserving orthogonal rotation between two embedding geometries, recoverable without paired data [dar-2025-mini-vec2vec-linear-universal-geometry-alignment]

models: GTR-base · method: Orthogonal Procrustes alignment
Intervention Lens: from Representation Surgery to String Counterfactuals (2024)measured

A representation intervention inverts back into gender-flipped text

Details

Avitan et al. invert BiasBios representations that have passed through a gender intervention (LEACE erasure or MiMiC/MiMiC+ steering) back into fluent text via a vec2text procedure [avitan-etal-2024-intervention-lens] The recovered text adds only modest fluency cost (Mistral-7B perplexity 18.3-19.1 vs 18.2 for inversion alone, far below the 22.6 of the originals) [avitan-etal-2024-intervention-lens] Human annotators judge the intended gender flip present with substantial reliability (MiMiC female-to-male labeled male 82% vs 3% for originals) [avitan-etal-2024-intervention-lens] A PMI analysis shows the interventions also shift subtler class-correlated vocabulary (female-to-male adds developer/managers; male-to-female adds domestic/maternal) beyond pronouns [avitan-etal-2024-intervention-lens]

models: GTR-base · method: LEACE (LEAst-squares Concept Erasure), MiMiC / MiMiC+ (moment-matching steering), Representation Inversion / Intervention Lens (vec2text-based)

E5

mini-vec2vec: Scaling Universal Geometry Alignment with Linear Transformations (2025)measured

mini-vec2vec aligns embedding models with a cheap orthogonal map

Details

Dar replaces vec2vec's adversarial unpaired embedding translation with a three-stage linear pipeline: k-means cluster-landmark matching, Quadratic-Assignment-Problem permutation solving, and ICP-style orthogonal Procrustes refinement [dar-2025-mini-vec2vec-linear-universal-geometry-alignment] Evaluated on the four text encoders vec2vec used (gtr, e5, stella, granite) over the 12 directed pairs, it matches or exceeds vec2vec's top-1 accuracy and average rank on nearly every pair [dar-2025-mini-vec2vec-linear-universal-geometry-alignment] It never collapses on the pairs where adversarial vec2vec was weakest, runs in under 10 minutes on CPU versus 1-7 GPU-days, and shows <=0.01 run-to-run standard deviation [dar-2025-mini-vec2vec-linear-universal-geometry-alignment] This reduces vec2vec's implicit geometric claim to a literal distance-preserving orthogonal rotation between two embedding geometries, recoverable without paired data [dar-2025-mini-vec2vec-linear-universal-geometry-alignment]

models: E5-base-v2 · method: Orthogonal Procrustes alignment
Characterizing Linear Alignment Across Language Models (2026)measured

Independently-trained embedding models share linear structure (CKA 0.60-0.88)

Details

Gorbett & Jana measure linear CKA across five independently-trained embedding models (OpenAI text-embedding-3-small, Cohere embed, Gemini embedding-001, Qwen3-Embedding-8B, e5-mistral-7b-instruct), finding similarity 0.595-0.881 [gorbett-jana-2026-characterizing-linear-alignment-across-language-models] In a separate experiment they fit learned affine maps between instruction-tuned generative LMs (Llama-3-8B, Qwen2.5-7B/14B, Llama-3.2-1B, OLMo-7B and others) [gorbett-jana-2026-characterizing-linear-alignment-across-language-models] These maps preserve classification accuracy and OOD-AUROC across model pairs (e.g. 94.5%->93.1%) [gorbett-jana-2026-characterizing-linear-alignment-across-language-models] They enable zero-shot cross-model text generation via a frozen target head, with generation quality correlating with tokenizer exact-match rate (r=0.898) and vocabulary Jaccard overlap (r=0.822) [gorbett-jana-2026-characterizing-linear-alignment-across-language-models]

models: E5-mistral-7b-instruct · method: Centered Kernel Alignment (CKA), Singular Vector Canonical Correlation Analysis (SVCCA)
Aligning Sentence Embeddings to Human Concepts via Sparse Autoencoders (2026)measured

A Top-k SAE on sentence embeddings gives near-orthogonal, steerable latents

Details

Shin & Kim train a Top-k SAE (12x expansion, k=32) on real E5-large-v2 sentence embeddings over ~3.8M WikiText-103 sentences [shin-kim-2026-aligning-sentence-embeddings-saes] Decoder directions are near-orthogonal (mean pairwise cosine 0.0408) at 92.6% explained variance [shin-kim-2026-aligning-sentence-embeddings-saes] Zeroing specific interpretable latents (auto-labeled via GPT-4o-mini) measurably re-ranks retrieval results (an original top-1 result dropping to 9th place) [shin-kim-2026-aligning-sentence-embeddings-saes] The authors flag it as a single-backbone, English-only, proof-of-concept study [shin-kim-2026-aligning-sentence-embeddings-saes]

models: E5-large-v2 · method: TopK (k-sparse) Sparse Autoencoders, Causal interventions (steering)

Stella

mini-vec2vec: Scaling Universal Geometry Alignment with Linear Transformations (2025)measured

mini-vec2vec aligns embedding models with a cheap orthogonal map

Details

Dar replaces vec2vec's adversarial unpaired embedding translation with a three-stage linear pipeline: k-means cluster-landmark matching, Quadratic-Assignment-Problem permutation solving, and ICP-style orthogonal Procrustes refinement [dar-2025-mini-vec2vec-linear-universal-geometry-alignment] Evaluated on the four text encoders vec2vec used (gtr, e5, stella, granite) over the 12 directed pairs, it matches or exceeds vec2vec's top-1 accuracy and average rank on nearly every pair [dar-2025-mini-vec2vec-linear-universal-geometry-alignment] It never collapses on the pairs where adversarial vec2vec was weakest, runs in under 10 minutes on CPU versus 1-7 GPU-days, and shows <=0.01 run-to-run standard deviation [dar-2025-mini-vec2vec-linear-universal-geometry-alignment] This reduces vec2vec's implicit geometric claim to a literal distance-preserving orthogonal rotation between two embedding geometries, recoverable without paired data [dar-2025-mini-vec2vec-linear-universal-geometry-alignment]

models: Stella-base-en-v2 · method: Orthogonal Procrustes alignment

Granite Embedding

mini-vec2vec: Scaling Universal Geometry Alignment with Linear Transformations (2025)measured

mini-vec2vec aligns embedding models with a cheap orthogonal map

Details

Dar replaces vec2vec's adversarial unpaired embedding translation with a three-stage linear pipeline: k-means cluster-landmark matching, Quadratic-Assignment-Problem permutation solving, and ICP-style orthogonal Procrustes refinement [dar-2025-mini-vec2vec-linear-universal-geometry-alignment] Evaluated on the four text encoders vec2vec used (gtr, e5, stella, granite) over the 12 directed pairs, it matches or exceeds vec2vec's top-1 accuracy and average rank on nearly every pair [dar-2025-mini-vec2vec-linear-universal-geometry-alignment] It never collapses on the pairs where adversarial vec2vec was weakest, runs in under 10 minutes on CPU versus 1-7 GPU-days, and shows <=0.01 run-to-run standard deviation [dar-2025-mini-vec2vec-linear-universal-geometry-alignment] This reduces vec2vec's implicit geometric claim to a literal distance-preserving orthogonal rotation between two embedding geometries, recoverable without paired data [dar-2025-mini-vec2vec-linear-universal-geometry-alignment]

models: Granite-Embedding-278M-Multilingual · method: Orthogonal Procrustes alignment

Hierarchical-Equality MLP (custom, Geiger et al. 2023)

Finding Alignments Between Interpretable Causal Variables and Distributed Neural Representations (2024)measured

A learned rotation subspace beats brute-force causal-alignment search

Details

Geiger et al. test DAS on two tasks with known causal structure; on a hierarchical-equality ReLU network its learned orthogonal-rotation subspace reaches interchange-intervention accuracy up to 1.00 vs 0.60 brute-force and 0.73 best localist [geiger-etal-2023-das] On monotonicity NLI (BERT-base fine-tuned on MultiNLI then MoNLI) DAS reaches 1.00 IIA at layer 9 vs 0.64 and 0.51 [geiger-etal-2023-das] A randomly-initialized network stays near chance (~0.50) unless the hidden dimension is blown up to 4096, confirming DAS is not fabricating structure [geiger-etal-2023-das] The paper is explicit that DAS USES an assumed linear structure as a methodological device and does not itself claim to have discovered that any concept is linearly encoded [geiger-etal-2023-das]

models: 3-layer ReLU feedforward network (hierarchical equality task) · method: Distributed Alignment Search (DAS)

DINOv2

Probing the 3D Awareness of Visual Foundation Models (2024)measured

DINOv2 and Stable Diffusion encode depth/normals via nonlinear dense probes

Details

El Banani et al. probe frozen features from DINOv2 (ViT-B/14, ViT-L/14, with-registers), CLIP, MAE, iBOT, Stable Diffusion, DeiT III, SAM and MiDaS for single-view depth and surface normals [el-banani-etal-2024-probing-the-3d-awareness-of-visual-foundation-models] The depth/normal probe is a nonlinear multiscale DPT-style dense decoder, not a linear probe, chosen because 3D properties need not be linearly encoded [el-banani-etal-2024-probing-the-3d-awareness-of-visual-foundation-models] DINOv2 and Stable Diffusion support markedly more accurate depth/normal decoding than CLIP or MAE, whose features capture only coarse scene-layout priors (depth-normal correlation 0.37 image-level vs 0.13 pixel-level for DINOv2) [el-banani-etal-2024-probing-the-3d-awareness-of-visual-foundation-models] Despite strong single-view decodability, all models show weak multiview 3D consistency, so single-view and multiview-consistent 3D awareness are dissociable [el-banani-etal-2024-probing-the-3d-awareness-of-visual-foundation-models]

models: DINOv2 ViT-B/14, DINOv2-B (ViT-Base, 4 register tokens) · method: Mlp probing
Understanding Geometric Representations in Self-Supervised Vision Transformers via Subspace Intervention (2026)measured

SVD-extracted ViT-L task subspaces recover most full-feature performance

Details

Zhou et al. SVD-decompose converged linear-probe weights for depth, normals and segmentation on frozen DINOv2 ViT-L/14, MAE ViT-Large and iBOT ViT-Large features (ViT-Base only as auxiliary validation) [zhou-etal-2026-understanding-geometric-representations-in-self-supervised-vision-transformers-via-subspace-intervention] Projecting features onto the extracted top-k subspace recovers the vast majority of full-feature performance (DINOv2 alignment 0.916-0.948) while random and orthogonal-residual subspaces collapse to near noise [zhou-etal-2026-understanding-geometric-representations-in-self-supervised-vision-transformers-via-subspace-intervention] DINOv2 concentrates 72.5% of its geometric energy in intermediate layers with subspace similarity >0.93 across seeds at rank <=16 [zhou-etal-2026-understanding-geometric-representations-in-self-supervised-vision-transformers-via-subspace-intervention] MAE saturates over 98% of its linear potential by rank 32 while DINOv2 needs rank >=64, a difference in intrinsic task-subspace dimensionality [zhou-etal-2026-understanding-geometric-representations-in-self-supervised-vision-transformers-via-subspace-intervention]

models: DINOv2 ViT-L/14 · method: Linear probing, Causal interventions (steering)

CLIP (Contrastive Language-Image Pretraining)

Probing the 3D Awareness of Visual Foundation Models (2024)measured

DINOv2 and Stable Diffusion encode depth/normals via nonlinear dense probes

Details

El Banani et al. probe frozen features from DINOv2 (ViT-B/14, ViT-L/14, with-registers), CLIP, MAE, iBOT, Stable Diffusion, DeiT III, SAM and MiDaS for single-view depth and surface normals [el-banani-etal-2024-probing-the-3d-awareness-of-visual-foundation-models] The depth/normal probe is a nonlinear multiscale DPT-style dense decoder, not a linear probe, chosen because 3D properties need not be linearly encoded [el-banani-etal-2024-probing-the-3d-awareness-of-visual-foundation-models] DINOv2 and Stable Diffusion support markedly more accurate depth/normal decoding than CLIP or MAE, whose features capture only coarse scene-layout priors (depth-normal correlation 0.37 image-level vs 0.13 pixel-level for DINOv2) [el-banani-etal-2024-probing-the-3d-awareness-of-visual-foundation-models] Despite strong single-view decodability, all models show weak multiview 3D consistency, so single-view and multiview-consistent 3D awareness are dissociable [el-banani-etal-2024-probing-the-3d-awareness-of-visual-foundation-models]

models: CLIP ViT-L/14 · method: Mlp probing
Canonicalizing Multimodal Contrastive Representation Learning (2026)measured

Independent multimodal models share one orthogonal map across both modalities

Details

Gupta et al. show independently-trained multimodal contrastive models' embedding spaces are related, up to a mean shift, by a single orthogonal map Q applied identically to both the image and text encoders [gupta-etal-2026-canonicalizing-multimodal-contrastive-representation-learning] They prove that kernel agreement on a small image-text anchor set forces this single-shared-orthogonal-map relationship [gupta-etal-2026-canonicalizing-multimodal-contrastive-representation-learning] It is verified across CLIP, SigLIP and FLAVA models with different architectures and training data [gupta-etal-2026-canonicalizing-multimodal-contrastive-representation-learning] This is a discovered (not imposed) geometric relationship, solved in closed form via the orthogonal Procrustes SVD [gupta-etal-2026-canonicalizing-multimodal-contrastive-representation-learning]

models: CLIP ViT-B/32, CLIP ViT-L/14 · method: Orthogonal Procrustes alignment, Representation Alignment
Cross-Modal Redundancy and the Geometry of Vision-Language Embeddings (2026)measured

Iso-energy SAE isolates a bimodal cross-modal subspace

Details

- An iso-energy-regularized aligned SAE splits its dictionary into bimodal atoms (a modality-agnostic shared subspace carrying essentially all cross-modal alignment) and unimodal atoms (per-modality cones that account for the modality gap). [dhimoila-etal-2026-cross-modal-redundancy] - Ablating the unimodal atoms nearly eliminates the modality gap while barely changing retrieval recall (CLIP-B/32 delta_r 0.224 to 0.125 from SAE to SAE-A; functional alignment rho 0.327 to 4.232). [dhimoila-etal-2026-cross-modal-redundancy] - Restricting vector-arithmetic edits to the bimodal subspace keeps them in-distribution and improves OOD retrieval (CLIP 0.97 to 0.77, SigLIP2 0.99 to 0.61); reconstruction R-squared >=0.859. [dhimoila-etal-2026-cross-modal-redundancy] - The paper measures retrieval (delta_r), not post-ablation zero-shot classification; tested on CLIP and OpenCLIP ViT-B/32 and ViT-L/14, SigLIP and SigLIP2. [dhimoila-etal-2026-cross-modal-redundancy]

models: CLIP ViT-B/32, CLIP ViT-L/14 · method: Iso-Energy regularized sparse autoencoder (SAE-A), Causal interventions (steering)
Linearly Mapping from Image to Text Space (2023)measured

One unconstrained linear map injects vision embeddings into frozen GPT-J

Details

Merullo et al. fit a single unconstrained linear projection (no orthogonality constraint, trained by SGD) mapping vision-encoder embeddings (CLIP RN50x16, NFRN50, BEiT-Large) into frozen 6B GPT-J's token-embedding space [merullo-etal-2023-linearly-mapping-image-to-text-space] Only the linear map is trained; the mapped vectors are fed as soft prompts into the untouched, frozen GPT-J [merullo-etal-2023-linearly-mapping-image-to-text-space] This causally drives competitive zero-shot image captioning (NoCaps/COCO CIDEr-D) and few-shot VQA with no fine-tuning of either model [merullo-etal-2023-linearly-mapping-image-to-text-space] It demonstrates a measured linear relationship between two independently-trained representation spaces plus a genuine behavioral effect [merullo-etal-2023-linearly-mapping-image-to-text-space]

models: CLIP ResNet-50x16 · method: Learned linear/affine map between two representation spaces
Text-to-Concept (and Back) via Cross-Model Alignment (2023)measured

An affine map between two vision models matches their PCA components

Details

Moayeri et al. fit an affine least-squares map between the activation spaces of independently-trained vision models (supervised/robust ResNets, Swin/DeiT/ConViT, self-supervised MoCo/DINO-ViT-S/SimCLR ResNets and ViTs, and CLIP), scored by R^2 [moayeri-etal-2023-text-to-concept-and-back-via-cross-model-alignment] The fitted map explains R^2 above 0.6, and the top principal components of the two aligned spaces correspond approximately one-to-one [moayeri-etal-2023-text-to-concept-and-back-via-cross-model-alignment] Aligning a vision encoder into CLIP's concept space enables zero-shot concept-bottleneck classification at up to 93.8% accuracy and over 92% concept-to-text relevance [moayeri-etal-2023-text-to-concept-and-back-via-cross-model-alignment] The analysis is passive, with no causal intervention on either model [moayeri-etal-2023-text-to-concept-and-back-via-cross-model-alignment]

models: CLIP ResNet-50, CLIP ViT-B/32 · method: Learned linear/affine map between two representation spaces
Parts of Speech-Grounded Subspaces in Vision-Language Models (2023)measured

Fitted part-of-speech CLIP subspaces disentangle content from appearance

Details

Oldfield et al. FIT part-of-speech-grounded subspaces in CLIP's joint space using WordNet word lists, via a closed-form class-contrastive trace-maximization objective (a constructed subspace, not one found in free activations) [oldfield-etal-2023-pos-clip-subspaces] Because CLIP embeddings live on a hypersphere, the objective is solved in the tangent space at the intrinsic mean and mapped back, outperforming the flat-Euclidean variant [oldfield-etal-2023-pos-clip-subspaces] Class-invariance metrics confirm each noun/adjective/verb/adverb subspace captures variance nearly exclusively from its own class, beating PCA and Principal Geodesic Analysis [oldfield-etal-2023-pos-clip-subspaces] Projecting a prompt embedding onto the orthogonal complement of the adjective (or artist) subspace before CLIP-conditioned generation selectively removes an artist style or a theme like gore [oldfield-etal-2023-pos-clip-subspaces]

models: CLIP ViT-B/32 · method: Class-contrastive trace maximization, Causal interventions (steering)
Decipher the Modality Gap in Multimodal Contrastive Learning: From Convergent Representations to Pairwise Alignment (2025)measured

Modality gap converges to angle between collapsed hyperplanes

Details

- The modality gap originates from dimension collapse: image and text representations each collapse onto distinct low-dimensional hyperplanes. [yi-etal-2025-decipher-the-modality-gap-in-multimodal-contrastive-learning] - At the contrastive optimum the modality means become orthogonal to the shared subspace, so the gap converges to the smallest angle between the two hyperplanes (Theorem 3). [yi-etal-2025-decipher-the-modality-gap-in-multimodal-contrastive-learning] - On CLIP ViT-B/32 the gap angle is 74.69 deg (CIFAR-10), 74.19 and 71.02 (ImageNet); a shared-subspace projection reduces it to 5.37/30.39/50.40, with an estimated ~212-dim shared space. [yi-etal-2025-decipher-the-modality-gap-in-multimodal-contrastive-learning] - Single model CLIP ViT-B/32; ViT-L/14 and RN50 are cited but not tested, and interventions are post-hoc geometric (projection/translation/rotation). [yi-etal-2025-decipher-the-modality-gap-in-multimodal-contrastive-learning]

models: CLIP ViT-B/32 · method: Geometric analysis, Causal interventions (steering)

Vision Transformer (ViT)

Probing the 3D Awareness of Visual Foundation Models (2024)measured

DINOv2 and Stable Diffusion encode depth/normals via nonlinear dense probes

Details

El Banani et al. probe frozen features from DINOv2 (ViT-B/14, ViT-L/14, with-registers), CLIP, MAE, iBOT, Stable Diffusion, DeiT III, SAM and MiDaS for single-view depth and surface normals [el-banani-etal-2024-probing-the-3d-awareness-of-visual-foundation-models] The depth/normal probe is a nonlinear multiscale DPT-style dense decoder, not a linear probe, chosen because 3D properties need not be linearly encoded [el-banani-etal-2024-probing-the-3d-awareness-of-visual-foundation-models] DINOv2 and Stable Diffusion support markedly more accurate depth/normal decoding than CLIP or MAE, whose features capture only coarse scene-layout priors (depth-normal correlation 0.37 image-level vs 0.13 pixel-level for DINOv2) [el-banani-etal-2024-probing-the-3d-awareness-of-visual-foundation-models] Despite strong single-view decodability, all models show weak multiview 3D consistency, so single-view and multiview-consistent 3D awareness are dissociable [el-banani-etal-2024-probing-the-3d-awareness-of-visual-foundation-models]

models: MAE ViT-Base (Masked Autoencoder) · method: Mlp probing
Robust Representation Learning in Masked Autoencoders (2026)measured

MAE per-class patch-token subspaces rotate apart with depth

Details

Shrivastava et al. take the pretrained MAE ViT-Base encoder and, per class, SVD the visible patch-token embeddings at each layer to get a k-dimensional class subspace [shrivastava-etal-2026-robust-representation-learning-mae] The smallest principal angle between per-class subspaces grows with depth, so class-specific subspaces rotate apart and become well-separated without labels, corroborated by t-SNE around layers 7-9 [shrivastava-etal-2026-robust-representation-learning-mae] This class structure is linked to MAE's persistent global attention (mean attention distance ~80-120px at all layers, unlike standard ViT's local-to-global progression) [shrivastava-etal-2026-robust-representation-learning-mae] Fine-tuned MAE stays above 80% accuracy under Gaussian blur and above 60% at 90% occlusion, with correctly-classified images showing higher clean-vs-perturbed cosine similarity; no causal intervention is performed [shrivastava-etal-2026-robust-representation-learning-mae]

models: MAE ViT-Base (Masked Autoencoder) · method: Subspace-angle alignment
Probing the Mid-level Vision Capabilities of Self-Supervised Learning (2024)measured

Mid-level vision decodability weakly correlates with recognition accuracy

Details

Chen, Marks & Cheng introduce eight frozen-feature benchmarks for mid-level vision (segmentation, geometric/3D grouping) across 22 self-supervised models spanning contrastive, clustering, pretext and masked paradigms [chen-etal-2024-probing-the-mid-level-vision-capabilities-of-self-supervised-learning] The benchmarks decode with a nonlinear DPT dense decoder on frozen features, not a linear probe [chen-etal-2024-probing-the-mid-level-vision-capabilities-of-self-supervised-learning] Mid-level task performance correlates only weakly with each model's high-level ImageNet accuracy (generic segmentation strongest ~R^2 0.70; 3D understanding weakly correlated) [chen-etal-2024-probing-the-mid-level-vision-capabilities-of-self-supervised-learning] Several models are strongly imbalanced across the two capability classes, so mid-level decodability is not simply a byproduct of overall recognition quality [chen-etal-2024-probing-the-mid-level-vision-capabilities-of-self-supervised-learning]

models: MAE ViT-Base (Masked Autoencoder) · method: Mlp probing
Understanding Geometric Representations in Self-Supervised Vision Transformers via Subspace Intervention (2026)measured

SVD-extracted ViT-L task subspaces recover most full-feature performance

Details

Zhou et al. SVD-decompose converged linear-probe weights for depth, normals and segmentation on frozen DINOv2 ViT-L/14, MAE ViT-Large and iBOT ViT-Large features (ViT-Base only as auxiliary validation) [zhou-etal-2026-understanding-geometric-representations-in-self-supervised-vision-transformers-via-subspace-intervention] Projecting features onto the extracted top-k subspace recovers the vast majority of full-feature performance (DINOv2 alignment 0.916-0.948) while random and orthogonal-residual subspaces collapse to near noise [zhou-etal-2026-understanding-geometric-representations-in-self-supervised-vision-transformers-via-subspace-intervention] DINOv2 concentrates 72.5% of its geometric energy in intermediate layers with subspace similarity >0.93 across seeds at rank <=16 [zhou-etal-2026-understanding-geometric-representations-in-self-supervised-vision-transformers-via-subspace-intervention] MAE saturates over 98% of its linear potential by rank 32 while DINOv2 needs rank >=64, a difference in intrinsic task-subspace dimensionality [zhou-etal-2026-understanding-geometric-representations-in-self-supervised-vision-transformers-via-subspace-intervention]

models: MAE ViT-Large (Masked Autoencoder) · method: Linear probing, Causal interventions (steering)
Discovering Universal Geometry in Embeddings with ICA (2023)measured

ICA after PCA-whitening recovers a universal sparse axis-aligned decomposition

Details

Yamagiwa et al. apply PCA-whitening then ICA to their own text8-trained word2vec (SGNS), cross-lingual fastText (7 languages), BERT contextual embeddings, and image encoders (ViT, ResMLP, Swin, ResNet, RegNet) [yamagiwa-etal-2023-discovering-universal-geometry-ica] The independent axes are individually interpretable and sparse, unlike PCA arbitrary rotation [yamagiwa-etal-2023-discovering-universal-geometry-ica] Specific axes can be matched across languages, models and modalities via cross-model component correlation, a privileged non-arbitrary decomposition [yamagiwa-etal-2023-discovering-universal-geometry-ica] No causal intervention is performed; axis interpretability is partly qualitative and universality depends on whitening choices [yamagiwa-etal-2023-discovering-universal-geometry-ica]

models: ViT-Base · method: ICA axis decomposition

iBOT

Probing the 3D Awareness of Visual Foundation Models (2024)measured

DINOv2 and Stable Diffusion encode depth/normals via nonlinear dense probes

Details

El Banani et al. probe frozen features from DINOv2 (ViT-B/14, ViT-L/14, with-registers), CLIP, MAE, iBOT, Stable Diffusion, DeiT III, SAM and MiDaS for single-view depth and surface normals [el-banani-etal-2024-probing-the-3d-awareness-of-visual-foundation-models] The depth/normal probe is a nonlinear multiscale DPT-style dense decoder, not a linear probe, chosen because 3D properties need not be linearly encoded [el-banani-etal-2024-probing-the-3d-awareness-of-visual-foundation-models] DINOv2 and Stable Diffusion support markedly more accurate depth/normal decoding than CLIP or MAE, whose features capture only coarse scene-layout priors (depth-normal correlation 0.37 image-level vs 0.13 pixel-level for DINOv2) [el-banani-etal-2024-probing-the-3d-awareness-of-visual-foundation-models] Despite strong single-view decodability, all models show weak multiview 3D consistency, so single-view and multiview-consistent 3D awareness are dissociable [el-banani-etal-2024-probing-the-3d-awareness-of-visual-foundation-models]

models: iBOT ViT-B/16 · method: Mlp probing
Understanding Geometric Representations in Self-Supervised Vision Transformers via Subspace Intervention (2026)measured

SVD-extracted ViT-L task subspaces recover most full-feature performance

Details

Zhou et al. SVD-decompose converged linear-probe weights for depth, normals and segmentation on frozen DINOv2 ViT-L/14, MAE ViT-Large and iBOT ViT-Large features (ViT-Base only as auxiliary validation) [zhou-etal-2026-understanding-geometric-representations-in-self-supervised-vision-transformers-via-subspace-intervention] Projecting features onto the extracted top-k subspace recovers the vast majority of full-feature performance (DINOv2 alignment 0.916-0.948) while random and orthogonal-residual subspaces collapse to near noise [zhou-etal-2026-understanding-geometric-representations-in-self-supervised-vision-transformers-via-subspace-intervention] DINOv2 concentrates 72.5% of its geometric energy in intermediate layers with subspace similarity >0.93 across seeds at rank <=16 [zhou-etal-2026-understanding-geometric-representations-in-self-supervised-vision-transformers-via-subspace-intervention] MAE saturates over 98% of its linear potential by rank 32 while DINOv2 needs rank >=64, a difference in intrinsic task-subspace dimensionality [zhou-etal-2026-understanding-geometric-representations-in-self-supervised-vision-transformers-via-subspace-intervention]

models: iBOT ViT-L/16 · method: Linear probing, Causal interventions (steering)

OLMo

Language Models Compare Quantities Using Number-specific and Unit-specific Heuristics (2026)measured

DAS finds number- and unit-specific quantity-comparison subspaces

Details

Sasaki et al. use Distributed Alignment Search (1024 intervention dimensions, 512 per variable) to locate linear subspaces aligned with a NumLogDiff and a UnitLogDiff variable in quantity-comparison activations [sasaki-etal-2026-language-models-compare-quantities-using-number-specific-and-unit-specific-heuristics] Interchange Intervention Accuracy exceeds 0.9 in many middle layers versus near-chance baselines [sasaki-etal-2026-language-models-compare-quantities-using-number-specific-and-unit-specific-heuristics] A linear surrogate over these variables predicts the model's comparison preference at R^2 approximately 0.817 overall (0.763 near the decision boundary) for Qwen3-4B-Base [sasaki-etal-2026-language-models-compare-quantities-using-number-specific-and-unit-specific-heuristics] This supports a "bag of number-specific and unit-specific heuristics" account over exact-scale unit conversion [sasaki-etal-2026-language-models-compare-quantities-using-number-specific-and-unit-specific-heuristics]

models: OLMo 3 7B · method: Distributed Alignment Search (DAS), Causal interventions (steering)
Characterizing Linear Alignment Across Language Models (2026)measured

Independently-trained embedding models share linear structure (CKA 0.60-0.88)

Details

Gorbett & Jana measure linear CKA across five independently-trained embedding models (OpenAI text-embedding-3-small, Cohere embed, Gemini embedding-001, Qwen3-Embedding-8B, e5-mistral-7b-instruct), finding similarity 0.595-0.881 [gorbett-jana-2026-characterizing-linear-alignment-across-language-models] In a separate experiment they fit learned affine maps between instruction-tuned generative LMs (Llama-3-8B, Qwen2.5-7B/14B, Llama-3.2-1B, OLMo-7B and others) [gorbett-jana-2026-characterizing-linear-alignment-across-language-models] These maps preserve classification accuracy and OOD-AUROC across model pairs (e.g. 94.5%->93.1%) [gorbett-jana-2026-characterizing-linear-alignment-across-language-models] They enable zero-shot cross-model text generation via a frozen target head, with generation quality correlating with tokenizer exact-match rate (r=0.898) and vocabulary Jaccard overlap (r=0.822) [gorbett-jana-2026-characterizing-linear-alignment-across-language-models]

models: OLMo-7B · method: Centered Kernel Alignment (CKA), Singular Vector Canonical Correlation Analysis (SVCCA)
Scale Determines Whether Language Models Organize Representation Geometry for Prediction (2026)measured

Scale determines whether late-layer geometry stays organized by the readout

Details

Xu introduces Subspace PGA, a z-scored RSA metric comparing how well a layer cosine-distance structure survives projection onto the unembedding matrix top-k readout subspace versus 100 random subspaces [xu-2026-scale-determines-representation-geometry-organization-prediction] Across Pythia (70M-6.9B) and OLMo-1B/Phi-1.5/Gemma-2-2B, intermediate geometry is robustly organized around the readout subspace (peak z 9-24 at mid layers) [xu-2026-scale-determines-representation-geometry-organization-prediction] Small models (hidden dim <=1024) progressively lose this at late layers over training even as loss drops (Pythia-410M min z +0.6 to -32 across checkpoints), while dim>=2048 models preserve it [xu-2026-scale-determines-representation-geometry-organization-prediction] Removing a few top principal components restores positive z at every layer for dim>=768 models, supporting a masking interpretation; the paper frames the claim as correlational [xu-2026-scale-determines-representation-geometry-organization-prediction]

models: OLMo-1B · method: Subspace PGA (Predictive Geometric Alignment)

wav2vec 2.0

Beyond Decodability: Reconstructing Language Model Representations with an Encoding Probe (2026)measured

An encoding probe decomposes feature-group variance in wav2vec2 and BERT

Details

Shen et al. fit ridge-regression "encoding probes" that reconstruct wav2vec2-base and BERT-base activations from interpretable feature sets (acoustics, phonetics, speaker-identity, syntax, lexicon), reporting unexplained variance (1 minus R-squared) under feature-group ablation [shen-etal-2026-beyond-decodability-reconstructing-language-model-representations-with-an-encoding-probe] Speaker-identity's contribution to explained variance shifts substantially depending on whether wav2vec2 is left self-supervised or fine-tuned for ASR versus speaker-ID [shen-etal-2026-beyond-decodability-reconstructing-language-model-representations-with-an-encoding-probe] Syntactic and lexical feature groups contribute largely independently and additively, with unexplained-variance differences stable to within 0.002 across seeds [shen-etal-2026-beyond-decodability-reconstructing-language-model-representations-with-an-encoding-probe] No causal intervention is performed; the analysis is explicitly observational [shen-etal-2026-beyond-decodability-reconstructing-language-model-representations-with-an-encoding-probe]

models: Wav2Vec 2.0 Base (LibriSpeech-960h) · method: Closed-form affine-regression ablation of a continuous conditioning variable
Analyzing Language and Geographical Variation in Speech Representations Across 60 Indic Languages (2026)measured

Indic dialect geography nests linearly inside each language's subspace

Details

Kumar J et al. fine-tune Whisper-base and Wav2Vec2.0-base on 60 Indic languages/165 districts under joint versus language-only supervision [kumarj-etal-2026-analyzing-language-and-geographical-variation-in-speech-representations-across-60-indic-languages] Language-conditioned logistic-regression probes decode district structure within each language's subspace (Whisper-base mean F1 91.29 joint vs 80.89 language-only; Wav2Vec2 87.14 vs 77.05) [kumarj-etal-2026-analyzing-language-and-geographical-variation-in-speech-representations-across-60-indic-languages] A normalized conditional mutual-information analysis reveals an asymmetric hierarchical geometry (AUC_D|L=2.68 vs AUC_L|D=5.43) with district clusters nested inside language structure [kumarj-etal-2026-analyzing-language-and-geographical-variation-in-speech-representations-across-60-indic-languages]

models: Wav2Vec 2.0 Base (LibriSpeech-960h) · method: Linear probing
Self-Supervised Speech Models Encode Phonetic Context via Position-dependent Orthogonal Subspaces (2026)measured

Speech models encode neighbor-phone info in orthogonal positional subspaces

Details

Choi et al. show a single S3M frame (wav2vec 2.0, HuBERT, WavLM Large) encodes phonological vectors not only for the current phone but for its neighbors, extracted via difference-of-means at relative positions -2 to +2 [choi-etal-2026-position-dependent-orthogonal-subspaces] Cosine similarity between vectors from different relative positions is substantially lower than within the same position across 8 phonological features, an orthogonality-of-subspaces structure preserved layer-by-layer [choi-etal-2026-position-dependent-orthogonal-subspaces] Vector norm decreases monotonically with distance from the center phone, with WavLM showing the clearest trapezoidal effective context window [choi-etal-2026-position-dependent-orthogonal-subspaces] The position-dependent subspace in use switches at annotated TIMIT phonetic boundaries rather than a fixed temporal window; no causal intervention is performed [choi-etal-2026-position-dependent-orthogonal-subspaces]

models: wav2vec 2.0 Large (LV-60) · method: Difference-in-means direction, Geometric analysis
Self-Supervised Models of Speech Infer Universal Articulatory Kinematics (2023)measured

Speech SSL features linearly recover universal articulatory kinematics

Details

Cho et al. linearly project frozen features from HuBERT-Large, wav2vec2-Large, XLS-R (300M) and MMS (1B) onto electromagnetic-articulography ground truth across 62 speakers and 5 corpora (English, Mandarin, Italian) [cho-etal-2023-self-supervised-models-of-speech-infer-universal-articulatory-kinematics] Average correlation exceeds 0.8 regardless of the SSL model training language (English 0.835, XLS-R 0.830, MMS 0.832), far above a raw-acoustic baseline (0.66) [cho-etal-2023-self-supervised-models-of-speech-infer-universal-articulatory-kinematics] 44% of speakers exceed 0.85 correlation [cho-etal-2023-self-supervised-models-of-speech-infer-universal-articulatory-kinematics] Individual speakers separately-fit linear inversion systems are mutually alignable by affine transformations at near/above-0.85 correlation, evidencing a shared canonical articulatory subspace across speakers, genders and languages [cho-etal-2023-self-supervised-models-of-speech-infer-universal-articulatory-kinematics]

models: wav2vec 2.0 Large (LV-60), XLS-R (300M, 53 languages), MMS (1B, 1000+ languages) · method: Linear probing
Emergence of Phonemic, Syntactic, and Semantic Representations in Artificial Neural Networks (2026)measured

Structural probes show staged phonemic-then-lexical-then-syntactic subspaces

Details

Orhan et al. train Hewitt-Manning distance-based structural probes to recover phoneme, WordNet lexical-semantic and syntactic-tree subspaces from wav2vec2 (tiny/base/large) and text models (Llama-2-13B, Pythia), tracked across pretraining [orhan-etal-2026-emergence-of-phonemic-syntactic-and-semantic-representations-in-artificial-neural-networks] Both modalities show staged emergence: phonemic structure first, then lexical, then syntactic, mirroring child language-acquisition order but needing 2-4 orders of magnitude more data [orhan-etal-2026-emergence-of-phonemic-syntactic-and-semantic-representations-in-artificial-neural-networks] On Llama-2-13B the syntactic and semantic subspaces are near-orthogonal (mean-squared cosine of principal angles 0.038-0.042), so they are distinct low-dimensional subspaces [orhan-etal-2026-emergence-of-phonemic-syntactic-and-semantic-representations-in-artificial-neural-networks] A contrastive topology-preserving probe confirms the distance-based probe validity [orhan-etal-2026-emergence-of-phonemic-syntactic-and-semantic-representations-in-artificial-neural-networks]

models: Wav2Vec 2.0 Base (LibriSpeech-960h), wav2vec 2.0 Large · method: Structural probing (metric/distance probe)

Custom Research Transformer (small, purpose-built for interpretability studies)

From Memories to Maps: Mechanisms of In-Context Reinforcement Learning in Transformers (2026)measured

An in-context-RL transformer linearly decodes XY position by layer 2

Details

Fang & Rajan train a 3-layer 512-dim causal Transformer from scratch via decision-pretraining meta-RL on gridworld (5x5) and tree-maze tasks [fang-rajan-2026-from-memories-to-maps] XY position is reliably linearly decodable from layer-2 node representations, and kernel alignment to the latent environment structure grows with in-context length, strongest at layer 2 [fang-rajan-2026-from-memories-to-maps] Representations align across differently-cued gridworld environments via cross-context kernel alignment, a cognitive-map structure emerging from in-context experience without ground-truth coordinates [fang-rajan-2026-from-memories-to-maps] Memory-token attention ablation shows the model causally selects shortcut paths in over 60% of held-out test environments [fang-rajan-2026-from-memories-to-maps]

models: Custom 3-layer decision-pretraining Transformer (Fang & Rajan 2026) · method: Linear probing, Centered Kernel Alignment (CKA)

Impala-CNN Atari encoder

Proto-Value Networks: Scaling Representation Learning with Auxiliary Tasks (2023)measured

A frozen proto-value subspace is competitive with DQN at ~13x fewer steps

Details

Farebrother et al. train an Impala-CNN encoder on offline RL Unplugged Atari (46 games) to estimate the successor measure, proving its features span the top singular vectors (graph-Laplacian eigenvectors / proto-value functions) [farebrother-etal-2023-proto-value-networks] Freezing this encoder and training only a linear RL head online is competitive with DQN using 3.75M agent steps versus DQN's 50M steps (200M frames), about 13x fewer interactions (not 50x) [farebrother-etal-2023-proto-value-networks] Per-game score deltas are large (e.g. Asterix 15,401 vs 2,953; DemonAttack 78,671 vs 4,428; Pong 20.1 vs 11.5) [farebrother-etal-2023-proto-value-networks] The subspace is a designed spectral objective (proto-value functions), imposed by construction rather than freely emergent [farebrother-etal-2023-proto-value-networks]

models: Proto-Value Network Impala-CNN encoder (offline-pretrained on RL Unplugged Atari, Farebrother et al. 2023) · method: Successor-measure spectral basis (Laplacian/proto-value eigenvectors)

Filus et al. (2026) Continual-Learning ResNet-18 (Mammoth framework)

Lost or Hidden? A Concept-Level Forgetting in Supervised Continual Learning (2026)measured

A fitted linear translation map recovers apparently-deleted SAE concept latents across continual-learning checkpoints

Details

Filus, Faber, Corizzo & Kanan train a BatchTopK sparse autoencoder per task on frozen real ResNet-18 activations (via the Mammoth continual-learning framework) on 2seq-CIFAR10, 2seq-tiny-ImageNet, and 10seq-tiny-ImageNet, then check which task-t SAE latents stop firing after continual learning on later tasks ("apparent deletion") [filus-etal-2026-lost-or-hidden-concept-level-forgetting-supervised-continual-learning] A least-squares linear map T (with bias term) translating post-continual-learning frozen features back toward the earlier task's representation space is fit per checkpoint pair, and re-running the frozen earlier-task SAE on translated features recovers many apparently-deleted latents, distinguishing concepts that are genuinely lost from concepts that are merely hidden behind a recoverable linear reparameterization [filus-etal-2026-lost-or-hidden-concept-level-forgetting-supervised-continual-learning] A nonlinear MLP translator gives only marginal additional recovery over the linear map, and deletion ratios are highest under naive SGD and EWC continual-learning strategies and lowest under DER++ and LwF, with concept decodability (via a separate logistic-regression probe) degrading further as more tasks accumulate [filus-etal-2026-lost-or-hidden-concept-level-forgetting-supervised-continual-learning]

models: ResNet-18 (2seq-CIFAR10, task-incremental continual learning), ResNet-18 (2seq/10seq-tiny-ImageNet, task-incremental continual learning) · method:

Whisper

Analyzing Language and Geographical Variation in Speech Representations Across 60 Indic Languages (2026)measured

Indic dialect geography nests linearly inside each language's subspace

Details

Kumar J et al. fine-tune Whisper-base and Wav2Vec2.0-base on 60 Indic languages/165 districts under joint versus language-only supervision [kumarj-etal-2026-analyzing-language-and-geographical-variation-in-speech-representations-across-60-indic-languages] Language-conditioned logistic-regression probes decode district structure within each language's subspace (Whisper-base mean F1 91.29 joint vs 80.89 language-only; Wav2Vec2 87.14 vs 77.05) [kumarj-etal-2026-analyzing-language-and-geographical-variation-in-speech-representations-across-60-indic-languages] A normalized conditional mutual-information analysis reveals an asymmetric hierarchical geometry (AUC_D|L=2.68 vs AUC_L|D=5.43) with district clusters nested inside language structure [kumarj-etal-2026-analyzing-language-and-geographical-variation-in-speech-representations-across-60-indic-languages]

models: Whisper base · method: Linear probing

AlexNet

Net2Vec: Quantifying and Explaining How Concepts Are Encoded by Filters in Deep Neural Networks (2018)measured

Net2Vec shows most CNN concepts need several jointly-fit filters

Details

Fong & Vedaldi fit each Broden-annotated concept as a learned weight vector over multiple AlexNet filter-activation maps, scored by segmentation-mask IoU [fong-vedaldi-2018-net2vec] Most concepts require several filters jointly rather than aligning with any single filter, a multi-filter subspace generalization of single-unit dissection [fong-vedaldi-2018-net2vec] Individual filters are typically polysemantic, encoding more than one concept; no causal intervention is performed [fong-vedaldi-2018-net2vec]

models: AlexNet (ImageNet image classifier, supervised) · method: Multi-filter concept-vector fitting (Net2Vec)

GPT-OSS

Convergent Evolution: How Different Language Models Learn Similar Number Representations (2026)measured

Fourier periodicity in number embeddings is universal; linear separability is not

Details

Fu et al. distinguish spectral convergence (Fourier spikes at periods T=2,5,10 in number-token embeddings) from geometric convergence (linear mod-T decodability) [fu-etal-2026-convergent-evolution-number-representations] Spectral convergence appears everywhere: Transformer and non-Transformer LLMs, classical word embeddings, and even the raw training-corpus token-frequency distribution computed with no model [fu-etal-2026-convergent-evolution-number-representations] Geometric convergence is architecture-selective: controlled 300M models on identical data show Transformers and linear RNNs develop linearly separable mod-T classes while LSTMs, despite stronger Fourier spikes, stay at chance probing [fu-etal-2026-convergent-evolution-number-representations] Theorem 1 proves Fourier sparsity is necessary but not sufficient for mod-T separability, and a Fisher-LDA (T-1)-dimensional discriminant bound (Theorem 2) explains why near-identical spectra give very different probe accuracy [fu-etal-2026-convergent-evolution-number-representations] Controlled data perturbations that remove text-number co-occurrence degrade probing accuracy while leaving the Fourier spectrum unchanged, a data-level causal attribution [fu-etal-2026-convergent-evolution-number-representations] Built from a partial retrieval: the paper's later circular-representation findings for multi-token addition could not be verified and are not asserted here [fu-etal-2026-convergent-evolution-number-representations]

models: GPT-OSS-20B · method: Linear probing

DeepSeek

Convergent Evolution: How Different Language Models Learn Similar Number Representations (2026)measured

Fourier periodicity in number embeddings is universal; linear separability is not

Details

Fu et al. distinguish spectral convergence (Fourier spikes at periods T=2,5,10 in number-token embeddings) from geometric convergence (linear mod-T decodability) [fu-etal-2026-convergent-evolution-number-representations] Spectral convergence appears everywhere: Transformer and non-Transformer LLMs, classical word embeddings, and even the raw training-corpus token-frequency distribution computed with no model [fu-etal-2026-convergent-evolution-number-representations] Geometric convergence is architecture-selective: controlled 300M models on identical data show Transformers and linear RNNs develop linearly separable mod-T classes while LSTMs, despite stronger Fourier spikes, stay at chance probing [fu-etal-2026-convergent-evolution-number-representations] Theorem 1 proves Fourier sparsity is necessary but not sufficient for mod-T separability, and a Fisher-LDA (T-1)-dimensional discriminant bound (Theorem 2) explains why near-identical spectra give very different probe accuracy [fu-etal-2026-convergent-evolution-number-representations] Controlled data perturbations that remove text-number co-occurrence degrade probing accuracy while leaving the Fourier spectrum unchanged, a data-level causal attribution [fu-etal-2026-convergent-evolution-number-representations] Built from a partial retrieval: the paper's later circular-representation findings for multi-token addition could not be verified and are not asserted here [fu-etal-2026-convergent-evolution-number-representations]

models: DeepSeek-V3 · method: Linear probing

Mamba

Convergent Evolution: How Different Language Models Learn Similar Number Representations (2026)measured

Fourier periodicity in number embeddings is universal; linear separability is not

Details

Fu et al. distinguish spectral convergence (Fourier spikes at periods T=2,5,10 in number-token embeddings) from geometric convergence (linear mod-T decodability) [fu-etal-2026-convergent-evolution-number-representations] Spectral convergence appears everywhere: Transformer and non-Transformer LLMs, classical word embeddings, and even the raw training-corpus token-frequency distribution computed with no model [fu-etal-2026-convergent-evolution-number-representations] Geometric convergence is architecture-selective: controlled 300M models on identical data show Transformers and linear RNNs develop linearly separable mod-T classes while LSTMs, despite stronger Fourier spikes, stay at chance probing [fu-etal-2026-convergent-evolution-number-representations] Theorem 1 proves Fourier sparsity is necessary but not sufficient for mod-T separability, and a Fisher-LDA (T-1)-dimensional discriminant bound (Theorem 2) explains why near-identical spectra give very different probe accuracy [fu-etal-2026-convergent-evolution-number-representations] Controlled data perturbations that remove text-number co-occurrence degrade probing accuracy while leaving the Fourier spectrum unchanged, a data-level causal attribution [fu-etal-2026-convergent-evolution-number-representations] Built from a partial retrieval: the paper's later circular-representation findings for multi-token addition could not be verified and are not asserted here [fu-etal-2026-convergent-evolution-number-representations]

models: Mamba-2.8B · method: Linear probing

Falcon-Mamba

Convergent Evolution: How Different Language Models Learn Similar Number Representations (2026)measured

Fourier periodicity in number embeddings is universal; linear separability is not

Details

Fu et al. distinguish spectral convergence (Fourier spikes at periods T=2,5,10 in number-token embeddings) from geometric convergence (linear mod-T decodability) [fu-etal-2026-convergent-evolution-number-representations] Spectral convergence appears everywhere: Transformer and non-Transformer LLMs, classical word embeddings, and even the raw training-corpus token-frequency distribution computed with no model [fu-etal-2026-convergent-evolution-number-representations] Geometric convergence is architecture-selective: controlled 300M models on identical data show Transformers and linear RNNs develop linearly separable mod-T classes while LSTMs, despite stronger Fourier spikes, stay at chance probing [fu-etal-2026-convergent-evolution-number-representations] Theorem 1 proves Fourier sparsity is necessary but not sufficient for mod-T separability, and a Fisher-LDA (T-1)-dimensional discriminant bound (Theorem 2) explains why near-identical spectra give very different probe accuracy [fu-etal-2026-convergent-evolution-number-representations] Controlled data perturbations that remove text-number co-occurrence degrade probing accuracy while leaving the Fourier spectrum unchanged, a data-level causal attribution [fu-etal-2026-convergent-evolution-number-representations] Built from a partial retrieval: the paper's later circular-representation findings for multi-token addition could not be verified and are not asserted here [fu-etal-2026-convergent-evolution-number-representations]

models: Falcon-Mamba-7B · method: Linear probing

xLSTM

Convergent Evolution: How Different Language Models Learn Similar Number Representations (2026)measured

Fourier periodicity in number embeddings is universal; linear separability is not

Details

Fu et al. distinguish spectral convergence (Fourier spikes at periods T=2,5,10 in number-token embeddings) from geometric convergence (linear mod-T decodability) [fu-etal-2026-convergent-evolution-number-representations] Spectral convergence appears everywhere: Transformer and non-Transformer LLMs, classical word embeddings, and even the raw training-corpus token-frequency distribution computed with no model [fu-etal-2026-convergent-evolution-number-representations] Geometric convergence is architecture-selective: controlled 300M models on identical data show Transformers and linear RNNs develop linearly separable mod-T classes while LSTMs, despite stronger Fourier spikes, stay at chance probing [fu-etal-2026-convergent-evolution-number-representations] Theorem 1 proves Fourier sparsity is necessary but not sufficient for mod-T separability, and a Fisher-LDA (T-1)-dimensional discriminant bound (Theorem 2) explains why near-identical spectra give very different probe accuracy [fu-etal-2026-convergent-evolution-number-representations] Controlled data perturbations that remove text-number co-occurrence degrade probing accuracy while leaving the Fourier spectrum unchanged, a data-level causal attribution [fu-etal-2026-convergent-evolution-number-representations] Built from a partial retrieval: the paper's later circular-representation findings for multi-token addition could not be verified and are not asserted here [fu-etal-2026-convergent-evolution-number-representations]

models: xLSTM-7B · method: Linear probing

Kimi Linear

Convergent Evolution: How Different Language Models Learn Similar Number Representations (2026)measured

Fourier periodicity in number embeddings is universal; linear separability is not

Details

Fu et al. distinguish spectral convergence (Fourier spikes at periods T=2,5,10 in number-token embeddings) from geometric convergence (linear mod-T decodability) [fu-etal-2026-convergent-evolution-number-representations] Spectral convergence appears everywhere: Transformer and non-Transformer LLMs, classical word embeddings, and even the raw training-corpus token-frequency distribution computed with no model [fu-etal-2026-convergent-evolution-number-representations] Geometric convergence is architecture-selective: controlled 300M models on identical data show Transformers and linear RNNs develop linearly separable mod-T classes while LSTMs, despite stronger Fourier spikes, stay at chance probing [fu-etal-2026-convergent-evolution-number-representations] Theorem 1 proves Fourier sparsity is necessary but not sufficient for mod-T separability, and a Fisher-LDA (T-1)-dimensional discriminant bound (Theorem 2) explains why near-identical spectra give very different probe accuracy [fu-etal-2026-convergent-evolution-number-representations] Controlled data perturbations that remove text-number co-occurrence degrade probing accuracy while leaving the Fourier spectrum unchanged, a data-level causal attribution [fu-etal-2026-convergent-evolution-number-representations] Built from a partial retrieval: the paper's later circular-representation findings for multi-token addition could not be verified and are not asserted here [fu-etal-2026-convergent-evolution-number-representations]

models: Kimi Linear 48B-A3B · method: Linear probing

GloVe

Convergent Evolution: How Different Language Models Learn Similar Number Representations (2026)measured

Fourier periodicity in number embeddings is universal; linear separability is not

Details

Fu et al. distinguish spectral convergence (Fourier spikes at periods T=2,5,10 in number-token embeddings) from geometric convergence (linear mod-T decodability) [fu-etal-2026-convergent-evolution-number-representations] Spectral convergence appears everywhere: Transformer and non-Transformer LLMs, classical word embeddings, and even the raw training-corpus token-frequency distribution computed with no model [fu-etal-2026-convergent-evolution-number-representations] Geometric convergence is architecture-selective: controlled 300M models on identical data show Transformers and linear RNNs develop linearly separable mod-T classes while LSTMs, despite stronger Fourier spikes, stay at chance probing [fu-etal-2026-convergent-evolution-number-representations] Theorem 1 proves Fourier sparsity is necessary but not sufficient for mod-T separability, and a Fisher-LDA (T-1)-dimensional discriminant bound (Theorem 2) explains why near-identical spectra give very different probe accuracy [fu-etal-2026-convergent-evolution-number-representations] Controlled data perturbations that remove text-number co-occurrence degrade probing accuracy while leaving the Fourier spectrum unchanged, a data-level causal attribution [fu-etal-2026-convergent-evolution-number-representations] Built from a partial retrieval: the paper's later circular-representation findings for multi-token addition could not be verified and are not asserted here [fu-etal-2026-convergent-evolution-number-representations]

models: GloVe (Wikipedia + Gigaword, uncased) · method: Linear probing
Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection (2020)measured

The gender subspace spans dozens of directions, not one

Details

Ravfogel et al. introduce Iterative Null-space Projection (INLP) and apply it across three case studies [ravfogel-etal-2020-inlp] On GloVe, 35 INLP iterations drop SVM gender classification 100%->49.3% while a nonlinear MLP still recovers 85%, showing the gender subspace spans dozens of orthogonal directions rather than the single Bolukbasi direction [ravfogel-etal-2020-inlp] Word-similarity benchmarks improve after projection (SimLex-999 0.373->0.489), so general lexical semantics is preserved [ravfogel-etal-2020-inlp] On DeepMoji race-correlated sentiment, removing the race subspace shrinks the true-positive-rate fairness gap 0.45->0.15 at some accuracy cost [ravfogel-etal-2020-inlp] On Bias-in-Bios, removing the gender subspace from BERT CLS representations (300 directions) cuts the gender-TPR gap 48% while dropping profession accuracy only 80.9%->75.2% [ravfogel-etal-2020-inlp]

models: GloVe (Wikipedia + Gigaword, uncased) · method: INLP subspace + AlterRep counterfactual push
Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings but do not Remove Them (2019)measured

Hard-debiasing a real word2vec/GloVe gender subspace leaves the residual geometry recoverably gendered

Details

Gonen & Goldberg apply Bolukbasi et al. (2016)'s hard-debiasing projection (removing the linear gender direction) to real word2vec-GoogleNews and GloVe-Wikipedia embeddings [gonen-goldberg-2019-lipstick-on-a-pig-debiasing-methods-cover-up-gender-bias] K-means clustering of the top gender-biased words on the "debiased" embeddings still recovers the original male/female clustering with 92-98.4% accuracy [gonen-goldberg-2019-lipstick-on-a-pig-debiasing-methods-cover-up-gender-bias] An RBF-kernel SVM trained on original (biased) word vectors to classify male- vs female-stereotyped words transfers to the debiased vectors with comparable accuracy, and nearest-neighbor profession word lists remain measurably gender-skewed after debiasing [gonen-goldberg-2019-lipstick-on-a-pig-debiasing-methods-cover-up-gender-bias] These findings show gender bias in these embeddings is not confined to the single linear direction removed by hard-debiasing, so linear-subspace projection alone does not eliminate the bias structure, only its visibility along that one axis [gonen-goldberg-2019-lipstick-on-a-pig-debiasing-methods-cover-up-gender-bias]

models: GloVe (Wikipedia + Gigaword, uncased) · method:
What Does Debiasing Really Remove? A Geometric Study of PCA-Based Gender Debiasing in Word Embeddings (2026)measured

Direct gender bias is rank-1; associative WEAT bias is distributed

Details

Kresin et al. build the Bolukbasi gender-difference matrix from definitional word pairs, form its covariance and analyze the PCA spectrum on GloVe, word2vec and FastText embeddings (the paper does not specify the exact training corpora) [kresin-etal-2026-what-does-debiasing-really-remove] The spectrum is strongly low-rank: PC1 alone captures ~48% of variance, the first 5 ~80%, and the first 10 exceed 93% [kresin-etal-2026-what-does-debiasing-really-remove] Direct projection-based bias collapses to near-zero after removing only PC1 and stays flat, so it is genuinely rank-1-dominated across all three embeddings [kresin-etal-2026-what-does-debiasing-really-remove] WEAT associative bias is not reduced by removing any single dominant component and stays distributed across many higher-dimensional directions [kresin-etal-2026-what-does-debiasing-really-remove] Mean vector displacement rises monotonically and neighbor stability declines as more components are removed, a distortion trade-off with no universal optimal k [kresin-etal-2026-what-does-debiasing-really-remove]

models: GloVe (Wikipedia + Gigaword, uncased) · method: PCA
Linear Adversarial Concept Erasure (2022)measured

One adversarial direction erases gender where INLP needs dozens

Details

Ravfogel et al. formulate concept erasure as a minimax game between a rank-k projection and a re-optimizing linear predictor, proving how many dimensions the erasure-optimal subspace needs (R-LACE) [ravfogel-etal-2022-rlace] On GloVe, rank-1 R-LACE drops SVM gender accuracy to chance while INLP fails even after removing a 20-dimensional subspace [ravfogel-etal-2022-rlace] On BERT Bias-in-Bios, rank-1 R-LACE drops gender accuracy 99.32% to 52.48% while INLP rank-1 barely moves it, needing ~100 dimensions to match [ravfogel-etal-2022-rlace] The gap is task-dependent: INLP is provably suboptimal for linear-regression objectives but provably identical to R-LACE for Rayleigh-quotient (PLS/CCA) objectives; both are linear-only (nonlinear classifiers still recover gender >90%) [ravfogel-etal-2022-rlace]

models: GloVe (Wikipedia + Gigaword, uncased) · method: R-LACE (Relaxed Linear Adversarial Concept Erasure)
Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation (2020)measured

Kernelizing the gender bias-subspace finds no benefit from nonlinearity

Details

Vargas & Cotterell prove Bolukbasi et al. hard-debiasing bias-subspace construction is exactly PCA on a mean-centered design matrix, making explicit the untested linear-subspace hypothesis for gender bias [vargas-cotterell-2020-linear-subspace-hypothesis-gender-bias] They generalize it to kernel PCA and test five kernels against the linear method on word2vec and GloVe across four benchmarks (WEAT, analogy correlation, indirect-bias SVM, SimLex) [vargas-cotterell-2020-linear-subspace-hypothesis-gender-bias] Across every benchmark and kernel, nonlinear kernels perform on par with or worse than the linear kernel, so added nonlinearity buys no extra bias removal [vargas-cotterell-2020-linear-subspace-hypothesis-gender-bias] This gives direct empirical support for treating the gender-bias subspace as linear; no novel causal intervention is performed [vargas-cotterell-2020-linear-subspace-hypothesis-gender-bias]

models: GloVe (Wikipedia + Gigaword, uncased) · method: Kernel-PCA linearity test

Convergent Evolution controlled 300M architecture comparison (Fu, Zhou, Belkin, Sharan & Jia 2026)

Convergent Evolution: How Different Language Models Learn Similar Number Representations (2026)measured

Fourier periodicity in number embeddings is universal; linear separability is not

Details

Fu et al. distinguish spectral convergence (Fourier spikes at periods T=2,5,10 in number-token embeddings) from geometric convergence (linear mod-T decodability) [fu-etal-2026-convergent-evolution-number-representations] Spectral convergence appears everywhere: Transformer and non-Transformer LLMs, classical word embeddings, and even the raw training-corpus token-frequency distribution computed with no model [fu-etal-2026-convergent-evolution-number-representations] Geometric convergence is architecture-selective: controlled 300M models on identical data show Transformers and linear RNNs develop linearly separable mod-T classes while LSTMs, despite stronger Fourier spikes, stay at chance probing [fu-etal-2026-convergent-evolution-number-representations] Theorem 1 proves Fourier sparsity is necessary but not sufficient for mod-T separability, and a Fisher-LDA (T-1)-dimensional discriminant bound (Theorem 2) explains why near-identical spectra give very different probe accuracy [fu-etal-2026-convergent-evolution-number-representations] Controlled data perturbations that remove text-number co-occurrence degrade probing accuracy while leaving the Fourier spectrum unchanged, a data-level causal attribution [fu-etal-2026-convergent-evolution-number-representations] Built from a partial retrieval: the paper's later circular-representation findings for multi-token addition could not be verified and are not asserted here [fu-etal-2026-convergent-evolution-number-representations]

models: Convergent-Evolution 300M controlled Transformer, Convergent-Evolution 300M controlled LSTM, Convergent-Evolution 300M controlled Linear RNN (Gated DeltaNet) · method: Linear probing

Leela Chess Zero

The Algorithm Is Not the Behavior: Learned Priors Override Look-Ahead in a Chess-Playing Neural Network (2025)measured

Leela Chess Zero's correct moves appear early, then get overridden

Details

Sandmann, Lapuschkin & Samek extend the logit lens (functionally equivalent to zero-ablating later sublayers) to Leela Chess Zero's Post-LN policy network [sandmann-etal-2025-the-algorithm-is-not-the-behavior-learned-priors-override-look-ahead-in-a-chess-playing-neural-network] Playing strength and puzzle-solving ability rise monotonically with depth, but policy distributions follow non-smooth, non-monotonic trajectories [sandmann-etal-2025-the-algorithm-is-not-the-behavior-learned-priors-override-look-ahead-in-a-chess-playing-neural-network] Correct puzzle solutions are discovered in intermediate layers but subsequently discarded, with move rankings poorly correlated to the final output until late in the network [sandmann-etal-2025-the-algorithm-is-not-the-behavior-learned-priors-override-look-ahead-in-a-chess-playing-neural-network] This contrasts with the smooth distributional convergence typical of language models, evidencing iterative inference where a late stage overrides an already-computed answer [sandmann-etal-2025-the-algorithm-is-not-the-behavior-learned-priors-override-look-ahead-in-a-chess-playing-neural-network]

models: Leela Chess Zero (policy network) · method: Logit Lens, Causal interventions (steering)

DeepMoji

Null It Out: Guarding Protected Attributes by Iterative Nullspace Projection (2020)measured

The gender subspace spans dozens of directions, not one

Details

Ravfogel et al. introduce Iterative Null-space Projection (INLP) and apply it across three case studies [ravfogel-etal-2020-inlp] On GloVe, 35 INLP iterations drop SVM gender classification 100%->49.3% while a nonlinear MLP still recovers 85%, showing the gender subspace spans dozens of orthogonal directions rather than the single Bolukbasi direction [ravfogel-etal-2020-inlp] Word-similarity benchmarks improve after projection (SimLex-999 0.373->0.489), so general lexical semantics is preserved [ravfogel-etal-2020-inlp] On DeepMoji race-correlated sentiment, removing the race subspace shrinks the true-positive-rate fairness gap 0.45->0.15 at some accuracy cost [ravfogel-etal-2020-inlp] On Bias-in-Bios, removing the gender subspace from BERT CLS representations (300 directions) cuts the gender-TPR gap 48% while dropping profession accuracy only 80.9%->75.2% [ravfogel-etal-2020-inlp]

models: DeepMoji encoder · method: INLP subspace + AlterRep counterfactual push

Principal Odor Map GNN

A Principal Odor Map Unifies Diverse Tasks in Human Olfactory Perception (2023)measured

A GNN odor-map's geometry matches human perceptual odor similarity

Details

Lee et al. train a message-passing GNN on ~5,000 molecules labeled with 138 human odor descriptors (cross-validated AUROC 0.89), taking the 256-dim penultimate layer as a Principal Odor Map [lee-etal-2023-principal-odor-map] POM pairwise distances correlate R=0.73 with an independently-measured human perceptual odor map, versus R=-0.12 for a Morgan-fingerprint embedding [lee-etal-2023-principal-odor-map] Same-label molecules cluster tighter in the POM (cluster density 0.51 vs 0.68 for fingerprints) [lee-etal-2023-principal-odor-map] On 320 novel odorants rated by a trained panel, the POM model beats the median panelist for 53% of molecules and on 58% of odor descriptors [lee-etal-2023-principal-odor-map] The evidence is an RSA-style distance correlation to perceptual ground truth; no representation-space intervention is performed [lee-etal-2023-principal-odor-map]

models: Principal Odor Map (Message Passing Neural Network) · method: Representational Similarity Analysis (RSA)

word2vec

Lipstick on a Pig: Debiasing Methods Cover up Systematic Gender Biases in Word Embeddings but do not Remove Them (2019)measured

Hard-debiasing a real word2vec/GloVe gender subspace leaves the residual geometry recoverably gendered

Details

Gonen & Goldberg apply Bolukbasi et al. (2016)'s hard-debiasing projection (removing the linear gender direction) to real word2vec-GoogleNews and GloVe-Wikipedia embeddings [gonen-goldberg-2019-lipstick-on-a-pig-debiasing-methods-cover-up-gender-bias] K-means clustering of the top gender-biased words on the "debiased" embeddings still recovers the original male/female clustering with 92-98.4% accuracy [gonen-goldberg-2019-lipstick-on-a-pig-debiasing-methods-cover-up-gender-bias] An RBF-kernel SVM trained on original (biased) word vectors to classify male- vs female-stereotyped words transfers to the debiased vectors with comparable accuracy, and nearest-neighbor profession word lists remain measurably gender-skewed after debiasing [gonen-goldberg-2019-lipstick-on-a-pig-debiasing-methods-cover-up-gender-bias] These findings show gender bias in these embeddings is not confined to the single linear direction removed by hard-debiasing, so linear-subspace projection alone does not eliminate the bias structure, only its visibility along that one axis [gonen-goldberg-2019-lipstick-on-a-pig-debiasing-methods-cover-up-gender-bias]

models: word2vec (Google News, 300d) · method:
What Does Debiasing Really Remove? A Geometric Study of PCA-Based Gender Debiasing in Word Embeddings (2026)measured

Direct gender bias is rank-1; associative WEAT bias is distributed

Details

Kresin et al. build the Bolukbasi gender-difference matrix from definitional word pairs, form its covariance and analyze the PCA spectrum on GloVe, word2vec and FastText embeddings (the paper does not specify the exact training corpora) [kresin-etal-2026-what-does-debiasing-really-remove] The spectrum is strongly low-rank: PC1 alone captures ~48% of variance, the first 5 ~80%, and the first 10 exceed 93% [kresin-etal-2026-what-does-debiasing-really-remove] Direct projection-based bias collapses to near-zero after removing only PC1 and stays flat, so it is genuinely rank-1-dominated across all three embeddings [kresin-etal-2026-what-does-debiasing-really-remove] WEAT associative bias is not reduced by removing any single dominant component and stays distributed across many higher-dimensional directions [kresin-etal-2026-what-does-debiasing-really-remove] Mean vector displacement rises monotonically and neighbor stability declines as more components are removed, a distortion trade-off with no universal optimal k [kresin-etal-2026-what-does-debiasing-really-remove]

models: word2vec (Google News, 300d) · method: PCA
Exploiting Similarities among Languages for Machine Translation (2013)measured

A linear map from a seed dictionary aligns two word-embedding spaces

Details

Mikolov, Le & Sutskever train monolingual skip-gram embeddings separately per language (the headline English-Spanish result uses WMT11-trained English embeddings, not Google News) [mikolov-le-sutskever-2013-exploiting-similarities-for-mt] They fit a linear transformation from a small seed bilingual dictionary and map one embedding space onto the other [mikolov-le-sutskever-2013-exploiting-similarities-for-mt] The map achieves high real word-translation precision (around 90% precision at rank 5 for English-Spanish) [mikolov-le-sutskever-2013-exploiting-similarities-for-mt] It is a foundational supervised precursor to later unsupervised cross-lingual alignment work (MUSE) [mikolov-le-sutskever-2013-exploiting-similarities-for-mt]

models: word2vec skip-gram (WMT11, English), word2vec (Spanish monolingual corpus) · method: Supervised bilingual-dictionary linear map fitting
Geometry of Polysemy (2016)measured

Monosemous word vectors sit at the intersection of their context subspaces

Details

Mu, Bhat & Viswanath fit a low-rank PCA subspace to each of many contexts around a target word in 300-dim word2vec skip-gram embeddings (Wikipedia) [mu-etal-2016-geometry-of-polysemy] For monosemous words, cosine similarity to the many context subspaces is far above a random-context baseline (e.g. "typhoon" 0.693 vs 0.305) [mu-etal-2016-geometry-of-polysemy] A word's vector thus occupies the geometric intersection of its senses' low-rank context regions, used downstream for sense clustering (K-Grassmeans) [mu-etal-2016-geometry-of-polysemy] No causal intervention is performed [mu-etal-2016-geometry-of-polysemy]

models: word2vec (trained on Wikipedia) · method: Grassmannian context-subspace intersection scoring
Understanding Linearity of Cross-Lingual Word Embedding Mappings (2020)measured

Analogy preservation is necessary and sufficient for a linear cross-lingual map

Details

Peng et al. prove a necessary-and-sufficient equivalence between within-language analogy (parallelogram) preservation and linearity of the ground-truth cross-lingual embedding map [peng-etal-2020-understanding-linearity-cross-lingual-mappings] They test it against real monolingual embeddings across 12 languages (fastText-Wiki, fastText-CommonCrawl, word2vec-CoNLL) using a new 5-category cross-lingual analogy dataset [peng-etal-2020-understanding-linearity-cross-lingual-mappings] Mapping-linearity quality tracks analogy-preservation quality empirically [peng-etal-2020-understanding-linearity-cross-lingual-mappings] No causal intervention is performed [peng-etal-2020-understanding-linearity-cross-lingual-mappings]

models: word2vec (CoNLL corpus) · method: Analytical derivation
Exploring the Linear Subspace Hypothesis in Gender Bias Mitigation (2020)measured

Kernelizing the gender bias-subspace finds no benefit from nonlinearity

Details

Vargas & Cotterell prove Bolukbasi et al. hard-debiasing bias-subspace construction is exactly PCA on a mean-centered design matrix, making explicit the untested linear-subspace hypothesis for gender bias [vargas-cotterell-2020-linear-subspace-hypothesis-gender-bias] They generalize it to kernel PCA and test five kernels against the linear method on word2vec and GloVe across four benchmarks (WEAT, analogy correlation, indirect-bias SVM, SimLex) [vargas-cotterell-2020-linear-subspace-hypothesis-gender-bias] Across every benchmark and kernel, nonlinear kernels perform on par with or worse than the linear kernel, so added nonlinearity buys no extra bias removal [vargas-cotterell-2020-linear-subspace-hypothesis-gender-bias] This gives direct empirical support for treating the gender-bias subspace as linear; no novel causal intervention is performed [vargas-cotterell-2020-linear-subspace-hypothesis-gender-bias]

models: word2vec (Google News, 300d) · method: Kernel-PCA linearity test
Discovering Universal Geometry in Embeddings with ICA (2023)measured

ICA after PCA-whitening recovers a universal sparse axis-aligned decomposition

Details

Yamagiwa et al. apply PCA-whitening then ICA to their own text8-trained word2vec (SGNS), cross-lingual fastText (7 languages), BERT contextual embeddings, and image encoders (ViT, ResMLP, Swin, ResNet, RegNet) [yamagiwa-etal-2023-discovering-universal-geometry-ica] The independent axes are individually interpretable and sparse, unlike PCA arbitrary rotation [yamagiwa-etal-2023-discovering-universal-geometry-ica] Specific axes can be matched across languages, models and modalities via cross-model component correlation, a privileged non-arbitrary decomposition [yamagiwa-etal-2023-discovering-universal-geometry-ica] No causal intervention is performed; axis interpretability is partly qualitative and universality depends on whitening choices [yamagiwa-etal-2023-discovering-universal-geometry-ica]

models: word2vec SGNS (text8, 300d) · method: ICA axis decomposition

OpenAI Embeddings

Characterizing Linear Alignment Across Language Models (2026)measured

Independently-trained embedding models share linear structure (CKA 0.60-0.88)

Details

Gorbett & Jana measure linear CKA across five independently-trained embedding models (OpenAI text-embedding-3-small, Cohere embed, Gemini embedding-001, Qwen3-Embedding-8B, e5-mistral-7b-instruct), finding similarity 0.595-0.881 [gorbett-jana-2026-characterizing-linear-alignment-across-language-models] In a separate experiment they fit learned affine maps between instruction-tuned generative LMs (Llama-3-8B, Qwen2.5-7B/14B, Llama-3.2-1B, OLMo-7B and others) [gorbett-jana-2026-characterizing-linear-alignment-across-language-models] These maps preserve classification accuracy and OOD-AUROC across model pairs (e.g. 94.5%->93.1%) [gorbett-jana-2026-characterizing-linear-alignment-across-language-models] They enable zero-shot cross-model text generation via a frozen target head, with generation quality correlating with tokenizer exact-match rate (r=0.898) and vocabulary Jaccard overlap (r=0.822) [gorbett-jana-2026-characterizing-linear-alignment-across-language-models]

models: text-embedding-3-small · method: Centered Kernel Alignment (CKA), Singular Vector Canonical Correlation Analysis (SVCCA)

Cohere

Characterizing Linear Alignment Across Language Models (2026)measured

Independently-trained embedding models share linear structure (CKA 0.60-0.88)

Details

Gorbett & Jana measure linear CKA across five independently-trained embedding models (OpenAI text-embedding-3-small, Cohere embed, Gemini embedding-001, Qwen3-Embedding-8B, e5-mistral-7b-instruct), finding similarity 0.595-0.881 [gorbett-jana-2026-characterizing-linear-alignment-across-language-models] In a separate experiment they fit learned affine maps between instruction-tuned generative LMs (Llama-3-8B, Qwen2.5-7B/14B, Llama-3.2-1B, OLMo-7B and others) [gorbett-jana-2026-characterizing-linear-alignment-across-language-models] These maps preserve classification accuracy and OOD-AUROC across model pairs (e.g. 94.5%->93.1%) [gorbett-jana-2026-characterizing-linear-alignment-across-language-models] They enable zero-shot cross-model text generation via a frozen target head, with generation quality correlating with tokenizer exact-match rate (r=0.898) and vocabulary Jaccard overlap (r=0.822) [gorbett-jana-2026-characterizing-linear-alignment-across-language-models]

models: Cohere Embed · method: Centered Kernel Alignment (CKA), Singular Vector Canonical Correlation Analysis (SVCCA)

Gemini

Characterizing Linear Alignment Across Language Models (2026)measured

Independently-trained embedding models share linear structure (CKA 0.60-0.88)

Details

Gorbett & Jana measure linear CKA across five independently-trained embedding models (OpenAI text-embedding-3-small, Cohere embed, Gemini embedding-001, Qwen3-Embedding-8B, e5-mistral-7b-instruct), finding similarity 0.595-0.881 [gorbett-jana-2026-characterizing-linear-alignment-across-language-models] In a separate experiment they fit learned affine maps between instruction-tuned generative LMs (Llama-3-8B, Qwen2.5-7B/14B, Llama-3.2-1B, OLMo-7B and others) [gorbett-jana-2026-characterizing-linear-alignment-across-language-models] These maps preserve classification accuracy and OOD-AUROC across model pairs (e.g. 94.5%->93.1%) [gorbett-jana-2026-characterizing-linear-alignment-across-language-models] They enable zero-shot cross-model text generation via a frozen target head, with generation quality correlating with tokenizer exact-match rate (r=0.898) and vocabulary Jaccard overlap (r=0.822) [gorbett-jana-2026-characterizing-linear-alignment-across-language-models]

models: Gemini embedding-001 · method: Centered Kernel Alignment (CKA), Singular Vector Canonical Correlation Analysis (SVCCA)

Aya

High-Dimensional Interlingual Representations of Large Language Models (2025)measured

Multilingual LLMs share a core interlingual subspace with language-specific fragments

Details

Wilie et al. study Aya Expanse 8B, Llama-3.1-8B and Gemma-2-9B with a kNN-based Interlingual Local Overlap (ILO) score plus t-SNE of hidden states [wilie-etal-2025-high-dimensional-interlingual-representations] Representations organize as a partially-shared core interlingual region (translation-equivalent inputs share local neighborhoods) surrounded by fragmented language-specific components that do not transfer [wilie-etal-2025-high-dimensional-interlingual-representations] Overlap varies systematically by resource level and linguistic family, with graph "bridge"/"reachability" metrics showing which languages connect the core to peripheral fragments [wilie-etal-2025-high-dimensional-interlingual-representations] The alignment is causally load-bearing: ILO overlap is preserved or disrupted depending on which layers are frozen during single-language fine-tuning [wilie-etal-2025-high-dimensional-interlingual-representations]

models: Aya Expanse 8B · method: Interlingual Local Overlap (ILO)
Understanding Subword Compositionality of Large Language Models (2025)measured

Composed-subword vs whole-word spaces show family-dependent isometry

Details

Peng, Chai & Sogaard fit an orthogonal Procrustes map between composed-subword and whole-word embedding spaces across instruction-tuned LLMs, scored by Precision@1 retrieval [peng-chai-sogaard-2025-subword-compositionality] Simple addition consistently outperforms other subword-composition operations [peng-chai-sogaard-2025-subword-compositionality] Three family patterns emerge: Aya-expanse and Gemma show high composed-vs-whole-word isometry, Llama 3/3.1 very little, others moderate but dropping late in the network [peng-chai-sogaard-2025-subword-compositionality] The analysis is purely observational with no causal intervention [peng-chai-sogaard-2025-subword-compositionality]

models: Aya Expanse 8B · method: Orthogonal Procrustes alignment, Linear probing
Language Lives in Sparse Dimensions: Toward Interpretable and Efficient Multilingual Control for Large Language Models (2025)measured

A sparse axis-aligned dimension set causally controls output language

Details

Zhong et al. hypothesize the English-centric cross-lingual transition is governed by a small, layer-consistent set of dimensions, identified from as few as ~50 sentences by comparing corpus-mean activations [zhong-etal-2025-language-lives-in-sparse-dimensions] Keeping the top-400 dimensions (~8-10% of hidden size), cross-language overlap tracks typological similarity (Chinese/Japanese share 193/400) and the monolingual and parallel methods agree 77.6% [zhong-etal-2025-language-lives-in-sparse-dimensions] Overwriting only those dimensions at one intermediate layer with a scaled target-language mean switches the output language while preserving semantic content (BLEU) across Llama-2/3.1 and Aya23-8B [zhong-etal-2025-language-lives-in-sparse-dimensions] It outperforms neuron-level baselines by up to 12.69 points at much lower data/compute cost, holding across most intermediate layers [zhong-etal-2025-language-lives-in-sparse-dimensions]

models: Aya-23-8B · method: Sparse dimension identification (axis-aligned language-specific dimensions)

Wan

Activation Steering of Video Generation Models via Reduced-Order Linear Optimal Control (2026)measured

An SVD contrastive subspace enables closed-loop control of video diffusion

Details

Hong et al. propose LA-LQR for text-to-video diffusion steering: per (layer, timestep) they extract an orthonormal basis of a contrastive prompt-pair difference matrix via streaming randomized SVD, finding a 64-dimensional latent captures most contrastive-difference energy [hong-etal-2026-la-lqr-activation-steering-video-generation-reduced-order-linear-optimal-control] Local linearized dynamics within this subspace generalize across prompts, with Jacobian distances across 20 prompts far below the distance to random matrices [hong-etal-2026-la-lqr-activation-steering-video-generation-reduced-order-linear-optimal-control] An LQR feedback gain computes a closed-loop correction tracking the projected contrastive coordinate toward a setpoint, applying perturbation proportional to online tracking error [hong-etal-2026-la-lqr-activation-steering-video-generation-reduced-order-linear-optimal-control] Applied to Wan2.1-T2V-14B and HunyuanVideo, it reduces unsafe generations on T2VSafetyBench and SafeSora while preserving fidelity better than fixed-magnitude steering [hong-etal-2026-la-lqr-activation-steering-video-generation-reduced-order-linear-optimal-control] The experiment/model section was not fully reachable at render, so the exact model identities are deferred for re-verification [hong-etal-2026-la-lqr-activation-steering-video-generation-reduced-order-linear-optimal-control]

models: Wan2.1-T2V-14B · method: Reduced-Order Linear Optimal Control Steering (LA-LQR)

HunyuanVideo

Activation Steering of Video Generation Models via Reduced-Order Linear Optimal Control (2026)measured

An SVD contrastive subspace enables closed-loop control of video diffusion

Details

Hong et al. propose LA-LQR for text-to-video diffusion steering: per (layer, timestep) they extract an orthonormal basis of a contrastive prompt-pair difference matrix via streaming randomized SVD, finding a 64-dimensional latent captures most contrastive-difference energy [hong-etal-2026-la-lqr-activation-steering-video-generation-reduced-order-linear-optimal-control] Local linearized dynamics within this subspace generalize across prompts, with Jacobian distances across 20 prompts far below the distance to random matrices [hong-etal-2026-la-lqr-activation-steering-video-generation-reduced-order-linear-optimal-control] An LQR feedback gain computes a closed-loop correction tracking the projected contrastive coordinate toward a setpoint, applying perturbation proportional to online tracking error [hong-etal-2026-la-lqr-activation-steering-video-generation-reduced-order-linear-optimal-control] Applied to Wan2.1-T2V-14B and HunyuanVideo, it reduces unsafe generations on T2VSafetyBench and SafeSora while preserving fidelity better than fixed-magnitude steering [hong-etal-2026-la-lqr-activation-steering-video-generation-reduced-order-linear-optimal-control] The experiment/model section was not fully reachable at render, so the exact model identities are deferred for re-verification [hong-etal-2026-la-lqr-activation-steering-video-generation-reduced-order-linear-optimal-control]

models: HunyuanVideo-1.5 · method: Reduced-Order Linear Optimal Control Steering (LA-LQR)

JEPA (Joint Embedding Predictive Architecture)

Social-JEPA: Emergent Geometric Isomorphism in Independently Trained World Models (2026)measured

Two independently-trained JEPA world models converge to a linear isomorphism

Details

Zhang et al. train pairs of ViT-S/16 I-JEPA encoders fully independently on different views of the same scenes (smallNORB, nuScenes multi-camera, ImageNet augmentation views) [zhang-etal-2026-social-jepa] They define geometric isomorphism as an invertible linear map z2 approximately W z1, fit by closed-form ridge regression and quantified by MSE, R-squared, linear CKA, distance-structure consistency and neighborhood overlap [zhang-etal-2026-social-jepa] Best case (smallNORB) reaches MSE 0.036, R-squared 0.891, DSC 0.872, a multiply-corroborated approximate linear isometry between independently-learned spaces [zhang-etal-2026-social-jepa] They prove the JEPA loss is invariant under GL(d) reparameterization of the encoder, giving a theoretical reason to expect this convergence class [zhang-etal-2026-social-jepa] The fitted map transfers a linear probe zero-shot with no gradient steps and enables teacher-student representation migration at 0.28x the FLOPs of training from scratch [zhang-etal-2026-social-jepa]

models: ViT-S/16 (I-JEPA objective, independently trained per view: smallNORB/nuScenes/ImageNet-1k) · method: Orthogonal Procrustes alignment, Centered Kernel Alignment (CKA)

SigLIP

Canonicalizing Multimodal Contrastive Representation Learning (2026)measured

Independent multimodal models share one orthogonal map across both modalities

Details

Gupta et al. show independently-trained multimodal contrastive models' embedding spaces are related, up to a mean shift, by a single orthogonal map Q applied identically to both the image and text encoders [gupta-etal-2026-canonicalizing-multimodal-contrastive-representation-learning] They prove that kernel agreement on a small image-text anchor set forces this single-shared-orthogonal-map relationship [gupta-etal-2026-canonicalizing-multimodal-contrastive-representation-learning] It is verified across CLIP, SigLIP and FLAVA models with different architectures and training data [gupta-etal-2026-canonicalizing-multimodal-contrastive-representation-learning] This is a discovered (not imposed) geometric relationship, solved in closed form via the orthogonal Procrustes SVD [gupta-etal-2026-canonicalizing-multimodal-contrastive-representation-learning]

models: SigLIP · method: Orthogonal Procrustes alignment, Representation Alignment
Cross-Modal Redundancy and the Geometry of Vision-Language Embeddings (2026)measured

Iso-energy SAE isolates a bimodal cross-modal subspace

Details

- An iso-energy-regularized aligned SAE splits its dictionary into bimodal atoms (a modality-agnostic shared subspace carrying essentially all cross-modal alignment) and unimodal atoms (per-modality cones that account for the modality gap). [dhimoila-etal-2026-cross-modal-redundancy] - Ablating the unimodal atoms nearly eliminates the modality gap while barely changing retrieval recall (CLIP-B/32 delta_r 0.224 to 0.125 from SAE to SAE-A; functional alignment rho 0.327 to 4.232). [dhimoila-etal-2026-cross-modal-redundancy] - Restricting vector-arithmetic edits to the bimodal subspace keeps them in-distribution and improves OOD retrieval (CLIP 0.97 to 0.77, SigLIP2 0.99 to 0.61); reconstruction R-squared >=0.859. [dhimoila-etal-2026-cross-modal-redundancy] - The paper measures retrieval (delta_r), not post-ablation zero-shot classification; tested on CLIP and OpenCLIP ViT-B/32 and ViT-L/14, SigLIP and SigLIP2. [dhimoila-etal-2026-cross-modal-redundancy]

models: SigLIP, SigLIP 2 · method: Iso-Energy regularized sparse autoencoder (SAE-A), Causal interventions (steering)

FLAVA

Canonicalizing Multimodal Contrastive Representation Learning (2026)measured

Independent multimodal models share one orthogonal map across both modalities

Details

Gupta et al. show independently-trained multimodal contrastive models' embedding spaces are related, up to a mean shift, by a single orthogonal map Q applied identically to both the image and text encoders [gupta-etal-2026-canonicalizing-multimodal-contrastive-representation-learning] They prove that kernel agreement on a small image-text anchor set forces this single-shared-orthogonal-map relationship [gupta-etal-2026-canonicalizing-multimodal-contrastive-representation-learning] It is verified across CLIP, SigLIP and FLAVA models with different architectures and training data [gupta-etal-2026-canonicalizing-multimodal-contrastive-representation-learning] This is a discovered (not imposed) geometric relationship, solved in closed form via the orthogonal Procrustes SVD [gupta-etal-2026-canonicalizing-multimodal-contrastive-representation-learning]

models: FLAVA · method: Orthogonal Procrustes alignment, Representation Alignment

Atomistic/Equivariant Graph Neural Network

Information Routing in Atomistic Foundation Models: How Task Alignment and Equivariance Shape Linear Disentanglement (2026)measured

Atomistic models route geometric information by task and symmetry type

Details

Steier introduces Compositional Probe Decomposition: linearly project out compositional (atom-count/formula-level) signal via QR/ridge residualization, then measure remaining geometric information accessible to a linear ridge probe [steier-2026-information-routing-in-atomistic-foundation-models] Validated on a structural-isomer benchmark (94.6% on geometric residuals versus chance for compositional projections), CPD across ten atomistic models finds a 6.6x linear-accessibility gradient driven mainly by task alignment [steier-2026-information-routing-in-atomistic-foundation-models] Task alignment matters: PaiNN's geometric R-squared drops 0.53->0.31 when retrained on energy instead of HOMO-LUMO gap; MACE drops 0.44->0.08 [steier-2026-information-routing-in-atomistic-foundation-models] Inside MACE, information routes by symmetry type across distinct linear subspaces (L=1 channels favor dipole, L=0 channels favor HOMO-LUMO gap), a pattern absent in ViSNet [steier-2026-information-routing-in-atomistic-foundation-models] Nonlinear probes give misleadingly inflated results on residualized representations, motivating exclusively linear probes here [steier-2026-information-routing-in-atomistic-foundation-models]

models: PaiNN (polarizable atom interaction neural network), MACE (higher-order equivariant message-passing potential), ViSNet · method: Compositional probe decomposition (CPD)

OpenCLIP

Cross-Modal Redundancy and the Geometry of Vision-Language Embeddings (2026)measured

Iso-energy SAE isolates a bimodal cross-modal subspace

Details

- An iso-energy-regularized aligned SAE splits its dictionary into bimodal atoms (a modality-agnostic shared subspace carrying essentially all cross-modal alignment) and unimodal atoms (per-modality cones that account for the modality gap). [dhimoila-etal-2026-cross-modal-redundancy] - Ablating the unimodal atoms nearly eliminates the modality gap while barely changing retrieval recall (CLIP-B/32 delta_r 0.224 to 0.125 from SAE to SAE-A; functional alignment rho 0.327 to 4.232). [dhimoila-etal-2026-cross-modal-redundancy] - Restricting vector-arithmetic edits to the bimodal subspace keeps them in-distribution and improves OOD retrieval (CLIP 0.97 to 0.77, SigLIP2 0.99 to 0.61); reconstruction R-squared >=0.859. [dhimoila-etal-2026-cross-modal-redundancy] - The paper measures retrieval (delta_r), not post-ablation zero-shot classification; tested on CLIP and OpenCLIP ViT-B/32 and ViT-L/14, SigLIP and SigLIP2. [dhimoila-etal-2026-cross-modal-redundancy]

models: OpenCLIP ViT-B/32, OpenCLIP ViT-L/14 · method: Iso-Energy regularized sparse autoencoder (SAE-A), Causal interventions (steering)

Kim et al. (2025) Custom Continual-Learning Networks

Measuring Representational Shifts in Continual Learning: A Linear Transformation Perspective (2025)measured

A fitted linear map between pre- and post-continual-learning representation spaces predicts forgetting

Details

Kim, Kim & Sohn define a representation discrepancy D as the minimum, worst-case misalignment between a layer's real representation space before and after continual-learning steps, minimized over a purely linear map T with no bias term, and fit this map on real fixed-dimension ResNets, a 9-layer fully-connected ReLU network, and a ViT trained sequentially on Split-CIFAR100 (50 tasks) and ImageNet32 (50 tasks) [kim-etal-2025-measuring-representational-shifts-continual-learning-linear-transformation] The linear-transformation discrepancy correlates strongly with measured representational forgetting from frozen-feature linear probing, R-squared 0.88 on Split-CIFAR100 and 0.74 on ImageNet32 [kim-etal-2025-measuring-representational-shifts-continual-learning-linear-transformation] Forgetting saturates at a training-step horizon of 10 tasks on both datasets, with the saturation point decreasing for deeper layers (deeper layers forget faster) and increasing with network width (wider networks forget slower) [kim-etal-2025-measuring-representational-shifts-continual-learning-linear-transformation]

models: Custom fixed-dimension ResNet (Split-CIFAR100, sequential fine-tuning), Custom fixed-dimension ResNet (ImageNet32, sequential fine-tuning) · method:

OthelloGPT

Tensor Product Representation Probes Reveal Shared Structure Across Linear Directions (2026)measured

A tensor-product probe matches linear-probe accuracy on real OthelloGPT with fewer parameters, revealing a shared low-rank role-filler factorization and a saddle-shaped role manifold

Details

A tensor-product representation (TPR) probe -- 64 role (square) embeddings and 3 filler (color) embeddings bound by a shared bilinear matrix -- is fit to real OthelloGPT's board-state representation, trained on 20 million real Othello game transcripts [lee-etal-2026-tensor-product-representation-probes] The TPR probe matches ordinary 192-independent-direction linear-probe accuracy (about 99%) while using only 57.5% of the parameters (56,582 vs. 98,304), demonstrating the 192 previously-independent linear directions share a common low-rank factorized basis [lee-etal-2026-tensor-product-representation-probes] Isomap applied to the learned role embeddings reveals a saddle-shaped manifold separating board rows from columns, while the first two principal components of the filler embeddings separate empty-vs-occupied and current-vs-opponent color [lee-etal-2026-tensor-product-representation-probes]

models: OthelloGPT · method: Tensor-product representation (TPR) probing

BART

Implicit Representations of Meaning in Neural Language Models (2021)measured

Splicing probed entity-state representations causally shifts generation

Details

Li, Nye & Andreas train linear probes to decode entity state (Alchemy beaker contents, TextWorld room/object state) from fine-tuned BART and T5 encoder representations [li-nye-andreas-2021-implicit-representations-of-meaning] They go beyond decoding with a causal splice WITHIN each model (not between BART and T5): replacing one context's encoded beaker-2 description with another context's encoding to build a synthetic "mixed" state [li-nye-andreas-2021-implicit-representations-of-meaning] Generating from the spliced representation lands in the human-predicted continuation set 57.7% (BART) / 75.4% (T5) of the time, versus 20.4%/37.9% and 16.1%/29.1% for the unmixed source contexts [li-nye-andreas-2021-implicit-representations-of-meaning] The probe only reads the representation while the splice is the intervention, so this is a genuine geometry-tied causal effect [li-nye-andreas-2021-implicit-representations-of-meaning]

models: BART-base · method: Linear probing, Causal interventions (steering)

T5

Implicit Representations of Meaning in Neural Language Models (2021)measured

Splicing probed entity-state representations causally shifts generation

Details

Li, Nye & Andreas train linear probes to decode entity state (Alchemy beaker contents, TextWorld room/object state) from fine-tuned BART and T5 encoder representations [li-nye-andreas-2021-implicit-representations-of-meaning] They go beyond decoding with a causal splice WITHIN each model (not between BART and T5): replacing one context's encoded beaker-2 description with another context's encoding to build a synthetic "mixed" state [li-nye-andreas-2021-implicit-representations-of-meaning] Generating from the spliced representation lands in the human-predicted continuation set 57.7% (BART) / 75.4% (T5) of the time, versus 20.4%/37.9% and 16.1%/29.1% for the unmixed source contexts [li-nye-andreas-2021-implicit-representations-of-meaning] The probe only reads the representation while the splice is the intervention, so this is a genuine geometry-tied causal effect [li-nye-andreas-2021-implicit-representations-of-meaning]

models: T5-Base · method: Linear probing, Causal interventions (steering)

Passive object-state world models (Liu & Chen 2026)

Event-Conditioned Diagnostics of Kinematic, Contact, and Object-Permanence Fields in Passive Object-State World Models (2026)measured

Physical-variable field directions in world models are causally ablatable

Details

Liu & Chen train small GRU, Transformer-lite and RSSM-lite transition models on a synthetic object-state prediction task (free-motion/collision/occlusion) [liu-chen-2026-event-conditioned-diagnostics] A 3-class linear probe classifies the physical-event regime at macro-F1=1.00 and a phase-aware field-readout probe on RSSM states reaches mean macro-F1=0.900, so kinematic/contact/permanence fields are overlapping probe-derived directions, not fixed subspaces [liu-chen-2026-event-conditioned-diagnostics] Projecting out the contact-aligned direction increases collision-window prediction loss in 6/6 architecture-seed cases (mean CFE 0.00319), beating random-direction, random-subspace and label-shuffle controls [liu-chen-2026-event-conditioned-diagnostics] Projecting out the object-permanence direction increases hard-occlusion loss (6/6 positive) but with a mixed margin against the random-subspace control, which the authors flag as more qualified [liu-chen-2026-event-conditioned-diagnostics]

models: GRU passive object-state world model (Liu & Chen 2026), RSSM-lite passive object-state world model (Liu & Chen 2026) · method: Linear probing, Causal Field Effect (CFE) ablation

Code Llama

Do Language Models Track Entities Across State Changes? (2026)measured

LLMs consolidate entity-state tracking into a local binary subspace

Details

Tang et al. train linear probes on last-token residual activations (the "boxes" dataset) to test global versus local entity-state tracking [tang-etal-2026-do-language-models-track-entities-across-state-changes] Global 8-way state probes reach only ~0.3 accuracy versus ~0.9 for local binary probes, so the model consolidates state locally rather than tracking a full global state [tang-etal-2026-do-language-models-track-entities-across-state-changes] INLP null-space projection of the state-tag direction causally changes completions [tang-etal-2026-do-language-models-track-entities-across-state-changes] A learned sparse boolean mask over the residual PCA basis ("subspace patching") reveals a shared positional-information subspace at layers 15-25 in CodeLlama-13B [tang-etal-2026-do-language-models-track-entities-across-state-changes]

models: CodeLlama-13B · method: Linear probing, Causal interventions (steering), PCA

ELECTRA

Can Language Models Encode Perceptual Structure Without Grounding? A Case Study in Color (2021)measured

Text-only LMs linearly recover CIELAB color-space structure

Details

Abdou et al. test whether text-only LMs encode perceptual color structure against the WCS/XKCD-derived 3D CIELAB space, with no visual grounding [abdou-etal-2021-color-perceptual-structure] RSA (Kendall's tau) between color-term embeddings and CIELAB distances is significant in the color-context configuration (BERT-large 0.24, ELECTRA 0.23, RoBERTa 0.19; the random null is not) [abdou-etal-2021-color-perceptual-structure] A lasso linear mapping onto 3D CIELAB with control-task selectivity is high for all three families (0.76-0.78), needing only ~10-40 dimensions to explain 0.4-0.7 of variance [abdou-etal-2021-color-perceptual-structure] Alignment scales with model size (BERT-mini tau 0.077 -> BERT-base 0.162) with a warm/cool recovery asymmetry; no causal intervention is performed [abdou-etal-2021-color-perceptual-structure]

models: ELECTRA-large (discriminator) · method: Representational Similarity Analysis (RSA), Linear probing

Brain-to-Speech Decoder (Conv1D + BiGRU + HiFi-GAN, trained on VOCALMIND sEEG)

Mechanistic Interpretability of Brain-to-Speech Models Across Speech Modes (2026)measured

Causal patching localizes vocalized/mimed/imagined speech transfer to a compact subspace in a real brain-to-speech decoder

Details

Maghsoudi & Mishra (2026) apply cross-mode activation patching, causal scrubbing, and coarse-to-fine causal tracing to a Conv1D-encoder + 3-layer bidirectional-GRU + HiFi-GAN decoder trained on real human sEEG (VOCALMIND dataset, vocalized/mimed/imagined speech) [maghsoudi-mishra-2026-mechanistic-interpretability-of-brain-to-speech-models] Patching vocalized-mode activations into the imagined-mode pathway raises reconstruction PCC from 0.725 to 0.954; the reverse direction collapses PCC to 0.177, establishing sufficiency and necessity [maghsoudi-mishra-2026-mechanistic-interpretability-of-brain-to-speech-models] Causal scrubbing localizes the transfer to a 16-channel convolutional subspace (channels 32-48) and a specific RNN time window (steps 21-84) -- KEEP-Conv (0.666 PCC) beats size-matched RAND-Conv (0.564) -- while tri-modal linear interpolation produces smooth, monotonic transitions with mimed speech landing intermediate, consistent with a shared continuous causal manifold across speech modes [maghsoudi-mishra-2026-mechanistic-interpretability-of-brain-to-speech-models]

models: Custom Conv1D-encoder + 3-layer BiGRU + HiFi-GAN brain-to-speech decoder (trained on real human sEEG, VOCALMIND) · method: Activation patching

NFNet (Normalizer-Free Networks)

Linearly Mapping from Image to Text Space (2023)measured

One unconstrained linear map injects vision embeddings into frozen GPT-J

Details

Merullo et al. fit a single unconstrained linear projection (no orthogonality constraint, trained by SGD) mapping vision-encoder embeddings (CLIP RN50x16, NFRN50, BEiT-Large) into frozen 6B GPT-J's token-embedding space [merullo-etal-2023-linearly-mapping-image-to-text-space] Only the linear map is trained; the mapped vectors are fed as soft prompts into the untouched, frozen GPT-J [merullo-etal-2023-linearly-mapping-image-to-text-space] This causally drives competitive zero-shot image captioning (NoCaps/COCO CIDEr-D) and few-shot VQA with no fine-tuning of either model [merullo-etal-2023-linearly-mapping-image-to-text-space] It demonstrates a measured linear relationship between two independently-trained representation spaces plus a genuine behavioral effect [merullo-etal-2023-linearly-mapping-image-to-text-space]

models: NFRN50 (Normalizer-Free ResNet-50) · method: Learned linear/affine map between two representation spaces

BEiT (BERT Pre-Training of Image Transformers)

Linearly Mapping from Image to Text Space (2023)measured

One unconstrained linear map injects vision embeddings into frozen GPT-J

Details

Merullo et al. fit a single unconstrained linear projection (no orthogonality constraint, trained by SGD) mapping vision-encoder embeddings (CLIP RN50x16, NFRN50, BEiT-Large) into frozen 6B GPT-J's token-embedding space [merullo-etal-2023-linearly-mapping-image-to-text-space] Only the linear map is trained; the mapped vectors are fed as soft prompts into the untouched, frozen GPT-J [merullo-etal-2023-linearly-mapping-image-to-text-space] This causally drives competitive zero-shot image captioning (NoCaps/COCO CIDEr-D) and few-shot VQA with no fine-tuning of either model [merullo-etal-2023-linearly-mapping-image-to-text-space] It demonstrates a measured linear relationship between two independently-trained representation spaces plus a genuine behavioral effect [merullo-etal-2023-linearly-mapping-image-to-text-space]

models: BEiT-Large (ImageNet-22k pretrained) · method: Learned linear/affine map between two representation spaces

DINO

Probing the Mid-level Vision Capabilities of Self-Supervised Learning (2024)measured

Mid-level vision decodability weakly correlates with recognition accuracy

Details

Chen, Marks & Cheng introduce eight frozen-feature benchmarks for mid-level vision (segmentation, geometric/3D grouping) across 22 self-supervised models spanning contrastive, clustering, pretext and masked paradigms [chen-etal-2024-probing-the-mid-level-vision-capabilities-of-self-supervised-learning] The benchmarks decode with a nonlinear DPT dense decoder on frozen features, not a linear probe [chen-etal-2024-probing-the-mid-level-vision-capabilities-of-self-supervised-learning] Mid-level task performance correlates only weakly with each model's high-level ImageNet accuracy (generic segmentation strongest ~R^2 0.70; 3D understanding weakly correlated) [chen-etal-2024-probing-the-mid-level-vision-capabilities-of-self-supervised-learning] Several models are strongly imbalanced across the two capability classes, so mid-level decodability is not simply a byproduct of overall recognition quality [chen-etal-2024-probing-the-mid-level-vision-capabilities-of-self-supervised-learning]

models: DINO ViT-B/16 · method: Mlp probing
Text-to-Concept (and Back) via Cross-Model Alignment (2023)measured

An affine map between two vision models matches their PCA components

Details

Moayeri et al. fit an affine least-squares map between the activation spaces of independently-trained vision models (supervised/robust ResNets, Swin/DeiT/ConViT, self-supervised MoCo/DINO-ViT-S/SimCLR ResNets and ViTs, and CLIP), scored by R^2 [moayeri-etal-2023-text-to-concept-and-back-via-cross-model-alignment] The fitted map explains R^2 above 0.6, and the top principal components of the two aligned spaces correspond approximately one-to-one [moayeri-etal-2023-text-to-concept-and-back-via-cross-model-alignment] Aligning a vision encoder into CLIP's concept space enables zero-shot concept-bottleneck classification at up to 93.8% accuracy and over 92% concept-to-text relevance [moayeri-etal-2023-text-to-concept-and-back-via-cross-model-alignment] The analysis is passive, with no causal intervention on either model [moayeri-etal-2023-text-to-concept-and-back-via-cross-model-alignment]

models: DINO ViT-S/16 · method: Learned linear/affine map between two representation spaces

Sentiment-classification RNNs (LSTM/GRU/UGRNN/vanilla)

How Recurrent Networks Implement Contextual Processing in Sentiment Analysis (2020)measured

Negation and intensification deflect sentiment RNNs into an orthogonal 2D subspace

Details

Maheswaranathan & Sussillo extend fixed-point analysis with input-Jacobian modulation to explain how negation and intensification modify sentiment-RNN meaning [maheswaranathan-sussillo-2020-contextual-processing] Modifier words deflect the hidden state into a 2D subspace orthogonal to the line attractor (rather than moving along it as valence words do), with the top-2 PCs explaining 96.2% of variance [maheswaranathan-sussillo-2020-contextual-processing] The modifier subspace has two internal timescales (tau 2.6 and 4.3 tokens), with negators and intensifiers in distinguishable regions [maheswaranathan-sussillo-2020-contextual-processing] Projecting the initial state out of the 2D modifier subspace drops accuracy while a random 2D projection has no effect, a placebo-controlled causal ablation, and an augmented bag-of-words baseline with modifier convolution recovers over 90% of the RNN's gain [maheswaranathan-sussillo-2020-contextual-processing]

models: LSTM (sentiment classification, Yelp/IMDB/SST), GRU (sentiment classification, Yelp/IMDB/SST), Update Gate RNN (sentiment classification, Yelp/IMDB/SST), Vanilla RNN (sentiment classification, Yelp/IMDB/SST) · method: Fixed-point dynamics analysis, Input-Jacobian modulation analysis, PCA

Google Massively Multilingual NMT

Investigating Multilingual NMT Representations at Scale (2019)measured

Multilingual NMT encoders converge across languages with depth

Details

Kudugunta et al. train one shared 375M Transformer NMT model on over 25B pairs across 103 languages, then compare per-language representations with SVCCA [kudugunta-etal-2019] For X->En, encoder representations of different source languages become MORE similar moving up the encoder; for En->X, the same English input's encoder representations DIVERGE by target language [kudugunta-etal-2019] Embedding-layer representations are highly stable under per-language-pair fine-tuning (mean canonical correlation >0.98), with later layers more sensitive [kudugunta-etal-2019] Encoder representations visually cluster by language family in Laplacian-eigenmap embeddings, but this is qualitative with no clustering statistic; no activation intervention is performed [kudugunta-etal-2019]

models: Google Multilingual NMT (Transformer-Big, 103 languages) · method: Singular Vector Canonical Correlation Analysis (SVCCA)

GPT-3

Mapping Language Models to Grounded Conceptual Spaces (2022)measured

Behavioral grounding survives rotation but collapses under random reassignment

Details

Patel & Pavlick test whether LMs' internal representations of color, cardinal directions and grid terms carry the same relational structure as external grounded spaces, via few-shot prompting with no probe training [patel-pavlick-2022-mapping-language-models-to-grounded-conceptual-spaces] GPT-3 (175B) performs similarly on the true grounding and a structure-preserving rotation (spatial 45%/76% vs 44%/75% Top-1/Top-3) but collapses under random reassignment (16-19%) [patel-pavlick-2022-mapping-language-models-to-grounded-conceptual-spaces] Color-grounding error drops from 328 (GPT-2 124M) to 96 (GPT-3 175B), and BERT-base badly underperforms GPT-3 across all conditions [patel-pavlick-2022-mapping-language-models-to-grounded-conceptual-spaces] This is a scale-dependent relational-isomorphism-up-to-rotation claim inferred behaviorally; no activation-space intervention is performed [patel-pavlick-2022-mapping-language-models-to-grounded-conceptual-spaces]

models: GPT-3 (175B, davinci) · method: Rotation-invariance isomorphism test

PhonSSM

State Space Models are Effective Sign Language Learners: Exploiting Phonological Compositionality for Vocabulary-Scale Recognition (2026)measured

PhonSSM's imposed orthogonal phonological subspaces are causally load-bearing

Details

Cheng, Jin & Zhang build PhonSSM, whose Phonological Decomposition Module projects sign features via four parallel MLPs into four 32-dim component subspaces (handshape/location/movement/orientation) [cheng-jin-zhang-2026-phonssm] The four subspaces are made orthogonal by an explicit orthogonality loss (imposed, not discovered): mean pairwise cosine similarity is 0.12 with the loss versus 0.67 without it [cheng-jin-zhang-2026-phonssm] Linear probes show clean dissociation (handshape branch 78.4% handshape vs 31.2% location, chance 8.3%) [cheng-jin-zhang-2026-phonssm] For 47 minimal pairs, swapping only the differing component embedding flips the prediction to the pair partner 73.2% of the time versus 12.4% for control swaps (p<0.001) [cheng-jin-zhang-2026-phonssm] PhonSSM reaches 72.1% on WLASL2000 (+18.4pp over skeleton SOTA) and 53.34% on a 5,565-sign merged dataset [cheng-jin-zhang-2026-phonssm]

models: PhonSSM (AGAN + PDM + bidirectional Mamba SSM + HPC, skeleton-based ASL) · method: Geometric analysis, Activation patching

ELMo

The Low-Dimensional Linear Geometry of Contextualized Word Representations (2021)measured

Linguistic categories occupy nested low-dimensional subspaces

Details

Hernandez & Andreas apply a rank-sweep minimal-subspace search to ELMo and BERT-base, finding POS, tense, number and dependency labels are linearly decodable from subspaces well below the hidden size [hernandez-andreas-2021] Finer sub-categories are decodable from a smaller subspace nested inside the coarser parent, roughly half its rank, in many (not all) layers [hernandez-andreas-2021] Neuron-ablation shows these subspaces are distributed across many neurons (POS survives reduction to 512 of BERT's 768 axes) [hernandez-andreas-2021] Ablating only the rank-4 nounspace lowers subject-noun prediction (.85 to .82) without affecting verbs, and vice versa for verbspace, despite each removing under 1% of dimensions [hernandez-andreas-2021]

models: ELMo (5.5B-word pretrained, 2-layer biLSTM) · method: Minimal subspace search (rank-sweep probing), Causal interventions (steering)

I3D (Inflated 3D ConvNet)

Phonological Perception of Sign Language Models (2026)measured

A pose-based sign model handshape geometry matches human perception

Details

Yin et al. compare a pixel-based (I3D) and pose-based (ST-GCN) sign-recognition model on ASL Citizen, testing whether they encode phonological structure rather than shortcuts [yin-etal-2026-sign-language-phonological-perception] On minimal pairs, ST-GCN wins on 81.08% (vs I3D 69.50%), stronger on handshape contrasts while I3D is stronger on location [yin-etal-2026-sign-language-phonological-perception] RSA correlates ST-GCN's handshape similarity with human perceptual confusion (r=0.49) and an articulatory Handshape-Distance metric (r=0.55), versus I3D 0.31/0.20 [yin-etal-2026-sign-language-phonological-perception] The evidence is an RSA distance-correlation to perceptual/articulatory ground truth with no causal intervention; full text was not fully available for deeper verification [yin-etal-2026-sign-language-phonological-perception]

models: I3D (fine-tuned on ASL Citizen, pixel-based) · method: Representational Similarity Analysis (RSA)

ST-GCN (Spatio-Temporal Graph Convolutional Network)

Phonological Perception of Sign Language Models (2026)measured

A pose-based sign model handshape geometry matches human perception

Details

Yin et al. compare a pixel-based (I3D) and pose-based (ST-GCN) sign-recognition model on ASL Citizen, testing whether they encode phonological structure rather than shortcuts [yin-etal-2026-sign-language-phonological-perception] On minimal pairs, ST-GCN wins on 81.08% (vs I3D 69.50%), stronger on handshape contrasts while I3D is stronger on location [yin-etal-2026-sign-language-phonological-perception] RSA correlates ST-GCN's handshape similarity with human perceptual confusion (r=0.49) and an articulatory Handshape-Distance metric (r=0.55), versus I3D 0.31/0.20 [yin-etal-2026-sign-language-phonological-perception] The evidence is an RSA distance-correlation to perceptual/articulatory ground truth with no causal intervention; full text was not fully available for deeper verification [yin-etal-2026-sign-language-phonological-perception]

models: ST-GCN (fine-tuned on ASL Citizen, pose-based) · method: Representational Similarity Analysis (RSA)

Contrastive Predictive Coding (CPC)

Self-supervised Predictive Coding Models Encode Speaker and Phonetic Information in Orthogonal Subspaces (2023)measured

Speech models split speaker and phonetic info into orthogonal subspaces

Details

Liu, Tang & Goldwater aggregate CPC-big/small and APC frame representations into per-speaker and per-phone means and run PCA on each [liu-tang-goldwater-2023-speaker-phonetic-orthogonal-subspaces] The top speaker and phone principal directions are nearly orthogonal (CPC-big: top-20 speaker directions average similarity to their most-aligned phone direction only 0.13, max 0.26) [liu-tang-goldwater-2023-speaker-phonetic-orthogonal-subspaces] Collapsing the speaker subspace drives speaker-probe error near chance while improving phone discrimination (ABX error), beating an utterance-standardization baseline [liu-tang-goldwater-2023-speaker-phonetic-orthogonal-subspaces] The effect generalizes to speakers never seen when the subspace was estimated [liu-tang-goldwater-2023-speaker-phonetic-orthogonal-subspaces]

models: CPC-big (LibriLight 6k hrs), CPC-small (LibriSpeech-100h) · method: Geometric analysis, Orthogonal subspace collapse (aggregated-PCA)

Autoregressive Predictive Coding (APC)

Self-supervised Predictive Coding Models Encode Speaker and Phonetic Information in Orthogonal Subspaces (2023)measured

Speech models split speaker and phonetic info into orthogonal subspaces

Details

Liu, Tang & Goldwater aggregate CPC-big/small and APC frame representations into per-speaker and per-phone means and run PCA on each [liu-tang-goldwater-2023-speaker-phonetic-orthogonal-subspaces] The top speaker and phone principal directions are nearly orthogonal (CPC-big: top-20 speaker directions average similarity to their most-aligned phone direction only 0.13, max 0.26) [liu-tang-goldwater-2023-speaker-phonetic-orthogonal-subspaces] Collapsing the speaker subspace drives speaker-probe error near chance while improving phone discrimination (ABX error), beating an utterance-standardization baseline [liu-tang-goldwater-2023-speaker-phonetic-orthogonal-subspaces] The effect generalizes to speakers never seen when the subspace was estimated [liu-tang-goldwater-2023-speaker-phonetic-orthogonal-subspaces]

models: APC (LibriSpeech-360h) · method: Geometric analysis, Orthogonal subspace collapse (aggregated-PCA)

OPT

Model Editing as a Robust and Denoised Variant of DPO: A Case Study on Toxicity (2024)measured

A projected-out toxic subspace beats DPO more sample-efficiently

Details

Uppaal et al. introduce ProFS: from toxic/non-toxic pair embeddings, remove the corpus-mean direction, then SVD the centered differences and keep the top-k (ScreeNot-selected: k=2 GPT-2, k=10 larger models) as a toxic-subspace projector [uppaal-etal-2024-profs-detox-model-editing-dpo] Projecting it out of MLP value matrices once offline drops GPT-2 toxicity 48.0% to 26.8% using 500 pairs, below DPO 36.4% at 2,000 pairs, with comparable perplexity [uppaal-etal-2024-profs-detox-model-editing-dpo] Under label noise up to 50% flipped, DPO degrades monotonically while ProFS stays flat, a consequence of the SVD being provably invariant to individual pair sign-flips [uppaal-etal-2024-profs-detox-model-editing-dpo] The fraction of a single DPO gradient step explained by the toxic subspace rises with sample size and depth, framing ProFS as a denoised DPO step [uppaal-etal-2024-profs-detox-model-editing-dpo]

models: OPT-6.7B · method: ProFS (Projection Filter for Subspaces) weight-subspace projection

OLMo 2

Emotions Where Art Thou: Characterizing the Emotional Latent Space of LLMs (2025)measured

A 40-dim SVD emotional subspace with four interpretable axes steers emotion

Details

Reichman et al. extract a low-dimensional emotional subspace from mean-pooled hidden states of Llama-3.1-8B, OLMo-2-7B and Ministral-8B via centered SVD, using its top 40 dimensions [reichman-etal-2025-emotional-latent-space-llms] The four leading PCs align with valence, dominance, approach-avoidance and arousal, stable across layers (Spearman up to 0.92 for emotion ordering) [reichman-etal-2025-emotional-latent-space-llms] A linear cross-domain alignment map generalizes the subspace across 8 emotion datasets in 6 languages (aligned-subspace cosine 0.83-0.94) [reichman-etal-2025-emotional-latent-space-llms] A trained one-layer MLP mapping the 40D projection to a residual shift raises target-emotion accuracy from 9% to 83% (English, Llama) while preserving semantics [reichman-etal-2025-emotional-latent-space-llms]

models: OLMo 2 7B, OLMo 2 7B Instruct · method: PCA, Linear probing, Representation Alignment

WavLM

Eta-WavLM: Efficient Speaker Identity Removal in Self-Supervised Speech Representations Using a Simple Linear Equation (2025)measured

Subtracting a regression-fit speaker component removes speaker identity

Details

Ruggiero et al. fit a closed-form affine regression from a frozen ECAPA-TDNN speaker embedding (PCA-reduced to 128 dims) to WavLM-Large layer-15 frames, then subtract the predicted component at inference [ruggiero-etal-2025-eta-wavlm-efficient-speaker-identity-removal-in-self-supervised-speech-representations] Speaker-classification accuracy collapses 82.30% to 55.73% (paired t-test p=5e-5) [ruggiero-etal-2025-eta-wavlm-efficient-speaker-identity-removal-in-self-supervised-speech-representations] Voice-conversion quality improves over multiple baselines (LJSpeech WER 3.81 vs 4.56, MOS 4.00 vs 3.84) [ruggiero-etal-2025-eta-wavlm-efficient-speaker-identity-removal-in-self-supervised-speech-representations] Unlike orthogonal-subspace findings, this engineers and subtracts a regression-fit affine component, a continuous-variable analogue of LEACE-style removal [ruggiero-etal-2025-eta-wavlm-efficient-speaker-identity-removal-in-self-supervised-speech-representations]

models: WavLM Large · method: Closed-form affine-regression ablation of a continuous conditioning variable
Sparse Autoencoders Make Audio Foundation Models More Explainable (2025)measured

Audio-model SAEs raise DCI completeness while preserving informativeness

Details

Mariotte et al. apply TopK SAEs to pooled representations of AST, HuBERT-base, WavLM-base-plus and MERT-v1-95M on VocalSet singing-technique classification [mariotte-etal-2025-sparse-autoencoders-make-audio-foundation-models-more-explainable] SAE codes retain classification informativeness (AST 81.8% up to 95% sparsity) while raising DCI completeness for eGeMAPS acoustic factors relative to the dense baseline [mariotte-etal-2025-sparse-autoencoders-make-audio-foundation-models-more-explainable] They localize specific factors to specific layers: pitch to early layers for HuBERT/WavLM, formants to final layers [mariotte-etal-2025-sparse-autoencoders-make-audio-foundation-models-more-explainable]

models: WavLM-base-plus · method: Sparse Autoencoders (SAE), DCI disentanglement framework (Disentanglement / Completeness / Informativeness)
Self-Supervised Speech Models Encode Phonetic Context via Position-dependent Orthogonal Subspaces (2026)measured

Speech models encode neighbor-phone info in orthogonal positional subspaces

Details

Choi et al. show a single S3M frame (wav2vec 2.0, HuBERT, WavLM Large) encodes phonological vectors not only for the current phone but for its neighbors, extracted via difference-of-means at relative positions -2 to +2 [choi-etal-2026-position-dependent-orthogonal-subspaces] Cosine similarity between vectors from different relative positions is substantially lower than within the same position across 8 phonological features, an orthogonality-of-subspaces structure preserved layer-by-layer [choi-etal-2026-position-dependent-orthogonal-subspaces] Vector norm decreases monotonically with distance from the center phone, with WavLM showing the clearest trapezoidal effective context window [choi-etal-2026-position-dependent-orthogonal-subspaces] The position-dependent subspace in use switches at annotated TIMIT phonetic boundaries rather than a fixed temporal window; no causal intervention is performed [choi-etal-2026-position-dependent-orthogonal-subspaces]

models: WavLM Large · method: Difference-in-means direction, Geometric analysis

AST (Audio Spectrogram Transformer)

Sparse Autoencoders Make Audio Foundation Models More Explainable (2025)measured

Audio-model SAEs raise DCI completeness while preserving informativeness

Details

Mariotte et al. apply TopK SAEs to pooled representations of AST, HuBERT-base, WavLM-base-plus and MERT-v1-95M on VocalSet singing-technique classification [mariotte-etal-2025-sparse-autoencoders-make-audio-foundation-models-more-explainable] SAE codes retain classification informativeness (AST 81.8% up to 95% sparsity) while raising DCI completeness for eGeMAPS acoustic factors relative to the dense baseline [mariotte-etal-2025-sparse-autoencoders-make-audio-foundation-models-more-explainable] They localize specific factors to specific layers: pitch to early layers for HuBERT/WavLM, formants to final layers [mariotte-etal-2025-sparse-autoencoders-make-audio-foundation-models-more-explainable]

models: AST (Audio Spectrogram Transformer, AudioSet-finetuned) · method: Sparse Autoencoders (SAE), DCI disentanglement framework (Disentanglement / Completeness / Informativeness)

HuBERT

Sparse Autoencoders Make Audio Foundation Models More Explainable (2025)measured

Audio-model SAEs raise DCI completeness while preserving informativeness

Details

Mariotte et al. apply TopK SAEs to pooled representations of AST, HuBERT-base, WavLM-base-plus and MERT-v1-95M on VocalSet singing-technique classification [mariotte-etal-2025-sparse-autoencoders-make-audio-foundation-models-more-explainable] SAE codes retain classification informativeness (AST 81.8% up to 95% sparsity) while raising DCI completeness for eGeMAPS acoustic factors relative to the dense baseline [mariotte-etal-2025-sparse-autoencoders-make-audio-foundation-models-more-explainable] They localize specific factors to specific layers: pitch to early layers for HuBERT/WavLM, formants to final layers [mariotte-etal-2025-sparse-autoencoders-make-audio-foundation-models-more-explainable]

models: HuBERT-base · method: Sparse Autoencoders (SAE), DCI disentanglement framework (Disentanglement / Completeness / Informativeness)
Self-Supervised Speech Models Encode Phonetic Context via Position-dependent Orthogonal Subspaces (2026)measured

Speech models encode neighbor-phone info in orthogonal positional subspaces

Details

Choi et al. show a single S3M frame (wav2vec 2.0, HuBERT, WavLM Large) encodes phonological vectors not only for the current phone but for its neighbors, extracted via difference-of-means at relative positions -2 to +2 [choi-etal-2026-position-dependent-orthogonal-subspaces] Cosine similarity between vectors from different relative positions is substantially lower than within the same position across 8 phonological features, an orthogonality-of-subspaces structure preserved layer-by-layer [choi-etal-2026-position-dependent-orthogonal-subspaces] Vector norm decreases monotonically with distance from the center phone, with WavLM showing the clearest trapezoidal effective context window [choi-etal-2026-position-dependent-orthogonal-subspaces] The position-dependent subspace in use switches at annotated TIMIT phonetic boundaries rather than a fixed temporal window; no causal intervention is performed [choi-etal-2026-position-dependent-orthogonal-subspaces]

models: HuBERT Large (LibriLight 60k) · method: Difference-in-means direction, Geometric analysis
Self-Supervised Models of Speech Infer Universal Articulatory Kinematics (2023)measured

Speech SSL features linearly recover universal articulatory kinematics

Details

Cho et al. linearly project frozen features from HuBERT-Large, wav2vec2-Large, XLS-R (300M) and MMS (1B) onto electromagnetic-articulography ground truth across 62 speakers and 5 corpora (English, Mandarin, Italian) [cho-etal-2023-self-supervised-models-of-speech-infer-universal-articulatory-kinematics] Average correlation exceeds 0.8 regardless of the SSL model training language (English 0.835, XLS-R 0.830, MMS 0.832), far above a raw-acoustic baseline (0.66) [cho-etal-2023-self-supervised-models-of-speech-infer-universal-articulatory-kinematics] 44% of speakers exceed 0.85 correlation [cho-etal-2023-self-supervised-models-of-speech-infer-universal-articulatory-kinematics] Individual speakers separately-fit linear inversion systems are mutually alignable by affine transformations at near/above-0.85 correlation, evidencing a shared canonical articulatory subspace across speakers, genders and languages [cho-etal-2023-self-supervised-models-of-speech-infer-universal-articulatory-kinematics]

models: HuBERT Large (LibriLight 60k) · method: Linear probing

MERT

Sparse Autoencoders Make Audio Foundation Models More Explainable (2025)measured

Audio-model SAEs raise DCI completeness while preserving informativeness

Details

Mariotte et al. apply TopK SAEs to pooled representations of AST, HuBERT-base, WavLM-base-plus and MERT-v1-95M on VocalSet singing-technique classification [mariotte-etal-2025-sparse-autoencoders-make-audio-foundation-models-more-explainable] SAE codes retain classification informativeness (AST 81.8% up to 95% sparsity) while raising DCI completeness for eGeMAPS acoustic factors relative to the dense baseline [mariotte-etal-2025-sparse-autoencoders-make-audio-foundation-models-more-explainable] They localize specific factors to specific layers: pitch to early layers for HuBERT/WavLM, formants to final layers [mariotte-etal-2025-sparse-autoencoders-make-audio-foundation-models-more-explainable]

models: MERT-v1-95M · method: Sparse Autoencoders (SAE), DCI disentanglement framework (Disentanglement / Completeness / Informativeness)

StyleGAN

Do Not Escape From the Manifold: Discovering the Local Coordinates on the Latent Space of GANs (2021)measured

StyleGAN2's W space is globally warped; local bases avoid traversal collapse

Details

Choi et al. compute a Local Basis at each W-space point via SVD of the noise-to-W Jacobian, giving local principal-variation directions rather than one global set (as in GANSpace or SeFa) [choi-etal-2021-local-basis-gan-manifold] Grassmannian distance between top-k Local Basis subspaces shows nearby points are far more similar than random points, themselves distinct from a random-subspace null, so W space is globally warped rather than flat/affine [choi-etal-2021-local-basis-gan-manifold] Traversing along the Local Basis keeps FID low under strong perturbation while global methods' FID rises sharply (image collapse), a measured causal effect of respecting local geometry [choi-etal-2021-local-basis-gan-manifold]

models: StyleGAN2 (trained on FFHQ, 1024x1024) · method: Jacobian spectral direction analysis, Subspace-angle alignment, Causal interventions (steering)

MultiBERTs

Subspace Chronicles: How Linguistic Information Emerges, Shifts and Interacts during Language Model Training (2023)measured

Linguistic subspaces emerge then specialize in critical pretraining phases

Details

Muller-Eberstein et al. fit an information-theoretic linear probing suite for 9 syntax/semantics/reasoning tasks at every layer of MultiBERTs across 2M pretraining steps and 5 random seeds (0-4) [muller-eberstein-etal-2023-subspace-chronicles] They compare the fitted probes subspaces directly via a subspace-similarity metric rather than only per-task accuracy [muller-eberstein-etal-2023-subspace-chronicles] Task-specific subspaces first emerge and share information broadly, then shift and specialize into distinct subspaces during identifiable critical learning phases [muller-eberstein-etal-2023-subspace-chronicles] Syntax subspaces form within 0.5% of total training while semantic and reasoning subspaces specialize much later, giving a developmental trajectory [muller-eberstein-etal-2023-subspace-chronicles]

models: MultiBERTs (seed 0, tracked across 2M pretraining steps), MultiBERTs (seed 1, tracked across pretraining steps), MultiBERTs (seed 2, tracked across pretraining steps), MultiBERTs (seed 3, tracked across pretraining steps), MultiBERTs (seed 4, tracked across pretraining steps) · method: Subspace-similarity probing

EuroBERT

Revisiting Anisotropy in Language Transformers: The Geometry of Learning Dynamics (2026)measured

Anisotropy from frequency-blind sampling and self-reinforcing tangent gradients

Details

Bernas et al. give a two-part geometric account of anisotropy: they prove (Corollary 2.2) that as high-frequency tokens concentrate near their centroid, local-manifold curvature becomes statistically invisible [bernas-etal-2026] They prove (Proposition 2.3) the normal-to-tangent gradient magnitude ratio scales as O(t) in the concentration radius, so curvature-bearing normal updates are suppressed for high-frequency tokens, compounded by attention and residuals [bernas-etal-2026] Across the Pythia (160m/410m/1b/1.4b), SmolLM2 (360m/1.7b) and EuroBERT (210m/610m) suites, true gradients concentrate in an activation-derived tangent subspace far more than matched-rank random controls (energy ratios orders of magnitude above null) [bernas-etal-2026] The effect is strongest early in training and in early/middle layers, and weaker in EuroBERT, whose language-balanced training counteracts the frequency skew the mechanism depends on [bernas-etal-2026] The paper reframes anisotropy as a possibly adaptive implicit dimensionality reduction rather than a pure training pathology [bernas-etal-2026]

models: EuroBERT-210m, EuroBERT-610m · method: PCA, Gradient–Tangent Alignment Test

SmolLM2

Revisiting Anisotropy in Language Transformers: The Geometry of Learning Dynamics (2026)measured

Anisotropy from frequency-blind sampling and self-reinforcing tangent gradients

Details

Bernas et al. give a two-part geometric account of anisotropy: they prove (Corollary 2.2) that as high-frequency tokens concentrate near their centroid, local-manifold curvature becomes statistically invisible [bernas-etal-2026] They prove (Proposition 2.3) the normal-to-tangent gradient magnitude ratio scales as O(t) in the concentration radius, so curvature-bearing normal updates are suppressed for high-frequency tokens, compounded by attention and residuals [bernas-etal-2026] Across the Pythia (160m/410m/1b/1.4b), SmolLM2 (360m/1.7b) and EuroBERT (210m/610m) suites, true gradients concentrate in an activation-derived tangent subspace far more than matched-rank random controls (energy ratios orders of magnitude above null) [bernas-etal-2026] The effect is strongest early in training and in early/middle layers, and weaker in EuroBERT, whose language-balanced training counteracts the frequency skew the mechanism depends on [bernas-etal-2026] The paper reframes anisotropy as a possibly adaptive implicit dimensionality reduction rather than a pure training pathology [bernas-etal-2026]

models: SmolLM2-360M, SmolLM2-1.7B · method: PCA, Gradient–Tangent Alignment Test

Yi

Task Recognition and Task Learning Heads Align In-Context Hidden States with a Label-Unembedding Task Subspace (2026)measured

ICL task-recognition and task-learning heads project onto a label subspace

Details

Yang, Cho & Inoue introduce Task Subspace Logit Attribution, scoring each attention head output by projection onto the span of label-token unembedding vectors [yang-cho-inoue-2026-task-subspace-attention-heads] Task-recognition heads align with the task subspace while task-learning heads rotate the hidden state toward the correct-minus-incorrect label direction within it [yang-cho-inoue-2026-task-subspace-attention-heads] Across Llama3-8B layers, hidden-update vs TR-head subspace alignment correlates rho=0.94, and logit-difference vs TL-head alignment rho=0.53 [yang-cho-inoue-2026-task-subspace-attention-heads] Injecting the summed top-3% TR-head outputs into zero-shot prompts raises classification accuracy 9.2% to 40.4% (TR-based), while TL-based vectors dominate open-ended generation; ablating each collapses its own role, tested across 6 models [yang-cho-inoue-2026-task-subspace-attention-heads]

models: Yi-34B (base) · method: Direct Logit Attribution (DLA), Activation patching

InternVL

Uncovering and Shaping the Latent Representation of 3D Scene Topology in Vision-Language Models (2026)measured

SVD-based cross-scene averaging isolates a linear 3D-position subspace in real VLMs

Details

Wang & Gao (2026) posit an additive decomposition per object-token, h = u_id + u_sp + noise, in real Qwen2.5-VL-7B and InternVL3-8B; averaging an object's activations across many randomly-positioned synthetic 3D scenes cancels the position term, isolating an identity basis via SVD, whose orthogonal complement recovers a spatial subspace [wang-gao-2026-3d-scene-topology-in-vlms] PCA of the spatial-extracted residual recovers a 3D geometry matching true scene layout, formally converging to Laplacian eigenmaps of the scene graph [wang-gao-2026-3d-scene-topology-in-vlms] Injecting a probe-derived direction into layer-12 residuals of Qwen2.5-VL-7B causally shifts the x-coordinate probe readout monotonically with injection strength (alpha=+0.30: delta-x-hat=+0.091+/-0.021 vs. control +0.001+/-0.030), while a null-direction control shows no effect [wang-gao-2026-3d-scene-topology-in-vlms]

models: InternVL3-8B · method: SVD, Activation patching

Grid-walker toy transformer

Predictive Statistics Shape Emergent World Representations of Grid Walkers (2026)measured

Attention builds a shared world-state coordinate system; the MLP specializes it

Details

Brenner et al. train one decoder-only transformer (L=4, H=4, d_model=128, HookedTransformer) from scratch, six times, on six 2D-lattice random walkers defined by two endpoints {(0,0),(4,0)} and three horizons {20,200,1000} [brenner-etal-2026] Held-out affine probes show the residual stream after the first attention block linearly encodes the full geometric sufficient statistic [x/t, y/t, log t] (R^2=0.84-0.99), while the post-MLP representation encodes the smaller task-specific next-step statistic [brenner-etal-2026] Linear CKA between post-attention activations across different walkers is high (0.95-0.97), so attention builds one shared coordinate system, while post-MLP cross-walker CKA drops sharply, so the MLP specializes it [brenner-etal-2026] The first attention block builds the statistic via uniform causal averaging of value-writes, with a near-additive token/position decomposition and antipodal move codes [brenner-etal-2026] Positions sharing an identical next-step distribution remain linearly distinguishable via their longer-horizon futures, evidence the network keeps more than the immediate objective needs [brenner-etal-2026]

models: Grid-walker decoder transformer (L4/H4/d_model=128, HookedTransformer) · method: Linear probing, Centered Kernel Alignment (CKA), Analytical derivation

Wu, Geiger & Millière (2025) Variable-Binding Toy Transformer

How Do Transformers Learn Variable Binding in Symbolic Programs? (2025)measured

A trained transformer forms separate subspaces for constants and variables

Details

Wu, Geiger & Milliere train a 12-layer 8-head GPT-2-style transformer from scratch (37.8M params, RoPE) on synthetic variable-assignment programs to >99.9% accuracy [wu-geiger-milliere-2025-variable-binding-symbolic-programs] PCA plus L1-regularized probing finds separate residual-stream subspaces for numerical-constant (10 components) and variable-name (26 components) information [wu-geiger-milliere-2025-variable-binding-symbolic-programs] UMAP shows increasing cluster separation across training [wu-geiger-milliere-2025-variable-binding-symbolic-programs] Interchange interventions swapping only the selected subspace between original and counterfactual programs causally validate each subspace role [wu-geiger-milliere-2025-variable-binding-symbolic-programs]

models: 12-layer, 8-head causal Transformer (d_model=512, RoPE, synthetic variable-assignment-program task) · method: PCA, Activation patching, UMAP