MATH · IN · MODELS
structures / Hypotheses / Linear Centroids Hypothesis

Linear Centroids Hypothesis

CLAIMhypothesisadvancedhow it's classified →

Features of a deep network are linear directions not in its raw activation space, but in its centroid space — the space of Jacobian row-sums mu = J^T*1 summarizing each local affine expert's actual input-output map. Proposed by Walker, Humayun, Balestriero & Baraniuk (2026) as a mapping-aware replacement for the Linear Representation Hypothesis.

Replicationcomputed from the corpus — never hand-assigned
1 paper3 architecture classes · one study2 domains · one study5 model families · one study
Filled = two or more values reported by papers that share no author — replication. Outlined = two or more values, but all from a single study — breadth, not replication. Grey = a single value. Derived from paper authorship and each model's architecture class, domain and family; it updates itself when a paper is added.

Statement

“The features of a DN are represented by linear directions in its centroid spaces” (Walker, Humayun, Balestriero & Baraniuk, 2026, arXiv:2604.11962, Section 3.2). A centroid μν(x)(ℓ1←ℓ2)=(Jx(f(ℓ1←ℓ2)))⊤1\boldsymbol{\mu}_{\boldsymbol{\nu}(\boldsymbol{x})}^{(\ell_1\leftarrow\ell_2)} = \big(J_{\boldsymbol{x}}(f^{(\ell_1\leftarrow\ell_2)})\big)^\top\mathbf{1} is the row-sum of a sub-component’s local input-output Jacobian — a vector or summary of a local affine expert that characterizes the network’s learned mapping exactly (for piecewise-affine networks) or approximately (for smooth networks like transformers), rather than merely recording where one input happens to land — see Jacobian centroid extraction. LCH claims the network’s human- interpretable features are linear directions in this space, not necessarily in the space of raw intermediate activations that the Linear Representation Hypothesis is conventionally tested against.

Intuition

Two different inputs can produce very different raw activation coordinates while a sub-network still applies the same local transformation to both — the activation records where you ended up, not what map got you there. LRH looks for feature directions in the “where you ended up” space. LCH instead asks: what if features are directions in the space of “what the network actually computed locally”? The centroid is a specific mathematical object for capturing that: it is constant across every input sharing the same affine region, and changes only when the underlying computed mapping changes.

Properties

  • A rival, not merely a refinement, of LRH. The paper argues LRH “decouples the input-output maps learned by a DN from the organization of feature directions in its activation spaces” and identifies three specific limitations: it abstracts away from individual sub-components, it is unclear whether every linear direction in activation space corresponds to a genuine feature, and per-latent-space analysis leaves cross-component feature contextualization unaddressed.
  • Linear separability is necessary but not sufficient for a genuine (non-spurious) feature (Lemma 3.1). Proved via a ReLU-collapse counterexample: two input sets can be linearly separable in activation space purely as an artifact of the network’s piecewise-affine folding, without the network’s actual computation treating them as functionally distinct — motivating the move to a mapping-anchored (Jacobian-derived) notion of feature instead of a purely geometric (activation-space) separability criterion.
  • Same operational move for smooth (transformer) and piecewise-affine (ReLU) networks alike. For a piecewise-affine network the centroid is an exact algebraic quantity per affine region; for a smooth network it is the same Jacobian-row-sum construction evaluated at the local linearization, so the hypothesis is stated in a form that applies to both CNN and transformer architectures without a separate definition for each.
  • Costs roughly 10-15% more compute to extract than raw activations in the paper’s own benchmarking, since it additionally requires a Jacobian (or its practical proxy) at each site of interest rather than only a forward pass.
  • Framed by its own authors as compatible with, and supportive of, Platonic Representation Hypothesis rather than competing with it: because centroids track the functional mapping rather than incidental intermediate geometry, the paper argues they converge more reliably across independently-trained models of different scale than raw- activation dictionaries do.

Key papers

  • Walker, Humayun, Balestriero & Baraniuk (2026). The Linear Centroids Hypothesis: Features as Directions Learned by Local Experts. arXiv:2604.11962. Introduces LCH and tests it across four real-model settings: (1) PCA on ResNet50 (ImageNet-pretrained) fourth-layer centroids separates DTD texture classes into distinct linear directions more cleanly than PCA on raw activations; a paired FashionMNIST probe further shows centroid-space linear probes track a color feature only to the extent it is task-relevant, whereas activation-space probes remain linearly decodable even when the feature is spurious. (2) TopK sparse dictionaries trained on DINOv2/DINOv3 centroids (vs. on raw activations) show markedly higher cross-model max cosine similarity — the raw-activation dictionaries’ feature pairs cluster around a cosine similarity of about 0.4 in a bimodal distribution the authors read as poor cross-model correlation, while centroid-based dictionaries converge reliably across model scale. (3) On GPT2-Large, a centroid-perturbation attribution score (normalized centroid displacement under single-neuron ablation) independently re-identifies a previously-reported “an”- predicting neuron (Clement & Joseph) at the 99.8th percentile of scores, without using logit or output-probability information — an internal, representation-level circuit-discovery signal rather than a behavioral one. (4) Mass-mean probes (Marks & Tegmark’s construction) fit on Llama-3.1-8B centroids generalize better across independent truth- identifying datasets than the same probes fit on raw activations. See Jacobian centroid extraction for the shared underlying technique across all four settings.

Found in (1 observation · 5 families)

ResNet

The Linear Centroids Hypothesis: Features as Directions Learned by Local Experts (2026)measured

Jacobian centroids (row-sums of a sub-component's local input-output Jacobian) form cleaner, more transferable linear feature directions than raw activations across four real-model settings: ResNet50/DTD, DINOv2-vs-DINOv3 SAE dictionaries, GPT2-Large circuit discovery, and Llama-3.1-8B truth probes

Details

Walker, Humayun, Balestriero & Baraniuk (2026) propose the Linear Centroids Hypothesis and test it by re-running standard representation- analysis tools on centroids (mu = J^T*1, see [[jacobian-centroid-extraction]]) instead of raw activations, across four real, pretrained-model settings. (1) PCA on ResNet50's (ImageNet-pretrained) fourth-layer centroids separates two DTD texture classes into visually distinct linear directions; a companion FashionMNIST color-correlation probe shows centroid-space linear probes track a color feature only in proportion to its actual task-relevance, while activation-space probes stay linearly decodable for the feature even when it is spurious to the task. (2) TopK sparse dictionaries fit on DINOv2 and DINOv3 centroids show substantially higher cross-model max cosine similarity than dictionaries fit on the same models' raw activations, whose feature pairs cluster in a bimodal distribution around cosine similarity ~0.4 (read by the authors as poor cross-model correspondence) -- framed as additional support for [[platonic-representation]], since centroids are argued to track the functional mapping rather than incidental intermediate geometry. (3) On GPT2-Large, a centroid-displacement attribution score under single-neuron ablation independently re-identifies a previously reported "an"-predicting neuron (Clement & Joseph) at the 99.8th percentile of scores -- a representation-level circuit-discovery signal computed without reference to logits or output probabilities. (4) Mass-mean probes (Marks & Tegmark's construction) fit on Llama-3.1-8B centroids generalize better to held-out truth-identifying datasets than the same probes fit on raw activations. The paper also proves (Lemma 3.1, via a ReLU-collapse counterexample) that linear separability in raw activation space is necessary but not sufficient for two input sets to correspond to a genuine (non-spurious) feature -- motivating the shift to a Jacobian-anchored notion of feature in place of a purely geometric separability criterion. Several of the paper's own comparative numbers (exact probe accuracies/SDs, SAE firing-frequency rates, and the precise centroid-side cosine-similarity values) are reported only in figures that could not be independently extracted for this entry; the cosine-similarity ~0.4 baseline and the 99.8th-percentile neuron-attribution figure above are the two numerically quotable results recovered from the paper's running text.

models: ResNet-50 (supervised, ImageNet) · method: Jacobian centroid extraction, PCA, TopK (k-sparse) Sparse Autoencoders, Linear probing

DINOv2

The Linear Centroids Hypothesis: Features as Directions Learned by Local Experts (2026)measured

Jacobian centroids (row-sums of a sub-component's local input-output Jacobian) form cleaner, more transferable linear feature directions than raw activations across four real-model settings: ResNet50/DTD, DINOv2-vs-DINOv3 SAE dictionaries, GPT2-Large circuit discovery, and Llama-3.1-8B truth probes

Details

Walker, Humayun, Balestriero & Baraniuk (2026) propose the Linear Centroids Hypothesis and test it by re-running standard representation- analysis tools on centroids (mu = J^T*1, see [[jacobian-centroid-extraction]]) instead of raw activations, across four real, pretrained-model settings. (1) PCA on ResNet50's (ImageNet-pretrained) fourth-layer centroids separates two DTD texture classes into visually distinct linear directions; a companion FashionMNIST color-correlation probe shows centroid-space linear probes track a color feature only in proportion to its actual task-relevance, while activation-space probes stay linearly decodable for the feature even when it is spurious to the task. (2) TopK sparse dictionaries fit on DINOv2 and DINOv3 centroids show substantially higher cross-model max cosine similarity than dictionaries fit on the same models' raw activations, whose feature pairs cluster in a bimodal distribution around cosine similarity ~0.4 (read by the authors as poor cross-model correspondence) -- framed as additional support for [[platonic-representation]], since centroids are argued to track the functional mapping rather than incidental intermediate geometry. (3) On GPT2-Large, a centroid-displacement attribution score under single-neuron ablation independently re-identifies a previously reported "an"-predicting neuron (Clement & Joseph) at the 99.8th percentile of scores -- a representation-level circuit-discovery signal computed without reference to logits or output probabilities. (4) Mass-mean probes (Marks & Tegmark's construction) fit on Llama-3.1-8B centroids generalize better to held-out truth-identifying datasets than the same probes fit on raw activations. The paper also proves (Lemma 3.1, via a ReLU-collapse counterexample) that linear separability in raw activation space is necessary but not sufficient for two input sets to correspond to a genuine (non-spurious) feature -- motivating the shift to a Jacobian-anchored notion of feature in place of a purely geometric separability criterion. Several of the paper's own comparative numbers (exact probe accuracies/SDs, SAE firing-frequency rates, and the precise centroid-side cosine-similarity values) are reported only in figures that could not be independently extracted for this entry; the cosine-similarity ~0.4 baseline and the 99.8th-percentile neuron-attribution figure above are the two numerically quotable results recovered from the paper's running text.

models: DINOv2 ViT-L/14 · method: Jacobian centroid extraction, PCA, TopK (k-sparse) Sparse Autoencoders, Linear probing

DINOv3

The Linear Centroids Hypothesis: Features as Directions Learned by Local Experts (2026)measured

Jacobian centroids (row-sums of a sub-component's local input-output Jacobian) form cleaner, more transferable linear feature directions than raw activations across four real-model settings: ResNet50/DTD, DINOv2-vs-DINOv3 SAE dictionaries, GPT2-Large circuit discovery, and Llama-3.1-8B truth probes

Details

Walker, Humayun, Balestriero & Baraniuk (2026) propose the Linear Centroids Hypothesis and test it by re-running standard representation- analysis tools on centroids (mu = J^T*1, see [[jacobian-centroid-extraction]]) instead of raw activations, across four real, pretrained-model settings. (1) PCA on ResNet50's (ImageNet-pretrained) fourth-layer centroids separates two DTD texture classes into visually distinct linear directions; a companion FashionMNIST color-correlation probe shows centroid-space linear probes track a color feature only in proportion to its actual task-relevance, while activation-space probes stay linearly decodable for the feature even when it is spurious to the task. (2) TopK sparse dictionaries fit on DINOv2 and DINOv3 centroids show substantially higher cross-model max cosine similarity than dictionaries fit on the same models' raw activations, whose feature pairs cluster in a bimodal distribution around cosine similarity ~0.4 (read by the authors as poor cross-model correspondence) -- framed as additional support for [[platonic-representation]], since centroids are argued to track the functional mapping rather than incidental intermediate geometry. (3) On GPT2-Large, a centroid-displacement attribution score under single-neuron ablation independently re-identifies a previously reported "an"-predicting neuron (Clement & Joseph) at the 99.8th percentile of scores -- a representation-level circuit-discovery signal computed without reference to logits or output probabilities. (4) Mass-mean probes (Marks & Tegmark's construction) fit on Llama-3.1-8B centroids generalize better to held-out truth-identifying datasets than the same probes fit on raw activations. The paper also proves (Lemma 3.1, via a ReLU-collapse counterexample) that linear separability in raw activation space is necessary but not sufficient for two input sets to correspond to a genuine (non-spurious) feature -- motivating the shift to a Jacobian-anchored notion of feature in place of a purely geometric separability criterion. Several of the paper's own comparative numbers (exact probe accuracies/SDs, SAE firing-frequency rates, and the precise centroid-side cosine-similarity values) are reported only in figures that could not be independently extracted for this entry; the cosine-similarity ~0.4 baseline and the 99.8th-percentile neuron-attribution figure above are the two numerically quotable results recovered from the paper's running text.

models: DINOv3 ViT-B/16 · method: Jacobian centroid extraction, PCA, TopK (k-sparse) Sparse Autoencoders, Linear probing

GPT

The Linear Centroids Hypothesis: Features as Directions Learned by Local Experts (2026)measured

Jacobian centroids (row-sums of a sub-component's local input-output Jacobian) form cleaner, more transferable linear feature directions than raw activations across four real-model settings: ResNet50/DTD, DINOv2-vs-DINOv3 SAE dictionaries, GPT2-Large circuit discovery, and Llama-3.1-8B truth probes

Details

Walker, Humayun, Balestriero & Baraniuk (2026) propose the Linear Centroids Hypothesis and test it by re-running standard representation- analysis tools on centroids (mu = J^T*1, see [[jacobian-centroid-extraction]]) instead of raw activations, across four real, pretrained-model settings. (1) PCA on ResNet50's (ImageNet-pretrained) fourth-layer centroids separates two DTD texture classes into visually distinct linear directions; a companion FashionMNIST color-correlation probe shows centroid-space linear probes track a color feature only in proportion to its actual task-relevance, while activation-space probes stay linearly decodable for the feature even when it is spurious to the task. (2) TopK sparse dictionaries fit on DINOv2 and DINOv3 centroids show substantially higher cross-model max cosine similarity than dictionaries fit on the same models' raw activations, whose feature pairs cluster in a bimodal distribution around cosine similarity ~0.4 (read by the authors as poor cross-model correspondence) -- framed as additional support for [[platonic-representation]], since centroids are argued to track the functional mapping rather than incidental intermediate geometry. (3) On GPT2-Large, a centroid-displacement attribution score under single-neuron ablation independently re-identifies a previously reported "an"-predicting neuron (Clement & Joseph) at the 99.8th percentile of scores -- a representation-level circuit-discovery signal computed without reference to logits or output probabilities. (4) Mass-mean probes (Marks & Tegmark's construction) fit on Llama-3.1-8B centroids generalize better to held-out truth-identifying datasets than the same probes fit on raw activations. The paper also proves (Lemma 3.1, via a ReLU-collapse counterexample) that linear separability in raw activation space is necessary but not sufficient for two input sets to correspond to a genuine (non-spurious) feature -- motivating the shift to a Jacobian-anchored notion of feature in place of a purely geometric separability criterion. Several of the paper's own comparative numbers (exact probe accuracies/SDs, SAE firing-frequency rates, and the precise centroid-side cosine-similarity values) are reported only in figures that could not be independently extracted for this entry; the cosine-similarity ~0.4 baseline and the 99.8th-percentile neuron-attribution figure above are the two numerically quotable results recovered from the paper's running text.

models: GPT-2-Large · method: Jacobian centroid extraction, PCA, TopK (k-sparse) Sparse Autoencoders, Linear probing

Llama

The Linear Centroids Hypothesis: Features as Directions Learned by Local Experts (2026)measured

Jacobian centroids (row-sums of a sub-component's local input-output Jacobian) form cleaner, more transferable linear feature directions than raw activations across four real-model settings: ResNet50/DTD, DINOv2-vs-DINOv3 SAE dictionaries, GPT2-Large circuit discovery, and Llama-3.1-8B truth probes

Details

Walker, Humayun, Balestriero & Baraniuk (2026) propose the Linear Centroids Hypothesis and test it by re-running standard representation- analysis tools on centroids (mu = J^T*1, see [[jacobian-centroid-extraction]]) instead of raw activations, across four real, pretrained-model settings. (1) PCA on ResNet50's (ImageNet-pretrained) fourth-layer centroids separates two DTD texture classes into visually distinct linear directions; a companion FashionMNIST color-correlation probe shows centroid-space linear probes track a color feature only in proportion to its actual task-relevance, while activation-space probes stay linearly decodable for the feature even when it is spurious to the task. (2) TopK sparse dictionaries fit on DINOv2 and DINOv3 centroids show substantially higher cross-model max cosine similarity than dictionaries fit on the same models' raw activations, whose feature pairs cluster in a bimodal distribution around cosine similarity ~0.4 (read by the authors as poor cross-model correspondence) -- framed as additional support for [[platonic-representation]], since centroids are argued to track the functional mapping rather than incidental intermediate geometry. (3) On GPT2-Large, a centroid-displacement attribution score under single-neuron ablation independently re-identifies a previously reported "an"-predicting neuron (Clement & Joseph) at the 99.8th percentile of scores -- a representation-level circuit-discovery signal computed without reference to logits or output probabilities. (4) Mass-mean probes (Marks & Tegmark's construction) fit on Llama-3.1-8B centroids generalize better to held-out truth-identifying datasets than the same probes fit on raw activations. The paper also proves (Lemma 3.1, via a ReLU-collapse counterexample) that linear separability in raw activation space is necessary but not sufficient for two input sets to correspond to a genuine (non-spurious) feature -- motivating the shift to a Jacobian-anchored notion of feature in place of a purely geometric separability criterion. Several of the paper's own comparative numbers (exact probe accuracies/SDs, SAE firing-frequency rates, and the precise centroid-side cosine-similarity values) are reported only in figures that could not be independently extracted for this entry; the cosine-similarity ~0.4 baseline and the 99.8th-percentile neuron-attribution figure above are the two numerically quotable results recovered from the paper's running text.

models: Llama-3.1-8B · method: Jacobian centroid extraction, PCA, TopK (k-sparse) Sparse Autoencoders, Linear probing