Jacobian centroid extraction

methods / Theoretical / Analytical / Jacobian centroid extraction

Techniqueadvanced

Replaces a sub-network's raw hidden-state activation with the row-sum of its local input-output Jacobian (mu = J^T * 1, evaluated at the affine region containing the input) as the feature-representation space, then applies standard representation-analysis tools (PCA, sparse dictionary learning, probing) to these centroids instead of to activations.

Used in (1 observation)

Jacobian centroids (row-sums of a sub-component's local input-output Jacobian) form cleaner, more transferable linear feature directions than raw activations across four real-model settings: ResNet50/DTD, DINOv2-vs-DINOv3 SAE dictionaries, GPT2-Large circuit discovery, and Llama-3.1-8B truth probesmeasured

structure: Linear Centroids Hypothesis · models: ResNet-50 (supervised, ImageNet), DINOv2 ViT-L/14, DINOv3 ViT-B/16, GPT-2-Large, Llama-3.1-8B · paper: The Linear Centroids Hypothesis: Features as Directions Learned by Local Experts