Trains a linear classifier or regressor on frozen activations to test whether a concept is linearly decodable — the main operational test for the Linear Representation Hypothesis.

Extracts a candidate feature direction as the difference between mean activations of two contrastive prompt sets (e.g. harmful vs. harmless instructions) at a chosen layer and token position — no training required.

A small multi-layer perceptron (typically 1–2 hidden layers with a nonlinearity) trained on frozen activations to predict a target property; used in place of a linear probe when the property may be decodable only nonlinearly, at the cost of the interpretability a single weight direction provides.

Fits a linear transform B so that squared Euclidean distance/norm under B approximates a target graph metric (e.g. parse-tree edge distance and depth) across all pairs of tokens simultaneously — a metric-learning probe, not a single-output classifier or regressor.

Identifies the small set of attention heads with the highest average indirect effect (via causal mediation / activation patching) on a diverse set of in-context-learning tasks, then sums those heads' mean per-task output activations into a single 'function vector' representing the task.

Extracts a task-encoding vector directly as an intermediate-layer residual-stream activation — computed using a query-independent 'dummy query' so the vector doesn't leak information about the real query — then validates it by patching that single activation into a separate forward pass on a fresh query.

Approximates a transformer's subject-to-object relation decoding F(s,c) by a first-order Taylor expansion — the mean Jacobian W and bias b of F with respect to the subject representation s, estimated from a handful of examples — yielding a single affine map LRE(s)=βWs+b that is faithful (matches the model's real next-token prediction) and causal (inverting it edits which object the model predicts) for roughly half of tested relations, but conspicuously fails for others despite the model still answering those relations correctly.

Unsupervised direction extraction: given contrast pairs of statements with opposite truth values, finds a direction whose projected values satisfy logical negation consistency (p(true) + p(false) ≈ 1) without ever using ground-truth labels.

Extracts a diff-in-means direction independently in each of several differently-sized/architected models, then fits a ridge-regression map between one model's activation space and another's (on paired activations from the same prompts) and transfers the source model's direction into the target model's space by applying that same linear map — turning 'do two unrelated models share a behavioral direction' into a directly testable geometric transfer, rather than a same-model-only diff-in-means claim.

Finds image-specific (local) semantic edit directions by computing the Jacobian of a generative model's output map with respect to a bottleneck latent, then taking its SVD - the top right singular vectors are the directions that locally perturb the output the most, computed per-sample without any labels or auxiliary loss.

Extracts a 'reading vector' for a high-level concept or behavioral function as the first principal component of paired activation differences, collected from contrastive stimulus templates designed to isolate that concept — a PCA-of-differences variant of direction extraction, usable with or without labels.

Injects a fixed a priori basis (e.g. a Fourier or binary-digit matrix) into a probe's architecture as a candidate hidden representation scheme, learning only a low-dimensional projection into and out of it — testing whether a specific structured hypothesis fits activations far better than an unconstrained linear probe, rather than fitting the freest possible model.

Finds a direction (or an orthonormal basis of several directions) by gradient-based optimization of a custom loss with explicit ablation/addition/retain terms, rather than a closed-form statistic (diff-in-means) or a generic classifier fit (linear-probing).

Recovers a per-concept difference vector for each factor of variation from paired embeddings (e.g. same scene differing only in one attribute), then scores how well an embedding reconstructs as a sum of these factor vectors ($R^2$) and how mutually orthogonal the recovered factors are — turning "is this representation compositional" into two separate quantified numbers.

Thread extra 'concept' tokens through a frozen diffusion transformer's own text-conditioning weights, one-directionally (concepts read from image/prompt tokens, never write back), then localize each concept by a dot product between concept and image tokens taken specifically in the attention-output space (post output-projection) rather than the raw cross-attention softmax(QK^T)V space.

Decomposes an embedding into a sparse, non-negative linear combination of a fixed, human-interpretable concept dictionary (e.g. common words/phrases passed through the model's own text encoder) via per-sample convex optimization (LASSO/ADMM) - no encoder/decoder network is trained, unlike a sparse autoencoder.

A structural probe whose target metric space is a Poincaré ball rather than flat Euclidean space — a linear map into the tangent space at the origin, followed by the exponential map into the ball, then the hyperbolic distance/norm-from-origin replace the Euclidean d_B(u,v) and ||u||_B used by an ordinary structural probe.

Extracts a concept direction for a specific (relation, object) pair by inverting a linear relational embedding — a learned affine map from subject activations to object activations — rather than contrasting two labeled classes.

Learns a k-dimensional subspace whose closed-form solution is the leading eigenvectors of a signed combination of within-class scatter matrices - maximizing projected variance for a target class while jointly minimizing it for all other classes - generalizing single-class PCA and the Fukunaga-Koontz transform to more than two classes in one eigendecomposition.

Finds an edit direction (or small per-timestep function) in a frozen generative model's bottleneck by gradient-optimizing a directional CLIP loss between source and target text prompts, plus a reconstruction/regularization term, rather than contrasting labeled activation classes.

Linearly projects out the compositional (atom-count/formula-level) signal from a representation before measuring how much genuinely geometric/structural information remains via a Ridge probe, isolating structure-specific information from a confounding compositional shortcut.

Quantifies how cleanly a set of learned latent codes maps onto a set of known ground-truth generative factors along three separate axes: Disentanglement (does each code capture at most one factor), Completeness (is each factor captured by as few codes as possible), and Informativeness (can a factor be predicted from the codes at all) — typically operationalized via a Lasso/regression importance matrix between codes and factors.

Approximately inverts a diffusion model's deterministic sampling process (integrating the learned velocity/score field backward from a clean sample to noise) to recover intermediate diffusion-transformer states, then fits a linear probe on those recovered states — extending linear probing to generative diffusion models whose forward pass alone doesn't expose a clean encoder representation to probe.

Fits a linear direction separating two semantically opposite action classes (e.g. push vs. pull) from frozen video-model activations, then measures the consistency of that direction's sign/orientation across held-out clips, distinguishing an oriented axis of variation from mere cluster separability (which cosine-similarity or accuracy alone can conflate).

Selects a sparse subset of units — raw neurons or SAE latent features — as language-associated by computing, for each unit, the entropy of its per-language activation-probability distribution and keeping only low-entropy (single-language-dominant) units.

Extracts a per-group (e.g. per-language) direction as that group's own empirical mean activation, then tests its role by subtracting it (centering) and measuring the effect on downstream extrinsic tasks — a one-sided special case of diff-in-means, using an absolute group mean rather than a difference between two contrastive groups.

Compares two probe-derived directions using cosine similarity computed in a Mahalanobis-whitened (covariance-normalized) coordinate frame rather than the raw activation space, giving a far stronger predictor of cross-domain probe generalization than standard cosine similarity.

Sweeps candidate subspace dimension d, jointly training a rank-d linear projection plus classifier at each d, to find the smallest d at which held-out accuracy is within a fixed tolerance of the unconstrained ceiling — operationalizes 'how many dimensions does this concept need', distinct from simply asking whether it is decodable at all.

Fits a concept's encoding as a learned linear combination of multiple CNN filter-activation maps (a weight vector over filters), rather than searching for the single best-aligned filter, then validates the fit via segmentation-mask overlap (IoU) against the concept's ground-truth spatial extent.

Scores and selects specialized attention heads by sparsely reconstructing a batch of real head activations as a small combination of unembedding-matrix (vocabulary) directions via Simultaneous Orthogonal Matching Pursuit (SOMP) -- a multi-sample generalization of the Logit Lens that greedily selects the token direction most correlated with the current residual across all samples at once, rather than one sample at a time.

Fits a multi-component partial least squares regression from activations to a target numeric variable, identifying a low-dimensional linear subspace (not just a single direction) that predicts the variable, then uses the leading component as an intervention direction.

Jointly fits a structural (distance-based, existence) probe and a contrastive angular probe (direction-based, type/orientation) as one linear transform, so that squared distance encodes whether a relation exists and the angle between edge vectors encodes which type of relation it is.

A kernel-machine probe that iteratively reweights input features via the average gradient outer product (the feature matrix), learning a data-adaptive Mahalanobis metric; used as a probe more expressive than a plain linear classifier while still yielding an interpretable feature-importance direction.

Finds an edit direction in a frozen generative model's bottleneck by gradient-optimizing a single vector so the model's own denoising/reconstruction loss recovers concept-present images from a concept-stripped text prompt, requiring no labels, no classifier, and no auxiliary vision-language model.

Identifies a small, sparse set of ambient-basis (axis-aligned) dimensions responsible for a cross-condition transition — e.g. language-agnostic to language-specific representation — by comparing corpus-mean activations across two conditions and keeping the top-K dimensions with the largest absolute difference, then validating by directly overwriting only those dimensions at inference time.

Fits a bilinear role-filler probe (score = r^T B f, with separate learned role and filler embedding matrices bound by a shared matrix B) to a real trained model's hidden states, testing whether many independent linear probe directions actually share a common low-rank factorized basis -- a compositional/binding structure claim distinct from ordinary linear probing, which fits one direction per concept with no shared-structure constraint.