MATH · IN · MODELS
methods / Representation Alignment / Learned linear/affine map between two representation spaces

Learned linear/affine map between two representation spaces

Techniqueintermediate

Fits a linear or affine map (via least squares / SGD, with no orthogonality constraint) from one model's activation space onto another's, scored by R^2 or by downstream task performance — a general-linear alternative to orthogonal Procrustes for cross-model alignment and cross-modal transfer.

Used in (2 observations)

One unconstrained linear map injects vision embeddings into frozen GPT-Jmeasured
structure: Linear Subspace · models: CLIP ResNet-50x16, NFRN50 (Normalizer-Free ResNet-50), BEiT-Large (ImageNet-22k pretrained), GPT-J-6B · paper: Linearly Mapping from Image to Text Space
An affine map between two vision models matches their PCA componentsmeasured
structure: Linear Subspace · models: CLIP ResNet-50, CLIP ViT-B/32, DINO ViT-S/16, ResNet-50 (SimCLR contrastive pretraining, ImageNet), ResNet-50 (MoCo v2, unsupervised contrastive pretraining, ImageNet) · paper: Text-to-Concept (and Back) via Cross-Model Alignment