Orthogonal Procrustes alignment

methods / Representation Alignment / Orthogonal Procrustes alignment

Techniqueintermediate

Fits the orthogonal matrix $W^\star = \arg\min_{W\in O(d)} \|WX-Y\|_F = UV^\top$ (via the SVD of $YX^\top$) that best maps one representation onto another, then evaluates the fit functionally — e.g. via nearest-neighbor retrieval or downstream transfer — rather than only reporting a single aggregate similarity number.

Used in (11 observations)

Twelve trait directions form stable antisocial and prosocial clustersmeasured

structure: Linear Direction · models: Llama-3.1-8B, Qwen2.5-14B, Qwen-2.5-7B, Qwen2.5-0.5B, Llama-3.2-1B · paper: Intrinsic Guardrails: How Semantic Geometry of Personality Interacts with Emergent Misalignment in LLMs

Independent monolingual LMs learn alignable isomorphic geometriesmeasured

structure: Platonic Representation Hypothesis, Linear Subspace · models: Conneau et al. Monolingual MLM (English), Conneau et al. Monolingual MLM (French), Conneau et al. Monolingual MLM (German), Conneau et al. Monolingual MLM (Russian), Conneau et al. Monolingual MLM (Chinese) · paper: Emerging Cross-lingual Structure in Pretrained Language Models

mini-vec2vec aligns embedding models with a cheap orthogonal mapmeasured

structure: Platonic Representation Hypothesis, Linear Subspace · models: GTR-base, E5-base-v2, Stella-base-en-v2, Granite-Embedding-278M-Multilingual · paper: mini-vec2vec: Scaling Universal Geometry Alignment with Linear Transformations

Independently pretrained vision (DINOv2) and text (MiniLM) encoders develop graph-Laplacian eigenvalue spectra of near-identical intrinsic complexity, yet the functional map between their spectral eigenbases is far from diagonal or orthogonal -- a decoupling of 'how much structure' from 'how it is oriented' that bounds spectral cross-modal alignment methods and underperforms ambient-space alignment on retrievalmeasured

structure: Platonic Representation Hypothesis · models: DINOv2 ViT-B/14, all-MiniLM-L6-v2, all-mpnet-base-v2, CLIP ViT-B/32 · paper: On the Spectral Geometry of Cross-Modal Representations: A Functional Map Diagnostic for Multimodal Alignment

Two independently-trained JEPA world models converge to a linear isomorphismmeasured

structure: Platonic Representation Hypothesis, Linear Subspace · models: ViT-S/16 (I-JEPA objective, independently trained per view: smallNORB/nuScenes/ImageNet-1k) · paper: Social-JEPA: Emergent Geometric Isomorphism in Independently Trained World Models

Independent multimodal models share one orthogonal map across both modalitiesmeasured

structure: Linear Subspace · models: CLIP ViT-B/32, CLIP ViT-L/14, SigLIP, FLAVA · paper: Canonicalizing Multimodal Contrastive Representation Learning

A linear map from 10K pairs aligns English BERT with multilingual BERTmeasured

structure: Platonic Representation Hypothesis, Linear Subspace · models: BERT-base-cased, mBERT (BERT-base, Multilingual Cased) · paper: Cross-Lingual BERT Transformation for Zero-Shot Dependency Parsing

Composed-subword vs whole-word spaces show family-dependent isometrymeasured

structure: Linear Subspace · models: Llama-3-8B-Instruct, Llama-3.1-8B-Instruct, Aya Expanse 8B, Gemma 2 9B IT, Qwen2.5-7B-Instruct · paper: Understanding Subword Compositionality of Large Language Models

Orthogonal Procrustes alignment maps a Gemma Scope SAE 'Concept Atlas' onto three independently-trained Llama-3.1-8B models' activation spaces, with quantified retrieval/translation quality (AUROC 0.82-0.86, AP 0.39-0.49 vs. a 0.046 random baseline), and the transferred concept directions can be added back into the subject model's residual stream to steer generationmeasured

structure: Platonic Representation Hypothesis · models: Gemma-2-2B, Llama-3.1-8B, Llama-3.1-8B-Instruct, DeepSeek-R1-Distill-Llama-8B · paper: Atlas-Alignment: Making Interpretability Transferable Across Language Models

A Backpack model sense-vector topology partially transfers across languagesmeasured

structure: Linear Subspace · models: GPT-2-small, GPT-2-Medium, GPT-2-Large · paper: Multilinguality as Sense Adaptation

Othello-move representations from seven architecturally distinct real models (GPT-2, BART, T5, Flan-T5-XL, LLaMA-2-7B, Mistral-7B, Qwen2.5-7B) Procrustes-align into a shared geometric space at cosine similarity up to 97.2% (unsupervised), with PCA trajectories converging across models and nearest-neighbor tile embeddings recovering the board's actual physical adjacency structuremeasured

structure: Platonic Representation Hypothesis · models: GPT-2 Small, BART-base, Llama-2-7B, Mistral-7B · paper: Revisiting the Othello World Model Hypothesis