Subspace-angle alignment

methods / Theoretical / Analytical / Geometric analysis / Subspace-angle alignment

Techniqueadvanced

Compares two independently-fitted linear subspaces (e.g. the same probe trained on different languages) via their principal/subspace angles, testing whether they occupy approximately the same subspace rather than merely achieving similar task performance separately.

Used in (5 observations)

mBERT's syntactic probe subspace is approximately shared across languagesmeasured

structure: Tree Metric Embedding · models: mBERT (BERT-base, Multilingual Cased) · paper: Finding Universal Grammatical Relations in Multilingual BERT

After mean-centering, 88 languages share one XLM-R subspacemeasured

structure: Linear Subspace · models: XLM-RoBERTa base · paper: The Geometry of Multilingual Language Model Representations

MoE experts are functionally decorrelated but only partly subspace-separatedmeasured

structure: Linear Subspace · models: Mixtral-8x7B, Qwen1.5-MoE-A2.7B · paper: Geometric Asymmetry in MoE Specialization: Functional Decorrelation and Representational Overlap

MAE per-class patch-token subspaces rotate apart with depthmeasured

structure: Linear Subspace · models: MAE ViT-Base (Masked Autoencoder) · paper: Robust Representation Learning in Masked Autoencoders

StyleGAN2's W space is globally warped; local bases avoid traversal collapsemeasured

structure: Affine Subspace, Linear Subspace · models: StyleGAN2 (trained on FFHQ, 1024x1024) · paper: Do Not Escape From the Manifold: Discovering the Local Coordinates on the Latent Space of GANs