Functional-map spectral alignment

methods / Representation Alignment / Functional-map spectral alignment

Techniqueadvanced

Builds a k-nearest-neighbor graph over shared samples in each of two independently-trained representation spaces, computes the graph Laplacian eigenbasis of each (a spectral analogue of PCA restricted to a similarity graph rather than raw coordinates), then fits a linear operator C between the two truncated spectral bases and decomposes cross-modal compatibility into two independent diagnostics: whether the eigenvalue spectra match (shared manifold complexity) and whether C is diagonal/orthogonal (shared eigenvector orientation).

Used in (1 observation)

Independently pretrained vision (DINOv2) and text (MiniLM) encoders develop graph-Laplacian eigenvalue spectra of near-identical intrinsic complexity, yet the functional map between their spectral eigenbases is far from diagonal or orthogonal -- a decoupling of 'how much structure' from 'how it is oriented' that bounds spectral cross-modal alignment methods and underperforms ambient-space alignment on retrievalmeasured

structure: Platonic Representation Hypothesis · models: DINOv2 ViT-B/14, all-MiniLM-L6-v2, all-mpnet-base-v2, CLIP ViT-B/32 · paper: On the Spectral Geometry of Cross-Modal Representations: A Functional Map Diagnostic for Multimodal Alignment