Compositional generalization needs linear, additive, orthogonal per-concept factor vectors

Uselis, Dittadi & Oh recover per-concept difference vectors from paired stimuli across three datasets in many pretrained encoders (CLIP, OpenCLIP, MetaCLIP, MetaCLIP2, SigLIP, SigLIP2, DINOv1/v2/v3) [uselis-dittadi-oh-2026-compositional-generalization-requires-linear-orthogonal-representations-in-vision-embedding-models] A whitened, probe-span-projected reconstruction of embeddings as a linear sum of recovered factors reaches R^2=0.42-0.63 versus 0.12-0.42 for a randomly-initialized-encoder baseline [uselis-dittadi-oh-2026-compositional-generalization-requires-linear-orthogonal-representations-in-vision-embedding-models] Cross-concept factor vectors are confirmed mutually orthogonal via cosine similarity and effective-rank analysis [uselis-dittadi-oh-2026-compositional-generalization-requires-linear-orthogonal-representations-in-vision-embedding-models] The degree of linear factorization correlates with each model's own compositional-generalization accuracy across variants [uselis-dittadi-oh-2026-compositional-generalization-requires-linear-orthogonal-representations-in-vision-embedding-models]