SigLIP

Google DeepMind

Structures found in this family (4)

Dimensional collapse, Linear Direction, Anisotropy, Linear Subspace

By model (2)

SigLIP

Dimensional collapse, Linear Subspace, Anisotropy, Linear Direction

SigLIP 2

Dimensional collapse, Linear Direction, Anisotropy, Linear Subspace

Observations (6)

CLIP/SigLIP covariance splits into signal and shared-noise subspaces

Same concept, different SAE directions across modalities

Independent multimodal models share one orthogonal map across both modalities

Iso-energy SAE isolates a bimodal cross-modal subspace

Compositional generalization needs linear, additive, orthogonal per-concept factor vectors

VLM SAE concepts are single-modality yet gap-orthogonal

Papers

Your CLIP has 164 dimensions of noise: Exploring the embeddings covariance eigenspectrum of contrastively pretrained vision-language transformers (2026), Same Concept, Different Directions: Cross-Modal Feature Heterogeneity in Sparse Autoencoders (2026), Canonicalizing Multimodal Contrastive Representation Learning (2026), Cross-Modal Redundancy and the Geometry of Vision-Language Embeddings (2026), Compositional Generalization Requires Linear, Orthogonal Representations in Vision Embedding Models (2026), Interpreting the Linear Structure of Vision-Language Model Embedding Spaces (2025)