Iso-Energy regularized sparse autoencoder (SAE-A)

methods / Dictionary Learning / Sparse Autoencoders (SAE) / Iso-Energy regularized sparse autoencoder (SAE-A)

Iso-Energy regularized sparse autoencoder (SAE-A)

Techniqueadvanced

A sparse autoencoder trained with an added alignment loss term that forces each dictionary atom's second moment (energy) to be domain-invariant across paired image/text embeddings, so atoms partition cleanly into modality-specific and genuinely shared (bimodal) subspaces rather than an SAE's usual ungrounded mixture of both.

Used in (1 observation)

Iso-energy SAE isolates a bimodal cross-modal subspacemeasured

structure: Linear Subspace, Anisotropy · models: CLIP ViT-B/32, CLIP ViT-L/14, OpenCLIP ViT-B/32, OpenCLIP ViT-L/14, SigLIP, SigLIP 2 · paper: Cross-Modal Redundancy and the Geometry of Vision-Language Embeddings