Cross-Modal PLS-SVD Decomposition

methods / Dimensionality Reduction / Cross-Modal PLS-SVD Decomposition

Techniqueadvanced

Decomposes the cross-covariance between two modalities' embedding spaces (e.g. text and audio) via SVD into paired, covariance-ranked orthogonal axes, splitting each embedding into a shared/aligned 'head' and a modality-private 'tail'.

Used in (1 observation)

Audio-text embeddings split into shared head and private tailmeasured

structure: Linear Subspace, Anisotropy · models: CLAP (HTSAT-BERT-ZS, trained on WavCaps), DRCap's CLAP (trained on WavCaps + SoundVECaps) · paper: COMET: Concept Space Dissection of the Modality Gap in Audio-Text Multimodal Contrastive Embeddings