MATH · IN · MODELS

WavLM

Microsoft

Structures found in this family (6)

Linear Direction, Intrinsic-dimension profile across depth, Linear Subspace, Representational-similarity trajectory across depth, Dimensional collapse, Circle

By model (3)

WavLM Large
Linear Direction, Intrinsic-dimension profile across depth, Linear Subspace, Circle
WavLM Base · 94M
Intrinsic-dimension profile across depth, Representational-similarity trajectory across depth, Dimensional collapse
WavLM-base-plus · 95M
Intrinsic-dimension profile across depth, Linear Subspace

Observations (8)

Phonological features are linear vector-offset directions in speech models
Layer intrinsic-dimension peak aligns with the best brain-predicting layer
LID separates adversarial from benign speech perturbations, transcript-free
Subtracting a regression-fit speaker component removes speaker identity
Real speech-SSL models Wav2Vec2, WavLM, HuBERT, and Data2Vec show distinct layer-wise entropy and trajectory-curvature regimes, with Wav2Vec2 undergoing a late-layer entropy collapse
Audio-model SAEs raise DCI completeness while preserving informativeness
Speech models encode neighbor-phone info in orthogonal positional subspaces
Anchored projection recovers a 2D arousal-valence plane from categorical labels

Papers

[b]=[d]-[t]+[p]: Self-Supervised Speech Models Discover Phonological Vector Arithmetic (2026), Abstraction Induces the Brain Alignment of Language and Speech Models (2026), Dimensionality-Aware Anomaly Detection in Learned Representations of Self-Supervised Speech Models (2026), Eta-WavLM: Efficient Speaker Identity Removal in Self-Supervised Speech Representations Using a Simple Linear Equation (2025), InsideSSL: Understanding Self-Supervised Speech Representations using a Model-Centric Perspective (2026), Sparse Autoencoders Make Audio Foundation Models More Explainable (2025), Self-Supervised Speech Models Encode Phonetic Context via Position-dependent Orthogonal Subspaces (2026), Learning Arousal-Valence Representation from Categorical Emotion Labels of Speech (2023)