Data2Vec

Meta AI

Structures found in this family (3)

Representational-similarity trajectory across depth, Dimensional collapse, Circle

By model (1)

Data2Vec Audio Base (LibriSpeech-960h)

Representational-similarity trajectory across depth, Dimensional collapse, Circle

Observations (2)

Real speech-SSL models Wav2Vec2, WavLM, HuBERT, and Data2Vec show distinct layer-wise entropy and trajectory-curvature regimes, with Wav2Vec2 undergoing a late-layer entropy collapse

Music-task fine-tuning, not exposure, induces chroma equivalence

Papers

InsideSSL: Understanding Self-Supervised Speech Representations using a Model-Centric Perspective (2026), Musical Training, but not Mere Exposure to Music, Drives the Emergence of Chroma Equivalence in Artificial Neural Networks (2026)