Real speech-SSL models Wav2Vec2, WavLM, HuBERT, and Data2Vec show distinct layer-wise entropy and trajectory-curvature regimes, with Wav2Vec2 undergoing a late-layer entropy collapse

Von Neumann entropy (from the per-layer Gram matrix) and average curvature of the trajectory traced by adjacent-token transition vectors are computed layer-by-layer for real pretrained Wav2Vec2-base, WavLM-base, HuBERT-base, and Data2Vec-audio-base on the real LibriSpeech test-clean corpus [sadok-alameda-pineda-2026-insidessl] WavLM, HuBERT, and Data2Vec sustain high normalized entropy throughout the network (about 0.82 declining only to about 0.75 in final layers), while Wav2Vec2-base instead undergoes an entropy collapse toward near-baseline levels at layer 11, coinciding with a sharp rise in InfoNCE-based invariance loss (average 1.0 to 3.0) across its final two layers [sadok-alameda-pineda-2026-insidessl] Trajectory curvature starts high (about 1.4) in early layers and decreases to stabilize around 1.2 in deeper layers for most models, indicating the embedding manifold "unfolds" toward linear separability with depth; WavLM, HuBERT and UniSpeech show highly correlated curvature trajectories (Pearson correlation above 0.96), while linear probing shows phoneme-classification accuracy peaks at the depth where curvature transitions from the high- to low-curvature regime [sadok-alameda-pineda-2026-insidessl]