Definition

Given an input image, compute its ground-truth radial spatial-frequency energy in $k$ log-spaced Fourier bands via a windowed 2D FFT. At each layer $\ell$ of a frozen, pretrained vision encoder, fit a ridge-regression probe from that layer’s representation to each frequency band’s energy, reporting the probe’s linear recoverability (e.g. Pearson $r$) as spectral accessibility $\alpha_\ell$. Because raw dimensionality reduction alone can trivially destroy or preserve accessibility independent of any learned transformation, a Residual Spectral Loss (RSL) is defined as the accessibility gap between the real layer and a dimension-matched random-projection baseline of the same input, isolating the learned transformation’s own effect on frequency content from a pure compression artifact.

Relative to manifold-curvature-profile and intrinsic-dimension-profile

Intrinsic-dimension profile across depth tracks how many effective coordinates a representation needs at each layer, and Curvature profile of the representation manifold tracks how bent the representation manifold is; both are properties of the representation’s own geometry, agnostic to what the input actually was. This node instead asks a content-referenced question — how much of a specific, externally defined input property (spatial-frequency energy, computed directly from the pixels) remains linearly recoverable at each layer — making it a profile of input-content preservation, not of the representation’s intrinsic shape.

Key evidence

Kitessa & Zhao (2026) apply Residual Spectral Loss to three real frozen pretrained vision encoders (CLIP ViT-L/14, CLIP ViT-B/32, DINOv2 ViT-B) on real ImageNet and MS-COCO image subsets, finding a consistently non-monotonic accessibility profile: accessibility rises roughly 2-3x from the convolutional stem ($\alpha\approx0.26$-$0.42$) to a mid-layer peak ($\alpha\approx0.71$-$0.76$), then falls 17-27% toward the output. CLIP’s final projection layer is spectrally neutral (RSL $\approx\pm0.035$, matching the random-projection baseline), while DINOv2’s [CLS]-token pooling induces a large, statistically significant spectral loss (RSL $+0.10$ to $+0.22$ across bands, confidence interval excluding zero) concentrated at low-to-mid frequencies — a genuine architectural difference in how each model’s final aggregation step treats spatial- frequency content, isolated from mere compression by the random-baseline control. See kitessa-zhao-2026-a-residual-spectral-loss-probe-finds-a-non-monotonic-spatial-frequency-accessibility-profile-across-depth-in-real-clip-and-dinov2-vision-transformers-with-dinov2s-cls-pooling-inducing-a-spectrally-significant-loss-that-clips-final-projection-does-not.