MATH · IN · MODELS
methods / Theoretical / Analytical / Local Intrinsic Dimensionality (LID) under perturbation

Local Intrinsic Dimensionality (LID) under perturbation

Techniqueintermediate

Estimates the local intrinsic dimensionality around each activation vector from the rate of growth of its neighbor-distance distribution, tracked per-layer and compared between clean, benign-perturbed, and adversarially-perturbed inputs — a diagnostic use of local dimensionality as an anomaly signature rather than a static quality measure.

Used in (5 observations)

Concept-direction local ID tracks generality and restores distilled-diffusion diversitymeasured
structure: Intrinsic-dimension profile across depth · models: Stable Diffusion XL, SDXL-DMD (4-step distilled) · paper: ELROND: Exploring and Decomposing Intrinsic Capabilities of Diffusion Models
LID separates adversarial from benign speech perturbations, transcript-freemeasured
structure: Intrinsic-dimension profile across depth · models: WavLM Base, wav2vec2-base-960h · paper: Dimensionality-Aware Anomaly Detection in Learned Representations of Self-Supervised Speech Models
Lower local intrinsic dimension predicts vision-model alignment and generalizationmeasured
structure: Intrinsic-dimension profile across depth · models: ConvNeXt (image classifier, various sizes, ImageNet), ResNet (image classifier, various depths), Vision Transformer (ViT, image classifier, various sizes, ImageNet), ResMLP (image classifier, various sizes, ImageNet) · paper: Local Intrinsic Dimension of Representations Predicts Alignment and Generalization in AI Models and the Human Brain
Local dimension of real fine-tuned RoBERTa embeddings drops with successful dialogue-state tracking and rises with overfitting onset on real emotion-recognition fine-tuningmeasured
structure: Intrinsic-dimension profile across depth · models: RoBERTa-base (TripPy-R dialogue-state tracker, fine-tuned on MultiWOZ 2.1), RoBERTa-base (fine-tuned on EmoWOZ, 7-class emotion recognition) · paper: Less is More: Local Intrinsic Dimensions of Contextual Language Models
The local intrinsic dimension of real Llama-2 activations traces a hunchback shape across layers that predicts generation truthfulness on real QA datasetsmeasured
structure: Intrinsic-dimension profile across depth · models: Llama-2-7B, Llama-2-13B · paper: Characterizing Truthfulness in Large Language Model Generations with Local Intrinsic Dimension