Concept-direction local ID tracks generality and restores distilled-diffusion diversity

measured in 1 paper

Skiers et al. backpropagate differences between stochastic image realizations of the same prompt in real SDXL, then decompose the gradient directions via PCA or a sparse autoencoder into concept-specific latent directions [skiers-etal-2026-elrond-diffusion-concept-decomposition] The local intrinsic dimension of each concept's manifold tracks concept generality: general concepts (e.g. "Dog") show higher LID than their hyponyms (e.g. "Poodle"), validated against WordNet pairs [skiers-etal-2026-elrond-diffusion-concept-decomposition] Adding the discovered directions into the distilled, mode-collapsed SDXL-DMD student causally steers single concepts and, combined, restores output diversity toward the teacher (FID improves), most when directions come from the teacher [skiers-etal-2026-elrond-diffusion-concept-decomposition] Equal-norm random directions are semantically inert by comparison [skiers-etal-2026-elrond-diffusion-concept-decomposition]

Structure

Intrinsic-dimension profile across depth

Context

gradient-derived concept directions in diffusion text embeddings, local intrinsic dimension as concept-generality proxy, mode-collapse mitigation via direction injection, FID-measured causal restoration of output diversity

Confirmed in models

Stable Diffusion XL · Stable Diffusion, SDXL-DMD (4-step distilled) · Stable Diffusion

Method

Local Intrinsic Dimensionality (LID) under perturbation, PCA, Sparse Autoencoders (SAE), Causal interventions (steering)

Papers

ELROND: Exploring and Decomposing Intrinsic Capabilities of Diffusion Models — Skierś, Paweł, Trzciński, Tomasz, Deja, Kamil2026 · arXiv:2602.10216