MATH · IN · MODELS

Self-supervised h-space directions enable responsible diffusion generation

measured in 1 paper

Li et al. target the same additive h-space bottleneck as Asyrp but replace its CLIP-guided optimization with a self-supervised reconstruction technique [li-etal-2024-self-discovering-diffusion-directions] A single vector c is optimized so the frozen U-Net reconstructs a concept-present image when conditioned instead on a concept-stripped prompt plus c added to h-space [li-etal-2024-self-discovering-diffusion-directions] Because the model is frozen and the prompt omits the concept, gradient descent forces c to absorb exactly the missing semantic content, needing no labels or CLIP [li-etal-2024-self-discovering-diffusion-directions] A negative-prompt variant learns an anti-concept direction for safety mitigation; linearity and additive composability are demonstrated qualitatively and imposed by design [li-etal-2024-self-discovering-diffusion-directions]

Context

h-space, self-supervised direction optimization, reconstruction loss, fairness mitigation, safe generation, anti-concept direction, additive vector composition

Papers

Self-Discovering Interpretable Diffusion Latent Directions for Responsible Text-to-Image Generation — Li, Hang, Shen, Chengzhi, Torr, Philip, Tresp, Volker, Gu, Jindong2024 · arXiv:2311.17216