MATH · IN · MODELS

SAE features from SDXL-Turbo's one-step U-Net transfer zero-shot and edit images

measured in 1 paper

Surkov et al. train SAEs on transformer-block updates inside SDXL-Turbo's denoising U-Net [surkov-etal-2024-one-step-is-enough-sparse-autoencoders-for-text-to-image-diffusion-models] The resulting features generalize zero-shot, with no retraining, to 4-step SDXL-Turbo and the separately-trained multi-step SDXL-base model [surkov-etal-2024-one-step-is-enough-sparse-autoencoders-for-text-to-image-diffusion-models] Switching individual SAE features on or off during generation causally edits specific attributes of the generated image on the RIEBench benchmark [surkov-etal-2024-one-step-is-enough-sparse-autoencoders-for-text-to-image-diffusion-models] Different transformer blocks show measurable specialization by edit category [surkov-etal-2024-one-step-is-enough-sparse-autoencoders-for-text-to-image-diffusion-models]

Context

diffusion-models, sparse-autoencoders

Papers

One-Step Is Enough: Sparse Autoencoders for Text-to-Image Diffusion Models — Surkov, Viacheslav, Wendler, Chris, Mari, Antonio, Terekhov, Mikhail, Deschenaux, Justin, West, Robert, Gulcehre, Caglar, Bau, David2024 · arXiv:2410.22366