Activation-derived forget/retain subspaces edited into diffusion weights

measured in 1 paper

Moon et al. build forget and retain subspaces via per-layer SVD directly on Stable Diffusion v1.5's cross-attention activations rather than text-embedding space [moon-etal-2026-concept-unlearning-via-cross-attention-activation-projection-for-diffusion-models] An activation-basis binary probe achieves roughly 5x higher recall than a text-basis probe [moon-etal-2026-concept-unlearning-via-cross-attention-activation-projection-for-diffusion-models] A single closed-form linear projector removing the forget subspace while preserving the retain subspace is edited once into the cross-attention K/V weights across 10 concepts [moon-etal-2026-concept-unlearning-via-cross-attention-activation-projection-for-diffusion-models] On the Holistic Unlearning Benchmark it achieves the best per-category trade-off (e.g. IP H-Mean 0.683 vs 0.571 for CURE); the forget/retain subspaces are constructed for unlearning rather than discovered [moon-etal-2026-concept-unlearning-via-cross-attention-activation-projection-for-diffusion-models]

Structure

Linear Subspace

Context

deriving forget/retain subspaces from real model activations rather than from a text-embedding proxy, quantified as directly improving probe recall, a single closed-form linear projector permanently edited into diffusion cross-attention weights, analogous to LLM weight-subspace projection methods but applied to a different architecture/domain

Confirmed in models

Stable Diffusion v1.5 · Stable Diffusion

Method

ProFS (Projection Filter for Subspaces) weight-subspace projection

Papers

Concept Unlearning via Cross-Attention Activation Projection for Diffusion Models — Moon, Seungjun, Jun, Sangwoo, Lee, Jaeho, Kim, Sungroh2026 · arXiv:2605.25765