Fixed-dictionary sparse recovery

methods / Direction Extraction / Fixed-dictionary sparse recovery

Techniqueintermediate

Decomposes an embedding into a sparse, non-negative linear combination of a fixed, human-interpretable concept dictionary (e.g. common words/phrases passed through the model's own text encoder) via per-sample convex optimization (LASSO/ADMM) - no encoder/decoder network is trained, unlike a sparse autoencoder.

Used in (2 observations)

Dictionary-learned features in Flux.1 causally steer image generationmeasured

structure: Linear Direction · models: FLUX.1 · paper: Interpreting Large Text-to-Image Diffusion Models with Dictionary Learning

Sparse nonnegative concept decomposition enables causal CLIP editingmeasured

structure: Linear Direction, Anisotropy · models: OpenCLIP ViT-B/32, CLIP ResNet-50 · paper: Interpreting CLIP with Sparse Linear Concept Embeddings (SpLiCE)