Sparse nonnegative concept decomposition enables causal CLIP editing

- CLIP image embeddings decompose as sparse, nonnegative combinations (l0 ~5-20, recommended ~10-30) of a fixed dictionary of ~15k text-encoder concept vectors from LAION captions. [bhalla-etal-2024-splice] - The decomposition nearly preserves zero-shot accuracy (ImageNet 0.552 to 0.552, CIFAR-100 0.750 to 0.768) with cosine reconstruction ~0.75-0.88. [bhalla-etal-2024-splice] - Because embeddings are additive concept mixtures, zeroing a concept edits behavior surgically (removing "glasses" drops a glasses classifier 0.88 to 0.69 while gender stays 0.97 to 0.96). [bhalla-etal-2024-splice] - Tested on OpenCLIP ViT-B/32 (main) and OpenAI CLIP RN50 (appendix). [bhalla-etal-2024-splice]