SAE decoder cosine reveals feature splitting in dense embeddings and steers search

measured in 1 paper

O'Neill et al. train TopK SAEs directly on OpenAI text-embedding-3-small embeddings of 420,000+ scientific abstracts, one of the first SAE applications to dense embeddings rather than LLM activations [oneill-etal-2024-disentangling-dense-embeddings-sparse-autoencoders] A decoder-direction cosine-similarity matrix between different-sized SAEs shows larger SAEs split a smaller SAE's feature direction, with over 1,100 triplets recurring at cosine >0.95 [oneill-etal-2024-disentangling-dense-embeddings-sparse-autoencoders] Steering semantic search is proven algebraically equivalent to adding a scaled decoder feature vector to the output embedding [oneill-etal-2024-disentangling-dense-embeddings-sparse-autoencoders] Down-weighting one feature and up-weighting an orthogonal one achieves ~20% higher accuracy than query rewriting at a target cosine similarity of 0.75 [oneill-etal-2024-disentangling-dense-embeddings-sparse-autoencoders]

Structure

Linear Direction

Context

decoder-direction cosine-similarity matrix S_ij = w_i^T w_j / ||w_i|| ||w_j|| quantifying feature splitting across SAE dictionary sizes, applied to dense embeddings rather than LLM activations, feature families as a separate, co-occurrence/activation-based (not decoder-cosine-based) hierarchical grouping, empirically found to be linear subspaces rather than curved manifolds, proof that additively steering an SAE latent before decoding is algebraically equivalent to adding a scaled decoder direction to the embedding, causal steering of semantic search retrieval accuracy via decoder-direction addition/subtraction, beating query rewriting by ~20% at matched cosine similarity

Confirmed in models

text-embedding-3-small · OpenAI Embeddings

Method

TopK (k-sparse) Sparse Autoencoders, Causal interventions (steering)

Papers

Disentangling Dense Embeddings with Sparse Autoencoders — O'Neill, Charles, Ye, Christine, Iyer, Kartheik, Wu, John F.2024 · arXiv:2408.00657