Sparse autoencoders trained on real Gemma-2 (2B/9B) and Llama-3.1-8B base models uncover linear entity-recognition directions in the representation space that distinguish entities the model can vs. cannot recall facts about, and are causally relevant to refusal/hallucination behavior

- Ferrando, Obeso, Rajamanoharan & Nanda apply sparse autoencoders (Gemma Scope for Gemma-2-2B and Gemma-2-9B; LlamaScope for Llama-3.1-8B in the App. Q replication, all SAEs trained on base models) and find SAE latents at the final entity token that separate entities the model recognizes (can recall facts about) from entities it does not, generalizing across distinct entity types (players, movies, cities, songs); the paper frames these as linear directions in the representation space. [ferrando-etal-2024-do-i-know-this-entity] - These known-entity / unknown-entity directions are causally relevant: steering with the unknown-entity latent induces knowledge refusal even on known entities (near 100% on Gemma-2-2B), and steering with the known-entity latent can induce hallucination instead of refusal on unknown entities; effects are less pronounced on Gemma-2-9B and Llama-3.1-8B. [ferrando-etal-2024-do-i-know-this-entity] - Despite being derived from SAEs trained on the base model, these directions causally affect the chat (instruction-tuned) model's refusal behavior (main steering experiment on Gemma-2-2B chat; Gemma-2-9B-it in the appendices), suggesting that chat finetuning repurposes this pre-existing entity-recognition mechanism. [ferrando-etal-2024-do-i-know-this-entity]