A mid-layer head group decodes another agent's beliefs and steers theory-of-mind

Zhu et al. prompt Mistral-7B-Instruct (and DeepSeek-LLM-7B-Chat) with third-person belief narratives and fit logistic-regression probes on attention-head activations [zhu-etal-2024-language-models-represent-beliefs-of-self-and-others] A group of middle-layer attention heads decodes another agent's belief status at over 80% accuracy, distinct from the model's own self-belief representation [zhu-etal-2024-language-models-represent-beliefs-of-self-and-others] Steering these representations produces large changes in downstream Theory-of-Mind performance, generalizing across social-reasoning tasks [zhu-etal-2024-language-models-represent-beliefs-of-self-and-others]