Emotion clusters peak at mid-depth and tighten with scale

measured in 1 paper

Zhang & Zhong train two-layer MLP probes on frozen hidden states of Qwen3 (0.6B-8B) and LLaMA-3 (1B-8B), finding 7-class emotion clusters tight and well-separated for larger models but diffuse for smaller [zhang-zhong-2025-decoding-emotion-in-the-deep] Layer-wise probe accuracy rises from chance (0.143) to a peak of 0.78-0.80 at 50-75% relative depth rather than the final layer [zhang-zhong-2025-decoding-emotion-in-the-deep] An offset-aware probe shows emotion-specific signal decaying differentially across up to 400 generated tokens [zhang-zhong-2025-decoding-emotion-in-the-deep]

Structure

Linear Separability

Context

emotion representation, MLP probing, cluster separability, depth-of-peak decodability, temporal persistence

Confirmed in models

Qwen3-0.6B · Qwen, Qwen3-1.7B · Qwen, Qwen3-4B · Qwen, Qwen3-8B · Qwen, Llama-3.2-1B · Llama, Llama-3.2-3B · Llama, Llama-3.1-8B · Llama

Method

Mlp probing, PCA

Papers

Decoding Emotion in the Deep: A Systematic Study of How LLMs Represent, Retain, and Express Emotion — Zhang, Jingxiang, Zhong, Lujia2025 · arXiv:2510.04064