MATH · IN · MODELS

A 'state of mind' spectrum is linearly decodable across five encoders

measured in 1 paper

Zhao annotates 636 sentences with a continuous energy score and one of seven ordered tiers, an informally-sourced psychological taxonomy explicitly disclaimed as non-clinical [zhao-2026] Across five frozen representations (BGE-large, all-mpnet-base-v2, all-MiniLM-L6-v2, Qwen3-Embedding-0.6B, and Qwen2.5-3B-Instruct's layer 24), both the score (Ridge R^2 0.63-0.79) and the seven tiers (weighted F1 0.57-0.72) are linearly decodable above chance [zhao-2026] Permutation tests and a TF-IDF baseline (R^2 0.44) rule out simple lexical confounds [zhao-2026] Projecting out a single Ridge-derived direction (1.6-4.8% of variance) collapses decodability far more than a random direction, though the structure is only approximately one-dimensional [zhao-2026]

Context

affective/psychological spectrum, ordinal decodability, permutation testing, TF-IDF baseline, directional ablation, representation type comparison

Papers

Probing Spectrum-Like Organization of States of Mind in Transformer Representation Spaces — Zhao, Sophie2026 · arXiv:2512.22227