ICL answer tokens are affine functions in a PCA subspace

Lee & Vijayan collect residual-stream activations from six pretrained LLMs across six in-context relational tasks and reduce each layer to 30 principal components [lee-vijayan-2026-functional-subspace-vector-algebra] For each layer and component an answer token's projection is fit as an affine function of query and separator projections (a = alpha*q + beta*s + gamma) [lee-vijayan-2026-functional-subspace-vector-algebra] R^2 is high along a small, largely layer-consistent subset of components, rising in later layers, so ICL answer prediction reduces to an affine operation there [lee-vijayan-2026-functional-subspace-vector-algebra] Query, separator, and answer tokens form well-separated clusters in the top-3 high-R^2 component subspace across models and tasks [lee-vijayan-2026-functional-subspace-vector-algebra] Projections onto the highest-R^2 component differ significantly between correct and incorrect predictions (p<0.05) in mid/late layers, with no causal intervention performed [lee-vijayan-2026-functional-subspace-vector-algebra]