A generic switching direction (PCA + logistic-regression coefficients projected back to the residual stream) predicts and bidirectionally steers cluster switching in the semantic fluency task across five open instruction-tuned LLMs

- Leckie, Todd & Foster fit, at each layer, a logistic regression on the top-30 PCA loadings of anticipatory-position residual activations (switch vs. cluster) and project its coefficients back through the loading vectors into one unit vector d_L in the original residual space; activations at the anticipatory and event positions predict switching above chance (Gemma-2-9B maximum AUC 0.72 anticipatory at layer 26, 0.87 event at layer 16, with categories disjoint between train and test). [leckie-etal-2026-semantic-search-signatures] - Loading onto this anticipatory direction ramps up as more items from a cluster have been produced in all five models (Gemma-2-9B, Gemma-2-2B, Llama-3.1-8B, Llama-3.2-3B, Qwen2.5-7B; chi-squared at least 20.44, p below 0.0001). [leckie-etal-2026-semantic-search-signatures] - Adding the direction to the residual stream at layer L (scale s of +0.5 or -0.5 times the mean residual norm, +/-0.1 for Qwen2.5-7B) biases switching against magnitude-matched orthogonal random-noise controls; for Gemma-2-9B at layer 26 positive steering raises the switch rate from 0.29 to 0.70 and negative steering lowers it to 0.07. Effects depend on model and layer, positive steering is more consistent than negative, and negative steering was less effective for Llama-3.1-8B. [leckie-etal-2026-semantic-search-signatures] - Separately, item-level and category-label steering vectors built from the Jacobian-lens and unembedding rows (not a learned probe direction) also induce switching into a target category, reaching a target-category switching probability of 0.68 for Gemma-2-9B (item-level, layer 40); the models are the paper's own instruction-tuned checkpoints, whose exact Hub ids the paper does not give. [leckie-etal-2026-semantic-search-signatures]