Clinical concept centers: a residual-stream direction per ICD-9 disease chapter, built as a decoded SAE difference of means, is held-out causally steerable in eleven open Qwen and Gemma models
measured in 1 paper- Vaidyanathan et al. build one direction v_C in the residual stream per ICD-9-CM chapter (17 chapters, 8,617 MIMIC-IV discharge notes) by decoding the difference of mean SAE codes between in-chapter and out-of-chapter notes, in eleven open models (five Qwen, six Gemma, 4B-30B); the best single SAE feature reaches held-out one-vs-rest AUC 0.80-0.92 in the Qwen models and 0.70-0.90 in the Gemma models, against a full-rank logistic-probe ceiling of 0.90-0.93 macro AUC on ten of eleven models. [vaidyanathan-etal-2026] - Adding the chapter direction to the residual stream at the final token raises the correct ICD-letter probability on the held-out half in all eleven models (Qwen models with sound baselines +0.041 to +0.069, confidence intervals excluding zero; Gemma gains smaller, up to +0.0193, with the two Gemma-3-27B rows' intervals including zero); the dose-response is an inverted U and negative strengths lower the correct-letter probability. [vaidyanathan-etal-2026] - Matched-magnitude random directions achieve far less (and damage the pretrained Gemma models), injecting the unembedding row of the correct answer letter does not reproduce the effect, and the chapter directions have near-zero cosine with the answer-letter unembedding rows (+0.003 on Qwen3-8B-Base, layer 23), so the effect does not run through the output vocabulary; in free-form generation, steering with the wrong chapter hurts the model. [vaidyanathan-etal-2026] - The direction is fit on mean-pooled note activations and injected at the final token; a fresh SAE trained on II-Medical-8B recovers chapter directions with mean residual-space cosine +0.835 to the parent dictionary's. Causal effectiveness peaks at roughly 60-85% of depth in the Qwen models although separability is nearly uniform across depth; the measured gain in free-form generation steering is +1.1 to +2.3 extra correct diagnoses per hundred cases on three models. [vaidyanathan-etal-2026]