Late-layer SAE feature directions in Geneformer steer cell differentiation

Kendiukhov trains SAEs on Geneformer's layer-5 representations and performs trajectory-guided feature steering at early, middle, and late layers [kendiukhov-2026-exhaustive-circuit-mapping-of-a-single-cell-foundation-model] Late-layer (L17) feature steering pushes cell states toward maturity (fraction positive 1.0), while early/middle (L0, L11) steering mostly pushes away (0.00-0.58) [kendiukhov-2026-exhaustive-circuit-mapping-of-a-single-cell-foundation-model] Combinatorial triplet ablation finds three-way redundancy exceeding pairwise redundancy, with zero synergy detected at any order [kendiukhov-2026-exhaustive-circuit-mapping-of-a-single-cell-foundation-model] Exhaustive circuit tracing finds a heavy-tailed hub architecture, with 1.8% of features carrying disproportionate downstream connectivity [kendiukhov-2026-exhaustive-circuit-mapping-of-a-single-cell-foundation-model]