Sparse-autoencoder features on a real small genomic language model correspond to nucleotide identity and real transcription-factor binding motifs

Guan, He & Zhang (2025) train a sparse autoencoder (32x expansion, 8,192-feature dictionary, L1=0.1) on layer-3 activations of real HyenaDNA-small-32k, pretrained at single-nucleotide resolution on the human reference genome, extracted from real GRCh38 sequences [guan-he-zhang-2025-sparse-autoencoders-reveal-interpretable-structure-in-small-gene-language-models] Individual SAE features correspond to nucleotide-identity positions (e.g. a feature tracking cytosine) and to real transcription-factor binding sites, validated against independent ground-truth JASPAR motif annotations on chromosome 14 via nucleotide-level precision/recall/F1 (named motif matches include MA1596.1, MA2121.1 [C2H2 zinc finger class] and MA0052.5 [MADS-box class], with strand specificity) [guan-he-zhang-2025-sparse-autoencoders-reveal-interpretable-structure-in-small-gene-language-models] Purely correlational -- no steering, ablation, or intervention experiment is performed; the paper's own conclusion frames causal use of the SAE features as future work [guan-he-zhang-2025-sparse-autoencoders-reveal-interpretable-structure-in-small-gene-language-models]