Frozen real genomic language models show task-dependent linear accessibility of biological signal, quantified by silhouette, kNN and Fisher-separation geometry

Datta, Shatabda & Rahman (2026) probe five real frozen genomic language models (DNABERT-2, Nucleotide Transformer, HyenaDNA, GENERATOR-v2, Omni-DNA) with logistic-regression (linear), RBF-SVM and MLP (nonlinear) probes plus quantitative embedding-geometry metrics (silhouette score, 5-nearest-neighbor consistency, linear separability, Fisher separation ratio) on mean-pooled frozen embeddings [datta-shatabda-rahman-2026-frozen-but-not-always-accessible-genomic-language-models] Promoter-recognition signal is highly linearly accessible from frozen embeddings (frozen-probe recovery of fine-tuned performance: 98%/95%/100%/98% for HyenaDNA/NT/GENERATOR-v2/Omni-DNA), while splice-site signal is markedly less accessible (55-88% recovery), with Nucleotide Transformer's own embedding geometry showing the same split quantitatively: Promoter-All silhouette=0.044/kNN=0.883/linear-separability=0.917 versus Splice-All silhouette=-0.011/kNN=0.465/linear-separability=0.597 [datta-shatabda-rahman-2026-frozen-but-not-always-accessible-genomic-language-models] Layer-wise probing of Nucleotide Transformer on the Splice-All task shows intermediate layers (layer 20: ACC=0.639, F1=0.527, MCC=0.449) outperform the final layer (layer 24: ACC~0.60, F1~0.52); causal validation is via input-level in-silico mutagenesis (per-nucleotide substitution sensitivity), not internal activation patching [datta-shatabda-rahman-2026-frozen-but-not-always-accessible-genomic-language-models]