A protein-LM nativeness axis orders sequences; viral signal stays separable

measured in 1 paper

Bigot et al. run PCA on mean-pooled embeddings from five protein language models (ESM2, ESM3, ESMC, ProGen2, EvoDiff), finding a dominant nativeness axis (PC1, 54.3-73.1% variance) correlating with masked-reconstruction perplexity (rho=0.83-0.96) [bigot-etal-2026-viral-proteins-reveal-geometry-of-protein-language-models] The axis replicates within each of 10 biological groups via within-group PCA refits [bigot-etal-2026-viral-proteins-reveal-geometry-of-protein-language-models] Linear probes classify viral versus cellular proteins at AUC 0.97-1.00 even under homology-controlled splits, with 88.3-96.7% true-positive rate at 1% false-positive rate versus 26.9-39.2% for a perplexity-only classifier [bigot-etal-2026-viral-proteins-reveal-geometry-of-protein-language-models]

Structure

Linear Direction, Linear Separability

Context

a dominant PCA axis (nativeness) in pooled protein-LM embedding space that tracks masked-reconstruction perplexity, replicated across architecturally distinct model families, linear probing showing viral-vs-cellular separability exceeding perplexity-only classification, even under homology-controlled splits

Confirmed in models

ESM-2 (650M) · ESM-2, ESM-2 (15B) · ESM-2, ESM3 (1.4B, OPEN) · ESM3, ESM3 (98B, LARGE) · ESM3, ESMC (600M) · ESM Cambrian (ESMC), ESMC (6B) · ESM Cambrian (ESMC), ProGen2-base · ProGen2, EvoDiff OA-DM · EvoDiff

Method

PCA, Linear probing

Papers

Viral Proteins Reveal Geometry of Protein Language Models — Bigot, Arthur, Bhasin, Harmon, Park, Core Francisco, Shakhnovich, Eugene, Wang, Dianzhuo2026 · arXiv:2606.12609