MATH · IN · MODELS

scGPT attention weights encode a compact ~8-10D hematopoietic manifold

measured in 1 paper

Kendiukhov exports a fixed operator from scGPT's frozen attention value-projection weights (no forward-pass activations) and compresses it into a latent space whose effective dimensionality plateaus at ~8-10 [kendiukhov-2026-scgpt-hematopoietic-manifold] The manifold's geodesic distances correlate with an independent hematopoietic developmental ontology (erythroid rho=0.768 p=0.0017, granulocytic rho=0.568, trunk rho=0.611; internal rho=0.835) [kendiukhov-2026-scgpt-hematopoietic-manifold] It validates on a strict non-overlap external panel (Tabula Sapiens, 564,253 cells) and zero-shot transfer to an immune panel (trustworthiness 0.993, blocked-permutation p=0.0005) [kendiukhov-2026-scgpt-hematopoietic-manifold] A lightweight readout on the extracted manifold beats scVI, Palantir, DPT, CellTypist, and PCA on pseudotime ordering (|rho|=0.439 vs 0.331) with ~1,000x fewer parameters [kendiukhov-2026-scgpt-hematopoietic-manifold]

Context

hematopoietic manifold extracted from frozen attention weights (no forward-pass activations), effective dimensionality ~8-10 (trustworthiness/transfer objective plateau), geodesic-distance correlation to independent biological developmental ordering (branch rho 0.568-0.768), external non-overlap panel validation (Tabula Sapiens, trustworthiness 0.993, p=0.0005), lightweight extracted readout beats scVI/Palantir/DPT/CellTypist/PCA baselines

Papers

Discovery of a Hematopoietic Manifold in scGPT Yields a Method for Extracting Performant Algorithms from Biological Foundation Model Internals — Kendiukhov, Ihor2026