Definition
Rather than claiming the representation manifold is some named shape, this node covers papers that measure what topological features persist in a real model’s activation point-cloud — a computed quantity via persistent homology (tracking -dimensional holes/loops across a filtration of the point cloud, e.g. by distance scale or, for layer-to-layer comparison, a zigzag filtration connecting successive layers’ point clouds) — not a geometric object in its own right. Output is typically a persistence diagram or Betti-number profile: how many topological features of each dimension appear and how long they persist before merging or vanishing.
Relative to intrinsic-dimension-profile and manifold-curvature-profile
Intrinsic-dimension profile across depth tracks how many coordinates the manifold needs; Curvature profile of the representation manifold tracks how bent it is. This node tracks a third, independent question: what topological features (holes, loops, connected components) does the point cloud have, and how do they persist or evolve. All three are orthogonal measurements of the same underlying activation geometry — a manifold can be low-dimensional, flat, and topologically rich (many small holes) all at once, or any other combination; a paper reporting one of these three quantities makes no claim about either of the other two.
Key evidence: layer-wise topological tracking in LLMs
Gardinazzi, Viswanathan, Panerai, Ansuini, Cazzaniga & Biagetti (2024)
introduce zigzag persistent homology to track how topological features
of the layer-by-layer representation point-cloud persist and evolve
across depth in real pretrained Pythia models (up to Pythia-6.9B). The
resulting topological-stability signature — which layers’ removal would
least disturb the persistent topological structure — is used as a
practical layer-pruning criterion, validated against downstream task
performance. See
gardinazzi-etal-2024-zigzag-persistent-homology-tracks-topological-features-of-the-llm-representation-point-cloud-across-layers-and-flags-prunable-layers.
Key evidence: topology and intrinsic dimension jointly predicting generalization
Magai & Ayzenberg (2022) apply persistent homology and intrinsic
dimension estimation jointly to the data manifold formed by
penultimate-layer activations of real trained CNNs (CIFAR-10 and
related datasets), tracking how topological complexity and intrinsic
dimension evolve across network depth and training progress, and
showing both descriptors are predictive of generalization performance.
They extend the same measurement to a face-recognition system’s
embedding manifold under adversarial spoofing attacks, using the
topological descriptors as an anomaly signature. See
magai-ayzenberg-2022-persistent-homology-and-intrinsic-dimension-of-the-last-layer-data-manifold-track-generalization-in-cnns.
Key evidence: cross-model topological/geometric audit in biology
Kendiukhov (2026) runs an autonomous 141-hypothesis topological/geometric
audit of scGPT and Geneformer gene-embedding representations, finding
persistent homology statistically significant (p<0.05) in nearly every
transformer layer across both independently-trained models. See
a-141-hypothesis-automated-audit-of-scgpt-and-geneformer-finds-significant-persistent-homology-in-nearly-every-layer-and-a-cca-canonical-correlation-of-0-80-between-the-two-models.
Key papers
- Gardinazzi, Y., Viswanathan, K., Panerai, G., Ansuini, A., Cazzaniga, A. & Biagetti, M. (2024). Persistent Topological Features in Large Language Models. arXiv:2410.11042 — origin of the zigzag layer-tracking method and pruning application.
- Magai, G. & Ayzenberg, A. (2022). Topology and Geometry of Data Manifold in Deep Learning. arXiv:2204.08624 — joint persistent homology / intrinsic dimension tracking predicting generalization.
- Kendiukhov, I. (2026). What Topological and Geometric Structure Do Biological Foundation Models Learn? Evidence from 141 Hypotheses. arXiv:2602.22289 — large-scale automated audit finding significant persistent homology across two independently-trained genomics foundation models.