Persistent homology (Betti number analysis)

methods / Theoretical / Analytical / Persistent homology (Betti number analysis)

Persistent homology (Betti number analysis)

Techniqueadvanced

Computes Betti numbers (counts of connected components, independent loops, and enclosed voids) directly from a point cloud of activations via topological data analysis, to empirically distinguish a disc from a circle from a torus without assuming a parametric shape in advance.

Used in (7 observations)

A 141-hypothesis automated topology/geometry audit of scGPT and Geneformer gene-embedding representations finds statistically significant persistent homology in nearly every transformer layer and a canonical-correlation cross-model alignment of 0.80 between the two independently-trained modelsmeasured

structure: Persistent-homology / Betti profile across depth, Platonic Representation Hypothesis · models: scGPT (whole-human pretrained checkpoint), Geneformer · paper: What Topological and Geometric Structure Do Biological Foundation Models Learn? Evidence from 141 Hypotheses

A persistent circle recurs in the convolutional filter weights of real trained CNNs across depths, datasets, and training runs, and fixing an idealized circle as the first layer causally improves cross-dataset generalizationmeasured

structure: Circle · models: Custom convolutional net on MNIST (Carlsson & Brüel Gabrielsson), Custom convolutional net on CIFAR-10 (Carlsson & Brüel Gabrielsson), Custom convolutional net on SVHN (Brüel Gabrielsson & Carlsson), VGG (image classifier, various depths) · paper: Topological Approaches to Deep Learning, Exposition and Interpretation of the Topology of Neural Networks

Zigzag persistent homology tracks how topological features (p-dimensional holes) of a real pretrained LLM's representation point-cloud persist and evolve across layers, and the resulting topological-stability signature identifies layers safe to prunemeasured

structure: Persistent-homology / Betti profile across depth · models: Pythia-6.9B · paper: Persistent Topological Features in Large Language Models

Topological Polysemy (TPS), a persistent-homology-based singularity measure, correlates with real word-sense counts in a trained fastText embedding, and not with frequencymeasured

structure: Pinched Manifold (Singular Quotient) · models: fastText (self-trained on 65M-token SemEval-2010 corpus) · paper: Topology of Word Embeddings: Singularities Reflect Polysemy

Topological descriptors (persistent homology) and intrinsic-dimension estimates of the last-layer data manifold in real trained CNNs change measurably across layers and training, and predict generalization performancemeasured

structure: Persistent-homology / Betti profile across depth · models: ResNet-18 (supervised, ImageNet) · paper: Topology and Geometry of Data Manifold in Deep Learning

Modular-addition networks universally learn a torus projecting to a discmeasured

structure: Torus, Circle · models: Grokking Modular-Arithmetic Transformer (1 layer, 4 heads, d_model=128, mod 113), Clock/Pizza Transformer, Model A (1 layer, constant attention alpha=0, width 128, mod 59), Clock/Pizza Transformer, Model B (1 layer, normal attention alpha=1, width 128, mod 59) · paper: Progress Measures for Grokking via Mechanistic Interpretability, The Clock and the Pizza: Two Stories in Mechanistic Explanation of Neural Networks, On the Geometry and Topology of Representations: The Manifolds of Modular Addition

Persistent-homology H1 cycle strength rises sharply at grokkingmeasured

structure: Circle · models: Grokking Modular-Arithmetic Transformer (2 layers, 4 heads, pre-LN, d_model=128, mod 113/149/197), Grokking Modular-Arithmetic MLP (3 hidden layers, width 512, d_embed=128, mod 113/149/197) · paper: Topological Signatures of Grokking