methods / Corpus Statistics
Corpus Statistics
Analyze the training corpus's word co-occurrence statistics directly — not model activations — to explain why a representational geometry arises in the first place.
2 methods in this category
Spectral analysis of the co-occurrence/PMI matrix
Studies the eigenvectors/eigenvalues of the (normalized) word co-occurrence or pointwise mutual information matrix to predict the geometry of learned representations before training any model.
6 papers
Co-occurrence statistics ablation
Removes specific co-occurrence statistics from the training/estimation data and checks whether the resulting geometric structure is robust to the perturbation — a robustness check, not a discovery method.
3 papers