Co-occurrence statistics ablation

methods / Corpus Statistics / Co-occurrence statistics ablation

Techniqueadvanced

Removes specific co-occurrence statistics from the training/estimation data and checks whether the resulting geometric structure is robust to the perturbation — a robustness check, not a discovery method.

Used in (2 observations)

Analogy parallelograms emerge from Kronecker eigenvector structuremeasured

structure: Concept Crystals (Parallelogram/Trapezoid Structure) · models: · paper: On the Emergence of Linear Analogies in Word Embeddings

Ordered continuous concepts trace a rippled open 1D (Lissajous) manifoldmeasured

structure: 1D continuum manifold · models: Gemma-2-2B, EmbeddingGemma, word2vec (trained on Wikipedia), Claude 3.5 Haiku · paper: Symmetry in Language Statistics Shapes the Geometry of Model Representations, When Models Manipulate Manifolds: The Geometry of a Counting Task