Cluster-corrected isotropy analysis

methods / Theoretical / Analytical / Geometric analysis / Cluster-corrected isotropy analysis

Techniqueintermediate

K-means-clusters a representation space first, then recomputes cosine-similarity-based anisotropy statistics per-cluster (after center-shifting each cluster to its own mean) — testing whether a high global anisotropy reading is actually a between-cluster separation effect that a genuinely-isotropic within-cluster structure is hiding.

Used in (1 observation)

Clusters and low-dim manifolds explain apparent anisotropymeasured

structure: Anisotropy · models: BERT-base-uncased, DistilBERT-base-uncased, GPT-1 (OpenAI GPT), GPT-2-small, ELMo (AllenNLP biLM, 1B Word Benchmark) · paper: Isotropy in the Contextual Embedding Space: Clusters and Manifolds