IsoScore

methods / Theoretical / Analytical / Geometric analysis / IsoScore

Techniqueadvanced

A rigorously axiomatized isotropy metric — PCA-decorrelate the point cloud, normalize its per-axis variance vector, measure its Euclidean distance from perfect uniformity (the isotropy defect), and affinely rescale to [0,1] — designed to satisfy mean-agnosticism, scale-invariance, rotation-invariance, and monotonic sensitivity to how many dimensions are actually used, properties Rudman et al. (2021) show average random cosine similarity and the partition score each fail.

Used in (2 observations)

IsoScore shows contextual spaces use even fewer dimensionsmeasured

structure: Anisotropy · models: GPT-1 (OpenAI GPT), GPT-2-small, BERT-base-uncased, DistilBERT-base-uncased · paper: IsoScore: Measuring the Uniformity of Embedding Space Utilization

Dominant-dimension variance decides cosine vs rank metricsmeasured

structure: Anisotropy · models: all-MiniLM-L6-v2, all-MiniLM-L12-v2, all-mpnet-base-v2, paraphrase-mpnet-base-v2, BGE-base-en-v1.5, BGE-large-en-v1.5, E5-large-v2, multilingual-e5-large, e5-mistral-7b-instruct, SFR-Embedding-Mistral, BERT-base-uncased, RoBERTa-base, ELECTRA-base, mBERT (BERT-base, Multilingual Cased), GPT-2-small, Pythia-410M, Qwen2.5-1.5B, Qwen2.5-7B, Mistral-7B · paper: Anisotropy Decides Cosine vs. Rank Metrics for Text Embeddings