Dominant-dimension variance decides cosine vs rank metrics
measured in 1 paper- Across 19 text encoders, the variance share held by the single dominant ("rogue") dimension predicts whether rank/L1-type metrics or cosine win (rank correlation 0.86, linear 0.95). [parupudi-2026-anisotropy-decides-cosine-vs-rank-metrics-for-text-embeddings] - A dominance threshold ~0.01 splits crowded (9) from well-spread (10) encoders; on crowded encoders the best alternative beats cosine by 0.055 Spearman on average (87% of cells) versus 0.001 for well-spread. [parupudi-2026-anisotropy-decides-cosine-vs-rank-metrics-for-text-embeddings] - Projecting out the top principal directions (all-but-the-top) erases 87% of the advantage on crowded encoders versus ~10% for random directions, so geometry, not the training method, decides the optimal metric. [parupudi-2026-anisotropy-decides-cosine-vs-rank-metrics-for-text-embeddings] - Evaluated on 10 contrastive embedders (MiniLM-L6/L12, MPNet, paraphrase-MPNet, BGE-base/large, E5-large, multilingual-E5-large, E5-Mistral-7B, SFR-Embedding-Mistral) and 9 base LMs (BERT, RoBERTa, ELECTRA, mBERT, GPT-2, Pythia-410M, Qwen2.5-1.5B/7B, Mistral-7B). [parupudi-2026-anisotropy-decides-cosine-vs-rank-metrics-for-text-embeddings]