MATH · IN · MODELS

E5

Microsoft

Structures found in this family (3)

Platonic Representation Hypothesis, Linear Subspace, Anisotropy

By model (5)

E5-base-v2 · 109M
Platonic Representation Hypothesis, Linear Subspace
E5-large-v2 · 335M
Linear Subspace, Anisotropy
e5-mistral-7b-instruct · 7B
Anisotropy
E5-mistral-7b-instruct · 7B
Linear Subspace
multilingual-e5-large · 560M
Anisotropy

Observations (5)

mini-vec2vec aligns embedding models with a cheap orthogonal map
Independently-trained embedding models share linear structure (CKA 0.60-0.88)
vec2vec learns a fully unpaired translation between six independently-trained text embedding models (gtr, clip, e5, gte, stella, granite), recovering cosine similarity up to 0.92 and top-1 retrieval accuracy up to 100% with no paired training data, and an ablation shows the explicit Gram-matrix-preservation (VSP) loss is causally necessary -- removing it alone collapses top-1 accuracy from 0.91 to 0.00
A Top-k SAE on sentence embeddings gives near-orthogonal, steerable latents
Dominant-dimension variance decides cosine vs rank metrics

Papers

mini-vec2vec: Scaling Universal Geometry Alignment with Linear Transformations (2025), Characterizing Linear Alignment Across Language Models (2026), Harnessing the Universal Geometry of Embeddings (2026), Aligning Sentence Embeddings to Human Concepts via Sparse Autoencoders (2026), Anisotropy Decides Cosine vs. Rank Metrics for Text Embeddings (2026)