Normalizing-flow calibration (BERT-flow)

methods / Theoretical / Analytical / Geometric analysis / Normalizing-flow calibration (BERT-flow)

Techniqueadvanced

Learns an invertible neural mapping (a normalizing flow) from a frozen model's embedding space to a standard isotropic Gaussian latent space by maximizing the likelihood of the embeddings under the flow, unsupervised — correcting anisotropy without discarding information, unlike top-k singular-vector removal or simple standardization.

Used in (1 observation)

Normalizing flow removes BERT's anisotropic frequency holesmeasured

structure: Anisotropy · models: BERT-base-uncased, BERT-large-uncased · paper: On the Sentence Embeddings from Pre-trained Language Models