Neighbor Distance Minimization (NDM)

methods / Dimensionality Reduction / Neighbor Distance Minimization (NDM)

Techniqueadvanced

Unsupervised learning of an orthogonal rotation of activation space followed by a partition of the rotated coordinates into subspaces, by minimizing each point's nearest-neighbor distance inside every subspace, with subspaces merged when their mutual information is high.

Used in (1 observation)

Unsupervised orthogonal subspace partitions of the residual stream localize circuit variablesmeasured

structure: Linear Subspace · models: GPT-2 Small, Qwen2.5-1.5B, Gemma-2-2B · paper: Decomposing Representation Space into Interpretable Subspaces with Unsupervised Learning