Linguistic categories occupy nested low-dimensional subspaces

measured in 1 paper

Hernandez & Andreas apply a rank-sweep minimal-subspace search to ELMo and BERT-base, finding POS, tense, number and dependency labels are linearly decodable from subspaces well below the hidden size [hernandez-andreas-2021] Finer sub-categories are decodable from a smaller subspace nested inside the coarser parent, roughly half its rank, in many (not all) layers [hernandez-andreas-2021] Neuron-ablation shows these subspaces are distributed across many neurons (POS survives reduction to 512 of BERT's 768 axes) [hernandez-andreas-2021] Ablating only the rank-4 nounspace lowers subject-noun prediction (.85 to .82) without affecting verbs, and vice versa for verbspace, despite each removing under 1% of dimensions [hernandez-andreas-2021]

Structure

Linear Subspace

Context

part of speech, dependency relations, subspace nesting, distributed representation, causal ablation

Confirmed in models

ELMo (5.5B-word pretrained, 2-layer biLSTM) · ELMo, BERT-base-cased · BERT

Method

Minimal subspace search (rank-sweep probing), Causal interventions (steering)

Papers

The Low-Dimensional Linear Geometry of Contextualized Word Representations — Hernandez, Evan, Andreas, Jacob2021 · arXiv:2105.07109