Register-neuron identification

methods / Causal Validation / Register-neuron identification

Techniqueintermediate

Algorithmically identifies a small set of neurons whose average activation is highest specifically at high-norm outlier-token positions in a frozen vision transformer, then inspects their decoder weights for consistently large values in specific dimensions — distinguishing a directional (decoder-weight) claim from a purely magnitude-based (activation-norm) one.

Used in (1 observation)

Fewer than 25 register neurons with consistent directions cause ViT outlier tokensmeasured

structure: Linear Direction · models: OpenCLIP ViT-B/16 (LAION-2B), DINOv2 ViT-L/14, LLaVA-Llama-3-8B · paper: Vision Transformers Don't Need Trained Registers