Tuned Lens

Trains a per-layer affine map (a matrix and bias, the 'translator') that transforms a hidden state before reusing the model's own frozen unembedding and final LayerNorm, distilled via KL divergence to the model's own final-layer output distribution — a learned affine correction to the logit lens that removes its systematic bias and drift-related unreliability, while causal experiments show the resulting per-layer predictions rely on features the model itself actually uses.