Jacobian Lens (J-lens)

methods / Causal Validation / Jacobian Lens (J-lens)

Techniqueintermediate

Reads an intermediate residual-stream activation through the average first-order effect of that activation on the model's final-layer residual stream (an averaged Jacobian J_ℓ), followed by the model's own unembedding — a refinement of the logit lens that stays readable in earlier layers. Its rows W_U·J_ℓ are one direction per vocabulary token; the sparse non-negative span of these directions is called the J-space.

Used in (1 observation)

A generic switching direction (PCA + logistic-regression coefficients projected back to the residual stream) predicts and bidirectionally steers cluster switching in the semantic fluency task across five open instruction-tuned LLMsmeasured

structure: Linear Direction · models: Gemma-2-9B-it, Gemma-2-2B-it, Llama-3.1-8B-Instruct, Llama-3.2-3B-Instruct, Qwen2.5-7B-Instruct · paper: Signatures of Semantic Search in the Activations of Large Language Models