Mlp probing

methods / Direction Extraction / Mlp probing

Techniquebeginner

A small multi-layer perceptron (typically 1–2 hidden layers with a nonlinearity) trained on frozen activations to predict a target property; used in place of a linear probe when the property may be decodable only nonlinearly, at the cost of the interpretability a single weight direction provides.

Used in (13 observations)

Probes decode a grid cognitive map that reasoning then reorganizesmeasured

structure: Linear Separability · models: GPT-OSS-20B · paper: A Behavioural and Representational Evaluation of Goal-Directedness in Language Model Agents

DINOv2 and Stable Diffusion encode depth/normals via nonlinear dense probesmeasured

structure: Linear Subspace · models: DINOv2 ViT-B/14, DINOv2-B (ViT-Base, 4 register tokens), CLIP ViT-L/14, MAE ViT-Base (Masked Autoencoder), iBOT ViT-B/16, Stable Diffusion v2.1 · paper: Probing the 3D Awareness of Visual Foundation Models

Geographic representations are causally used for prediction, not just decodablemeasured

structure: Affine Subspace · models: DeBERTa-v2-xxlarge, GPT-Neo-1.3B · paper: More than Correlation: Do Large Language Models Learn Causal Representations of Space?

Nonlinear (small-MLP) probes reliably decode the full Othello board state from an 8-layer, 512-dim GPT-style transformer's internal activations at every move, and activation-patching interventions that flip the decoded board-state representation causally change the model's subsequent move predictionsmeasured

structure: Nonlinear World-Model Decodability · models: OthelloGPT · paper: Emergent World Representations: Exploring a Sequence Model Trained on a Synthetic Task

Morphology stays linearly separable while lexical identity turns nonlinear with depthmeasured

structure: Linear Separability · models: BERT-base-uncased, BERT-large-uncased, GPT-2-small, GPT-2-Large, GPT-2-XL, Pythia-6.9B, OLMo 2 7B, Gemma-2-2B, Qwen2.5-1.5B-Instruct, Llama-3.1-8B · paper: Finding Lexical Identity and Inflectional Morphology in Modern Language Models

Mid-level vision decodability weakly correlates with recognition accuracymeasured

structure: Linear Subspace · models: DINO ViT-B/16, MAE ViT-Base (Masked Autoencoder), ResNet-50 (SimCLR contrastive pretraining, ImageNet), ResNet-50 (MoCo v2, unsupervised contrastive pretraining, ImageNet) · paper: Probing the Mid-level Vision Capabilities of Self-Supervised Learning

Othello-GPT's board state is linear under the model's own relative framemeasured

structure: Linear Direction · models: OthelloGPT · paper: Emergent Linear Representations in World Models of Self-Supervised Sequence Models

A probe reads answer correctness before reasoning models state itmeasured

structure: Linear Separability · models: DeepSeek-R1-Distill-Llama-8B, DeepSeek-R1-Distill-Llama-70B, DeepSeek-R1-Distill-Qwen-1.5B, DeepSeek-R1-Distill-Qwen-7B, DeepSeek-R1-Distill-Qwen-32B, QwQ-32B, Llama-3.1-8B-Instruct · paper: Reasoning Models Know When They're Right: Probing Hidden States for Self-Verification

Stack depth is linearly decoded in tiny counter-language transformersmeasured

structure: Linear Direction · models: Counter-Language Transformer (1 layer, 4 heads, causal-masked encoder) · paper: Emergent Stack Representations in Modeling Counter Languages Using Transformers

A 'state of mind' spectrum is linearly decodable across five encodersmeasured

structure: Linear Direction · models: BGE-large-en-v1.5, all-mpnet-base-v2, all-MiniLM-L6-v2, Qwen3-Embedding-0.6B, Qwen2.5-3B-Instruct · paper: Probing Spectrum-Like Organization of States of Mind in Transformer Representation Spaces

TabPFN linearly encodes regression coefficients and arithmetic intermediatesmeasured

structure: Linear Subspace · models: TabPFN v2 (tabular in-context-learning foundation model) · paper: TabPFN Through The Looking Glass: An Interpretability Study of TabPFN and Its Internal Representations

MLP probes decode seven music-theory concepts from generative modelsmeasured

structure: Linear Separability · models: Jukebox, MusicGen-Small, MusicGen-Large · paper: Do Music Generation Models Encode Music Theory? (SynTheory)

Emotion clusters peak at mid-depth and tighten with scalemeasured

structure: Linear Separability · models: Qwen3-0.6B, Qwen3-1.7B, Qwen3-4B, Qwen3-8B, Llama-3.2-1B, Llama-3.2-3B, Llama-3.1-8B · paper: Decoding Emotion in the Deep: A Systematic Study of How LLMs Represent, Retain, and Express Emotion