Structural probing (metric/distance probe)

methods / Direction Extraction / Structural probing (metric/distance probe)

Techniqueintermediate

Fits a linear transform B so that squared Euclidean distance/norm under B approximates a target graph metric (e.g. parse-tree edge distance and depth) across all pairs of tokens simultaneously — a metric-learning probe, not a single-output classifier or regressor.

Used in (11 observations)

A structural probe's tree-metric encodes a phase-count gradient for finitenessmeasured

structure: Tree Metric Embedding · models: Gemma 3 27B, Llama-3.1-8B, Mistral-7B-v0.3, Qwen3-14B, Qwen3-4B · paper: A Minimalist Structural Probe for Phase-Count Abstraction in Transformer Language Models

AST-Probe recovers full code syntax trees from a 64-128 dimensional subspacemeasured

structure: Tree Metric Embedding · models: CodeBERT-base, GraphCodeBERT-base, CodeT5-base, RoBERTa-base, CodeBERTa-small · paper: AST-Probe: Recovering Abstract Syntax Trees from Hidden Representations of Pre-trained Language Models

BERT's tree-metric partially distinguishes constituency depth, with mixed significancemeasured

structure: Tree Metric Embedding · models: BERT-large-cased · paper: Evidence of Hierarchically-Complex Syntactic Structure Within BERT's Word Representations

mBERT's syntactic probe subspace is approximately shared across languagesmeasured

structure: Tree Metric Embedding · models: mBERT (BERT-base, Multilingual Cased) · paper: Finding Universal Grammatical Relations in Multilingual BERT

A polar probe splits in-context relational graphs into existence and type readoutsmeasured

structure: Tree Metric Embedding · models: Llama-3-8B, OLMo-7B, Pythia-70M, Pythia-160M, Pythia-410M, Pythia-1B, Pythia-1.4B, Pythia-2.8B, Pythia-6.9B · paper: Polar probe linearly decodes semantic structures from LLMs

mBERT's shared syntax metric needs a per-language transform only for distant languagesmeasured

structure: Tree Metric Embedding · models: mBERT (BERT-base, Multilingual Cased) · paper: Examining Cross-lingual Contextual Embeddings with Orthogonal Structural Probes

The structural-probe embedding recovers BERT syntax trees; heads track coreferencemeasured

structure: Tree Metric Embedding · models: BERT-base-cased · paper: Emergent Linguistic Structure in Artificial Neural Networks Trained by Self-Supervision

Structural probes show staged phonemic-then-lexical-then-syntactic subspacesmeasured

structure: Linear Subspace · models: Wav2Vec 2.0 Base (LibriSpeech-960h), wav2vec 2.0 Large, Llama-2-13B, Pythia-1B · paper: Emergence of Phonemic, Syntactic, and Semantic Representations in Artificial Neural Networks

A fitted linear metric embeds dependency trees in BERT/ELMo geometrymeasured

structure: Tree Metric Embedding · models: ELMo (5.5B-word pretrained, 2-layer biLSTM), BERT-base-cased, BERT-large-cased · paper: A Structural Probe for Finding Syntax in Word Representations, Probing BERT in Hyperbolic Spaces

Fine-tuning reshapes BERT's recoverable syntax tree, differently per taskmeasured

structure: Tree Metric Embedding · models: BERT-base-cased · paper: On the Evolution of Syntactic Information Encoded by BERT's Contextualized Representations

An RBF-kernelized structural probe beats the linear baseline on mBERT syntaxmeasured

structure: Tree Metric Embedding · models: mBERT (BERT-base, Multilingual Cased) · paper: A Non-Linear Structural Probe