Linear H-probes recover tree depth and pairwise tree distance from low-dimensional, causally important subspaces of language-model representations in synthetic tree-traversal tasks, with analogous but weaker structure in real mathematical reasoning traces

- Dawes et al. fit linear H-probes on residual-stream activations of the node tokens in the final answer path of synthetic binary-tree traversal tasks: a distance probe (a p-dimensional projection, p = 2-5, of 10 PCA components whose Euclidean distance reconstructs pairwise tree distance) and a ridge-regression depth direction; together they span a (p+1)-dimensional 'hierarchical subspace'. Probes are best around layers ~25-35 of the 14B R1-distilled model, are stable across five training splits and across model scales (R1-Distill-Qwen 1.5B/7B/14B; weaker in Qwen1.5-Chat 1.8B/7B/14B, which mostly fail the task), and zero-ablating the subspace lowers task accuracy by about as much as ablating the top principal components of the CoT distribution. [dawes-etal-2026-h-probes] - Transfer is partial: distance transfers strongly to deeper trees (depth 3-4) but depth transfers weakly (exact Pearson about 0.2-0.3). On GSM8K and HiBench the signal is analogous but weaker and noisier (e.g. GSM8K best exact-example Pearson r = 0.29 for distance, 0.49 for depth in the 14B reasoning model; HiBench ablation effects mixed), which the paper calls supportive but exploratory, so the real-world result is directional rather than a full replication. The authors also note PCA to 10 components means the signal is not fully captured by a low-rank space and that causal identification is not fully established. [dawes-etal-2026-h-probes]