Mahalanobis cosine between truth directions predicts cross-domain generalization

measured in 1 paper

Ying et al. test whether truth directions range from domain-general to domain-specific across five truth types plus sycophantic/expectation-inverted lying [ying-etal-2026-the-truthfulness-spectrum-hypothesis] Linear probes generalize well pairwise except on sycophantic/inverted lying (AUROC ~0.55 and ~0.28), while joint training recovers strong performance [ying-etal-2026-the-truthfulness-spectrum-hypothesis] Mahalanobis cosine similarity between probe directions predicts cross-domain generalization far better than standard cosine (R^2=0.98 vs 0.56) [ying-etal-2026-the-truthfulness-spectrum-hypothesis] Concept-erasure isolates domain-general, domain-specific, and shared truth directions with effective dimensionality under 100-200 in an 8192-d residual stream [ying-etal-2026-the-truthfulness-spectrum-hypothesis] Steering domain-specific directions selectively suppresses incorrect-answer probability (+0.05 to +0.10) while the domain-general direction backfires (-0.07 to -0.11) [ying-etal-2026-the-truthfulness-spectrum-hypothesis]

Structure

Linear Direction

Context

truthfulness, probe-generalization

Confirmed in models

Llama 3.3 70B Instruct · Llama, Llama-3.1-8B-Instruct · Llama, Llama-3.2-3B-Instruct · Llama, Qwen2.5-14B-Instruct · Qwen, Qwen2.5-7B-Instruct · Qwen

Method

Linear probing, LEACE (LEAst-squares Concept Erasure), Mahalanobis-cosine probe alignment, Causal interventions (steering)

Papers

The Truthfulness Spectrum Hypothesis — Ying, Zhuofan Josh, Ravfogel, Shauli, Kriegeskorte, Nikolaus, Hase, Peter2026 · arXiv:2602.20273