Rhetorical-question status is linearly decodable but methods find non-collinear directions

measured in 1 paper

Yao et al. test whether rhetorical-question status is linearly decodable in Qwen3-32B and Llama-3.3-70B-Instruct, projecting representations into a 64-dim PCA space used only for numerical stabilization [yao-etal-2026-rhetorical-questions-linear-probing] Comparing diff-in-means, logistic regression, and hinge-loss SVM extraction, layer-wise AUROC rises from ~0.6 (diffMean, early) to ~0.85-0.9 (trained probes, deep) [yao-etal-2026-rhetorical-questions-linear-probing] The two trained-probe directions nearly agree (cosine ~1), but the training-free diff-in-means direction is only weakly aligned with them (cosine <0.7 on RQ, ~0.5 on SRAQ) [yao-etal-2026-rhetorical-questions-linear-probing] Directions fit on different datasets align only weakly (cosine 0.2-0.4), so rhetorical question is not one dataset-general direction but a method- and corpus-dependent family [yao-etal-2026-rhetorical-questions-linear-probing]

Structure

Linear Direction

Context

method-dependent (diffMean vs. trained-probe) direction non-collinearity for the same concept, cross-dataset direction alignment (cosine 0.2-0.4) as a stricter test than within-dataset AUROC, PCA-truncated (k=64) representation space for numerical stability of trained probes, layer-wise and pooling-strategy (last-token vs. mean-pooled) AUROC trajectories

Confirmed in models

Qwen3-32B · Qwen, Llama 3.3 70B Instruct · Llama

Method

Linear probing, Difference-in-means direction, PCA

Papers

Rhetorical Questions in LLM Representations: A Linear Probing Study — Yao, Louie Hong, Anand, Vishesh, Zhuang, Yuan, Jiang, Tianyu2026 · arXiv:2604.14128