MATH · IN · MODELS

Cross-layer dispersion of hidden states detects hallucination

measured in 1 paper

- A per-token signature combines the log-pseudo-determinant of the regularized cross-layer hidden-state covariance (a generalized variance) with the circular variance of normalized per-layer hidden states and predictive entropy. [srey-etal-2026-learning-uncertainty-from-sequential-internal-dispersion-in-large-language-models] - Hallucination-prone tokens show higher cross-layer dispersion, and the signature beats the strongest baseline (Ministral-8B average AUC 78.11 vs 75.24, FPR@95 -7.53). [srey-etal-2026-learning-uncertainty-from-sequential-internal-dispersion-in-large-language-models] - Cross-layer (all-pairs) dispersion captures uncertainty that successive-layer methods miss and generalizes better out-of-distribution; it does not compute intrinsic dimension. [srey-etal-2026-learning-uncertainty-from-sequential-internal-dispersion-in-large-language-models] - Tested on Llama-3.2-3B-Instruct, Llama-3.1-8B-Instruct and Ministral-8B-Instruct, with Qwen-3-4B and Qwen-3-14B in the appendix. [srey-etal-2026-learning-uncertainty-from-sequential-internal-dispersion-in-large-language-models]

Structure

Context

hallucination-detection, uncertainty-quantification

Papers

Learning Uncertainty from Sequential Internal Dispersion in Large Language Models — Srey, Rado, Wu, Xin, Nguyen, Thien, Luu, Anh Tuan2026 · arXiv:2604.15741