Unsupervised direction extraction: given contrast pairs of statements with opposite truth values, finds a direction whose projected values satisfy logical negation consistency (p(true) + p(false) ≈ 1) without ever using ground-truth labels.
Used in (3 observations)
structure: Linear Direction · models: T5-11B, UnifiedQA-11B (T5-based), T0++, GPT-J-6B, RoBERTa-large-MNLI, DeBERTa-xxlarge-v2-MNLI · paper: Discovering Latent Knowledge in Language Models Without Supervision
structure: Linear Direction · models: Llama-2-7B, Llama-2-13B, Llama-2-70B, Llama-2-7B-Chat, Llama-2-13B-Chat, Mistral-7B-v0.1 · paper: The Geometry of Truth: Emergent Linear Structure in LLM Representations of True/False Datasets, On the Universal Truthfulness Hyperplane Inside LLMs
structure: Linear Direction · models: LLaMA-7B, Alpaca-7B, Vicuna-7B · paper: Inference-Time Intervention: Eliciting Truthful Answers from a Language Model