Contrast-Consistent Search finds a truth-like direction with no labels

Burns et al. introduce Contrast-Consistent Search: fit a linear probe on yes/no contrast pairs purely by a logical-consistency plus confidence loss, using no truth labels [burns-etal-2023] Across 6 models and 10 classification/QA datasets, CCS beats zero-shot accuracy by ~4% and approaches supervised logistic regression [burns-etal-2023] The direction found on one dataset transfers to structurally unrelated datasets and is often recoverable as the top principal component of contrast-pair differences [burns-etal-2023] True/false examples form two well-separated clusters along it, sometimes from as few as one contrast pair [burns-etal-2023] This unsupervised truth-like direction is broader than Marks & Tegmark's factual-statement truth direction and is kept as a separate observation [burns-etal-2023]