In Gemma 3 4B IT, a linear truth direction and a linear impossibility direction are close to orthogonal, and the impossibility direction partially overlaps (while remaining distinguishable from) a semantic-anomaly direction, so necessary falsehoods lie closer to semantic anomaly than to contingent falsehood

- Lee (arXiv v1) reports an exploratory activation study of Gemma 3 4B IT on 160 prompts (85 philosophical prompts plus a 75-statement modality set of 15 topics x 5 conditions; the probes below are trained on the modality set with whole topic families held out): a linear truth probe separates impossible from true statements (AUC 0.93) but not impossible from false statements (AUC 0.20), whereas an impossibility probe evaluated on held-out topic families separates necessary from contingent falsehood (AUC 1.00, peaking at layer 15 with balanced accuracy 0.97). [lee-2026-falsehood-impossibility-directions] - The truth and impossibility directions are close to orthogonal (absolute cosine at most 0.12 at every depth beyond 10) and the impossibility direction partially overlaps a semantic-anomaly direction (cosine around 0.4 in the middle layers, while remaining distinguishable); sparse-autoencoder features at the same layer repeat this pattern. The author stresses these are correlational observations from one small model, with surface TF-IDF baselines of 0.67-0.77, so they are a directional result. [lee-2026-falsehood-impossibility-directions] - Version note: these results are from arXiv v1 (title 'Falsehood and Impossibility Are Different Directions in an AI's Representation of Language'). The later v2 (18 Aug 2026) is retitled 'When Language Describes the Impossible: Internal Representations of Contradiction and Paradox in Gemma 3' and reports only coherent-vs-noncoherent group-held-out probes (peak balanced accuracy 0.79 at layer 18) and exploratory SAE features on the 85 philosophical prompts; it does not contain the truth/impossibility dissociation or the orthogonality result. [lee-2026-falsehood-impossibility-directions]