Linear Discriminant Analysis (LDA)

methods / Dimensionality Reduction / Linear Discriminant Analysis (LDA)

Techniqueintermediate

Projects data onto the eigenmodes of (between-cluster covariance) / (within-cluster covariance), maximizing separation between labeled clusters while suppressing variation within them — used here to strip distractor directions (e.g. word length) out of concept-difference vectors before checking for parallelogram/trapezoid structure.

Used in (4 observations)

Factual and opinion sycophancy occupy partially distinct directionsmeasured

structure: Linear Direction · models: Gemma 3 12B Instruct, Llama-3.1-8B-Instruct · paper: Dissociating the Internal Representations of Sycophancy in LLMs

A language difference-of-means direction causally steers target-language outputmeasured

structure: Linear Direction · models: XLM-RoBERTa base · paper: The Geometry of Multilingual Language Model Representations

Negation is a single linear direction decodable by layer 4measured

structure: Linear Direction · models: Llama-3.1-8B, Mistral-7B-v0.1 · paper: How Language Models Process Negation

SAE features form concept crystals revealed by LDA whiteningmeasured

structure: Concept Crystals (Parallelogram/Trapezoid Structure) · models: Gemma-2-2B, Gemma-2-9B · paper: The Geometry of Concepts: Sparse Autoencoder Feature Structure