Definition

Given a matrix $X$ of activation vectors (rows), a conceptor is the positive semi-definite matrix

$C(R,\alpha) = R(R + \alpha^{-2}I)^{-1}, \qquad R = \frac{X^\top X}{n}$

for correlation matrix $R$ and aperture $\alpha > 0$. $C$ is first of all a positive semi-definite linear operator ($0 \preceq C \preceq I$), a spectral contraction — not itself a region. Writing $R$‘s eigenvalues as $\lambda_i$, $C$‘s eigenvalues are $\mu_i = \lambda_i/(\lambda_i + \alpha^{-2}) \in [0,1)$ for finite $\alpha$ (strictly below $1$; the value $1$ is only reached in the $\alpha\to\infty$ limit on $\mathrm{im}\,R$) — continuous, unlike an orthogonal-projection matrix (see Linear Subspace), whose eigenvalues are restricted to exactly $\{0,1\}$. The operator acts as a soft projection: $x \mapsto Cx$ shrinks $x$‘s components along low-variance directions rather than zeroing them out entirely.

Limits (corrected). As $\alpha \to 0$, $\mu_i \to 0$ so $C \to 0$ (everything is filtered). As $\alpha \to \infty$, $\mu_i \to 1$ only for $\lambda_i > 0$ and $\mu_i = 0$ whenever $\lambda_i = 0$, so $C \;\longrightarrow\; P_{\mathrm{im}\,R},$ the orthogonal projector onto the range of $R$ — not $I$ in general. $C \to I$ holds only when $R \succ 0$ (full rank). This matters for activations: when $n < d$ the empirical $R = X^\top X/n$ has rank $\le n < d$ and is therefore singular, so the large-aperture limit is a projector onto the data’s principal subspace, not the identity.

Associated region (only after a level-set is chosen). $C$ can be tied to an ellipsoid, e.g. the level set $E_C = \{x : x^\top C^{\dagger} x \le 1\}$ (using the pseudo-inverse $C^{\dagger}$, with a stated convention on $\ker C$), but that is an additional definition layered on the operator, not something $C$ is on its own.

Relative to linear-subspace

Linear Subspace‘s orthogonal projection $P_V$ is the $\alpha \to \infty$-with-hard-cutoff limiting case: a direction either fully belongs to the represented concept ($\mu_i=1$) or is fully discarded ($\mu_i=0$), with no in-between. A conceptor keeps the same basic idea — capture a $k$-dimensional-ish region relevant to a concept — but replaces the binary membership test with a graded one that also encodes each direction’s relative importance (via its eigenvalue), not just whether it belongs. Two conceptors admit Boolean operations (AND/OR/NOT) that behave like set operations on the regions they represent, with no analogue for a plain linear subspace’s binary projection beyond ordinary subspace intersection/sum.

Key evidence

Origin paper, now with its own full entry. Jaeger (2014) is the source of the conceptor construction itself, already cited above as the Key Paper for both this node and Conceptor steering (soft projection) — but is added here as its own full paper/observation entry (unlike the analogous Bricken et al. 2023 SAE-origin case, rejected as redundant elsewhere in this map) because it reports a rich, doubly-satisfying finding not yet captured anywhere else in the corpus: conceptor ellipsoids computed directly from a real N=100-unit reservoir network’s own neuron-state correlation matrix are used causally, loaded back into the same network’s recurrent dynamics, to deterministically store, morph between, and cleanly regenerate multiple distinct attractor patterns from a single shared substrate — the founding demonstration that this operator is not just a post-hoc descriptive fit but a causally load-bearing control object in the very setting (reservoir computing) it was introduced for. See jaeger-2014-conceptor-ellipsoids-computed-from-real-reservoir-network-states-enable-causal-storage-and-recall-of-multiple-attractor-patterns-in-a-single-network.

Postmus & Abreu (2024) compute a conceptor from the cached activations of in-context demonstrations of a task (e.g. antonym-generation), then steer by soft-projecting new activations through it ($h' = \beta_c C h$) instead of adding a fixed steering vector. Across five function-vector tasks (antonyms, present-past, English-French, singular-plural, country-capital) plus capitalize, on GPT-J-6B and GPT-NeoX-20B, this consistently outperforms additive steering — e.g. country-capital accuracy on GPT-J rises from 32.0% (addition) / 63.9% (addition + mean-centering) to 81.6% / 85.3% (conceptor, without/with mean-centering). Combining two conceptors for different tasks via the AND Boolean operation also outperforms averaging their corresponding additive steering vectors on composite tasks. See Conceptor steering (soft projection) for the causal-intervention mechanism itself.

Triantafyllopoulos, Cho, Tao, Miao, Rai, Ungar, Guntuku, Ryant & Sedoc (2026) extend the operator to a bipolar variant — fit by pooling activations from both poles of a concept (e.g. positive+negative sentiment) rather than one direction at a time, capturing ”>=95% of the mean-difference direction” versus ~10% for a single pole — and test it on Gemma-2-2B-IT, Gemma-2-9B-IT, and Qwen-2.5-3B-Instruct. Bipolar-conceptor steering beats both additive and diff-of-means steering on win-ratio evaluation (Gemma-2-2B-IT: 0.70-0.82 vs. baselines’ 0.50-0.62) and produces far fewer degenerate outputs (13% vs. 58% additive / 30% diff-of-means). Composing two bipolar conceptors via Boolean AND/OR/NOT shifts multi- concept response distributions in the predicted direction (e.g. Political AND-NOT Sentiment raises “Sentiment only” responses from a 0.27 base rate to 0.33) — a second, independent confirmation that the operator’s Boolean algebra tracks real behavioral composition, on different models than Postmus & Abreu’s GPT-J/GPT-NeoX. See bipolar-conceptors-outperform-additive-steering-and-compose-via-boolean-operations-across-gemma-2-and-qwen-2.5.

How to detect it

Fit a conceptor to a set of task-demonstrating activations and check whether soft-projecting through it (rather than adding their mean as a fixed vector) causally steers behavior more reliably or accurately — if so, the underlying representation is better described as a region than a single point/direction.

Key papers

• Jaeger (2014). Controlling Recurrent Neural Networks by Conceptors. — origin of the conceptor construction in reservoir computing, prior to its use on LLMs.
• Postmus & Abreu (2024). Steering Large Language Models using Conceptors: Improving Addition-Based Activation Engineering. NeurIPS 2024 MINT Workshop, arXiv:2410.16314 — first application of conceptors to LLM activation steering.