Definition

For a scalar $a$ (e.g. an integer token value), a generalized helix with $k$ frequencies is

$B(a) = \big[\,a,\ \cos(\tfrac{2\pi}{T_1}a),\ \sin(\tfrac{2\pi}{T_1}a),\ \ldots,\ \cos(\tfrac{2\pi}{T_k}a),\ \sin(\tfrac{2\pi}{T_k}a)\,\big] \in \mathbb{R}^{2k+1}$

$\mathrm{helix}(a) = C\,B(a)^\top$

where $C$ is a coefficient matrix (fit by linear regression, typically after PCA-reducing the ambient representation) mapping the $(2k+1)$-dimensional basis into the model’s activation space. $k=1$ recovers an ordinary 3-coordinate helix (one straight axis plus one circle); $k>1$ (“generalized”) superposes several periods $T_1,\ldots,T_k$ simultaneously, so the same scalar $a$ is legible both as a position along an unbounded linear axis (via the raw $a$ term) and as an angle on each of several circles of different circumference, typically chosen to include both short periods (capturing high-frequency structure) and long periods tied to a numeral system’s base (e.g. $T=100$ for base-10 magnitudes).

Intuition

A plain circle can represent a genuinely cyclic quantity (day of week, angle) but cannot by itself distinguish $a$ from $a+T,a+2T,\ldots$ — winding around the same loop indefinitely loses the “how far” information a linear axis provides. A generalized helix keeps that linear axis and adds one or more circles on top of it, the way a spring’s coil (circular cross-section) rises steadily along its length (linear axis): each point is simultaneously “this far along” and “at this angle,” and stacking several different-period circles lets the same combined coordinate disambiguate values that any single period alone would confuse (e.g. two numbers 100 apart look identical on a period-10 circle, but not on a period-100 one).

Properties

• A multi-period generalization of Circle, not a compact loop itself. Because the basis includes an unbounded linear term $a$ alongside the periodic terms, the resulting curve does not close up into a loop the way a bare circle does — it is topologically an unbounded curve, with the periodic terms providing local disambiguation/redundancy layered on top of monotonic linear structure, not full compactness.
• Superposed periods can be read out independently or jointly. Because each frequency contributes its own $(\cos,\sin)$ pair to $B(a)$, a probe or intervention can target one period’s circular subspace in isolation (as in a bare-circle claim for a single $T$) or the full generalized-helix fit across all $k$ periods jointly; the two are related but empirically distinguishable claims about how much of the representation’s structure a single period captures versus how much additional structure the other periods add.
• First quantified case in this map: numbers represented as a period-[2,5,10,100] helix in transformer residual streams, causally implicated for addition via activation patching. Kantamneni & Tegmark (2025) fit $\mathrm{helix}(a)=C\,B(a)^\top$ for token values $a\in[0,99]$ in GPT-J-6B (also Pythia-6.9B and Llama3.1-8B), identifying four dominant Fourier periods $T=[2,5,10,100]$ from the layer-0 residual stream’s own Fourier spectrum (short periods for high-frequency structure, $T=100$ for both raw signal strength and the base-10 numeral system). Activation-patching experiments that reconstruct the residual stream from a fitted geometric form and measure the resulting logit-difference recovery (higher = more of the true causal effect reproduced) find the full ($k$-frequency) helical and circular fits are the strongest tested reconstructions on the primary addition task (fitted-form logit-difference of 7.21 and 6.83 respectively, versus 8.34 for patching the true unmodified activations, 6.13 for an equal-parameter PCA-only baseline with no periodic structure, and only 3.09 for a polynomial-only fit) — direct causal evidence that the periodic (helical/circular) component of the representation, not just its magnitude-carrying linear component, is doing real computational work. The same helical fit is also causally relevant (though the helix does not always outperform a plain PCA baseline) for subtraction, integer division, multiplication, and modular arithmetic, tested across the same four-way (Full/PCA/Helix/Circle/Polynomial) patching comparison. See kantamneni-tegmark-2025-numbers-are-represented-as-a-multi-period-generalized-helix-causally-implicated-in-addition-via-the-clock-algorithm.
• The same computational mechanism can be shared by a concept whose own representation is a different shape entirely. Feucht, Haklay, Bhalla, Wurgaft, Rager, Sarfati, Merullo, McGrath, Lewis, Lubana, Fel & Geiger (2026) find that Llama-3.1-8B’s cyclic concepts (months, weekdays, hours) are represented as circles (see Circle) at the input-token position, but the offset arithmetic needed to answer “six months after August” is computed via the same base-10 Fourier-period addition machinery as plain integer addition — periods $T\in\{2,5,10,20,50,100\}$ (extending Kantamneni & Tegmark’s $T\in\{2,5,10,100\}$ with $T=20,50$), localized to a shared, sparse 28-neuron circuit (~0.2% of the MLP) at layer 18, active across addition, months, weekdays, and hours alike. Cross-task activation patching (addition activations into cyclic-task forward passes and vice versa) directly demonstrates the shared mechanism, and the circular representation of the output concept only re-emerges very late (layers 22-25), well after the underlying arithmetic has already been computed via the helical/Fourier route — a dissociation between what a concept is represented as and what geometry is used to compute with it, not a claim that the helix and the circle are the same object. See feucht-haklay-etal-2026-cyclic-concept-arithmetic-is-computed-via-a-shared-base-10-fourier-addition-circuit-not-a-modular-clock-mechanism-dissociating-representational-and-computational-geometry.
• Distinguishable from, and an explicit generalization/synthesis of, prior single-period claims for numeric/cyclic quantities in this map. Where other observations in this map document a single circular period for a specific bounded cyclic quantity (e.g. days of the week, a 12-hour clock face), the generalized-helix claim is that an unbounded, non-cyclic quantity (an arbitrary integer) is represented via several periods simultaneously plus a linear term — a genuinely different structure from, not a relabeling of, a single-period circle claim, and only reduces to a bare circle in the special case $k=1$ with the linear term’s coefficient in $C$ negligible.

Exercises

Base

1. For $k=1$ with period $T$, write out $B(a)$ explicitly and explain in what sense this is “an ordinary helix.”

2. Why can a bare period-10 circle alone not distinguish $a=7$ from $a=17$, while a generalized helix with periods $T=[10,100]$ can?

Middle

3. Kantamneni & Tegmark’s activation-patching comparison finds the full helical fit (7.21) underperforms the plain PCA baseline (6.13) is false on the primary addition task but is true on some other arithmetic tasks (e.g. integer division, where Helix scores 4.55 against PCA’s 5.24). What would this pattern suggest about whether the helical structure is used uniformly across all arithmetic operations?