Persistent homology (Vietoris-Rips barcodes) on the point cloud of convolutional filter weights confirmed a persistent one-dimensional loop (a circle) recurring at nearly all depths (layers 1-13) across roughly 1,000 real trained CNNs on MNIST, CIFAR-10, SVHN, and pretrained VGG16/VGG19 on ImageNet [carlsson-bruel-gabrielsson-2018-topological-approaches-to-deep-learning][bruel-gabrielsson-carlsson-2019-exposition-and-interpretation-of-the-topology-of-neural-networks] Weaker two- or three-circle configurations also recur in some layers and datasets; both papers loosely call this motif a "Klein bottle" only as an inherited label from Carlsson et al. (2008)'s natural-image-patch study, without independently computing non-orientability (no H2 or orientability check on the weights themselves) [carlsson-bruel-gabrielsson-2018-topological-approaches-to-deep-learning][bruel-gabrielsson-carlsson-2019-exposition-and-interpretation-of-the-topology-of-neural-networks] Fixing the first convolutional layer to an idealized discretized circle, versus a random-Gaussian control and a normally-trained control, causally raised MNIST-to-SVHN transfer accuracy from 11-12% to 28% [bruel-gabrielsson-carlsson-2019-exposition-and-interpretation-of-the-topology-of-neural-networks] Appending idealized-circle-derived features to raw pixel input sped up training by a factor of 2 on MNIST and 3.5 on SVHN, and improved MNIST-to-SVHN transfer from 10% to 22% [carlsson-bruel-gabrielsson-2018-topological-approaches-to-deep-learning] Topological simplicity of the first-layer weight point cloud (persistence of its dominant loop) correlates with the trained network's test-set generalization accuracy [bruel-gabrielsson-carlsson-2019-exposition-and-interpretation-of-the-topology-of-neural-networks]

Persistent homology (Vietoris-Rips barcodes) on the point cloud of convolutional filter weights confirmed a persistent one-dimensional loop (a circle) recurring at nearly all depths (layers 1-13) across roughly 1,000 real trained CNNs on MNIST, CIFAR-10, SVHN, and pretrained VGG16/VGG19 on ImageNet [carlsson-bruel-gabrielsson-2018-topological-approaches-to-deep-learning][bruel-gabrielsson-carlsson-2019-exposition-and-interpretation-of-the-topology-of-neural-networks] Weaker two- or three-circle configurations also recur in some layers and datasets; both papers loosely call this motif a "Klein bottle" only as an inherited label from Carlsson et al. (2008)'s natural-image-patch study, without independently computing non-orientability (no H2 or orientability check on the weights themselves) [carlsson-bruel-gabrielsson-2018-topological-approaches-to-deep-learning][bruel-gabrielsson-carlsson-2019-exposition-and-interpretation-of-the-topology-of-neural-networks] Fixing the first convolutional layer to an idealized discretized circle, versus a random-Gaussian control and a normally-trained control, causally raised MNIST-to-SVHN transfer accuracy from 11-12% to 28% [bruel-gabrielsson-carlsson-2019-exposition-and-interpretation-of-the-topology-of-neural-networks] Appending idealized-circle-derived features to raw pixel input sped up training by a factor of 2 on MNIST and 3.5 on SVHN, and improved MNIST-to-SVHN transfer from 10% to 22% [carlsson-bruel-gabrielsson-2018-topological-approaches-to-deep-learning] Topological simplicity of the first-layer weight point cloud (persistence of its dominant loop) correlates with the trained network's test-set generalization accuracy [bruel-gabrielsson-carlsson-2019-exposition-and-interpretation-of-the-topology-of-neural-networks]

Wurgaft et al. fit a second manifold over Llama-3.1-8B's output token distributions (in Hellinger/sqrt-probability space) alongside the residual-stream concept manifold [wurgaft-etal-2026-manifold-steering] Geodesic distances between the activation and behavior manifolds correlate at r=0.99 (weekdays), 0.89 (months), 0.999 (letters), and 0.999 (ages), far above the straight-line Euclidean distances (0.89/0.53/0.71/0.36) [wurgaft-etal-2026-manifold-steering] Manifold steering along the activation geodesic produces natural on-manifold behavioral trajectories where linear steering does not, and a pullback test recovers the activation manifold from a target behavior trajectory [wurgaft-etal-2026-manifold-steering] The approach generalizes to synthetic multi-dimensional concept spaces and a Mountain Car visual world-model task [wurgaft-etal-2026-manifold-steering]

Feucht, Haklay et al. confirm Llama-3.1-8B represents cyclic concepts (months, weekdays, hours) as circles at the input-token position [feucht-haklay-etal-2026-arithmetic-in-the-wild] Distributed alignment search causally isolates the input concept and offset (rank k=8-16) with over 95% accuracy before layer 18, but the circular representation is not recoverable at the final-token position until layers 22-25 [feucht-haklay-etal-2026-arithmetic-in-the-wild] Offset arithmetic is computed with the same base-10 Fourier periods (T in {2,5,10,20,50,100}) found for plain integer addition, not a mod-12/mod-7/mod-24 clock mechanism [feucht-haklay-etal-2026-arithmetic-in-the-wild] Cross-task activation patching between addition and cyclic tasks recovers the pre-modulo sum, and the computation localizes to a shared 28-neuron layer-18 circuit (~0.2% of the MLP) [feucht-haklay-etal-2026-arithmetic-in-the-wild] This dissociates representational geometry (a circle) from computational geometry (base-10 Fourier addition) for the same concept [feucht-haklay-etal-2026-arithmetic-in-the-wild]

Singh & Chopra extract per-layer last-token activations from four open LLMs (LLaMA-3-8B, LLaMA-3.2-3B, Gemma-7B, Qwen3-4B) for color, pitch, taste and emotion stimuli, with no probing or training [singh-chopra-2026-transient-perceptual-geometry] Each layer's 2D MDS qualia map is aligned to independent human ground truth via RSA (Spearman) and generalized Procrustes analysis, with bootstrapped 95% CIs over 1000 iterations [singh-chopra-2026-transient-perceptual-geometry] Color forms a circular manifold resembling the human color wheel, pitch a smooth arc, and emotion an affective manifold aligned with the valence-arousal structure [singh-chopra-2026-transient-perceptual-geometry] Alignment follows a rise-peak-fall trajectory across depth (weak early, strongest in intermediate layers, attenuated late), consistent across all four models and domains [singh-chopra-2026-transient-perceptual-geometry] Emotion is a partial exception, staying comparatively stable once peaked, while taste shows a noisier, faster-decaying trajectory [singh-chopra-2026-transient-perceptual-geometry] Exact per-layer RSA/GPA values live only in figures not extractable from the fetched source, so this entry uses the paper's quoted qualitative claims [singh-chopra-2026-transient-perceptual-geometry]

Park et al. (ICLR 2025) define an arbitrary graph (grid or ring) over semantically unrelated concept tokens in-context and feed random-walk traces over it as exemplars [park-etal-2025-icl-representations] As context length scales, windowed activations projected onto top principal components suddenly reproduce the specified graph's topology rather than the pretrained semantic clustering of those tokens [park-etal-2025-icl-representations] A Dirichlet-energy metric against the ground-truth graph decreases with context size, quantifying the sudden reorganization, with a spectral-embedding argument linking it to the top PCA components [park-etal-2025-icl-representations] When reference concepts already carry correlated pretrained semantics (e.g. weekdays), the in-context graph appears only in PC3-4 rather than overriding the pretrained ring in PC1-2 [park-etal-2025-icl-representations] Intervening on concept projections along the identified principal components changes next-node prediction accuracy, a causal check beyond the PCA visualization (exact effect sizes not captured here) [park-etal-2025-icl-representations] Tested mainly on Llama-3.1-8B and replicated on Llama-3.2-1B, Llama-3.1-8B-Instruct, Gemma-2-2B and Gemma-2-9B [park-etal-2025-icl-representations]

Nishi et al. confirm via Isomap that pretrained transformers (Llama-3.1-405B-Instruct, GPT-2-Small, Mistral-7B) encode cyclic concepts like months and weekdays as genuine cyclic manifolds, not merely approximate clusters [nishi-etal-2024-representation-shattering-in-transformers] They train a synthetic transformer on a structured knowledge graph and quantify edit-induced distortion with a metric R(D*) = ||D*-D_empty||_F / ||D_empty||_F over pairwise-distance matrices [nishi-etal-2024-representation-shattering-in-transformers] Applying ROME or MEMIT edits measurably shatters the cyclic-manifold structure, distorting the relative positions of non-targeted entities [nishi-etal-2024-representation-shattering-in-transformers] The distortion scales with the edit's counterfactual distance from the targeted fact and degrades factual recall and downstream reasoning [nishi-etal-2024-representation-shattering-in-transformers] The effect is replicated with naturalistic corroboration on real pretrained Llama-3-8B-Instruct and Mamba, not just the synthetic model [nishi-etal-2024-representation-shattering-in-transformers]

Levy & Geva show Llama-3-8B and GPT-4o make numerical errors scattered by digit-edit-distance rather than distributed around the true value, suggesting digit-wise rather than value-space representation [levy-geva-2025-digit-representations-base-10] Circular probes fitting [cos(2*pi*x/b), sin(2*pi*x/b)] per digit reconstruct digits best in base 10 (91% Llama, 92% Mistral), far above every other base (all under ~20%) and above a matched linear probe [levy-geva-2025-digit-representations-base-10] Each digit position occupies its own circle, orthogonal to the others, in both models despite different tokenization (Llama whole-number, Mistral per-digit) [levy-geva-2025-digit-representations-base-10] Rotating a hidden state by +5 mod 10 along one digit's circular-probe directions shifts the generated digit (exact hit up to 15%, off-by-one up to 50%), versus under 1% for a matched linear intervention and 0.1% random [levy-geva-2025-digit-representations-base-10] Each digit's code is a single base-10 circle (one dominant frequency), distinct from the multi-frequency Fourier findings for addition computation [levy-geva-2025-digit-representations-base-10]

Bhalla et al. formalize manifold capture (a small fixed group of decoder atoms spanning the manifold, consistently reselected by the encoder) and identify three regimes: compact capture, tiling/shattering, and intermediate dilution [bhalla-etal-2026] Training five SAE architectures on Llama-3.1-8B layer-19 activations, variance explained by a restricted atom group plateaus well beyond each manifold's ambient dimension, so none achieve compact capture [bhalla-etal-2026] Features behave like overlapping population-code tuning curves that redundantly tile the manifold, placing all tested SAEs in the dilution regime [bhalla-etal-2026] Ising-coactivation analysis on SAE codes recovers known manifolds (temperature, colors, political bias) unsupervised and surfaces a new epistemic-uncertainty manifold [bhalla-etal-2026]

Cyclical temporal concepts (days of the week, months) are each independently represented as points on a circle in the residual stream, linearly decodable from activations [karkada-etal-2026] Despite the "toroidal" id, the confirmed finding is a single circle per cyclic variable, not a joint torus; no cited paper shows two such circles coexisting as orthogonal factors of one representation [karkada-etal-2026] Karkada et al. independently derive the same single-circle geometry analytically from translation-symmetric co-occurrence statistics [karkada-etal-2026] Engels et al. discover the same circles in GPT-2-small (layer 7) and Mistral-7B by clustering SAE dictionary elements, and causally validate them via activation patching on Mistral-7B and Llama-3-8B, where patching to a rotated point changes a day/month arithmetic answer [engels-etal-2024] Tiblias et al. recover the same circular date/month structure via a supervised distance-fit (sMDS) across Qwen2.5-3B, Llama-3.2-3B, and Gemma-2-2B, persisting at 3B/8B/70B Llama scale [tiblias-etal-2025] The circle is task-dependent: the identical date context collapses into a non-cyclic clustered or linear manifold when the completion cue asks for season or temperature rather than recency [tiblias-etal-2025] It is causally necessary: noise confined to the located 2D subspace degrades temporal-reasoning accuracy as much as noise across the full residual stream, while an equally-sized random subspace has negligible effect [tiblias-etal-2025] Bhalla et al. reproduce the days-of-week circle in Llama-3.1-8B (layer 19) via PCA and causally steer along its principal components to shift the predicted day-of-week token smoothly [bhalla-etal-2026]

Sun et al. recover two near-orthogonal valence/arousal axes inside Llama-3.1-8B-Instruct (replicated in Qwen3-8B/14B) by PCA-projecting contrastive emotion-steering vectors and ridge-regressing against human ratings [sun-etal-2026-valence-arousal] Projecting the emotion vectors onto this plane traces a circle: a circularity statistic reaches 2.76-4.08 with fitted radii ~0.37-0.39, analogous to Russell's circumplex [sun-etal-2026-valence-arousal] The valence axis recovers self-reported valence at r=0.97 and the NRC-VAD lexicon at r=0.71, with cross-model valence agreement r=0.95 [sun-etal-2026-valence-arousal] Adding valence/arousal directions at specific circle angles produces dose-dependent, angle-specific shifts in generated-text affect (e.g. 0deg: delta-valence +0.75; 180deg: -0.73) [sun-etal-2026-valence-arousal] The same arousal axis causally controls refusal (20%->86% on OKTest) and sycophancy, with random-direction controls within 2-3 points of baseline [sun-etal-2026-valence-arousal] Logit-clamping and top-neuron ablation along the direction crash refusal while preserving MATH-500/IFEval, and an independent refusal direction is near-orthogonal (86.5deg) to the VA plane [sun-etal-2026-valence-arousal] Van der Ben et al. independently replicate the valence/arousal PCA structure (PC1-valence r=0.72-0.83, PC2-arousal r=0.21-0.45) in Apertus-8B and Gemma-4-E4B-it without computing circularity or steering [vanderben-etal-2026-emotion-vectors-open-source-llms] They add that cross-architecture layer-depth trajectories diverge sharply (a 3-phase plateau in Apertus vs a smooth gradient in Gemma via linear CKA) [vanderben-etal-2026-emotion-vectors-open-source-llms]

Sadek & Bakarji train an autoencoder on J.S. Bach's Well-Tempered Clavier (24 preludes and fugues per book, all 24 major/minor keys) with reconstruction loss only, using no harmonic labels [sadek-bakarji-2026-circle-of-fifths] Projected onto the first two principal components, pieces cluster hierarchically by piece and by key, and the key clusters arrange into the classical circle-of-fifths ordering [sadek-bakarji-2026-circle-of-fifths] Relative major-minor key pairs (sharing a pitch collection) sit more than three times closer in latent space than non-relative pairs [sadek-bakarji-2026-circle-of-fifths] Classical circle-of-fifths distance between keys correlates strongly with learned latent distance, so the structure is a byproduct of compression rather than a supervised target [sadek-bakarji-2026-circle-of-fifths] Full text was inaccessible (OpenReview challenge, unparseable PDF mirrors), so claims rest on the peer-reviewed PMLR abstract and no causal-validation claim is made [sadek-bakarji-2026-circle-of-fifths]

Prieto et al. introduce Bag-of-Words Superposition (BOWS): tied-weight linear/ReLU autoencoders (W in R^{m x d}) reconstructing binary bag-of-words vectors of WikiText-103/OpenWebText, vocabulary d=10,000, latent widths m 200-1000 [prieto-etal-2026] They derive that a linear autoencoder's loss-optimal W^T W is the orthogonal projector onto the feature covariance's top-m principal components, making interference proportional to a feature's own signal rather than adversarial noise when Sigma is near-low-rank [prieto-etal-2026] A weight-norm identity (||W||_F^2 = m in the constructive regime vs ~d in the classical near-orthogonal ReLU-filtered regime) shows weight decay favors constructive packing whenever m << d [prieto-etal-2026] Under a tight bottleneck (m<6), autoencoder weight columns recover a clean circular arrangement of month-words tracking Sigma's top-2 eigenvectors (linear-decoder R^2=0.98+/-0.00015) [prieto-etal-2026] Under strong compression (m=200) or explicit weight decay, weight columns form semantically coherent clusters (verbs, proper names, sports); this clustering weakens as m grows toward 800 and reappears when weight decay is added [prieto-etal-2026] A custom single-block causal toy transformer (one encoder block, 8 heads, d_model=768, vocabulary 16,000) trained on a seen-so-far multi-hot token-recovery objective reproduces the same clusters and circular structure in its residual stream [prieto-etal-2026] The paper stresses this constructive-packing mechanism is distinct from value-coding (a single computed quantity linearly encoded), since both can yield circular geometry but only the former evidences constructive interference [prieto-etal-2026]

Prieto et al. introduce Bag-of-Words Superposition (BOWS): tied-weight linear/ReLU autoencoders (W in R^{m x d}) reconstructing binary bag-of-words vectors of WikiText-103/OpenWebText, vocabulary d=10,000, latent widths m 200-1000 [prieto-etal-2026] They derive that a linear autoencoder's loss-optimal W^T W is the orthogonal projector onto the feature covariance's top-m principal components, making interference proportional to a feature's own signal rather than adversarial noise when Sigma is near-low-rank [prieto-etal-2026] A weight-norm identity (||W||_F^2 = m in the constructive regime vs ~d in the classical near-orthogonal ReLU-filtered regime) shows weight decay favors constructive packing whenever m << d [prieto-etal-2026] Under a tight bottleneck (m<6), autoencoder weight columns recover a clean circular arrangement of month-words tracking Sigma's top-2 eigenvectors (linear-decoder R^2=0.98+/-0.00015) [prieto-etal-2026] Under strong compression (m=200) or explicit weight decay, weight columns form semantically coherent clusters (verbs, proper names, sports); this clustering weakens as m grows toward 800 and reappears when weight decay is added [prieto-etal-2026] A custom single-block causal toy transformer (one encoder block, 8 heads, d_model=768, vocabulary 16,000) trained on a seen-so-far multi-hot token-recovery objective reproduces the same clusters and circular structure in its residual stream [prieto-etal-2026] The paper stresses this constructive-packing mechanism is distinct from value-coding (a single computed quantity linearly encoded), since both can yield circular geometry but only the former evidences constructive interference [prieto-etal-2026]

Chughtai, Chan & Nanda train small MLPs and transformers on finite-group composition tasks and present a representation-theoretic (Fourier/irrep) algorithm for how such networks implement group composition [chughtai-chan-nanda-2023-toy-model-of-universality] Reverse-engineering real trained weights and logits, plus ablation experiments, confirms networks consistently learn this family of algorithms [chughtai-chan-nanda-2023-toy-model-of-universality] Evidence for universality is mixed: the theory fully characterizes the space of possible circuits, but the specific circuit and its training order vary across differently-initialized networks [chughtai-chan-nanda-2023-toy-model-of-universality] The algorithm covers non-abelian groups (e.g. S5, S6, A5) via higher-dimensional irreps, of which the cyclic circle is only a special sub-case [chughtai-chan-nanda-2023-toy-model-of-universality]

A 2026 study (arXiv:2607.29086) feeds isolated musical notes across a wide range of instrument timbres into real generative music foundation models (Jukebox and MERT, with consistent results reported for MusicGen), extracting layer-wise representations and fitting the combined linear-pitch-height-plus-circular-chroma (helical) geometry via PCA [do-music-foundation-models-embed-pitch-in-helical-structure-2026] The helix's clarity and rotations-per-octave depend systematically on instrument timbre -- e.g. Hammond organ yields a clean one-rotation-per-octave helix while electric clean guitar yields two rotations per octave -- with the effect traced to specific harmonics of the input signal, not merely to which model or layer is probed [do-music-foundation-models-embed-pitch-in-helical-structure-2026]

A 2026 study (arXiv:2607.29086) feeds isolated musical notes across a wide range of instrument timbres into real generative music foundation models (Jukebox and MERT, with consistent results reported for MusicGen), extracting layer-wise representations and fitting the combined linear-pitch-height-plus-circular-chroma (helical) geometry via PCA [do-music-foundation-models-embed-pitch-in-helical-structure-2026] The helix's clarity and rotations-per-octave depend systematically on instrument timbre -- e.g. Hammond organ yields a clean one-rotation-per-octave helix while electric clean guitar yields two rotations per octave -- with the effect traced to specific harmonics of the input signal, not merely to which model or layer is probed [do-music-foundation-models-embed-pitch-in-helical-structure-2026]

Sivasankar introduces Frequency Synchronization Degree (FSD), a permutation-tested metric of Fourier-circuit synchronization across a modular-addition transformer's embedding, MLP and unembedding components [sivasankar-2026-circuit-synchronization-precedes-generalization] The model is a 2-layer transformer (d_model=128, 4 heads, d_mlp=512) trained across five primes p in {53,71,97,113,131} over 11 configurations and 3 seeds [sivasankar-2026-circuit-synchronization-precedes-generalization] FSD reaches its ceiling 500-3,000 training steps before grokking (mean lead +1,722 steps, sign-test p~0.004), so the Fourier circuit is computation-complete well before generalization [sivasankar-2026-circuit-synchronization-precedes-generalization] Forking training at the FSD ceiling with higher weight decay (lambda in {1,2,3,4,5,10}) produces monotonically earlier grokking (delta-t proportional to 1/lambda), causally tying synchronization to accelerated generalization [sivasankar-2026-circuit-synchronization-precedes-generalization]

Nanda et al. show a 1-layer transformer trained on (a+b) mod 113 concentrates its embedding norm on 5 key Fourier frequencies, each embedding inputs as rotations that combine via trig identities into addition on the circle [nanda-etal-2023-grokking] Ablating all but the 5 key frequencies improves loss by 70% while ablating non-key frequencies does nothing, and Fourier-derived progress measures reveal grokking as three overlapping phases (memorization, circuit formation, cleanup) [nanda-etal-2023-grokking] Zhong et al. show Clock is one point in a wider phase space: networks split between the Clock circuit (multiplicative, needs attention) and a new Pizza circuit (absolute-value, a plain ReLU MLP), classified by gradient symmetricity and distance irrelevance [zhong-etal-2023] Circle-isolation interventions further reveal parallel Pizza ensembles and antipodal-pair-compensating mechanisms [zhong-etal-2023] Moisescu-Pareja et al. prove (Theorem 4.1) the first-layer representation is a torus T^2 when two phase variables are independent, or a rank-2 disc when perfectly correlated, with the disc always a linear projection of the torus [moisescu-pareja-etal-2025] Using PCA and persistent homology across 703 toy networks, Clock, Pizza, and MLP-Add are nearly indistinguishable and all learn the disc/vector-addition manifold, restoring the universality hypothesis [moisescu-pareja-etal-2025] Only MLP-Concat learns the genuine torus at layer 1, which later layers project to the same disc, so different architectures encode the same algorithm at different compression [moisescu-pareja-etal-2025]

Tang et al. apply persistent homology (Vietoris-Rips, degrees 0-1) to token-embedding and hidden-state point clouds from modular-addition models mod {113,149,197}, using a 2-layer transformer and a 3-hidden-layer MLP [tang-etal-2026-topological-signatures-of-grokking] H1 (loop/cycle) persistence rises sharply and reproducibly at the grokking transition (e.g. transformer p=197 max persistence ~0.075 to 0.20-0.25) [tang-etal-2026-topological-signatures-of-grokking] This is an independent topological confirmation of the circular structure Nanda et al. (2023) found via Fourier analysis, now also demonstrated in an MLP architecture [tang-etal-2026-topological-signatures-of-grokking] Local intrinsic dimension (TwoNN) collapses from ~20-25 to ~5 at the transition, alongside the topological cycle strengthening [tang-etal-2026-topological-signatures-of-grokking] Persistence statistics correlate with test accuracy (max H1 rho up to 0.81; total H0 rho down to -0.91), a correlational not causal linkage [tang-etal-2026-topological-signatures-of-grokking] A label-permutation ablation breaks the topology-generalization link above ~10-20% corruption, and an MNIST control shows no sharp topological transition [tang-etal-2026-topological-signatures-of-grokking]

Singh & Chopra extract per-layer last-token activations from four open LLMs (LLaMA-3-8B, LLaMA-3.2-3B, Gemma-7B, Qwen3-4B) for color, pitch, taste and emotion stimuli, with no probing or training [singh-chopra-2026-transient-perceptual-geometry] Each layer's 2D MDS qualia map is aligned to independent human ground truth via RSA (Spearman) and generalized Procrustes analysis, with bootstrapped 95% CIs over 1000 iterations [singh-chopra-2026-transient-perceptual-geometry] Color forms a circular manifold resembling the human color wheel, pitch a smooth arc, and emotion an affective manifold aligned with the valence-arousal structure [singh-chopra-2026-transient-perceptual-geometry] Alignment follows a rise-peak-fall trajectory across depth (weak early, strongest in intermediate layers, attenuated late), consistent across all four models and domains [singh-chopra-2026-transient-perceptual-geometry] Emotion is a partial exception, staying comparatively stable once peaked, while taste shows a noisier, faster-decaying trajectory [singh-chopra-2026-transient-perceptual-geometry] Exact per-layer RSA/GPA values live only in figures not extractable from the fetched source, so this entry uses the paper's quoted qualitative claims [singh-chopra-2026-transient-perceptual-geometry]

Park et al. (ICLR 2025) define an arbitrary graph (grid or ring) over semantically unrelated concept tokens in-context and feed random-walk traces over it as exemplars [park-etal-2025-icl-representations] As context length scales, windowed activations projected onto top principal components suddenly reproduce the specified graph's topology rather than the pretrained semantic clustering of those tokens [park-etal-2025-icl-representations] A Dirichlet-energy metric against the ground-truth graph decreases with context size, quantifying the sudden reorganization, with a spectral-embedding argument linking it to the top PCA components [park-etal-2025-icl-representations] When reference concepts already carry correlated pretrained semantics (e.g. weekdays), the in-context graph appears only in PC3-4 rather than overriding the pretrained ring in PC1-2 [park-etal-2025-icl-representations] Intervening on concept projections along the identified principal components changes next-node prediction accuracy, a causal check beyond the PCA visualization (exact effect sizes not captured here) [park-etal-2025-icl-representations] Tested mainly on Llama-3.1-8B and replicated on Llama-3.2-1B, Llama-3.1-8B-Instruct, Gemma-2-2B and Gemma-2-9B [park-etal-2025-icl-representations]

Cyclical temporal concepts (days of the week, months) are each independently represented as points on a circle in the residual stream, linearly decodable from activations [karkada-etal-2026] Despite the "toroidal" id, the confirmed finding is a single circle per cyclic variable, not a joint torus; no cited paper shows two such circles coexisting as orthogonal factors of one representation [karkada-etal-2026] Karkada et al. independently derive the same single-circle geometry analytically from translation-symmetric co-occurrence statistics [karkada-etal-2026] Engels et al. discover the same circles in GPT-2-small (layer 7) and Mistral-7B by clustering SAE dictionary elements, and causally validate them via activation patching on Mistral-7B and Llama-3-8B, where patching to a rotated point changes a day/month arithmetic answer [engels-etal-2024] Tiblias et al. recover the same circular date/month structure via a supervised distance-fit (sMDS) across Qwen2.5-3B, Llama-3.2-3B, and Gemma-2-2B, persisting at 3B/8B/70B Llama scale [tiblias-etal-2025] The circle is task-dependent: the identical date context collapses into a non-cyclic clustered or linear manifold when the completion cue asks for season or temperature rather than recency [tiblias-etal-2025] It is causally necessary: noise confined to the located 2D subspace degrades temporal-reasoning accuracy as much as noise across the full residual stream, while an equally-sized random subspace has negligible effect [tiblias-etal-2025] Bhalla et al. reproduce the days-of-week circle in Llama-3.1-8B (layer 19) via PCA and causally steer along its principal components to shift the predicted day-of-week token smoothly [bhalla-etal-2026]

Sun et al. recover two near-orthogonal valence/arousal axes inside Llama-3.1-8B-Instruct (replicated in Qwen3-8B/14B) by PCA-projecting contrastive emotion-steering vectors and ridge-regressing against human ratings [sun-etal-2026-valence-arousal] Projecting the emotion vectors onto this plane traces a circle: a circularity statistic reaches 2.76-4.08 with fitted radii ~0.37-0.39, analogous to Russell's circumplex [sun-etal-2026-valence-arousal] The valence axis recovers self-reported valence at r=0.97 and the NRC-VAD lexicon at r=0.71, with cross-model valence agreement r=0.95 [sun-etal-2026-valence-arousal] Adding valence/arousal directions at specific circle angles produces dose-dependent, angle-specific shifts in generated-text affect (e.g. 0deg: delta-valence +0.75; 180deg: -0.73) [sun-etal-2026-valence-arousal] The same arousal axis causally controls refusal (20%->86% on OKTest) and sycophancy, with random-direction controls within 2-3 points of baseline [sun-etal-2026-valence-arousal] Logit-clamping and top-neuron ablation along the direction crash refusal while preserving MATH-500/IFEval, and an independent refusal direction is near-orthogonal (86.5deg) to the VA plane [sun-etal-2026-valence-arousal] Van der Ben et al. independently replicate the valence/arousal PCA structure (PC1-valence r=0.72-0.83, PC2-arousal r=0.21-0.45) in Apertus-8B and Gemma-4-E4B-it without computing circularity or steering [vanderben-etal-2026-emotion-vectors-open-source-llms] They add that cross-architecture layer-depth trajectories diverge sharply (a 3-phase plateau in Apertus vs a smooth gradient in Gemma via linear CKA) [vanderben-etal-2026-emotion-vectors-open-source-llms]

Singh & Chopra extract per-layer last-token activations from four open LLMs (LLaMA-3-8B, LLaMA-3.2-3B, Gemma-7B, Qwen3-4B) for color, pitch, taste and emotion stimuli, with no probing or training [singh-chopra-2026-transient-perceptual-geometry] Each layer's 2D MDS qualia map is aligned to independent human ground truth via RSA (Spearman) and generalized Procrustes analysis, with bootstrapped 95% CIs over 1000 iterations [singh-chopra-2026-transient-perceptual-geometry] Color forms a circular manifold resembling the human color wheel, pitch a smooth arc, and emotion an affective manifold aligned with the valence-arousal structure [singh-chopra-2026-transient-perceptual-geometry] Alignment follows a rise-peak-fall trajectory across depth (weak early, strongest in intermediate layers, attenuated late), consistent across all four models and domains [singh-chopra-2026-transient-perceptual-geometry] Emotion is a partial exception, staying comparatively stable once peaked, while taste shows a noisier, faster-decaying trajectory [singh-chopra-2026-transient-perceptual-geometry] Exact per-layer RSA/GPA values live only in figures not extractable from the fetched source, so this entry uses the paper's quoted qualitative claims [singh-chopra-2026-transient-perceptual-geometry]

Cyclical temporal concepts (days of the week, months) are each independently represented as points on a circle in the residual stream, linearly decodable from activations [karkada-etal-2026] Despite the "toroidal" id, the confirmed finding is a single circle per cyclic variable, not a joint torus; no cited paper shows two such circles coexisting as orthogonal factors of one representation [karkada-etal-2026] Karkada et al. independently derive the same single-circle geometry analytically from translation-symmetric co-occurrence statistics [karkada-etal-2026] Engels et al. discover the same circles in GPT-2-small (layer 7) and Mistral-7B by clustering SAE dictionary elements, and causally validate them via activation patching on Mistral-7B and Llama-3-8B, where patching to a rotated point changes a day/month arithmetic answer [engels-etal-2024] Tiblias et al. recover the same circular date/month structure via a supervised distance-fit (sMDS) across Qwen2.5-3B, Llama-3.2-3B, and Gemma-2-2B, persisting at 3B/8B/70B Llama scale [tiblias-etal-2025] The circle is task-dependent: the identical date context collapses into a non-cyclic clustered or linear manifold when the completion cue asks for season or temperature rather than recency [tiblias-etal-2025] It is causally necessary: noise confined to the located 2D subspace degrades temporal-reasoning accuracy as much as noise across the full residual stream, while an equally-sized random subspace has negligible effect [tiblias-etal-2025] Bhalla et al. reproduce the days-of-week circle in Llama-3.1-8B (layer 19) via PCA and causally steer along its principal components to shift the predicted day-of-week token smoothly [bhalla-etal-2026]

Sun et al. recover two near-orthogonal valence/arousal axes inside Llama-3.1-8B-Instruct (replicated in Qwen3-8B/14B) by PCA-projecting contrastive emotion-steering vectors and ridge-regressing against human ratings [sun-etal-2026-valence-arousal] Projecting the emotion vectors onto this plane traces a circle: a circularity statistic reaches 2.76-4.08 with fitted radii ~0.37-0.39, analogous to Russell's circumplex [sun-etal-2026-valence-arousal] The valence axis recovers self-reported valence at r=0.97 and the NRC-VAD lexicon at r=0.71, with cross-model valence agreement r=0.95 [sun-etal-2026-valence-arousal] Adding valence/arousal directions at specific circle angles produces dose-dependent, angle-specific shifts in generated-text affect (e.g. 0deg: delta-valence +0.75; 180deg: -0.73) [sun-etal-2026-valence-arousal] The same arousal axis causally controls refusal (20%->86% on OKTest) and sycophancy, with random-direction controls within 2-3 points of baseline [sun-etal-2026-valence-arousal] Logit-clamping and top-neuron ablation along the direction crash refusal while preserving MATH-500/IFEval, and an independent refusal direction is near-orthogonal (86.5deg) to the VA plane [sun-etal-2026-valence-arousal] Van der Ben et al. independently replicate the valence/arousal PCA structure (PC1-valence r=0.72-0.83, PC2-arousal r=0.21-0.45) in Apertus-8B and Gemma-4-E4B-it without computing circularity or steering [vanderben-etal-2026-emotion-vectors-open-source-llms] They add that cross-architecture layer-depth trajectories diverge sharply (a 3-phase plateau in Apertus vs a smooth gradient in Gemma via linear CKA) [vanderben-etal-2026-emotion-vectors-open-source-llms]

Joseph et al. analyze frozen pretrained V-JEPA 2 and VideoMAE-v2 video encoders and find motion direction carried by direction-selective MLP units whose sinusoidal tuning curves tile the full angular range at an intermediate-depth Physics Emergence Zone [joseph-etal-2026-interpreting-physics-video-world-models] A sawtooth pattern in probe accuracy under successive feature orthogonalization is consistent with paired sine/cosine encodings, a circular population code rather than a single linear direction (linear-probe R^2=0.97) [joseph-etal-2026-interpreting-physics-video-world-models] Scalar physical quantities (speed, acceleration) are linearly decodable from early layers onward, a simpler shape than the direction variable's population code [joseph-etal-2026-interpreting-physics-video-world-models] Local-attention suppression at the emergence zone drops direction-decoding R^2 (0.97->0.14) and intuitive-physics accuracy (78.3%->61.7%) while leaving ImageNet nearly unchanged (33.7%->33.1%), a clean double dissociation [joseph-etal-2026-interpreting-physics-video-world-models] Steering the direction variable required jointly manipulating dozens of orthogonal probe dimensions rather than a single vector, consistent with population-code geometry [joseph-etal-2026-interpreting-physics-video-world-models]

Joseph et al. analyze frozen pretrained V-JEPA 2 and VideoMAE-v2 video encoders and find motion direction carried by direction-selective MLP units whose sinusoidal tuning curves tile the full angular range at an intermediate-depth Physics Emergence Zone [joseph-etal-2026-interpreting-physics-video-world-models] A sawtooth pattern in probe accuracy under successive feature orthogonalization is consistent with paired sine/cosine encodings, a circular population code rather than a single linear direction (linear-probe R^2=0.97) [joseph-etal-2026-interpreting-physics-video-world-models] Scalar physical quantities (speed, acceleration) are linearly decodable from early layers onward, a simpler shape than the direction variable's population code [joseph-etal-2026-interpreting-physics-video-world-models] Local-attention suppression at the emergence zone drops direction-decoding R^2 (0.97->0.14) and intuitive-physics accuracy (78.3%->61.7%) while leaving ImageNet nearly unchanged (33.7%->33.1%), a clean double dissociation [joseph-etal-2026-interpreting-physics-video-world-models] Steering the direction variable required jointly manipulating dozens of orthogonal probe dimensions rather than a single vector, consistent with population-code geometry [joseph-etal-2026-interpreting-physics-video-world-models]

Nishi et al. confirm via Isomap that pretrained transformers (Llama-3.1-405B-Instruct, GPT-2-Small, Mistral-7B) encode cyclic concepts like months and weekdays as genuine cyclic manifolds, not merely approximate clusters [nishi-etal-2024-representation-shattering-in-transformers] They train a synthetic transformer on a structured knowledge graph and quantify edit-induced distortion with a metric R(D*) = ||D*-D_empty||_F / ||D_empty||_F over pairwise-distance matrices [nishi-etal-2024-representation-shattering-in-transformers] Applying ROME or MEMIT edits measurably shatters the cyclic-manifold structure, distorting the relative positions of non-targeted entities [nishi-etal-2024-representation-shattering-in-transformers] The distortion scales with the edit's counterfactual distance from the targeted fact and degrades factual recall and downstream reasoning [nishi-etal-2024-representation-shattering-in-transformers] The effect is replicated with naturalistic corroboration on real pretrained Llama-3-8B-Instruct and Mamba, not just the synthetic model [nishi-etal-2024-representation-shattering-in-transformers]

Nishi et al. confirm via Isomap that pretrained transformers (Llama-3.1-405B-Instruct, GPT-2-Small, Mistral-7B) encode cyclic concepts like months and weekdays as genuine cyclic manifolds, not merely approximate clusters [nishi-etal-2024-representation-shattering-in-transformers] They train a synthetic transformer on a structured knowledge graph and quantify edit-induced distortion with a metric R(D*) = ||D*-D_empty||_F / ||D_empty||_F over pairwise-distance matrices [nishi-etal-2024-representation-shattering-in-transformers] Applying ROME or MEMIT edits measurably shatters the cyclic-manifold structure, distorting the relative positions of non-targeted entities [nishi-etal-2024-representation-shattering-in-transformers] The distortion scales with the edit's counterfactual distance from the targeted fact and degrades factual recall and downstream reasoning [nishi-etal-2024-representation-shattering-in-transformers] The effect is replicated with naturalistic corroboration on real pretrained Llama-3-8B-Instruct and Mamba, not just the synthetic model [nishi-etal-2024-representation-shattering-in-transformers]

Nishi et al. confirm via Isomap that pretrained transformers (Llama-3.1-405B-Instruct, GPT-2-Small, Mistral-7B) encode cyclic concepts like months and weekdays as genuine cyclic manifolds, not merely approximate clusters [nishi-etal-2024-representation-shattering-in-transformers] They train a synthetic transformer on a structured knowledge graph and quantify edit-induced distortion with a metric R(D*) = ||D*-D_empty||_F / ||D_empty||_F over pairwise-distance matrices [nishi-etal-2024-representation-shattering-in-transformers] Applying ROME or MEMIT edits measurably shatters the cyclic-manifold structure, distorting the relative positions of non-targeted entities [nishi-etal-2024-representation-shattering-in-transformers] The distortion scales with the edit's counterfactual distance from the targeted fact and degrades factual recall and downstream reasoning [nishi-etal-2024-representation-shattering-in-transformers] The effect is replicated with naturalistic corroboration on real pretrained Llama-3-8B-Instruct and Mamba, not just the synthetic model [nishi-etal-2024-representation-shattering-in-transformers]

Modell et al. model a feature as a metric space whose representation is a homeomorphic image on a manifold in the unit sphere, and prove that under a continuous-correspondence hypothesis cosine similarity approximates a monotone function of on-manifold geodesic distance rather than ambient Euclidean distance [modell-etal-2025] The theorem is validated on known manifold features via Kendall's tau and Chatterjee's xi between predicted and true geodesic distance [modell-etal-2025] The hue circle and day-of-year circle are found in OpenAI text-embedding-3-large raw embeddings, not SAE features [modell-etal-2025] The months circle is an SAE feature in Mistral-7B and the year axis is an SAE feature in GPT-2-small layer 7 (following Engels et al. 2025), isometric only on a log scale log(2019 - year) [modell-etal-2025]

Cyclical temporal concepts (days of the week, months) are each independently represented as points on a circle in the residual stream, linearly decodable from activations [karkada-etal-2026] Despite the "toroidal" id, the confirmed finding is a single circle per cyclic variable, not a joint torus; no cited paper shows two such circles coexisting as orthogonal factors of one representation [karkada-etal-2026] Karkada et al. independently derive the same single-circle geometry analytically from translation-symmetric co-occurrence statistics [karkada-etal-2026] Engels et al. discover the same circles in GPT-2-small (layer 7) and Mistral-7B by clustering SAE dictionary elements, and causally validate them via activation patching on Mistral-7B and Llama-3-8B, where patching to a rotated point changes a day/month arithmetic answer [engels-etal-2024] Tiblias et al. recover the same circular date/month structure via a supervised distance-fit (sMDS) across Qwen2.5-3B, Llama-3.2-3B, and Gemma-2-2B, persisting at 3B/8B/70B Llama scale [tiblias-etal-2025] The circle is task-dependent: the identical date context collapses into a non-cyclic clustered or linear manifold when the completion cue asks for season or temperature rather than recency [tiblias-etal-2025] It is causally necessary: noise confined to the located 2D subspace degrades temporal-reasoning accuracy as much as noise across the full residual stream, while an equally-sized random subspace has negligible effect [tiblias-etal-2025] Bhalla et al. reproduce the days-of-week circle in Llama-3.1-8B (layer 19) via PCA and causally steer along its principal components to shift the predicted day-of-week token smoothly [bhalla-etal-2026]

Nishi et al. confirm via Isomap that pretrained transformers (Llama-3.1-405B-Instruct, GPT-2-Small, Mistral-7B) encode cyclic concepts like months and weekdays as genuine cyclic manifolds, not merely approximate clusters [nishi-etal-2024-representation-shattering-in-transformers] They train a synthetic transformer on a structured knowledge graph and quantify edit-induced distortion with a metric R(D*) = ||D*-D_empty||_F / ||D_empty||_F over pairwise-distance matrices [nishi-etal-2024-representation-shattering-in-transformers] Applying ROME or MEMIT edits measurably shatters the cyclic-manifold structure, distorting the relative positions of non-targeted entities [nishi-etal-2024-representation-shattering-in-transformers] The distortion scales with the edit's counterfactual distance from the targeted fact and degrades factual recall and downstream reasoning [nishi-etal-2024-representation-shattering-in-transformers] The effect is replicated with naturalistic corroboration on real pretrained Llama-3-8B-Instruct and Mamba, not just the synthetic model [nishi-etal-2024-representation-shattering-in-transformers]

Levy & Geva show Llama-3-8B and GPT-4o make numerical errors scattered by digit-edit-distance rather than distributed around the true value, suggesting digit-wise rather than value-space representation [levy-geva-2025-digit-representations-base-10] Circular probes fitting [cos(2*pi*x/b), sin(2*pi*x/b)] per digit reconstruct digits best in base 10 (91% Llama, 92% Mistral), far above every other base (all under ~20%) and above a matched linear probe [levy-geva-2025-digit-representations-base-10] Each digit position occupies its own circle, orthogonal to the others, in both models despite different tokenization (Llama whole-number, Mistral per-digit) [levy-geva-2025-digit-representations-base-10] Rotating a hidden state by +5 mod 10 along one digit's circular-probe directions shifts the generated digit (exact hit up to 15%, off-by-one up to 50%), versus under 1% for a matched linear intervention and 0.1% random [levy-geva-2025-digit-representations-base-10] Each digit's code is a single base-10 circle (one dominant frequency), distinct from the multi-frequency Fourier findings for addition computation [levy-geva-2025-digit-representations-base-10]

Modell et al. model a feature as a metric space whose representation is a homeomorphic image on a manifold in the unit sphere, and prove that under a continuous-correspondence hypothesis cosine similarity approximates a monotone function of on-manifold geodesic distance rather than ambient Euclidean distance [modell-etal-2025] The theorem is validated on known manifold features via Kendall's tau and Chatterjee's xi between predicted and true geodesic distance [modell-etal-2025] The hue circle and day-of-year circle are found in OpenAI text-embedding-3-large raw embeddings, not SAE features [modell-etal-2025] The months circle is an SAE feature in Mistral-7B and the year axis is an SAE feature in GPT-2-small layer 7 (following Engels et al. 2025), isometric only on a log scale log(2019 - year) [modell-etal-2025]

Cyclical temporal concepts (days of the week, months) are each independently represented as points on a circle in the residual stream, linearly decodable from activations [karkada-etal-2026] Despite the "toroidal" id, the confirmed finding is a single circle per cyclic variable, not a joint torus; no cited paper shows two such circles coexisting as orthogonal factors of one representation [karkada-etal-2026] Karkada et al. independently derive the same single-circle geometry analytically from translation-symmetric co-occurrence statistics [karkada-etal-2026] Engels et al. discover the same circles in GPT-2-small (layer 7) and Mistral-7B by clustering SAE dictionary elements, and causally validate them via activation patching on Mistral-7B and Llama-3-8B, where patching to a rotated point changes a day/month arithmetic answer [engels-etal-2024] Tiblias et al. recover the same circular date/month structure via a supervised distance-fit (sMDS) across Qwen2.5-3B, Llama-3.2-3B, and Gemma-2-2B, persisting at 3B/8B/70B Llama scale [tiblias-etal-2025] The circle is task-dependent: the identical date context collapses into a non-cyclic clustered or linear manifold when the completion cue asks for season or temperature rather than recency [tiblias-etal-2025] It is causally necessary: noise confined to the located 2D subspace degrades temporal-reasoning accuracy as much as noise across the full residual stream, while an equally-sized random subspace has negligible effect [tiblias-etal-2025] Bhalla et al. reproduce the days-of-week circle in Llama-3.1-8B (layer 19) via PCA and causally steer along its principal components to shift the predicted day-of-week token smoothly [bhalla-etal-2026]

Limberg, Zhang, Schulz & Weinzierl train a VAE (pGESAM) encoding EnCodec audio embeddings into a 2D timbre latent space disentangled from pitch, with a downstream Transformer generating audio from a chosen (timbre, pitch) pair [limberg-etal-2025-pitch-conditioned-instrument-sound-synthesis-interactive-timbre-latent-space] The 2D circular geometry is imposed by explicit loss terms, not emergent: an L2-norm hinge loss constrains latent means within the unit circle and an attractive/repulsive neighbor loss forms the instrument clusters [limberg-etal-2025-pitch-conditioned-instrument-sound-synthesis-interactive-timbre-latent-space] Disentanglement is quantified by a component-wise variance ratio: within-instrument-across-pitch variance (~1.1e-7) is six orders of magnitude below across-instrument-within-pitch variance (0.179) [limberg-etal-2025-pitch-conditioned-instrument-sound-synthesis-interactive-timbre-latent-space] V_pitch ~0.179 approaches the ~0.25 value expected for points spread near-uniformly across the unit disc [limberg-etal-2025-pitch-conditioned-instrument-sound-synthesis-interactive-timbre-latent-space] A four-configuration loss ablation attributes geometry to specific terms: removing the neighbor loss collapses V_pitch to ~2e-3 (points crowd the origin), while removing the family classifier slightly raises V_inst [limberg-etal-2025-pitch-conditioned-instrument-sound-synthesis-interactive-timbre-latent-space] A 2D scatter confirms macro-clusters by instrument family and micro-clusters by individual instrument appearing as near-single points [limberg-etal-2025-pitch-conditioned-instrument-sound-synthesis-interactive-timbre-latent-space]

Limberg, Zhang, Schulz & Weinzierl train a VAE (pGESAM) encoding EnCodec audio embeddings into a 2D timbre latent space disentangled from pitch, with a downstream Transformer generating audio from a chosen (timbre, pitch) pair [limberg-etal-2025-pitch-conditioned-instrument-sound-synthesis-interactive-timbre-latent-space] The 2D circular geometry is imposed by explicit loss terms, not emergent: an L2-norm hinge loss constrains latent means within the unit circle and an attractive/repulsive neighbor loss forms the instrument clusters [limberg-etal-2025-pitch-conditioned-instrument-sound-synthesis-interactive-timbre-latent-space] Disentanglement is quantified by a component-wise variance ratio: within-instrument-across-pitch variance (~1.1e-7) is six orders of magnitude below across-instrument-within-pitch variance (0.179) [limberg-etal-2025-pitch-conditioned-instrument-sound-synthesis-interactive-timbre-latent-space] V_pitch ~0.179 approaches the ~0.25 value expected for points spread near-uniformly across the unit disc [limberg-etal-2025-pitch-conditioned-instrument-sound-synthesis-interactive-timbre-latent-space] A four-configuration loss ablation attributes geometry to specific terms: removing the neighbor loss collapses V_pitch to ~2e-3 (points crowd the origin), while removing the family classifier slightly raises V_inst [limberg-etal-2025-pitch-conditioned-instrument-sound-synthesis-interactive-timbre-latent-space] A 2D scatter confirms macro-clusters by instrument family and micro-clusters by individual instrument appearing as near-single points [limberg-etal-2025-pitch-conditioned-instrument-sound-synthesis-interactive-timbre-latent-space]

Modell et al. model a feature as a metric space whose representation is a homeomorphic image on a manifold in the unit sphere, and prove that under a continuous-correspondence hypothesis cosine similarity approximates a monotone function of on-manifold geodesic distance rather than ambient Euclidean distance [modell-etal-2025] The theorem is validated on known manifold features via Kendall's tau and Chatterjee's xi between predicted and true geodesic distance [modell-etal-2025] The hue circle and day-of-year circle are found in OpenAI text-embedding-3-large raw embeddings, not SAE features [modell-etal-2025] The months circle is an SAE feature in Mistral-7B and the year axis is an SAE feature in GPT-2-small layer 7 (following Engels et al. 2025), isometric only on a log scale log(2019 - year) [modell-etal-2025]

McCracken et al. propose that networks solving modular addition universally implement an approximate Chinese Remainder Theorem algorithm built from approximate cosets [mccracken-etal-2025-universal-abstract-algorithm-modular-addition] Across neurons, neuron clusters, and whole trained MLPs and transformers, they measure that neurons activate on approximate cosets in early layers or on linear combinations of them in deeper layers, rather than exclusively on single cosets [mccracken-etal-2025-universal-abstract-algorithm-modular-addition] The measured feature count matches the theory's predicted O(log n) scaling [mccracken-etal-2025-universal-abstract-algorithm-modular-addition] The work extends the Fourier/group-representation-theoretic universality cluster of Nanda et al. (2023) and Chughtai et al. (2023) with a coset-structure measurement, and performs no causal intervention [mccracken-etal-2025-universal-abstract-algorithm-modular-addition]

Nanda et al. show a 1-layer transformer trained on (a+b) mod 113 concentrates its embedding norm on 5 key Fourier frequencies, each embedding inputs as rotations that combine via trig identities into addition on the circle [nanda-etal-2023-grokking] Ablating all but the 5 key frequencies improves loss by 70% while ablating non-key frequencies does nothing, and Fourier-derived progress measures reveal grokking as three overlapping phases (memorization, circuit formation, cleanup) [nanda-etal-2023-grokking] Zhong et al. show Clock is one point in a wider phase space: networks split between the Clock circuit (multiplicative, needs attention) and a new Pizza circuit (absolute-value, a plain ReLU MLP), classified by gradient symmetricity and distance irrelevance [zhong-etal-2023] Circle-isolation interventions further reveal parallel Pizza ensembles and antipodal-pair-compensating mechanisms [zhong-etal-2023] Moisescu-Pareja et al. prove (Theorem 4.1) the first-layer representation is a torus T^2 when two phase variables are independent, or a rank-2 disc when perfectly correlated, with the disc always a linear projection of the torus [moisescu-pareja-etal-2025] Using PCA and persistent homology across 703 toy networks, Clock, Pizza, and MLP-Add are nearly indistinguishable and all learn the disc/vector-addition manifold, restoring the universality hypothesis [moisescu-pareja-etal-2025] Only MLP-Concat learns the genuine torus at layer 1, which later layers project to the same disc, so different architectures encode the same algorithm at different compression [moisescu-pareja-etal-2025]

Mathewson probes NLLB-200 (distilled 600M), a 200-language encoder-decoder translation Transformer, extending coverage beyond decoder-only LLMs [mathewson-2026-nllb-multilingual-geometry] Embedding the 11 Berlin & Kay basic color terms across 136 languages, PCA's first two components recover a circular hue arrangement with a warm/cool opposition matching color-wheel structure [mathewson-2026-nllb-multilingual-geometry] A third principal component forms an axis orthogonal to the hue plane separating achromatic terms (white/black/grey) from the chromatic ring [mathewson-2026-nllb-multilingual-geometry] The shape (hue circle plus orthogonal lightness axis) matches the Llama-3.1-8B color paraboloid qualitatively, but here the two dimensions are orthogonally separate rather than fused into a paraboloid [mathewson-2026-nllb-multilingual-geometry] The finding is purely representational (PCA projection only), with no causal steering, across 136 typologically diverse languages [mathewson-2026-nllb-multilingual-geometry]

Carvalho & Bernardes train an LSTM-VAE on 371 Bach chorales, comparing six input encodings (piano roll, MIDI-like, ABC, Tonnetz, and two DFT-based) for circle-of-fifths alignment of the PCA-reduced latent space [carvalho-bernardes-2023-tonal-vae-latent-spaces] Alignment is quantified via Davis-Bouldin, Dunn Index, and a circular Kendall's Tau against the true circle-of-fifths sequence [carvalho-bernardes-2023-tonal-vae-latent-spaces] Pitch DFT wins on all three metrics (Kendall's Tau .44+/-.32 vs next-best .15+/-.14), with ~16% of chorales reaching |Tau|>.9, disproportionately major-key and longer pieces [carvalho-bernardes-2023-tonal-vae-latent-spaces] Tonal alignment dissociates from reconstruction quality: ABC reconstructs best (82.6%) but aligns weakly (Tau .11+/-.09), while worse-reconstructing Pitch DFT (75.6%) aligns best [carvalho-bernardes-2023-tonal-vae-latent-spaces] The choice of input encoding materially determines whether circle-of-fifths geometry emerges at all; the study is entirely correlational with no intervention [carvalho-bernardes-2023-tonal-vae-latent-spaces]

Swaroop shows ReLU MLPs trained on modular arithmetic (one hidden layer, width 256, modulus 97) develop near-binary square-wave input weights, with intermediate values only near sign-change boundaries [swaroop-2026-latent-algorithmic-structure-precedes-grokking] Output weights have dominant Fourier phases satisfying phi_out = phi_a + phi_b, with frequency and phase extracted per-neuron via a discrete Fourier transform [swaroop-2026-latent-algorithmic-structure-precedes-grokking] An idealized MLP built purely from these extracted square-wave/cosine components reaches 95.5% accuracy even when derived from a model that itself scores only 0.23% [swaroop-2026-latent-algorithmic-structure-precedes-grokking] This is direct evidence that the Fourier/geometric weight structure is the causally load-bearing algorithmic content, present before the model generalizes [swaroop-2026-latent-algorithmic-structure-precedes-grokking]

Grasse & Tata test whether chroma equivalence (the circular, octave-periodic component of pitch, the Shepard/Drobisch helix, distinct from linear pitch height) emerges in trained audio networks via representational similarity analysis on NSynth notes [grasse-tata-2026-chroma-equivalence-anns] All self-supervised pretrained models (Wav2Vec 2.0, Data2Vec, others) show significant pitch-height encoding but none show significant chroma equivalence, including after self-supervised fine-tuning on a speech+music mixture [grasse-tata-2026-chroma-equivalence-anns] Mere exposure to music in the training data is therefore not sufficient to induce chroma equivalence [grasse-tata-2026-chroma-equivalence-anns] Supervised fine-tuning on polyphonic piano-note transcription (MAESTRO) does induce significant chroma equivalence in both base models, while fine-tuning on speech recognition (a control) does not [grasse-tata-2026-chroma-equivalence-anns] A hard-coded constant-Q transform baseline trivially exhibits chroma equivalence, serving as a positive control for the RSA methodology; no causal intervention is performed [grasse-tata-2026-chroma-equivalence-anns] Exact Spearman coefficients live only in bar-chart figures not recoverable from the text, so the entry rests on the pre-registered-model RSA and Bonferroni-corrected significance testing [grasse-tata-2026-chroma-equivalence-anns]

Grasse & Tata test whether chroma equivalence (the circular, octave-periodic component of pitch, the Shepard/Drobisch helix, distinct from linear pitch height) emerges in trained audio networks via representational similarity analysis on NSynth notes [grasse-tata-2026-chroma-equivalence-anns] All self-supervised pretrained models (Wav2Vec 2.0, Data2Vec, others) show significant pitch-height encoding but none show significant chroma equivalence, including after self-supervised fine-tuning on a speech+music mixture [grasse-tata-2026-chroma-equivalence-anns] Mere exposure to music in the training data is therefore not sufficient to induce chroma equivalence [grasse-tata-2026-chroma-equivalence-anns] Supervised fine-tuning on polyphonic piano-note transcription (MAESTRO) does induce significant chroma equivalence in both base models, while fine-tuning on speech recognition (a control) does not [grasse-tata-2026-chroma-equivalence-anns] A hard-coded constant-Q transform baseline trivially exhibits chroma equivalence, serving as a positive control for the RSA methodology; no causal intervention is performed [grasse-tata-2026-chroma-equivalence-anns] Exact Spearman coefficients live only in bar-chart figures not recoverable from the text, so the entry rests on the pre-registered-model RSA and Bonferroni-corrected significance testing [grasse-tata-2026-chroma-equivalence-anns]

Tang et al. apply persistent homology (Vietoris-Rips, degrees 0-1) to token-embedding and hidden-state point clouds from modular-addition models mod {113,149,197}, using a 2-layer transformer and a 3-hidden-layer MLP [tang-etal-2026-topological-signatures-of-grokking] H1 (loop/cycle) persistence rises sharply and reproducibly at the grokking transition (e.g. transformer p=197 max persistence ~0.075 to 0.20-0.25) [tang-etal-2026-topological-signatures-of-grokking] This is an independent topological confirmation of the circular structure Nanda et al. (2023) found via Fourier analysis, now also demonstrated in an MLP architecture [tang-etal-2026-topological-signatures-of-grokking] Local intrinsic dimension (TwoNN) collapses from ~20-25 to ~5 at the transition, alongside the topological cycle strengthening [tang-etal-2026-topological-signatures-of-grokking] Persistence statistics correlate with test accuracy (max H1 rho up to 0.81; total H0 rho down to -0.91), a correlational not causal linkage [tang-etal-2026-topological-signatures-of-grokking] A label-permutation ablation breaks the topology-generalization link above ~10-20% corruption, and an MNIST control shows no sharp topological transition [tang-etal-2026-topological-signatures-of-grokking]

Tian trains 2-layer quadratic-activation MLPs (widths 20/512/1024, 5 seeds) on modular addition over Z_p (p in {23,71,127}) and factorizes the trained weights against a closed-form semi-ring / ring-homomorphism construction [tian-2024-composing-global-solutions-algebraic-objects] About 95% of gradient-descent solutions are factorable with small factorization error (~0.04 relative to solution norm) [tian-2024-composing-global-solutions-algebraic-objects] Each Fourier frequency admits both order-6 (6 hidden nodes) and order-4 (4 hidden nodes) solution components, not a fixed count, with the mix depending on the modulus [tian-2024-composing-global-solutions-algebraic-objects] For d=127 the overwhelming majority of frequencies take the order-4 form (only ~1.26 of 63 frequencies are pure order-6), so six-nodes-per-frequency is only the sufficient count for a pure order-6 construction [tian-2024-composing-global-solutions-algebraic-objects] The framing is purely algebraic (commutative semi-ring, ring homomorphisms); the paper makes no literal circular-geometry claim, treating the circle only as membership in the modular-addition cluster [tian-2024-composing-global-solutions-algebraic-objects] The measurement is observational weight structure with no activation-level causal intervention [tian-2024-composing-global-solutions-algebraic-objects]

Cyclical temporal concepts (days of the week, months) are each independently represented as points on a circle in the residual stream, linearly decodable from activations [karkada-etal-2026] Despite the "toroidal" id, the confirmed finding is a single circle per cyclic variable, not a joint torus; no cited paper shows two such circles coexisting as orthogonal factors of one representation [karkada-etal-2026] Karkada et al. independently derive the same single-circle geometry analytically from translation-symmetric co-occurrence statistics [karkada-etal-2026] Engels et al. discover the same circles in GPT-2-small (layer 7) and Mistral-7B by clustering SAE dictionary elements, and causally validate them via activation patching on Mistral-7B and Llama-3-8B, where patching to a rotated point changes a day/month arithmetic answer [engels-etal-2024] Tiblias et al. recover the same circular date/month structure via a supervised distance-fit (sMDS) across Qwen2.5-3B, Llama-3.2-3B, and Gemma-2-2B, persisting at 3B/8B/70B Llama scale [tiblias-etal-2025] The circle is task-dependent: the identical date context collapses into a non-cyclic clustered or linear manifold when the completion cue asks for season or temperature rather than recency [tiblias-etal-2025] It is causally necessary: noise confined to the located 2D subspace degrades temporal-reasoning accuracy as much as noise across the full residual stream, while an equally-sized random subspace has negligible effect [tiblias-etal-2025] Bhalla et al. reproduce the days-of-week circle in Llama-3.1-8B (layer 19) via PCA and causally steer along its principal components to shift the predicted day-of-week token smoothly [bhalla-etal-2026]

Sun et al. recover two near-orthogonal valence/arousal axes inside Llama-3.1-8B-Instruct (replicated in Qwen3-8B/14B) by PCA-projecting contrastive emotion-steering vectors and ridge-regressing against human ratings [sun-etal-2026-valence-arousal] Projecting the emotion vectors onto this plane traces a circle: a circularity statistic reaches 2.76-4.08 with fitted radii ~0.37-0.39, analogous to Russell's circumplex [sun-etal-2026-valence-arousal] The valence axis recovers self-reported valence at r=0.97 and the NRC-VAD lexicon at r=0.71, with cross-model valence agreement r=0.95 [sun-etal-2026-valence-arousal] Adding valence/arousal directions at specific circle angles produces dose-dependent, angle-specific shifts in generated-text affect (e.g. 0deg: delta-valence +0.75; 180deg: -0.73) [sun-etal-2026-valence-arousal] The same arousal axis causally controls refusal (20%->86% on OKTest) and sycophancy, with random-direction controls within 2-3 points of baseline [sun-etal-2026-valence-arousal] Logit-clamping and top-neuron ablation along the direction crash refusal while preserving MATH-500/IFEval, and an independent refusal direction is near-orthogonal (86.5deg) to the VA plane [sun-etal-2026-valence-arousal] Van der Ben et al. independently replicate the valence/arousal PCA structure (PC1-valence r=0.72-0.83, PC2-arousal r=0.21-0.45) in Apertus-8B and Gemma-4-E4B-it without computing circularity or steering [vanderben-etal-2026-emotion-vectors-open-source-llms] They add that cross-architecture layer-depth trajectories diverge sharply (a 3-phase plateau in Apertus vs a smooth gradient in Gemma via linear CKA) [vanderben-etal-2026-emotion-vectors-open-source-llms]

Zhou, Zhang & Duan fine-tune WavLM-Large on categorical speech-emotion classification only (never continuous arousal-valence labels), then map its features into a 2D arousal-valence plane via anchored dimensionality reduction [zhou-etal-2023-learning-arousal-valence-representation-categorical-emotion-labels-speech] Each categorical emotion is first pinned to its known arousal-valence coordinate from psychology (Russell & Mehrabian), then a weighted kNN-graph, UMAP-style optimization refines each utterance around its anchor [zhou-etal-2023-learning-arousal-valence-representation-categorical-emotion-labels-speech] The learned structure is a 2D plane with two continuous axes (neutral at the origin), not a closed-loop circumplex with S^1 cyclic topology [zhou-etal-2023-learning-arousal-valence-representation-categorical-emotion-labels-speech] On IEMOCAP the method reaches CCC up to 0.674 (valence) and 0.679 (arousal), matching or exceeding fully-supervised AV regression (0.317 valence baseline) without any continuous AV labels [zhou-etal-2023-learning-arousal-valence-representation-categorical-emotion-labels-speech] Cluster-center MAE to the psychological anchors (0.141-0.376 across IEMOCAP, EmoDB, MEAD) confirms alignment to the Russell axes rather than an arbitrary 2D layout [zhou-etal-2023-learning-arousal-valence-representation-categorical-emotion-labels-speech] An ablation removing the anchors (plain unsupervised UMAP) collapses CCC to near zero, so the imposed anchors, not the graph optimization, create the alignment, making any emergent/recoverable framing an over-claim [zhou-etal-2023-learning-arousal-valence-representation-categorical-emotion-labels-speech]