Permutation-tracking transformers encode state via a parity axis orthogonal to a triangle
measured in 1 paperLi, Guo & Andreas fine-tune Pythia-160M on the symmetric-group (S3, S5) word problem and find models converge on one of exactly two mechanisms, identified via probing and activation patching [li-etal-2025] The Associative Algorithm computes the product via a hierarchical parallel-prefix scan, with state-probe accuracy and patch-restoration length growing exponentially with depth [li-etal-2025] The Parity-Associative Algorithm first computes parity via dedicated parity heads, then refines with an AA-like scan, and an intermediate pretraining curriculum controls which mechanism emerges [li-etal-2025] For PAA on S3, PCA reveals an orthogonal parity axis crossed with a 2D triangle of three equidistant clusters (a 2-simplex direct-summed with the parity direction); the clean 3-direction decomposition breaks down for S5 [li-etal-2025]