Fixed-point dynamics analysis

methods / Theoretical / Analytical / Fixed-point dynamics analysis

Techniqueadvanced

Numerically locates approximate fixed points of a recurrent network's zero-input update rule, then linearizes the dynamics around each one via its Jacobian and studies the Jacobian's eigenvalues/eigenvectors — converting an opaque nonlinear recurrent system into a locally-linear, interpretable dynamical description.

Used in (6 observations)

A memory RNN forms a line attractor without any bifurcationmeasured

structure: Line Attractor · models: Piecewise-linear RNN (N=40, delayed-addition short-term-memory task) · paper: Understanding and Controlling the Geometry of Memory Organization in RNNs

Ghost points of destroyed saddle-nodes govern abrupt RNN learningmeasured

structure: Ghost Point (saddle-node bifurcation remnant) · models: Rank-one RNN (N=100, working-memory tasks, Dinc et al. 2025), Full-rank RNN (N=100, working-memory tasks, Dinc et al. 2025) · paper: A Ghost Mechanism: An Analytical Model of Abrupt Learning in Recurrent Networks

Negation and intensification deflect sentiment RNNs into an orthogonal 2D subspacemeasured

structure: Linear Subspace · models: LSTM (sentiment classification, Yelp/IMDB/SST), GRU (sentiment classification, Yelp/IMDB/SST), Update Gate RNN (sentiment classification, Yelp/IMDB/SST), Vanilla RNN (sentiment classification, Yelp/IMDB/SST) · paper: How Recurrent Networks Implement Contextual Processing in Sentiment Analysis

Sentiment RNNs converge on a readout-aligned line-attractor mechanismmeasured

structure: Line Attractor · models: LSTM (sentiment classification, Yelp/IMDB/SST), GRU (sentiment classification, Yelp/IMDB/SST), Update Gate RNN (sentiment classification, Yelp/IMDB/SST), Vanilla RNN (sentiment classification, Yelp/IMDB/SST) · paper: Reverse Engineering Recurrent Networks for Sentiment Classification Reveals Line Attractor Dynamics

Fixed-point analysis reverse-engineers RNN dynamics across three tasksmeasured

structure: Line Attractor · models: Continuous-time RNN (3-bit flip-flop memory task) · paper: Opening the Black Box: Low-Dimensional Dynamics in High-Dimensional Recurrent Neural Networks

Trained recurrent (RNN/GRU/Mamba) policies' hidden states spontaneously converge to a stable, low-dimensional closed-orbit limit cycle that is contractive under finite-time Lyapunov analysis, and selectively randomizing only the top CCA-aligned dimensions between neural state and behavior causally collapses navigation performance back to baseline while randomizing all other dimensions preserves itmeasured

structure: Limit cycle (stable periodic attractor) · models: GRU policy (PPO, Procgen Jumper), Mamba policy (PPO, Procgen Jumper) · paper: Unraveling the Hidden Dynamical Structure in Recurrent Neural Policies