MATH · IN · MODELS

Fixed-point analysis reverse-engineers RNN dynamics across three tasks

measured in 1 paper

Sussillo & Barak introduce fixed-point-finding plus local Jacobian linearization to reverse-engineer trained continuous-time RNNs across three tasks [sussillo-barak-2013-opening-the-black-box] For a 3-bit flip-flop memory task the method recovers 8 stable point attractors (the 2^3 memory states) among 26 fixed points, the remainder saddles [sussillo-barak-2013-opening-the-black-box] For an input-driven sine-wave generator the oscillation arises from linear dynamics around unstable fixed points (each a complex-conjugate unstable pair), not a genuine limit cycle, with frequency set by the imaginary part of the unstable eigenvalue [sussillo-barak-2013-opening-the-black-box] For a 2-point moving-average task the network builds a 2-dimensional manifold of slow points, an approximate plane attractor rather than a 1D line/integrator [sussillo-barak-2013-opening-the-black-box] This is the foundational origin of the fixed-point-dynamics-analysis methodology; the study is purely descriptive with no causal ablation [sussillo-barak-2013-opening-the-black-box]

Context

fixed-point topology, Jacobian linearization, discrete memory states, limit cycle

Papers

Opening the Black Box: Low-Dimensional Dynamics in High-Dimensional Recurrent Neural Networks — Sussillo, David, Barak, Omri2013