Sentiment-classification RNNs (LSTM/GRU/UGRNN/vanilla)

Maheswaranathan, Williams, Golub, Ganguli & Sussillo (academic)

Structures found in this family (2)

Linear Subspace, Line Attractor

By model (4)

GRU (sentiment classification, Yelp/IMDB/SST)

Linear Subspace, Line Attractor

LSTM (sentiment classification, Yelp/IMDB/SST)

Linear Subspace, Line Attractor

Update Gate RNN (sentiment classification, Yelp/IMDB/SST)

Linear Subspace, Line Attractor

Vanilla RNN (sentiment classification, Yelp/IMDB/SST)

Linear Subspace, Line Attractor

Observations (2)

Negation and intensification deflect sentiment RNNs into an orthogonal 2D subspace

Sentiment RNNs converge on a readout-aligned line-attractor mechanism

Papers

How Recurrent Networks Implement Contextual Processing in Sentiment Analysis (2020), Reverse Engineering Recurrent Networks for Sentiment Classification Reveals Line Attractor Dynamics (2019)