MATH
·
IN
·
MODELS
The map
Structures
Hypotheses
Classification
Matrix
Methods bubble
Genealogy
Families
Papers
Methods
Why
Adaptive Input Transformer LM (Baevski & Auli)
Facebook AI Research
Structures found in this family (1)
Linear Direction
By model (1)
Adaptive Input Transformer LM (16-layer, WikiText-103)
· 247M
Linear Direction
Observations (1)
FFN value vectors are vocabulary-space directions in upper layers
Papers
Transformer Feed-Forward Layers Are Key-Value Memories (2020)