MATH
·
IN
·
MODELS
The map
Structures
Hypotheses
Classification
Matrix
Methods bubble
Genealogy
Families
Papers
Methods
Why
GELU Toy Transformer
Neel Nanda (independent/TransformerLens)
Structures found in this family (1)
Linear Direction
By model (1)
GELU-1L
· ~3M
Linear Direction
Observations (1)
Gated SAEs eliminate shrinkage and halve the required L0
Papers
Improving Dictionary Learning with Gated Sparse Autoencoders (2024)