MATH
·
IN
·
MODELS
The map
Structures
Hypotheses
Classification
Matrix
Methods bubble
Genealogy
Families
Papers
Methods
Why
Falcon
TII
Structures found in this family (1)
Anisotropy
By model (2)
Falcon-7B
· 7B
Anisotropy
Falcon-7B-Instruct
· 7B
Anisotropy
Observations (1)
Decoder-only models peak anisotropy in middle layers
Papers
The Shape of Learning: Anisotropy and Intrinsic Dimensions in Transformer-Based Models (2024)