GPT-Neo

EleutherAI

Structures found in this family (3)

Affine Subspace, Intrinsic-dimension profile across depth, Linear Direction

By model (3)

GPT-Neo-1.3B · 1.3B

Affine Subspace, Intrinsic-dimension profile across depth

GPT-Neo-2.7B · 2.7B

Intrinsic-dimension profile across depth, Linear Direction

GPT-Neo-125M · 125M

Intrinsic-dimension profile across depth

Observations (3)

Geographic representations are causally used for prediction, not just decodable

Lower input-sequence intrinsic dimension predicts more verbatim memorization

MEND transforms rank-one gradient factors with a hypernetwork

Papers

More than Correlation: Do Large Language Models Learn Causal Representations of Space? (2023), Memorization in Language Models through the Lens of Intrinsic Dimension (2025), Fast Model Editing at Scale (2022)