Shai et al. (2024) Custom Epsilon-Machine Transformers
Research (Shai, Marzen, Teixeira, Gietelink Oldenziel & Riechers; custom-trained from scratch, arXiv:2405.15943, NeurIPS 2024 spotlight)
Structures found in this family (1)
By model (1)
Custom GPT-2-style transformer (trained on Mess3 epsilon-machine sequences)
Observations (1)
Papers
Transformers Represent Belief State Geometry in Their Residual Stream (2024)