methods / Theoretical / Analytical / Latent-trajectory shape metrics (entropy, effective rank, anisotropy, intrinsic dimension)
Latent-trajectory shape metrics (entropy, effective rank, anisotropy, intrinsic dimension)
A four-metric suite — token/state entropy, effective rank of the activation covariance, anisotropy, and intrinsic dimension — applied jointly to a sequence of latent states (e.g. a recurrent model's iterated latent 'thoughts') to characterize how tightly organized versus diffuse a trajectory through representation space is, rather than any single scalar.