MATH · IN · MODELS
methods / Theoretical / Analytical / Latent-trajectory shape metrics (entropy, effective rank, anisotropy, intrinsic dimension)

Latent-trajectory shape metrics (entropy, effective rank, anisotropy, intrinsic dimension)

Techniqueintermediate

A four-metric suite — token/state entropy, effective rank of the activation covariance, anisotropy, and intrinsic dimension — applied jointly to a sequence of latent states (e.g. a recurrent model's iterated latent 'thoughts') to characterize how tightly organized versus diffuse a trajectory through representation space is, rather than any single scalar.

Used in (1 observation)

structure: Dimensional collapse · models: Huginn-3.5B (recurrent-depth latent-reasoning model) · paper: Latent Thinking Optimization: Your Latent Reasoning Language Model Secretly Encodes Reward Signals in its Latent Thoughts