Time-series foundation models linearly recover generative parameters by depth

measured in 1 paper

Pandey et al. fit per-layer linear probes on frozen Chronos and MOMENT to regress synthetic-series generative parameters [pandey-etal-2025-internal-semantics-time-series-foundation-models] Early layers capture local time-domain patterns (AR(1), trend, level shifts), deeper layers specialize in dispersion and change-points, and spectral/warping concepts stay hardest throughout [pandey-etal-2025-internal-semantics-time-series-foundation-models] Probe accuracy improves when a parameter varies smoothly along the UMAP manifold, and Chronos yields more linearly recoverable representations than MOMENT [pandey-etal-2025-internal-semantics-time-series-foundation-models] In compositional settings probe performance degrades and embedding arithmetic shows interference; no causal intervention is performed [pandey-etal-2025-internal-semantics-time-series-foundation-models]

Structure

Linear Direction

Context

per-layer linear probing of generative concept parameters (MSE), cross-concept decodability hierarchy (local time-domain > dispersion/change-point > spectral/warping), probe accuracy tracks UMAP manifold smoothness of the concept parameter, compositional interference between simultaneously-varied concepts, cross-model comparison (Chronos vs. MOMENT) of representation organization

Confirmed in models

Chronos · Chronos, MOMENT · MOMENT

Method

Linear probing

Papers

On the Internal Semantics of Time-Series Foundation Models — Pandey, Atharva, Neog, Abhilash, Jajoo, Gautam2025 · arXiv:2511.15324