MATH · IN · MODELS
methods / Theoretical / Analytical / Geometric analysis / Successor-measure spectral basis (Laplacian/proto-value eigenvectors)

Successor-measure spectral basis (Laplacian/proto-value eigenvectors)

Techniqueadvanced

Trains a deep encoder to estimate the successor measure (discounted future-state occupancy) of a real RL agent's transitions, then extracts the top-d singular/eigenvectors of the resulting matrix as a task-agnostic auxiliary basis — provably equal to graph-Laplacian eigenvectors (proto-value functions) under a symmetric transition matrix.

Used in (1 observation)

structure: Linear Subspace · models: Proto-Value Network Impala-CNN encoder (offline-pretrained on RL Unplugged Atari, Farebrother et al. 2023) · paper: Proto-Value Networks: Scaling Representation Learning with Auxiliary Tasks