Sparse logit-lens directions in VLA FFN value vectors steer robot actions

measured in 1 paper

Haon et al. project FFN value vectors in OpenVLA-7B and Pi0 onto the vocabulary basis (logit-lens), revealing sparse directions for concepts like "speed", "direction", "up", and "slow" [haon-etal-2025-mechanistic-interpretability-steering-vla] Fewer than 25% of OpenVLA's 352,255 FFN value vectors are rewired for action prediction, the rest inherited semantic directions from the vision-language backbone [haon-etal-2025-mechanistic-interpretability-steering-vla] Upweighting a semantic-direction cluster steers both models zero-shot: "fast" interventions give larger end-effector displacement than "slow" [haon-etal-2025-mechanistic-interpretability-steering-vla] Full-depth "up" cluster injections give the largest mean Y-displacement, validated in LIBERO-Long (OpenVLA) and on a physical UR5 arm (Pi0-FAST) against a random-cluster control [haon-etal-2025-mechanistic-interpretability-steering-vla]

Structure

Linear Direction

Context

FFN value-vector directions, logit-lens decoding of VLA policies, zero-shot action steering

Confirmed in models

OpenVLA-7B · OpenVLA, pi0 (PaliGemma VLA backbone, predecessor of pi0.5) · Pi (Physical Intelligence VLA)

Method

Logit Lens, Causal interventions (steering)

Papers

Mechanistic Interpretability for Steering Vision-Language-Action Models — Häon, Bear, Stocking, Kaylene, Chuang, Ian, Tomlin, Claire2025 · arXiv:2509.00328