A model-free Sokoban agent linearly encodes a causal plan

Bush et al. probe a DRC(3,3) ConvLSTM agent trained model-free on Sokoban and find linear probes recover an internal plan predicting the agent's future actions [bush-etal-2025-interpreting-emergent-planning-in-model-free-reinforcement-learning] Causal interventions on the probed representation change subsequent behavior in the predicted direction, showing the plan is causally implemented [bush-etal-2025-interpreting-emergent-planning-in-model-free-reinforcement-learning] Extra test-time computation improves plan quality and solve rate [bush-etal-2025-interpreting-emergent-planning-in-model-free-reinforcement-learning] The evolving plan representation resembles a parallelized bidirectional search rather than a single forward rollout [bush-etal-2025-interpreting-emergent-planning-in-model-free-reinforcement-learning]