methods / Causal Validation / Transformer-Patcher — One Mistake, One Added Neuron
Transformer-Patcher — One Mistake, One Added Neuron
Corrects a mistake by literally appending one new key-value neuron to the last FFN layer — a new row k_p in the up-projection matrix (a trigger direction detected via inner product with the input query) and a new column v_p in the down-projection matrix (a value direction added to the output), trained by gradient descent with a margin-based memory loss to prevent the new neuron from firing on unrelated inputs. Same outer-product/direction mechanism as rank-one weight editing, implemented via matrix growth instead of matrix modification.