MATH · IN · MODELS
methods / Causal Validation / KnowledgeEditor — Constrained Hyper-Network Weight Editing

KnowledgeEditor — Constrained Hyper-Network Weight Editing

Techniqueadvanced

A bidirectional-LSTM hyper-network conditions five small feed-forward networks per weight matrix that predict two outer-product vector pairs (a gated gradient-scale and a bias term), producing a rank-one-structured update applied to the gradient of the target edit's loss, trained under a KL-divergence constraint (in output-distribution space, not parameter space) that keeps unrelated predictions unchanged. Predates and is the direct architectural ancestor of MEND.

Used in (1 observation)

structure: Linear Direction · models: BERT-base-uncased, BART-base · paper: Editing Factual Knowledge in Language Models