MATH · IN · MODELS
methods / Theoretical / Analytical / Representation-shattering distortion metric

Representation-shattering distortion metric

Techniqueintermediate

A normalized Frobenius-norm distance between two entity-token pairwise-distance matrices — one from an edited model, one from the unedited baseline — quantifying how much a knowledge-editing intervention distorts the relative geometric positions of entities that were never themselves targeted by the edit.

Used in (1 observation)

structure: Circle · models: Custom Transformer (trained from scratch on a structured synthetic knowledge graph), Llama-3-8B-Instruct, Mamba-2.8B, Llama-3.1-405B-Instruct, GPT-2-small, Mistral-7B · paper: Representation Shattering in Transformers: A Synthetic Study with Knowledge Editing