An RBF-kernelized structural probe beats the linear baseline on mBERT syntax
measured in 1 paperWhite, Pimentel, Saphra & Cotterell kernelize Hewitt-Manning's linear structural probe with an RBF kernel at identical parameter count [white-etal-2021-a-non-linear-structural-probe] Tested on mBERT across 6 UD-2.4 languages (Basque, English, Finnish, Korean, Tamil, Turkish), the RBF probe gives a significant UUAS improvement over the linear baseline in all six (paired permutation test p<0.05; Tamil 48.52 to 56.96) [white-etal-2021-a-non-linear-structural-probe] This is direct evidence that syntax-tree-distance geometry in mBERT is partly encoded non-linearly (curved), refining rather than merely replicating the linear tree-metric claim [white-etal-2021-a-non-linear-structural-probe]
Structure
Context
non-linear structural probe, curved syntax-tree metric, RBF kernel probing
Confirmed in models
Papers
A Non-Linear Structural Probe — White, Jennifer C., Pimentel, Tiago, Saphra, Naomi, Cotterell, Ryan