MATH · IN · MODELS

An entity-familiarity direction reads and steers refusal in Polish LLMs

measured in 1 paper

Brzezinka reads final-prompt-token residual-stream activations across 14 models (Bielik, PLLuM, Gemma-4, Qwen3 families) on a 1,440-entity Polish dataset [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] A linear familiarity probe separates real from fabricated entities at AUROC 0.86-0.93, beating a lexical baseline (0.786) [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] In Polish-adapted Bielik/PLLuM models the score additionally tracks entity popularity (Spearman rho up to 0.567), with matched before/after controls isolating Polish continual-pretraining as the driver [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] Cross-language transfer retains 96-101% of within-language AUROC [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering] In Gemma-4-12B (the only natively refusing model), adding a one-dimensional familiarity direction moves refusal rate monotonically (0.24 to 1.00 on known entities; 0.73 to 0.00 on unknown) [brzezinka-2026-graded-entity-familiarity-readouts-in-language-models-polish-adaptation-cross-language-robustness-and-refusal-steering]

Context

entity-familiarity, refusal, cross-lingual-transfer

Papers

Graded Entity-Familiarity Readouts in Language Models: Polish Adaptation, Cross-Language Robustness, and Refusal Steering — Brzezinka, Kamil2026 · arXiv:2607.13568