Visual concept directions split into localized-entity vs distributed-abstract encoding

measured in 1 paper

Deng et al. extract per-layer diff-in-means concept vectors for Entity, Visual-Style, Emotion, and Abstract categories across six MLLMs [deng-etal-2026-causal-probing-internal-visual-representations-mllms] Entity directions are sharply localized (Gini averaging 0.071, as low as 0.012) with a bimodal layer profile and extreme logit boosts [deng-etal-2026-causal-probing-internal-visual-representations-mllms] Abstract-concept directions are globally distributed (Gini up to 0.429), weakly steerable (success rate as low as 0.160) with negligible logit boost [deng-etal-2026-causal-probing-internal-visual-representations-mllms] Distributedness increases with scale: larger models localize concrete categories more but spread abstract concepts across more layers [deng-etal-2026-causal-probing-internal-visual-representations-mllms] Reverse steering establishes causal necessity, revealing a compensatory internal logit-boost surge (10^6-10^8) under entity suppression [deng-etal-2026-causal-probing-internal-visual-representations-mllms]

Structure

Linear Direction

Context

Gini coefficient quantifying layer-wise localization vs. distribution of a concept direction's causal effect, four-category visual concept taxonomy (Entity, Visual Style, Emotion, Abstract Concept), entity directions sharply localized with a bimodal layer profile; abstract-concept directions globally distributed and right-skewed, scale increases localization for concrete categories but widens distribution for abstract concepts, reverse steering (ablation) establishes causal necessity, revealing a compensatory latent-activation surge under suppression, perception/reasoning disconnect in visual logical reasoning via auxiliary-line steering

Confirmed in models

Qwen2.5-VL 7B Instruct · Qwen, Qwen3-VL 8B · Qwen, Qwen3-VL 32B · Qwen, LLaVA-OneVision-1.5 8B · LLaVA-OneVision, Gemma 3 4B Instruct · Gemma, Gemma 3 27B Instruct · Gemma

Method

Difference-in-means direction, Causal interventions (steering), Activation Steering (Addition)

Papers

Causal Probing for Internal Visual Representations in Multimodal Large Language Models — Deng, Zehao, Ju, Tianjie, Wu, Zheng, He, Liangbo, Lan, Jun, Zhu, Huijia, Wang, Weiqiang, Zhang, Zhuosheng2026 · arXiv:2605.05593