DINOv3

Meta AI

Structures found in this family (3)

Linear Separability, Linear Direction, Linear Centroids Hypothesis

By model (1)

DINOv3 ViT-B/16 · 86M

Linear Separability, Linear Direction, Linear Centroids Hypothesis

Observations (3)

Patch-cosine Gram structure groups objects, strongest in self-supervised ViTs

Compositional generalization needs linear, additive, orthogonal per-concept factor vectors

Jacobian centroids (row-sums of a sub-component's local input-output Jacobian) form cleaner, more transferable linear feature directions than raw activations across four real-model settings: ResNet50/DTD, DINOv2-vs-DINOv3 SAE dictionaries, GPT2-Large circuit discovery, and Llama-3.1-8B truth probes

Papers

Human-like Object Grouping in Self-Supervised Vision Transformers (2026), Compositional Generalization Requires Linear, Orthogonal Representations in Vision Embedding Models (2026), The Linear Centroids Hypothesis: Features as Directions Learned by Local Experts (2026)