MATH · IN · MODELS

DINOv2

Meta AI

Structures found in this family (10)

Linear Direction, Linear Separability, Minkowski Sum of Tile Polytopes (Minkowski Representation Hypothesis), Linear Subspace, Polytope (Simplex), Aristotelian Representation Hypothesis, Platonic Representation Hypothesis, Spatial-frequency (spectral) accessibility profile across depth, Intrinsic-dimension profile across depth, Linear Centroids Hypothesis

Hypotheses argued for by this family (1)

Minkowski Representation Hypothesis

By model (5)

DINOv2 ViT-B/14
Linear Direction, Linear Separability, Linear Subspace, Aristotelian Representation Hypothesis, Platonic Representation Hypothesis, Intrinsic-dimension profile across depth
DINOv2 ViT-L/14
Aristotelian Representation Hypothesis, Linear Direction, Linear Subspace, Linear Centroids Hypothesis
DINOv2-B (ViT-Base, 4 register tokens)
Minkowski Sum of Tile Polytopes (Minkowski Representation Hypothesis), Linear Subspace
DINOv2 ViT-B · 86M
Spatial-frequency (spectral) accessibility profile across depth
DINOv2 ViT-B/16
Polytope (Simplex)

Observations (16)

A value-like success direction is recoverable from frozen VLAs
Patch-cosine Gram structure groups objects, strongest in self-supervised ViTs
DINOv2 activation space is consistent with a Minkowski sum of tile polytopes
DINOv2 and Stable Diffusion encode depth/normals via nonlinear dense probes
SSL encoders collapse variance along each task's decision axis, aiding few-shot transfer
After correcting CKA and mutual k-NN alignment for a proven width/depth-dependent null-baseline confound (O(d/n) for global spectral metrics vs O(k/n) for local-neighborhood metrics), the Platonic Representation Hypothesis's global cross-modal convergence largely vanishes while its local-neighborhood convergence survives, across real image-text and video-language pretrained model pairs
Independently pretrained vision (DINOv2) and text (MiniLM) encoders develop graph-Laplacian eigenvalue spectra of near-identical intrinsic complexity, yet the functional map between their spectral eigenbases is far from diagonal or orthogonal -- a decoupling of 'how much structure' from 'how it is oriented' that bounds spectral cross-modal alignment methods and underperforms ambient-space alignment on retrieval
Fewer than 25 register neurons with consistent directions cause ViT outlier tokens
A non-monotonic spatial-frequency accessibility profile across depth in real CLIP and DINOv2 vision transformers
A shared PCA basis's leading components separate figure from ground
MST intrinsic dimension of frozen SSL features predicts probe accuracy
SAE directions show CLIP encodes abstract concepts, DINOv2 fragmented visual features
SVD-extracted ViT-L task subspaces recover most full-feature performance
Compositional generalization needs linear, additive, orthogonal per-concept factor vectors
Multi-object scene embeddings decompose hierarchically as sums of object embeddings
Jacobian centroids (row-sums of a sub-component's local input-output Jacobian) form cleaner, more transferable linear feature directions than raw activations across four real-model settings: ResNet50/DTD, DINOv2-vs-DINOv3 SAE dictionaries, GPT2-Large circuit discovery, and Llama-3.1-8B truth probes

Papers

What Frozen VLAs Already Know About Success: A Probing Study of Value-Like Structure in Foundation Robot Policies (2026), Human-like Object Grouping in Self-Supervised Vision Transformers (2026), Into the Rabbit Hull: From Task-Relevant Concepts in DINO to Minkowski Geometry (2025), Probing the 3D Awareness of Visual Foundation Models (2024), Directional Neural Collapse Explains Few-Shot Transfer in Self-Supervised Learning (2026), Revisiting the Platonic Representation Hypothesis: An Aristotelian View (2026), On the Spectral Geometry of Cross-Modal Representations: A Functional Map Diagnostic for Multimodal Alignment (2026), Vision Transformers Don't Need Trained Registers (2025), Beyond Compression: Quantifying Spectral Accessibility in Vision Representations (2026), From Local Cues to Global Percepts: Emergent Gestalt Organization in Self-Supervised Vision Models (2025), IdEst: Assessing Self-Supervised Learning Representations via Intrinsic Dimension (2026), Interpretable and Testable Vision Features via Sparse Autoencoders (2025), Understanding Geometric Representations in Self-Supervised Vision Transformers via Subspace Intervention (2026), Compositional Generalization Requires Linear, Orthogonal Representations in Vision Embedding Models (2026), How Can Embedding Models Bind Concepts? (2026), The Linear Centroids Hypothesis: Features as Directions Learned by Local Experts (2026)