TabPFN

Prior Labs

Structures found in this family (3)

Linear Subspace, Linear Direction, Dimensional collapse

By model (1)

TabPFN v2 (tabular in-context-learning foundation model)

Linear Subspace, Linear Direction, Dimensional collapse

Observations (4)

Real TabPFNv2, TabICLv2, and Mitra build class-conditional prototype/vote geometry, causally confirmed via mechanism transplantation

TabICLv2 predicts by a literal class centroid; TabPFNv2 votes

TabPFN linearly encodes regression coefficients and arithmetic intermediates

Early-layer TabPFN-v2 hidden states are low-rank under standard scalar tokenization, and RBF-expanded tokenization raises shallow-layer rank

Papers

A Mechanistic Study of Tabular Foundation Models (2026), A Mechanistic Study of Tabular Foundation Models (2026), TabPFN Through The Looking Glass: An Interpretability Study of TabPFN and Its Internal Representations (2026), LimiX-2M: Mitigating Low-Rank Collapse and Attention Bottlenecks in Tabular Foundation Models (2026)