Graph properties decode linearly from GNN embeddings, collapsing with depth

measured in 1 paper

Pelletreau-Duris et al. fit linear-regression probes mapping GCN/GAT/GIN activations to dozens of graph-theoretic properties at node and graph level [pelletreau-duris-etal-2024-do-graph-neural-network-states-contain-graph-properties] Size-correlated global properties decode near-perfectly early (R^2=1.00) but collapse to about 0.04-0.07 by the final task-specialized layer, while node betweenness centrality stays roughly flat [pelletreau-duris-etal-2024-do-graph-neural-network-states-contain-graph-properties] A 0.992 correlation between task accuracy and maximum probing score links how well embeddings encode graph structure to downstream performance [pelletreau-duris-etal-2024-do-graph-neural-network-states-contain-graph-properties]

Structure

Linear Separability

Context

linear regression (R^2) diagnostic-classifier probing of graph-theoretic properties from GNN embeddings, depth-dependent collapse of globally-correlated (size) graph properties toward task-specialized final layers, a strong correlation between probe-recoverable graph-property information and downstream task accuracy

Confirmed in models

GCN / GAT / GIN (trained on synthetic Grid-House and ClinTox molecular graphs) · GNN diagnostic-classifier probing suite (GCN/GAT/GIN on Grid-House and ClinTox)

Method

Linear probing

Papers

Do Graph Neural Network States Contain Graph Properties? — Pelletreau-Duris, Tom, van Bakel, Ruud, Cochez, Michael2024 · arXiv:2411.02168