Attention-weighted graph curvature analysis

methods / Theoretical / Analytical / Geometric analysis / Attention-weighted graph curvature analysis

Attention-weighted graph curvature analysis

Techniqueadvanced

Computes a discrete graph curvature (Balanced Forman Curvature) on the 'effective graph' formed by re-weighting a real trained graph transformer's own input edges by its learned attention weights, tracking how this attention-weighted curvature differs from the raw input graph's own curvature -- distinct from [[discrete-ricci-curvature-estimation]], which builds its graph from a k-NN of activations rather than re-weighting a pre-existing input graph's edges by attention.

Used in (1 observation)

A real trained graph transformer's attention-weighted effective graph undergoes curvature collapse, concentrating on and worsening bottleneck edges rather than smoothing themmeasured

structure: Curvature profile of the representation manifold · models: Graph Transformer (GT/GraphiT/SAN), trained on LRGB peptides-func/peptides-struct (Tori, Bini, Sorbi, Marchand-Maillet & Ginis) · paper: Probing Graph Neural Network Activation Patterns Through Graph Topology