Idealized-geometry weight substitution

methods / Causal Validation / Idealized-geometry weight substitution

Techniqueintermediate

Replaces a component's learned parameters with a hand-constructed discretization of an independently hypothesized geometric object (e.g. points evenly spaced around a circle), often freezing them, then measures whether downstream task performance changes relative to normally-trained or randomly-initialized weights.

Used in (1 observation)

A persistent circle recurs in the convolutional filter weights of real trained CNNs across depths, datasets, and training runs, and fixing an idealized circle as the first layer causally improves cross-dataset generalizationmeasured

structure: Circle · models: Custom convolutional net on MNIST (Carlsson & Brüel Gabrielsson), Custom convolutional net on CIFAR-10 (Carlsson & Brüel Gabrielsson), Custom convolutional net on SVHN (Brüel Gabrielsson & Carlsson), VGG (image classifier, various depths) · paper: Topological Approaches to Deep Learning, Exposition and Interpretation of the Topology of Neural Networks