Decision-boundary margin is small along low-frequency discriminative directions
measured in 1 paper- Measuring margins along frequency subspaces (a 2D-DCT-basis decomposition via subspace-constrained DeepFool) shows the decision boundary hugs the data closely along low-frequency discriminative directions and lies far away along high-frequency directions. [ortiz-jimenez-etal-2020-hold-me-tight] - The quantitative margins (median 2.50 along the discriminative direction vs 12.36 orthogonal, 102.0 random-orthogonal, 27.90 random) come from a synthetic overparameterized MLP on a toy dataset (T_1), not from the real classifiers. [ortiz-jimenez-etal-2020-hold-me-tight] - The low-frequency-small / high-frequency-large pattern holds for MNIST (LeNet) and ImageNet (ResNet-50), but CIFAR-10 (DenseNet-121, ResNet-18, VGG-19) is an exception with more uniformly distributed margins. [ortiz-jimenez-etal-2020-hold-me-tight] - Frequency-flip and low-pass-filter interventions swap or remove the margins accordingly, showing discriminative data features causally shape boundary geometry. [ortiz-jimenez-etal-2020-hold-me-tight]