MATH · IN · MODELS

Custom research feedforward MLP

Structures found in this family (4)

Linear region arrangement (polyhedral tessellation of input space), Task-induced symmetry class of a learned representation, Curvature profile of the representation manifold, Decision boundary (as a codimension-1 hypersurface)

By model (9)

3-layer FC classifier, 2D input, 4-class synthetic data (Balestriero & Baraniuk)
Linear region arrangement (polyhedral tessellation of input space)
California Housing regression MLP (Gaines & Bi)
Linear region arrangement (polyhedral tessellation of input space)
MNIST fully-connected net [784,5,8,8,8,10] (Gaines & Bi)
Linear region arrangement (polyhedral tessellation of input space)
DQN navigation agent (Halvagal, Lee & Chung)
Task-induced symmetry class of a learned representation
PPO navigation agent (Halvagal, Lee & Chung)
Task-induced symmetry class of a learned representation
Custom feedforward MLP (binary image classification, Hehl et al. 2025) · width 15-50, depth 7-15 (swept)
Curvature profile of the representation manifold
Dense/fully-connected classifier (MNIST)
Decision boundary (as a codimension-1 hypersurface)
Overparameterized MLP on synthetic dataset T_1
Decision boundary (as a codimension-1 hypersurface)
4-layer MLP, 200 units/layer, on 1000 MNIST images (Patel & Montufar)
Linear region arrangement (polyhedral tessellation of input space)

Observations (7)

A real trained 3-layer ReLU classifier's composed linear-region partition is directly visualized on 2D data, and the per-layer regions are neither convex nor connected
A real ReLU network's connectivity graph of polyhedral linear regions converges toward its 2d average-degree bound, with data-containing regions more connected
DQN and PPO agents trained on the same real navigation task develop representations invariant to different symmetry classes, depending on learning objective
Activation-graph Ricci curvature contracts flow-like, reversing at overfitting onset
Decision-boundary margins keep shrinking past the error plateau
Decision-boundary margin is small along low-frequency discriminative directions
Local-complexity density of the ReLU linear-region boundary drops during the terminal phase of training on real MNIST images, coinciding with increased adversarial robustness

Papers

Mad Max: Affine Spline Insights into Deep Learning (expanded from A Spline Theory of Deep Networks, ICML 2018) (2018), Characterizing the Discrete Geometry of ReLU Networks (2026), Task-Induced Representational Invariances Depend on Learning Objective in Deep RL (2026), Neural Feature Geometry Evolves as Discrete Ricci Flow (2025), Understanding the Decision Boundary of Deep Neural Networks: An Empirical Study (2020), Hold Me Tight! Influence of Discriminative Features on Deep Network Boundaries (2020), On the Local Complexity of Linear Regions in Deep ReLU Networks (2024)