MATH · IN · MODELS

Custom research feedforward MLP

Structures found in this family (6)

Linear region arrangement (polyhedral tessellation of input space), Task-induced symmetry class of a learned representation, Curvature profile of the representation manifold, Decision boundary (as a codimension-1 hypersurface), Dimensional collapse, Representation-Weight-Gradient Alignment (Canonical Representation Hypothesis)

By model (11)

Andriopoulos et al. 2025 multivariate regression MLPs (control tasks and synthetic data)
Dimensional collapse
3-layer FC classifier, 2D input, 4-class synthetic data (Balestriero & Baraniuk)
Linear region arrangement (polyhedral tessellation of input space)
California Housing regression MLP (Gaines & Bi)
Linear region arrangement (polyhedral tessellation of input space)
MNIST fully-connected net [784,5,8,8,8,10] (Gaines & Bi)
Linear region arrangement (polyhedral tessellation of input space)
DQN navigation agent (Halvagal, Lee & Chung)
Task-induced symmetry class of a learned representation
PPO navigation agent (Halvagal, Lee & Chung)
Task-induced symmetry class of a learned representation
Custom feedforward MLP (binary image classification, Hehl et al. 2025) · width 15-50, depth 7-15 (swept)
Curvature profile of the representation manifold
Dense/fully-connected classifier (MNIST)
Decision boundary (as a codimension-1 hypersurface)
Overparameterized MLP on synthetic dataset T_1
Decision boundary (as a codimension-1 hypersurface)
4-layer MLP, 200 units/layer, on 1000 MNIST images (Patel & Montufar)
Linear region arrangement (polyhedral tessellation of input space)
Ziyin et al. 2024 fully connected nets on synthetic two-layer-teacher data (fc1/fc2)
Representation-Weight-Gradient Alignment (Canonical Representation Hypothesis)

Observations (9)

A real trained 3-layer ReLU classifier's composed linear-region partition is directly visualized on 2D data, and the per-layer regions are neither convex nor connected
A real ReLU network's connectivity graph of polyhedral linear regions converges toward its 2d average-degree bound, with data-containing regions more connected
DQN and PPO agents trained on the same real navigation task develop representations invariant to different symmetry classes, depending on learning objective
Activation-graph Ricci curvature contracts flow-like, reversing at overfitting onset
Decision-boundary margins keep shrinking past the error plateau
Neural regression collapse coincides with last-layer feature intrinsic dimension below that of the targets (ID_H < ID_Y), which over-compresses and hurts generalization, while non-collapsed regression models keep ID_H > ID_Y
Decision-boundary margin is small along low-frequency discriminative directions
Local-complexity density of the ReLU linear-region boundary drops during the terminal phase of training on real MNIST images, coinciding with increased adversarial robustness
Latent representations, weights, and neuron gradients become mutually aligned during training, unifying Neural Collapse and the Neural Feature Ansatz

Papers

Mad Max: Affine Spline Insights into Deep Learning (expanded from A Spline Theory of Deep Networks, ICML 2018) (2018), Characterizing the Discrete Geometry of ReLU Networks (2026), Task-Induced Representational Invariances Depend on Learning Objective in Deep RL (2026), Neural Feature Geometry Evolves as Discrete Ricci Flow (2025), Understanding the Decision Boundary of Deep Neural Networks: An Empirical Study (2020), Geometric Analysis of Neural Regression Collapse via Intrinsic Dimension (2025), Hold Me Tight! Influence of Discriminative Features on Deep Network Boundaries (2020), On the Local Complexity of Linear Regions in Deep ReLU Networks (2024), Formation of Representations in Neural Networks (2024)