Generator/decoder Jacobian pullback metric

methods / Theoretical / Analytical / Geometric analysis / Generator/decoder Jacobian pullback metric

Techniqueadvanced

Derives a Riemannian metric on a generative model's fixed latent space by pulling back the ambient (Euclidean or data-space) metric through the decoder/generator's own Jacobian, then computes curvature, geodesics, or eigen-structure of that metric to characterize how the latent space's geometry differs from flat Euclidean space -- distinct from directions-only Jacobian-SVD methods, which extract edit directions rather than a full metric tensor.

Used in (6 observations)

A real VAE's decoder-induced pullback metric and its geodesics substantially improve interpolation and density estimation over naive Euclidean latent distancemeasured

structure: Curvature profile of the representation manifold · models: MLP/convolutional VAE, trained on MNIST digit subsets and video frames (Arvanitidis, Hansen & Hauberg) · paper: Latent Space Oddity: On the Curvature of Deep Generative Models

A real trained importance-weighted autoencoder's pullback-Riemannian geodesics outperform Euclidean latent interpolation on MNIST, a simulated robot arm, and real human motion-capture datameasured

structure: Curvature profile of the representation manifold · models: Importance-Weighted Autoencoder (2D latent, binarized MNIST), Importance-Weighted Autoencoder (2D latent, simulated 6-DOF KUKA robot arm joint angles), Importance-Weighted Autoencoder (2D latent, CMU motion-capture walking data) · paper: Metrics for Deep Generative Models

Local intrinsic dimension of a real StyleGAN2 latent manifold varies across latent space and correlates with disentanglementmeasured

structure: Intrinsic-dimension profile across depth · models: StyleGAN2 (trained on FFHQ, 1024x1024) · paper: Analyzing the Latent Space of GAN through Local Dimension Estimation

Real CelebA- and SVHN-trained VAE manifolds are surprisingly close to zero curvature, so linear latent interpolation nearly matches true geodesicsmeasured

structure: Curvature profile of the representation manifold · models: Convolutional VAE (32-dim latent), trained on CelebA and SVHN (Shao, Kumar & Fletcher) · paper: The Riemannian Geometry of Deep Generative Models

Class-distinguishing latent regions of real disentangled VAE variants carry measurably higher pullback-metric curvature than a vanilla VAE's latent spacemeasured

structure: Curvature profile of the representation manifold · models: Vanilla VAE and disentangling VAE variants (Szabo et al., Mathieu et al., Jha et al.), trained on MNIST, MultiPIE and 3D Chairs (Shukla, Bhagat, Anand & Turaga) · paper: Geometry of Deep Generative Models for Disentangled Representations

Image variation in several real pretrained GANs concentrates along surprisingly few, position-homogeneous major axes of the pullback metricmeasured

structure: Curvature profile of the representation manifold · models: BigGAN-deep (256px, class-conditional on ImageNet), PGGAN (trained on CelebA, 256px), StyleGAN2 (Face256, trained on FFHQ) · paper: The Geometry of Deep Generative Image Models and its Applications