A real VAE's decoder-induced pullback metric and its geodesics substantially improve interpolation and density estimation over naive Euclidean latent distance

measured in 1 paper

A stochastic Riemannian pullback metric, derived from a real trained VAE decoder's own Jacobian (accounting for decoder variance), is computed on the latent space of VAEs trained on real MNIST digit subsets and a real video-frame dataset [arvanitidis-hansen-hauberg-2018-latent-space-oddity] Geodesic distances and interpolation paths computed under this metric substantially improve interpolation quality, density estimation, sampling and clustering relative to naive Euclidean latent-space distance [arvanitidis-hansen-hauberg-2018-latent-space-oddity] Decoder variance estimates are found to be systematically poor near the data manifold, and an architecture modification is proposed to correct this before computing the pullback metric [arvanitidis-hansen-hauberg-2018-latent-space-oddity]

Structure

Curvature profile of the representation manifold

Context

Riemannian pullback metric, VAE latent geometry, stochastic curvature estimation

Confirmed in models

MLP/convolutional VAE, trained on MNIST digit subsets and video frames (Arvanitidis, Hansen & Hauberg) · Custom Research VAE (purpose-built variational autoencoder for representation-geometry studies)

Method

Generator/decoder Jacobian pullback metric

Papers

Latent Space Oddity: On the Curvature of Deep Generative Models — Arvanitidis, Georgios, Hansen, Lars Kai, Hauberg, Søren2018 · arXiv:1710.11379