Learned linear latent walks in GANs realize shift, zoom and color edits, with extent tracking training-data variability

measured in 1 paper

A single learned vector w added to the latent code (G(z + alpha*w)), fit self-supervised to programmatic edits, produces shift, zoom and color transformations in BigGAN and, in the W space, in StyleGAN [jahanian-chai-isola-2020] The paper reports that the linear walk is as effective as more complex nonlinear walks, but the linear walk undershoots the targeted transformation while keeping outputs more realistic (lower FID increase than the nonlinear walk) [jahanian-chai-isola-2020] Across 100 sampled ImageNet classes the achieved mean shift Delta-mu correlates positively with the dataset standard deviation (slope p < 0.001); for linear walks Pearson r is 0.59 (luminance), 0.28 (shift X), 0.39 (shift Y), 0.37 (zoom) [jahanian-chai-isola-2020] Pushing alpha past about 150 pixels for vertical shift or an 8x zoom raises FID, so transformations saturate and leave the image manifold [jahanian-chai-isola-2020]

Structure

Linear Direction

Context

latent walk, steerability, dataset bias, camera shift and zoom

Confirmed in models

BigGAN (class-conditional, 1000 ImageNet categories; resolution not stated in Jahanian et al.) · BigGAN, StyleGAN (trained on FFHQ, 1024x1024) · StyleGAN, StyleGAN (trained on LSUN Car) · StyleGAN, StyleGAN (trained on LSUN Cat) · StyleGAN, PGGAN (trained on CelebA-HQ, 1024x1024) · PGGAN (Progressive GAN)

Method

Self-supervised latent walk optimization, Causal interventions (steering)

Papers

On the "steerability" of generative adversarial networks — Jahanian, Ali, Chai, Lucy, Isola, Phillip2020 · arXiv:1907.07171