Bayesian posterior entropy-rate estimation

methods / Theoretical / Analytical / Bayesian posterior entropy-rate estimation

Techniqueadvanced

Recovers the Bayes-optimal class posterior at each diffusion noise level from a trained diffusion model's own conditional and unconditional noise predictions (a classifier-free-guidance-style Bayesian update), then differentiates the resulting conditional entropy with respect to diffusion time to obtain a semantic-information-production rate -- a purely information-theoretic diagnostic requiring no auxiliary classifier.

Used in (1 observation)

Semantic information production peaks earlier for less visually similar CIFAR-10 class pairsmeasured

structure: Semantic information production rate along the diffusion trajectory · models: DDPM (CIFAR-10, custom-trained unconditional + class-conditional pair) · paper: Measuring Semantic Information Production in Generative Diffusion Models