• DDPM → DDIM difference (Denoising Diffusion Probabilistic Models → Denoising Diffusion Implicit Models)

  • process
  1. assumption : markovian → non-markovian (but train: same with DDPM)
  2. determinstic : calculated $x_0$ with predict reverse diffusion with $x_t$ & 0 random noise difference : predict t-1 step with calculated $x_0$ (DDIM) or directly (DDPM) → $\eta$ ratio control ($\sigma=0$ : DDIM)

• DDPM → DDIM result

  • Generate higher-quality samples using a much fewer number of steps.
  • Have “consistency” property since the generative process is deterministic, meaning that multiple samples conditioned on the same latent variable should have similar high-level features.
  • Because of the consistency, DDIM can do semantically meaningful interpolation in the latent variable.

• Implicit models

  • DDPM (from Denoising Diffusion Probabilistic Models)

    

  • DDIM

    • non-Markovian

    

    • chosen mean to ensure $q_\sigma(x_t|x_0) = N(\sqrt\alpha x_0, \sqrt{1-\alpha_t}I)$ for all t (Lemma 1) non-Markovian : depend on $x_{t-1}$ and $x_0$

      

      predict $\epsilon_t$ : denoising generative process calculate $x_0$ given $x_t$ with $x_t = \sqrt{\alpha_t}x_0 + \sqrt{1-\alpha_t}\epsilon$ → $x_0 = f(x_t)$

      

    • Training : same with DDPM optimize $\theta$ with forward $q$, reverse $p$

      

      which is $J_{\sigma} = L_{\gamma} + C$ → assume optimal : $J_\sigma = L_\gamma$ (Theorem 1)

      

    • Sampling

      

      deterministic when $\sigma_t = 0$ → consistency

      cf. non-markovian : sampling from changing $\sigma$ cf. accelerated generation process : sampling trajectory

      

      cf. relevance to neural ordinary differential equation (ODEs)

• Result

  • Reverse process with sub-sequence selection $\tau$ select timesteps such that linear or Quadratic for some c

  • Closed form equations for each sampling step

    

    

  

  • Consistency (& accelerated generation)

    

    

  • Interpolation

    

    

    

cf. result