The release's default sampler — linear multistep over DDIM's step.
Parameters
configStableDiffusionConfigDDIMScheduler uses.Notes
Reference: Liu et al., "Pseudo Numerical Methods for Diffusion
Models on Manifolds", ICLR, 2022 (arXiv:2202.09778); the constants are the released
Stable Diffusion configuration, whose _class_name is this.
The update is DDIM's at applied not to the current epsilon but to an Adams-Bashforth combination of the last four:
with lower-order openers while the history fills — the plain prediction first, then a trapezoid, then a two-term rule, then a three-term one. Four evaluations of information per step is what buys the same image in fifty steps that DDIM needs more of.
It carries state, unlike DDIMScheduler. The history,
the step counter and the sample the opener began from live on the
object, so one instance cannot sample two trajectories at once.
Sequential reuse is safe: timesteps means "start a sample"
and clears the history, and step checks that the timestep it
is handed is the one at the counter's position. That check is the
one that matters — the order of the correction is chosen by position,
so stepping out of order applies the wrong rule to right-looking
numbers. reset is there for the caller who wants to say so
without asking for a trajectory.
The second step repeats the first timestep rather than advancing —
counter == 1 in the reference — which looks like an off-by-one
and is the trapezoidal opener needing both endpoints.
Examples
>>> import lucid
>>> from lucid.models.generative.stable_diffusion import (
... PNDMScheduler, StableDiffusionConfig)
>>> scheduler = PNDMScheduler(StableDiffusionConfig())
>>> scheduler.timesteps(10)[:4]
[901, 801, 801, 701]
>>> scheduler.counter
0Used by 2
Constructors
1Instance methods
3Clear the history so a new trajectory starts clean.
One reverse step, using the history.
Parameters
Returns
Notes
The previous timestep is derived from the stride rather than
passed, because the multistep rule is only valid on the uniform
grid timesteps produces. Call that first; this raises
otherwise rather than assuming a stride.
Examples
>>> import lucid
>>> from lucid.models.generative.stable_diffusion import (
... DDIMScheduler, PNDMScheduler, StableDiffusionConfig)
>>> config = StableDiffusionConfig()
>>> scheduler, ddim = PNDMScheduler(config), DDIMScheduler(config)
>>> lucid.manual_seed(0)
>>> clean = lucid.randn((1, 4, 8, 8))
>>> noise = lucid.randn((1, 4, 8, 8))
Every rule's weights sum to one, so a prediction that is exactly
right at every step combines to itself and each step lands back on
the forward process. A whole trajectory run that way ends where
DDIMScheduler noises the clean latent to timestep 0.
>>> steps = scheduler.timesteps(4)
>>> latent = ddim.add_noise(clean, noise, steps[0])
>>> for t in steps:
... latent = scheduler.step(noise, t, latent)
>>> bool(lucid.allclose(latent, ddim.add_noise(clean, noise, 0), atol=1e-5))
True
The position is checked on every call: a fresh trajectory has to
start at 751, and anything else raises rather than apply the wrong
order of correction.
>>> _ = scheduler.timesteps(4)
>>> scheduler.step(noise, 501, latent)
Traceback (most recent call last):
...
RuntimeError: expected timestep 751 at position 0 of the trajectory ...The decreasing subsequence to sample over.
Parameters
num_inference_stepsintReturns
list of intDescending indices, one longer than requested: the second time is visited twice.
Notes
Calling this starts a sample — it clears the history, the counter and the saved opener sample.
The repeat is the trapezoidal opener, and the reference puts it
in the list rather than in the step. Its slices —
[t[:-1], t[-2:-1], t[-1:]] — are taken on the ascending
array before reversal, so the duplicated entry lands second
in the descending order: 901, 801, 801, 701, and not
…, 101, 101, 1. Reading those slices as descending puts the
opener at the end, where it is the wrong rule at the wrong time.
Examples
>>> import lucid
>>> from lucid.models.generative.stable_diffusion import (
... PNDMScheduler, StableDiffusionConfig)
>>> scheduler = PNDMScheduler(StableDiffusionConfig())
>>> scheduler.timesteps(4)
[751, 501, 501, 251, 1]
Four intervals, five evaluations — the second time repeats for the
opener. Asking again starts a new sample, so the counter a previous
trajectory advanced is cleared.
>>> latent = lucid.randn((1, 4, 8, 8))
>>> _ = scheduler.step(lucid.zeros((1, 4, 8, 8)), 751, latent)
>>> scheduler.counter
1
>>> _ = scheduler.timesteps(4)
>>> scheduler.counter
0