class

ChainedScheduler

ChainedScheduler(schedulers: list[_LRScheduler])

source

Apply multiple schedulers simultaneously, compounding their LR changes.

At every step call each scheduler in the list is stepped in order. Because each scheduler writes its own learning rates back into the shared optimizer, the net effect is the composition of all individual schedule factors:

\eta_t = \eta_0 \cdot \prod_{k} f_k(t)

where $f_k(t)$ is the multiplicative change applied by scheduler $k$ at epoch $t$ .

Parameters

schedulerslist of _LRScheduler

List of scheduler instances to step together. All schedulers must wrap the same optimizer.

Attributes

schedulerslist of _LRScheduler

The chained schedulers stepped in order.

Notes

ChainedScheduler differs from SequentialLR in that all schedulers are active at every step rather than one at a time. This is useful when you want, for example, a warmup schedule and an exponential decay to both apply simultaneously.

Only the last scheduler in the chain is consulted by get_last_lr, so the returned value reflects that scheduler's view of the current LR (which is also what is stored in the optimizer).

Examples

>>> import lucid.optim as optim
>>> optimizer = optim.SGD(model.parameters(), lr=0.1)
>>> warmup = optim.LinearLR(optimizer, start_factor=0.1, total_iters=5)
>>> decay  = optim.ExponentialLR(optimizer, gamma=0.95)
>>> scheduler = optim.ChainedScheduler([warmup, decay])
>>> for epoch in range(50):
...     train(...)
...     optimizer.step()
...     scheduler.step()

Methods (3)

dunder

init

→None

__init__(schedulers: list[_LRScheduler])

source

Initialise the ChainedScheduler. See the class docstring for parameter semantics.

step

→None

step()

source

Step all chained schedulers simultaneously.

Calls _LRScheduler.step on every scheduler in order. Each scheduler writes its updated learning rates into the shared optimizer, so the final LR reflects the composition of all schedules.

Examples

>>> for epoch in range(50):
...     optimizer.step()
...     scheduler.step()

get_last_lr

→list of float

get_last_lr()

source

Return the last learning rates from the final scheduler in the chain.

Returns

list of float

Current learning rate of each optimizer param group, as reported by the last scheduler in schedulers.