class

SequentialLR

SequentialLR(optimizer: Optimizer, schedulers: list[_LRScheduler], milestones: list[int], last_epoch: int = -1)

source

Chain multiple schedulers and switch between them at epoch milestones.

SequentialLR runs its list of schedulers one at a time, activating the next scheduler whenever the epoch counter crosses a milestone. Before the first milestone, schedulers[0] is active; between milestone k-1 and k, schedulers[k] is active.

Parameters

optimizerOptimizer

The optimizer shared by all schedulers.

schedulerslist of _LRScheduler

Ordered list of scheduler instances to activate in sequence. Must have len(schedulers) == len(milestones) + 1.

milestoneslist of int

Sorted list of epoch indices at which to switch to the next scheduler.

last_epochint= -1

The index of the last epoch (default: -1).

Attributes

optimizerOptimizer

The shared optimizer.

schedulerslist of _LRScheduler

All schedulers in activation order.

milestoneslist of int

Epoch indices marking scheduler transitions.

last_epochint

Number of step calls completed.

Notes

A common pattern is to combine a short warmup with a long cosine decay:

.. code-block:: python

warmup   = optim.LinearLR(optimizer, start_factor=0.1, total_iters=5)
cosine   = optim.CosineAnnealingLR(optimizer, T_max=95)
combined = optim.SequentialLR(
    optimizer,
    schedulers=[warmup, cosine],
    milestones=[5],
)

Examples

>>> import lucid.optim as optim
>>> optimizer = optim.SGD(model.parameters(), lr=0.1)
>>> s1 = optim.ConstantLR(optimizer, factor=0.1, total_iters=5)
>>> s2 = optim.ExponentialLR(optimizer, gamma=0.9)
>>> scheduler = optim.SequentialLR(optimizer, schedulers=[s1, s2], milestones=[5])
>>> for epoch in range(50):
...     train(...)
...     optimizer.step()
...     scheduler.step()

Methods (3)

dunder

init

→None

__init__(optimizer: Optimizer, schedulers: list[_LRScheduler], milestones: list[int], last_epoch: int = -1)

source

Initialise the SequentialLR. See the class docstring for parameter semantics.

step

→None

step()

source

Advance the active scheduler by one epoch, switching if a milestone is reached.

Increments the internal epoch counter, checks whether any milestone has been crossed, and delegates to the currently active scheduler's _LRScheduler.step.

Examples

>>> for epoch in range(100):
...     optimizer.step()
...     scheduler.step()

get_last_lr

→list of float

get_last_lr()

source

Return the last learning rates from the currently active scheduler.

Returns

list of float

Current learning rate of each optimizer param group, as reported by the active scheduler.