Lucid3.0
class

ReduceLROnPlateau

ReduceLROnPlateau(optimizer: Optimizer, mode: str = 'min', factor: float = 0.1, patience: int = 10, verbose: bool = False, threshold: float = 0.0001, min_lr: float = 0)
source

Reduce the learning rate when a monitored metric stops improving.

Once the metric fails to improve by more than threshold for patience consecutive epochs, the learning rate is multiplied by factor. This is useful when progress stalls and a smaller step size may help escape a plateau:

η←max⁡(η⋅factor,  ηmin⁡)\eta \leftarrow \max(\eta \cdot \text{factor},\; \eta_{\min})

Parameters

optimizerOptimizer
Wrapped optimizer.
modestr= 'min'
One of "min" or "max". In "min" mode the LR is reduced when the metric has stopped decreasing; in "max" mode it is reduced when the metric has stopped increasing (default: "min").
factorfloat= 0.1
Multiplicative factor by which the learning rate is reduced (default: 0.1). Must be less than 1.
patienceint= 10
Number of epochs with no improvement after which the LR is reduced (default: 10).
verbosebool= False
Print a message when the LR is reduced (default: False).
thresholdfloat= 0.0001
Minimum change to qualify as an improvement (default: 1e-4).
min_lrfloat= 0
A lower bound on the learning rate (default: 0).

Attributes

optimizerOptimizer
The optimizer being scheduled.
modestr
"min" or "max".
factorfloat
LR reduction factor.
patienceint
Epochs to wait before reducing.
thresholdfloat
Minimum improvement threshold.
min_lrfloat
Floor on the learning rate.

Notes

Unlike the epoch-based schedulers that inherit from _LRScheduler, ReduceLROnPlateau does not inherit from _LRScheduler because it requires a metric value at each step call rather than advancing by a fixed epoch count.

A typical use-case is to monitor validation loss:

.. code-block:: python

scheduler = optim.ReduceLROnPlateau(optimizer, mode="min", patience=5)
for epoch in range(epochs):
    train(...)
    val_loss = validate(...)
    scheduler.step(val_loss)

Examples

>>> import lucid.optim as optim
>>> optimizer = optim.Adam(model.parameters(), lr=1e-3)
>>> scheduler = optim.ReduceLROnPlateau(
...     optimizer, mode="min", factor=0.5, patience=5
... )
>>> for epoch in range(100):
...     val_loss = evaluate(...)
...     scheduler.step(val_loss)

Methods (2)

dunder

__init__

→None
__init__(optimizer: Optimizer, mode: str = 'min', factor: float = 0.1, patience: int = 10, verbose: bool = False, threshold: float = 0.0001, min_lr: float = 0)
source

Initialise the ReduceLROnPlateau. See the class docstring for parameter semantics.

fn

step

→None
step(metrics: float)
source

Update the scheduler with the latest monitored metric value.

Checks whether the metric has improved relative to the stored best. If it has not improved for more than patience epochs, the learning rate is reduced by factor.

Parameters

metricsfloat
The current value of the monitored metric (e.g. validation loss or validation accuracy, depending on mode).

Examples

>>> scheduler.step(val_loss)