A value the package carries from one prediction to the next.
Core ML keeps it: the caller neither passes it in nor gets it back, and each prediction sees what the last one wrote. A decoder's key-value cache is the case this exists for.
Lucid's side has to be a pair — an input the model reads and an output
it returns — rather than a buffer it mutates. The tracer records a
pure graph, so an in-place buffer write does not appear in it at all;
a package built from that would agree on the first call and stop
accumulating on every one after, which export refuses.
The state begins at zero. Nothing carries the example's values into it, so a model that needs a different starting point has to be given one through an ordinary input.
Attributes
inputstroutputstrExamples
The cache stops being an input the caller passes; Core ML keeps it
and each prediction sees what the last one wrote:
>>> import shutil, tempfile
>>> import lucid, lucid.nn as nn, lucid.coreml as cml
>>> class Decoder(nn.Module):
... def forward(self, x, cache):
... cache = cache + x # what this step writes back
... return cache, cache * 2.0
>>> token, room = lucid.ones(1, 4) * 3.0, tempfile.mkdtemp()
>>> package = cml.export(
... Decoder().eval(), {"x": token, "cache": lucid.zeros(1, 4)},
... f"{room}/decoder.mlpackage", precision=cml.Precision.FLOAT16,
... state=[cml.State(input="cache", output="output_0")],
... )
>>> package.input_names
['x']
>>> package.predict(token).tolist(), package.predict(token).tolist()
([[6.0, 6.0, 6.0, 6.0]], [[12.0, 12.0, 12.0, 12.0]])
>>> package.reset_state() # start a fresh sequence
>>> package.predict(token).tolist()
[[6.0, 6.0, 6.0, 6.0]]
>>> package.close()
>>> shutil.rmtree(room)Used by 2
Constructors
1Initialise the driver; the executable itself is lazy.
Parameters
modelfused_step for semantics.loss_fnfused_step for semantics.optimizerfused_step for semantics.grad_scalerGradScalerfused_step. When provided and enabled, the
trace records the scale → unscale → found_inf → conditional
update plumbing entirely inside the executable so the
user-facing step is identical to a no-scaler call.Raises
ValueErroroptimizer exposes no trainable parameters (every
param_group is empty).