Keep only the largest weights, and a bit saying where they were.
The smallest ratio of each weight tensor by magnitude is set to
zero, and the package stores the survivors plus one bit per element.
Below about half sparsity that costs more than it saves — the mask is
an eighth of a byte per weight whether the weight survives or not —
so the useful settings start around 0.5 and the saving grows from
there.
This is magnitude pruning applied at export, not training: the model
is not fine-tuned afterwards, so accuracy falls faster than it would
for a network pruned and then retrained. verify measures it.
Attributes
ratiofloatFraction of each weight set to zero, in
[0, 1).Examples
>>> import shutil, tempfile
>>> import lucid, lucid.nn as nn, lucid.coreml as cml
>>> model = nn.Sequential(
... nn.Conv2d(3, 64, 3, padding=1), nn.ReLU(), nn.Conv2d(64, 64, 3, padding=1)
... ).eval()
>>> x, room = lucid.randn(1, 3, 16, 16), tempfile.mkdtemp()
>>> with cml.export(model, x, f"{room}/sparse.mlpackage",
... weights=cml.Sparsify(ratio=0.5)) as package:
... cost = package.verify(model, x, relative=True)
>>> cost > 1e-3 # pruned and not retrained: not free
True
>>> shutil.rmtree(room)