Store each weight as an index into a small table of values.
A layer's weights are clustered into 2 ** bits representative
values; the package keeps the table and one key per weight, and Core
ML expands it on the way into the operation that uses it. At four
bits that is a quarter of the space float16 takes and an eighth of
float32, for weights that were never using their full range to begin
with.
Unlike WeightPrecision.INT8, which spaces its levels evenly and
spends them wherever the range happens to be, the table is fitted to
the weights — so a layer whose values crowd around zero keeps its
resolution there. The cost is the same in kind: fewer distinct values
than the model was trained with, and verify will say how much.
Attributes
bitsintOne of
1, 2, 3, 4, 6, 8 — palette sizes 2, 4, 8, 16, 64, 256. Anything else is refused rather than rounded, since the
choice is a size/accuracy trade the caller is making
deliberately.
At eight bits, WeightPrecision.INT8 is usually the better
instrument: it stores the same byte per weight but carries a
scale per output channel, and on a trained ResNet-50 it measured
both smaller and closer to the model.Examples
>>> import shutil, tempfile
>>> import lucid, lucid.nn as nn, lucid.coreml as cml
>>> model = nn.Sequential(
... nn.Conv2d(3, 64, 3, padding=1), nn.ReLU(), nn.Conv2d(64, 64, 3, padding=1)
... ).eval()
>>> x, room = lucid.randn(1, 3, 16, 16), tempfile.mkdtemp()
>>> with cml.export(model, x, f"{room}/6bit.mlpackage",
... weights=cml.Palettize(bits=6)) as package:
... print(package.palettized, package.deployment_target)
True DeploymentTarget.IOS18
>>> shutil.rmtree(room)
Below six bits, fit the palette during a fine-tune rather than
afterwards — see lucid.coreml.CompressionAware:
>>> aware = cml.CompressionAware(model, weights=cml.Palettize(bits=2))
>>> aware.covered # the first layer is too small to palettize
['2.weight']