data
MaskFormerOutput
extends
SemanticSegmentationOutputMaskFormerOutput(logits: Tensor, loss: Tensor | None = None, hidden_states: tuple[Tensor, ...] | None = None, class_queries_logits: Tensor | None = None, masks_queries_logits: Tensor | None = None)Segmentation output that also carries the raw per-query predictions.
logits is the semantic map the paper's semantic inference
produces by marginalising the queries. That marginalisation is
lossy: it collapses N queries into K class channels, so the general
mask-classification inference of §3.4 — which needs each query's
class distribution and its own mask — cannot be recovered from it.
Both are returned here so panoptic and instance inference are
expressible without re-running the model.
Attributes
Examples
Mask classification returns three tensors, not one. logits is the
per-pixel semantic map the two query tensors were resolved into, and
is what you would compare against a segmentation label.
>>> import lucid
>>> from lucid.models.vision.maskformer import maskformer_resnet50
>>> model = maskformer_resnet50().eval()
>>> out = model(lucid.randn(1, 3, 224, 224))
>>> out.logits.shape # (B, K, H, W)
(1, 150, 224, 224)
The queries are the model's own representation: 100 of them, each
carrying a class distribution and a mask. The class axis is 151 and
not 150 because the last slot is "no object" — a query that matched
nothing, which is how a fixed number of queries covers a variable
number of regions.
>>> out.class_queries_logits.shape # (B, Q, K + 1)
(1, 100, 151)
>>> out.masks_queries_logits.shape # (B, Q, H/4, W/4)
(1, 100, 56, 56)