class

Mask2FormerForSemanticSegmentation

extendsPretrainedModel

Mask2FormerForSemanticSegmentation(config: Mask2FormerConfig)

source edit

Mask2Former universal segmentation model (Cheng et al., CVPR 2022).

A unified mask-classification architecture that achieves state-of-the-art results across semantic, instance, and panoptic segmentation with the same code path. Its key advance over MaskFormer is masked attention: each transformer decoder layer restricts cross-attention to the foreground region of the previous layer's predicted mask, which sharply accelerates convergence and improves small-object segmentation.

The pipeline is: a Swin backbone -> a multi-scale deformable-attention pixel decoder producing per-pixel mask features at stride 4 plus three coarser memory levels -> $L$ masked-attention decoder layers cycling through the three memory levels with learnable query embeddings -> sibling class and mask-embedding heads.

Parameters

configMask2FormerConfig

Frozen architecture spec. Use the Swin-backbone factories (mask2former_swin_tiny / small / base / large) for the paper-cited ADE20K variants.

Attributes

configMask2FormerConfig

Stored copy of the config that built this model.

pixel_level_module_PixelLevelModule

Swin encoder + MSDeformAttn pixel decoder.

transformer_module_TransformerModule

Learnable queries + a 9-layer masked-attention decoder.

class_predictornn.Linear

Per-query

(K + 1)

class head (+1 for "no object").

Notes

See Cheng et al., "Masked-attention Mask Transformer for Universal Image Segmentation", CVPR 2022 (arXiv:2112.01527). For semantic inference the model drops the no-object slot and collapses queries via the softmax(class)[..., :-1] ⊗ sigmoid(mask) einsum into a $(B, K, H, W)$ logit map, upsampled to the input resolution.

Examples

>>> import lucid
>>> from lucid.models.vision.mask2former import mask2former_swin_tiny
>>> model = mask2former_swin_tiny()
>>> x = lucid.randn(1, 3, 384, 384)
>>> out = model(x)
>>> out.logits.shape   # (B, K, H, W)
(1, 150, 384, 384)

Used by 2

Constructors

dunder

init

→None

__init__(config: Mask2FormerConfig)

source edit

Instance methods

forward

→SemanticSegmentationOutput

forward(x: Tensor, targets: dict[str, Tensor] | None = None)

source edit

Run Mask2Former for semantic segmentation.

Parameters

xTensor

(B, C, H, W) image batch.

targetsdict[str, Tensor] | None= None

Currently ignored (inference path).

Returns

SemanticSegmentationOutput

SemanticSegmentationOutput with logits of shape

>>> import lucid >>> from lucid.models.vision.mask2former import mask2former_swin_tiny >>> model = mask2former_swin_tiny() >>> x = lucid.randn(1, 3, 384, 384) >>> out = model(x) >>> out.logits.shape # (B, K, H, W) (1, 150, 384, 384)