data
MaskedLMOutput
extends
ModelOutputMaskedLMOutput(logits: Tensor, loss: Tensor | None = None, hidden_states: tuple[Tensor, ...] | None = None, attentions: tuple[Tensor, ...] | None = None)Output of any masked-LM head (BERT, RoFormer, …).
Attributes
logitsTensorPer-position vocabulary logits, shape
(B, T, vocab_size).loss(Tensor or None, optional)Scalar cross-entropy loss reduced over masked positions only
(
ignore_index=-100 for non-masked tokens).hidden_states(tuple[Tensor, ...] or None, optional)Per-layer hidden states.
attentions(tuple[Tensor, ...] or None, optional)Per-layer attention weights.
Notes
Returned by every {Family}ForMaskedLM registration —
BERTForMaskedLM, RoFormerForMaskedLM, and future encoder LM
heads. Loss is computed via MaskedLMMixin.compute_lm_loss.
Examples
>>> model = AutoModelForMaskedLM.from_pretrained("bert_base_mlm")
>>> out = model(input_ids, labels=labels)
>>> out.loss.item()
2.34