data

CausalLMOutput

extendsModelOutput

CausalLMOutput(logits: Tensor, loss: Tensor | None = None, past_key_values: Cache | tuple[tuple[Tensor, Tensor], ...] | None = None, hidden_states: tuple[Tensor, ...] | None = None, attentions: tuple[Tensor, ...] | None = None)

source edit

Output of any causal (left-to-right) language model head.

Attributes

logitsTensor

Per-position vocabulary logits, shape (B, T, vocab_size).

loss(Tensor or None, optional)

Scalar cross-entropy loss when labels were supplied. Labels are typically shifted internally so loss reflects next-token prediction.

past_key_values(tuple[tuple[Tensor, Tensor], ...] or None, optional)

Per-layer (key, value) cache for incremental decoding. Each tensor shape (B, num_heads, T, head_dim).

hidden_states(tuple[Tensor, ...] or None, optional)

Per-layer hidden states.

attentions(tuple[Tensor, ...] or None, optional)

Per-layer attention weights.

Notes

Produced by GPT-1, GPT-2 (GPT2LMHeadModel), and any future decoder-only LM head. CausalLMMixin consumes the logits field; future cache-aware sampling will plumb through past_key_values.

Examples

>>> model = AutoModelForCausalLM.from_pretrained("gpt2_small")
>>> out = model(input_ids)
>>> out.logits.shape
(1, 128, 50257)

Used by 4

Constructors

dunder

init

→None

__init__(logits: Tensor, loss: Tensor | None = None, past_key_values: Cache | tuple[tuple[Tensor, Tensor], ...] | None = None, hidden_states: tuple[Tensor, ...] | None = None, attentions: tuple[Tensor, ...] | None = None)

source edit

data

CausalLMOutput

extendsModelOutput

CausalLMOutput(logits: Tensor, loss: Tensor | None = None, past_key_values: Cache | tuple[tuple[Tensor, Tensor], ...] | None = None, hidden_states: tuple[Tensor, ...] | None = None, attentions: tuple[Tensor, ...] | None = None)

source edit

Output of any causal (left-to-right) language model head.

Attributes

logitsTensor

Per-position vocabulary logits, shape (B, T, vocab_size).

loss(Tensor or None, optional)

Scalar cross-entropy loss when labels were supplied. Labels are typically shifted internally so loss reflects next-token prediction.

past_key_values(tuple[tuple[Tensor, Tensor], ...] or None, optional)

Per-layer (key, value) cache for incremental decoding. Each tensor shape (B, num_heads, T, head_dim).

hidden_states(tuple[Tensor, ...] or None, optional)

Per-layer hidden states.

attentions(tuple[Tensor, ...] or None, optional)

Per-layer attention weights.

Notes

Produced by GPT-1, GPT-2 (GPT2LMHeadModel), and any future decoder-only LM head. CausalLMMixin consumes the logits field; future cache-aware sampling will plumb through past_key_values.

Examples

>>> model = AutoModelForCausalLM.from_pretrained("gpt2_small")
>>> out = model(input_ids)
>>> out.logits.shape
(1, 128, 50257)

Used by 4

Constructors

dunder

init

→None

__init__(logits: Tensor, loss: Tensor | None = None, past_key_values: Cache | tuple[tuple[Tensor, Tensor], ...] | None = None, hidden_states: tuple[Tensor, ...] | None = None, attentions: tuple[Tensor, ...] | None = None)

source edit