gpt2_xlarge

→GPT2Model

gpt2_xlarge(pretrained: bool | str = False, weights: GPT2XLargeWeights | None = None, overrides: object = {})

source edit

Construct a GPT-2 XL (1.5B) decoder trunk.

Largest of the four canonical OpenAI variants: $L=48$ transformer blocks, $H=1600$ hidden, $A=25$ attention heads, intermediate width 6400. Roughly 1.5B parameters — fully released in November 2019 after staged disclosure; remained the largest publicly available autoregressive LM until GPT-3 (Brown 2020).

Model Size

Parameters

pretrainedbool= False

Reserved for future weight registration; currently a no-op.

weights(GPT2XLargeWeights, optional, keyword - only)= None

Explicit weights enum member; takes precedence over pretrained.

**overridesobject= {}

Optional GPT2Config field overrides forwarded into the underlying config.

Returns

GPT2Model

Decoder trunk configured with the GPT-2 XL size and any overrides.

Notes

Reference: Radford et al., "Language Models are Unsupervised Multitask Learners", OpenAI Technical Report, 2019.

Examples

>>> import lucid
>>> from lucid.models.text.gpt2 import gpt2_xlarge
>>> model = gpt2_xlarge().eval()
>>> input_ids = lucid.tensor([[15496, 11, 995, 13]])
>>> out = model(input_ids)
>>> out.last_hidden_state.shape   # (1, 4, 1600)
(1, 4, 1600)

Used by 1

lucid.models.text.gpt2

>>> import lucid >>> from lucid.models.text.gpt2 import gpt2_xlarge >>> model = gpt2_xlarge().eval() >>> input_ids = lucid.tensor([[15496, 11, 995, 13]]) >>> out = model(input_ids) >>> out.last_hidden_state.shape # (1, 4, 1600) (1, 4, 1600)