gpt2_xlarge_lm(pretrained: bool | str = False, weights: GPT2XLargeLMWeights | None = None, overrides: object = {})Construct a GPT-2 XL (1.5B) model with the tied causal-LM head.
Same trunk as gpt2_xlarge (L=48, H=1600, A=25, ~1.5B parameters),
wrapped with the tied LM head for pre-training and free-form generation.
Model Size
Parameters
pretrainedbool= FalseReserved for future weight registration; currently a no-op.
weights(GPT2XLargeLMWeights, optional, keyword - only)= NoneExplicit weights enum member; takes precedence over
pretrained.**overridesobject= {}Optional
GPT2Config field overrides forwarded into the
underlying config.Returns
GPT2LMHeadModelGPT-2 XL trunk wrapped with the tied LM head.
Notes
Reference: Radford et al., "Language Models are Unsupervised Multitask Learners", OpenAI Technical Report, 2019.
Examples
>>> import lucid
>>> from lucid.models.text.gpt2 import gpt2_xlarge_lm
>>> model = gpt2_xlarge_lm().eval()
>>> input_ids = lucid.tensor([[15496, 11, 995, 13]])
>>> out = model(input_ids)
>>> out.logits.shape # (1, 4, 50257)
(1, 4, 50257)