gpt2_large_lm(pretrained: bool | str = False, weights: GPT2LargeLMWeights | None = None, overrides: object = {})Construct a GPT-2 large (774M) model with the tied causal-LM head.
Same trunk as gpt2_large (L=36, H=1280, A=20, ~774M parameters),
wrapped with the tied LM head for pre-training and free-form generation.
Model Size
Parameters
pretrainedbool= FalseReserved for future weight registration; currently a no-op.
weights(GPT2LargeLMWeights, optional, keyword - only)= NoneExplicit weights enum member; takes precedence over
pretrained.**overridesobject= {}Optional
GPT2Config field overrides forwarded into the
underlying config.Returns
GPT2LMHeadModelGPT-2 large trunk wrapped with the tied LM head.
Notes
Reference: Radford et al., "Language Models are Unsupervised Multitask Learners", OpenAI Technical Report, 2019.
Examples
>>> import lucid
>>> from lucid.models.text.gpt2 import gpt2_large_lm
>>> model = gpt2_large_lm().eval()
>>> input_ids = lucid.tensor([[15496, 11, 995, 13]])
>>> out = model(input_ids)
>>> out.logits.shape # (1, 4, 50257)
(1, 4, 50257)