gpt2_medium

→GPT2Model

gpt2_medium(pretrained: bool | str = False, weights: GPT2MediumWeights | None = None, overrides: object = {})

source edit

Construct a GPT-2 medium (355M) decoder trunk.

Second of the four canonical OpenAI variants: $L=24$ transformer blocks, $H=1024$ hidden, $A=16$ attention heads, intermediate width 4096. Roughly 355M parameters — the inflection point at which zero-shot performance on reading comprehension / summarisation begins to substantially outperform random baselines (Radford 2019 Table 3).

Model Size

Parameters

pretrainedbool= False

Reserved for future weight registration; currently a no-op.

weights(GPT2MediumWeights, optional, keyword - only)= None

Explicit weights enum member; takes precedence over pretrained.

**overridesobject= {}

Optional GPT2Config field overrides forwarded into the underlying config.

Returns

GPT2Model

Decoder trunk configured with the GPT-2 medium size and any overrides.

Notes

Reference: Radford et al., "Language Models are Unsupervised Multitask Learners", OpenAI Technical Report, 2019.

Examples

>>> import lucid
>>> from lucid.models.text.gpt2 import gpt2_medium
>>> model = gpt2_medium().eval()
>>> input_ids = lucid.tensor([[15496, 11, 995, 13]])
>>> out = model(input_ids)
>>> out.last_hidden_state.shape   # (1, 4, 1024)
(1, 4, 1024)

Used by 1

lucid.models.text.gpt2

>>> import lucid >>> from lucid.models.text.gpt2 import gpt2_medium >>> model = gpt2_medium().eval() >>> input_ids = lucid.tensor([[15496, 11, 995, 13]]) >>> out = model(input_ids) >>> out.last_hidden_state.shape # (1, 4, 1024) (1, 4, 1024)