gpt2_small_cls(pretrained: bool | str = False, weights: GPT2SmallWeights | None = None, overrides: object = {})Construct a GPT-2 small (124M) model with a sequence-classification head.
Same trunk as gpt2_small (L=12, H=768, A=12), augmented with a
last-token linear classifier of output width config.num_labels.
The canonical recipe for fine-tuning GPT-2 on GLUE-style sentence
classification tasks when decoder-style features are preferred to
bidirectional ones.
Model Size
Parameters
pretrainedbool or str= FalseEncoder-weight selector.
False → fully random init; True
→ loads the pretrained gpt2_small decoder trunk
(GPT2SmallWeights.DEFAULT) into the .transformer
submodule; a tag string selects a specific encoder checkpoint.
The classifier head is always randomly initialised (fine-tuning
starting point — no GLUE-fine-tuned head ships).weights(GPT2SmallWeights, optional, keyword - only)= NoneExplicit encoder-weights enum member; takes precedence over
pretrained.**overridesobject= {}Optional
GPT2Config field overrides forwarded into the
underlying config. Pass num_labels=N to set the number of
classes (default 2). Overrides that change the trunk shape are
incompatible with loading pretrained encoder weights.Returns
GPT2ForSequenceClassificationGPT-2 small trunk wrapped with the last-token classifier head (encoder pretrained when requested; head random).
Notes
Reference: Radford et al., "Language Models are Unsupervised Multitask Learners", OpenAI Technical Report, 2019.
Examples
>>> import lucid
>>> from lucid.models.text.gpt2 import gpt2_small_cls
>>> model = gpt2_small_cls(num_labels=3).eval()
>>> input_ids = lucid.tensor([[15496, 11, 995, 13]])
>>> out = model(input_ids)
>>> out.logits.shape # (1, num_labels=3)
(1, 3)