roformer_cls(pretrained: bool | str = False, weights: RoFormerWeights | None = None, overrides: object = {})Construct a RoFormer model with a sequence-classification head.
Same trunk as roformer (L=12, H=768, A=12), augmented with a
dropout-regularised linear classifier on the pooled first-token
embedding. Use for GLUE-style fine-tunes when the RoPE relative
position bias is preferred to absolute position embeddings.
Model Size
Parameters
pretrainedbool or str= FalseEncoder-weight selector.
False → fully random init; True
→ loads the pretrained roformer encoder
(RoFormerWeights.DEFAULT) into the .roformer trunk; a
tag string selects a specific encoder checkpoint. The classifier
head is always randomly initialised (fine-tuning starting point —
no fine-tuned head ships).weights(RoFormerWeights, optional, keyword - only)= NoneExplicit encoder-weights enum member; takes precedence over
pretrained.**overridesobject= {}Optional
RoFormerConfig field overrides forwarded into the
underlying config. Pass num_labels=N to set the number of
classes (default 2). Overrides that change the encoder shape are
incompatible with loading pretrained encoder weights.Returns
RoFormerForSequenceClassificationRoFormer trunk wrapped with the pooled classifier head (encoder pretrained when requested; head random).
Notes
Reference: Su et al., "RoFormer: Enhanced Transformer with Rotary Position Embedding", Neurocomputing, vol. 568, 2024 (arXiv:2104.09864).
Examples
>>> import lucid
>>> from lucid.models.text.roformer import roformer_cls
>>> model = roformer_cls(num_labels=3).eval()
>>> input_ids = lucid.tensor([[101, 7592, 102]])
>>> out = model(input_ids)
>>> out.logits.shape # (1, num_labels=3)
(1, 3)