roformer_token_cls(pretrained: bool | str = False, weights: RoFormerWeights | None = None, overrides: object = {})Construct a RoFormer model with a per-token classification head.
Same trunk as roformer (L=12, H=768, A=12), augmented with a
dropout-regularised per-position linear classifier of output width
config.num_labels. Use for sequence-labelling tasks (NER, POS,
chunking) when a RoPE-trained encoder is preferred.
Model Size
Parameters
pretrainedbool or str= FalseEncoder-weight selector.
False → fully random init; True
→ loads the pretrained roformer encoder
(RoFormerWeights.DEFAULT) into the .roformer trunk; a
tag string selects a specific encoder checkpoint. The per-token
classifier head is always randomly initialised (fine-tuning
starting point — no NER/POS-fine-tuned head ships).weights(RoFormerWeights, optional, keyword - only)= NoneExplicit encoder-weights enum member; takes precedence over
pretrained.**overridesobject= {}Optional
RoFormerConfig field overrides forwarded into the
underlying config. Pass num_labels=N to set the tag set size.
Overrides that change the encoder shape are incompatible with
loading pretrained encoder weights.Returns
RoFormerForTokenClassificationRoFormer trunk wrapped with the per-token classifier head (encoder pretrained when requested; head random).
Notes
Reference: Su et al., "RoFormer: Enhanced Transformer with Rotary Position Embedding", Neurocomputing, vol. 568, 2024 (arXiv:2104.09864).
Examples
>>> import lucid
>>> from lucid.models.text.roformer import roformer_token_cls
>>> model = roformer_token_cls(num_labels=9).eval()
>>> input_ids = lucid.tensor([[101, 2198, 7592, 102]])
>>> out = model(input_ids)
>>> out.logits.shape # (1, T=4, num_labels=9)
(1, 4, 9)