bert_base_token_cls

→BERTForTokenClassification

bert_base_token_cls(pretrained: bool | str = False, weights: BERTBaseNERWeights | None = None, overrides: object = {})

source edit

Construct a BERT-Base per-token classification model (CoNLL-2003 NER).

A per-position linear classifier over the encoder sequence output, for sequence-labelling tasks: named-entity recognition, POS tagging, chunking.

Model Size

Parameters

pretrainedbool or str= False

Weight selector. False → random init (uncased _CFG_BASE, num_labels=2); True → the CoNLL-2003 NER checkpoint (BERTBaseNERWeights.CONLL2003, from dslim/bert-base-NER) — an inference-ready 9-way BIO tagger (O, B/I-PER, B/I-ORG, B/I-LOC, B/I-MISC; test F1 ≈ 91.3). This is a cased BERT-Base (vocab 28 996); when pretrained is requested the model is built with the matching cased config, so tokenize with the cased BERTTokenizer (do_lower_case=False).

weights(BERTBaseNERWeights, optional, keyword - only)= None

Explicit weights enum member; takes precedence over pretrained.

**overridesobject= {}

Optional BERTConfig field overrides. When given, they override the (random-init) _CFG_BASE and disable pretrained loading's cased-config switch — pass your own vocab_size / num_labels for custom fine-tuning.

Returns

BERTForTokenClassification

BERT-Base wrapped with the per-token classifier head (fully fine-tuned on CoNLL-2003 when pretrained is requested).

Notes

Reference: Devlin, Chang, Lee, and Toutanova, "BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding", NAACL 2019 (arXiv:1810.04805) §4.3.

Examples

>>> import lucid
>>> from lucid.models.text.bert import bert_base_token_cls
>>> model = bert_base_token_cls(pretrained=True).eval()   # CoNLL-2003 NER (9 tags)
>>> input_ids = lucid.tensor([[101, 2198, 7592, 102]])
>>> out = model(input_ids)
>>> out.logits.shape   # (1, T=4, num_labels=9)
(1, 4, 9)

Used by 1

lucid.models.text.bert