bert_medium

→BERTModel

bert_medium(pretrained: bool | str = False, weights: BERTMediumWeights | None = None, overrides: object = {})

source edit

Construct a BERT-Medium encoder.

Largest of the Turc et al., 2019 distillation sizes: $L=8$ layers, $H=512$ hidden, $A=8$ heads, intermediate width 2048. Roughly 41M parameters — closest in depth to Base while keeping the Small width.

Model Size

Parameters

pretrainedbool= False

Reserved for future weight registration; currently a no-op.

weights(BERTMediumWeights, optional, keyword - only)= None

Explicit weights enum member; takes precedence over pretrained.

**overridesobject= {}

Optional BERTConfig field overrides forwarded into the underlying config.

Returns

BERTModel

Encoder trunk configured with the BERT-Medium size and any overrides.

Notes

Reference: Turc, Chang, Lee, and Toutanova, "Well-Read Students Learn Better", 2019 (arXiv:1908.08962).

Examples

>>> import lucid
>>> from lucid.models.text.bert import bert_medium
>>> model = bert_medium().eval()
>>> out = model(lucid.tensor([[101, 7592, 102]]))
>>> out.last_hidden_state.shape   # (1, 3, 512)
(1, 3, 512)

Used by 1

lucid.models.text.bert

>>> import lucid >>> from lucid.models.text.bert import bert_medium >>> model = bert_medium().eval() >>> out = model(lucid.tensor([[101, 7592, 102]])) >>> out.last_hidden_state.shape # (1, 3, 512) (1, 3, 512)