data
GPTConfig
extends
LanguageModelConfigGPTConfig(vocab_size: int = 40478, hidden_size: int = 768, num_hidden_layers: int = 12, num_attention_heads: int = 12, intermediate_size: int = 3072, hidden_act: TextActivation = 'gelu_new', max_position_embeddings: int = 512, pad_token_id: int | None = None, bos_token_id: int | None = None, eos_token_id: int | None = None, hidden_dropout: float = 0.1, attention_dropout: float = 0.1, initializer_range: float = 0.02, layer_norm_eps: float = 1e-05, tie_word_embeddings: bool = True, use_cache: bool = True, num_labels: int = 2, classifier_dropout: float | None = None)Configuration for every GPT-1 variant.
Args mirror the HuggingFace OpenAIGPTConfig field set so checkpoint
porting is a flat key rename.
Used by 3
Constructors
1dunder
__init__
→None__init__(vocab_size: int = 40478, hidden_size: int = 768, num_hidden_layers: int = 12, num_attention_heads: int = 12, intermediate_size: int = 3072, hidden_act: TextActivation = 'gelu_new', max_position_embeddings: int = 512, pad_token_id: int | None = None, bos_token_id: int | None = None, eos_token_id: int | None = None, hidden_dropout: float = 0.1, attention_dropout: float = 0.1, initializer_range: float = 0.02, layer_norm_eps: float = 1e-05, tie_word_embeddings: bool = True, use_cache: bool = True, num_labels: int = 2, classifier_dropout: float | None = None)