dreamer_discrete(pretrained: bool = False, overrides: object = {})Construct Dreamer as the paper configures it for Atari and DMLab.
A categorical policy over the game's buttons, a 10-step imagination horizon, the divergence scaled to 0.1, and a discount head — the four changes the paper's Discrete control paragraph makes "to account for the higher complexity of these tasks".
Model Size
Parameters
pretrainedbool= FalseTrue raises.**overridesobject= {}DreamerConfig field overrides. action_dim is
the number of actions the game exposes.Returns
DreamerModelThe trunk, configured for discrete control.
Notes
Reference: Hafner, Lillicrap, Ba, and Norouzi, "Dream to Control: Learning Behaviors by Latent Imagination", ICLR, 2020 (arXiv:1912.01603), Appendix A.
The action is sampled with straight-through gradients, as the paper specifies. That is a biased estimator, and it is the one this paper used; DreamerV2 later replaced it with the score function for discrete actions.
Epsilon-greedy exploration and tanh reward bounding belong to the rollout and the environment respectively — see this module's docstring.
Examples
>>> from lucid.models.generative.dreamer import dreamer_discrete
>>> model = dreamer_discrete(action_dim=18).eval()
>>> model.config.action_space, model.config.horizon, model.config.kl_weight
('discrete', 10, 0.1)