data
DreamerBehaviorOutput
extends
ModelOutputDreamerBehaviorOutput(actor_loss: Tensor, value_loss: Tensor, lambda_return: Tensor, imagined_reward: Tensor, imagined_value: Tensor, imagined_action: Tensor, imagined_pcont: Tensor | None = None)What the imagination pass produces — the actor's and critic's terms.
Attributes
actor_lossTensor-mean(w * V_lambda), minimised by the actor's optimiser only.value_lossTensorHalf the discounted squared error between the critic and the
(detached) target.
lambda_returnTensorThe targets themselves,
(N, H) for N imagined trajectories
over a horizon of H.imagined_rewardTensorReward the model predicted along the imagined trajectories,
(N, H + 1).imagined_valueTensorThe critic's estimate along the same states,
(N, H + 1).imagined_actionTensorActions the actor proposed,
(N, H, action_dim).imagined_pcontTensor or NonePredicted continuation probability at each imagined state,
(N, H + 1); None when the discount is held constant.Notes
actor_loss and value_loss must be given separate optimisers
over disjoint parameter groups — see the module docstring.
Examples
>>> import lucid
>>> from lucid.models.generative.dreamer import (
... DreamerConfig, DreamerForWorldModeling)
>>> cfg = DreamerConfig(action_dim=2, cnn_depth=2, stoch_size=4,
... deter_size=8, hidden_size=8, actor_hidden=8,
... value_hidden=8, reward_hidden=8, horizon=3)
>>> model = DreamerForWorldModeling(cfg)
>>> out = model(lucid.randn((1, 3, 3, 64, 64)), lucid.randn((1, 3, 2)),
... lucid.randn((1, 3)))
>>> out.behavior.lambda_return.shape
(3, 3)