data
DreamerV2BehaviorOutput
extends
ModelOutputDreamerV2BehaviorOutput(actor_loss: Tensor, value_loss: Tensor, lambda_return: Tensor, entropy: Tensor, imagined_reward: Tensor, imagined_value: Tensor, imagined_action: Tensor, imagined_discount: Tensor | None = None)What the imagination pass produces.
Attributes
actor_loss, value_lossTensorScalars, each for its own optimiser.
lambda_returnTensorThe targets,
(N, H), computed from the target critic.entropyTensorMean policy entropy over the scored states — the quantity the
bonus is paid on, exposed because it is the first thing to look
at when a policy stops exploring.
imagined_reward, imagined_valueTensorAlong the imagined trajectory,
(N, H + 1).imagined_actionTensorWhat the actor proposed,
(N, H, action_dim).imagined_discountTensor or NonePredicted continuation probability,
(N, H + 1); None when
the discount is held constant.Examples
>>> import lucid
>>> from lucid.models import dreamer_v2_world_model
>>> model = dreamer_v2_world_model(action_dim=2, cnn_depth=2, stoch_size=3,
... discrete=4, deter_size=8, hidden_size=8, actor_hidden=8,
... value_hidden=8, reward_hidden=8, horizon=3, pcont=False)
>>> out = model(lucid.randn((1, 4, 3, 64, 64)), lucid.randn((1, 4, 2)),
... lucid.randn((1, 4)))
>>> out.behavior.lambda_return.shape
(4, 3)