Uniform actions, in whichever shape the action space has.
What fills the buffer before there is anything worth learning from. Both papers seed with a handful of random episodes for exactly this reason: a world model fitted on nothing has no opinion for a policy to improve on.
Parameters
action_dimintdiscrete.discretebool= Falseaction_dim choices instead of a vector in
(-1, 1).Notes
The flag is not cosmetic. Seeding a discrete agent's buffer with
continuous noise trains its dynamics on action vectors it will never
be given again — a uniform draw in (-1, 1) looks nothing like a
one-hot — so the model is fitted on one input distribution and asked
to act under another.
Examples
>>> from lucid.utils.rollout import RandomPolicy
>>> policy = RandomPolicy(3)
>>> policy.reset()
>>> policy(None).shape
(3,)
>>> onehot = RandomPolicy(4, discrete=True)
>>> onehot.reset()
>>> float(onehot(None).sum())
1.0Used by 1
Constructors
2Initialise the policy. See the class docstring for parameters.
Draw a uniform action, reading the observation only for its device.
The draw ignores what is in the frame — that is the point of a random policy — but not where it is. Seeding an accelerator environment with actions left on the CPU builds episodes whose five tensors do not agree, and the mismatch is only reported much later, inside whichever op a world model happens to reach first.