Lucid3.0
class

ContinuousBernoulli

extendsDistribution
ContinuousBernoulli(probs: Tensor | float | None = None, logits: Tensor | float | None = None, validate_args: bool | None = None)
source

Continuous Bernoulli distribution on [0,1][0, 1].

A continuous relaxation of the lucid.distributions.Bernoulli distribution introduced by Loaiza-Ganem & Cunningham (2019) to fix the well-known bias incurred by VAE decoders that use a Bernoulli likelihood on real-valued image pixels in [0,1][0, 1]. Unlike the naive "Bernoulli on [0,1]" likelihood, the Continuous Bernoulli has a proper normalising constant and yields unbiased ELBO gradients.

Parameters

probsTensor or float= None
Parameter λ∈[0,1]\lambda \in [0, 1]. Note that E[X]≠λ\mathbb{E}[X] \neq \lambda in general — see Notes for the closed-form mean. Mutually exclusive with logits.
logitsTensor or float= None
Log-odds ℓ=log⁡(λ/(1−λ))\ell = \log(\lambda / (1 - \lambda)). Mutually exclusive with probs.
validate_argsbool= None
If True, validate parameter constraints at construction time.

Notes

Probability density on x∈[0,1]x \in [0, 1]:

p(x;λ)=C(λ) λx(1−λ)1−xp(x; \lambda) = C(\lambda)\, \lambda^x (1 - \lambda)^{1 - x}

where the normalising constant is

C(λ)={2tanh⁡−1(1−2λ)1−2λλ≠122λ=12C(\lambda) = \begin{cases} \dfrac{2 \tanh^{-1}(1 - 2\lambda)}{1 - 2\lambda} & \lambda \neq \tfrac{1}{2} \\[4pt] 2 & \lambda = \tfrac{1}{2} \end{cases}

Mean (closed form):

E[X]={λ2λ−1+12tanh⁡−1(1−2λ)λ≠1212λ=12\mathbb{E}[X] = \begin{cases} \dfrac{\lambda}{2\lambda - 1} + \dfrac{1}{2 \tanh^{-1}(1 - 2\lambda)} & \lambda \neq \tfrac{1}{2} \\[4pt] \tfrac{1}{2} & \lambda = \tfrac{1}{2} \end{cases}

Important properties:

  • Support: [0,1][0, 1] (continuous, not just {0,1}\{0, 1\}).
  • Reduces to Uniform(0,1)\mathrm{Uniform}(0, 1) at λ=1/2\lambda = 1/2.
  • At λ→0\lambda \to 0 or λ→1\lambda \to 1 the density concentrates near 0 or 1 respectively.
  • Reparameterisable via inverse CDF.

The Continuous Bernoulli is the correct likelihood for decoders that output a value in [0,1][0, 1] (e.g. MNIST pixel intensities) when the model is to remain a valid probabilistic model. Using a plain Bernoulli on continuous data produces an improper density and a systematically biased ELBO.

Examples

>>> import lucid
>>> from lucid.distributions import ContinuousBernoulli
>>> d = ContinuousBernoulli(probs=0.7)
>>> d.rsample((4,))
Tensor([...])
>>> d.log_prob(lucid.tensor(0.5))
Tensor(...)

Methods (5)

dunder

__init__

→None
__init__(probs: Tensor | float | None = None, logits: Tensor | float | None = None, validate_args: bool | None = None)
source

Initialise a Continuous Bernoulli distribution.

Parameters

probsTensor | float | None= None
Parameter p∈[0,1]p \in [0, 1]. Mutually exclusive with logits.
logitsTensor | float | None= None
Log-odds l=log⁡(p/(1−p))∈Rl = \log(p/(1-p)) \in \mathbb{R}. Mutually exclusive with probs.
validate_argsbool | None= None
If True, validate parameter constraints at construction time.
prop

mean

→Tensor
mean: Tensor
source

p / (2p−1) + 1 / (2 atanh(1−2p)); ½ when p = ½.

prop

variance

→Tensor
variance: Tensor
source

E[X²] − E[X]²; E[X²] computed via the same normaliser trick.

fn

rsample

→Tensor
rsample(sample_shape: tuple[int, ...] = ())
source

Reparameterised sample via the closed-form icdf.

fn

log_prob

→Tensor
log_prob(value: Tensor)
source

x · l − softplus(l) + log C(p).