ContinuousBernoulli

Probability density on $x \in [0, 1]$:

$$p(x; \lambda) = C(\lambda)\, \lambda^x (1 - \lambda)^{1 - x}$$

where the normalising constant is

$$C(\lambda) = \begin{cases} \dfrac{2 \tanh^{-1}(1 - 2\lambda)}{1 - 2\lambda} & \lambda \neq \tfrac{1}{2} \\[4pt] 2 & \lambda = \tfrac{1}{2} \end{cases}$$

Mean (closed form):

$$\mathbb{E}[X] = \begin{cases} \dfrac{\lambda}{2\lambda - 1} + \dfrac{1}{2 \tanh^{-1}(1 - 2\lambda)} & \lambda \neq \tfrac{1}{2} \\[4pt] \tfrac{1}{2} & \lambda = \tfrac{1}{2} \end{cases}$$

Important properties:

• Support: $[0, 1]$ (continuous, not just $\{0, 1\}$).
• Reduces to $\mathrm{Uniform}(0, 1)$ at $\lambda = 1/2$.
• At $\lambda \to 0$ or $\lambda \to 1$ the density concentrates near 0 or 1 respectively.
• Reparameterisable via inverse CDF.

The Continuous Bernoulli is the correct likelihood for decoders that output a value in $[0, 1]$ (e.g. MNIST pixel intensities) when the model is to remain a valid probabilistic model. Using a plain Bernoulli on continuous data produces an improper density and a systematically biased ELBO.