entr

→Tensor

entr(x: Tensor)

source

Element-wise entropy kernel $-x \log x$ .

Returns the per-element contribution to Shannon entropy. This is the standard building block for evaluating $H(p) = \sum_i \mathrm{entr}(p_i)$ for a discrete distribution, with the limit convention $0 \log 0 = 0$ applied at the boundary.

Parameters

xTensor

Input tensor; any floating-point dtype. Values outside

[0, \infty)

produce NaN.

Returns

Tensor

Element-wise $-x \log x$ with the limit convention at x = 0; same shape and dtype as x.

Notes

Definition:

\mathrm{entr}(x) = \begin{cases} -x \log x, & x > 0 \\[2pt] 0, & x = 0 \\[2pt] \mathrm{NaN}, & x < 0. \end{cases}

The function is concave with maximum $1/e \approx 0.3679$ at $x = 1/e$ . It vanishes at both x = 0 (limit) and x = 1. Related quantities include rel_entr (point-wise relative entropy) and kl_div (KL kernel).

Examples

>>> import lucid
>>> from lucid.special import entr
>>> entr(lucid.tensor([0.0, 0.5, 1.0, 2.0]))
Tensor([0.0000, 0.3466, 0.0000, -1.3863])