fn
sinusoidal_pos_embedding_op
→TensorImplPtrint sinusoidal_pos_embedding_op(int seq_len, int embed_dim, Dtype dtype, Device device)Generate the fixed sinusoidal position-encoding matrix from Vaswani et al. 2017, "Attention Is All You Need".
Returns the non-trainable matrix
\quad \text{PE}(p, 2i+1) = \cos\!\left(\frac{p}{10000^{2i/D}}\right),$$ where $p \in [0, L)$ indexes positions and $i \in [0, D/2)$ indexes frequency bands. No autograd node is created — the result is a constant tensor used as an additive bias on token embeddings.Parameters
seq_lenstd::int64_tSequence length
L. Must be non-negative.embed_dimstd::int64_tEmbedding dimension
D. Must be strictly positive.dtypeDtypeFloating-point dtype of the output.
deviceDeviceDevice on which to allocate the output.
Returns
TensorImplPtrOutput tensor of shape (seq_len, embed_dim).
Raises
LucidErrorIf
seq_len < 0 or embed_dim <= 0.