module

data

21 members

lucid.utils.data

lucid.utils.data: dataset and dataloader utilities.

Classes

Abstract base class for map-style datasets.

Base class for iterable-style (stream) datasets.

Dataset wrapping one or more Tensors, indexed along their first axis.

Dataset formed by concatenating several map-style datasets end-to-end.

Concatenate multiple `IterableDataset` instances end-to-end.

Bundle several map-style datasets so each index returns a stacked tuple.

View into a parent dataset restricted to a list of indices.

Abstract base class for index samplers used by `DataLoader`.

Yield indices in fixed order `0, 1, ..., len(data_source) - 1`.

Yield indices in random order, with or without replacement.

Yield a random permutation of a fixed list of indices each epoch.

Yield indices drawn proportionally to user-supplied weights.

Group an inner sampler's indices into mini-batches.

Subset-and-shuffle sampler for distributed training.

Combine a dataset with a sampler to provide iteration over mini-batches.

Per-worker context published inside a `DataLoader` worker process.

Randomly split a dataset into non-overlapping `Subset` views.

Composable collate dispatcher with user-overridable type handlers.

Collate a list of samples into a batched tensor or nested structure.

Recursively convert a single sample's elements to Lucid Tensors.

Return a `WorkerInfo` object in a worker process, else `None`.