class

ChainDataset

extendsIterableDataset

ChainDataset(datasets: list[IterableDataset])

source

Concatenate multiple IterableDataset instances end-to-end.

Iteration walks the chain in registration order: every element from the first child is yielded first, then every element from the second, and so on. The iterable equivalent of ConcatDataset for map-style datasets.

Each child must be an IterableDataset; map-style datasets must be iterated through a sampler-driven DataLoader instead.

Parameters

datasetslist of IterableDataset

Iterable datasets to chain in order. Each element must be an IterableDataset instance — passing a Dataset raises TypeError at construction time.

Notes

Unlike ConcatDataset, no random access is possible: total length is unknown a priori and the chain may be infinite if any child is. Useful for stitching together streaming shards (e.g. multiple files in a sharded log) into one logical stream.

Examples

>>> a = CountUp(3)              # yields 0, 1, 2
>>> b = CountUp(2)              # yields 0, 1
>>> chain = ChainDataset([a, b])
>>> list(chain)
[0, 1, 2, 0, 1]

Methods (2)

dunder

init

→None

__init__(datasets: list[IterableDataset])

source

Build a chain over the given iterable datasets.

Parameters

datasetslist of IterableDataset

Iterable datasets to chain in order. Each must be an IterableDataset instance.

Raises

TypeError

If any element is not an IterableDataset.

dunder

iter

→Iterator[Tensor | tuple[Tensor, ...]]

__iter__()

source

Iterate through each child dataset in registration order.