partitions (DataSourceStreamReader)

Returns a list of InputPartition objects given the start and end offsets.

Each InputPartition represents a data split that can be processed by one Spark task. When called with an empty offset range where start == end, this method should return an empty sequence.

Added in Databricks Runtime 15.2

Syntax

partitions(start: dict, end: dict)

Parameters

Parameter	Type	Description
start	dict	The start offset of the microbatch to plan partitioning.
end	dict	The end offset of the microbatch to plan partitioning.

Returns

Sequence[InputPartition]

A sequence of partitions for this data source. Each partition value must be an instance of InputPartition or a subclass of it.