readBetweenOffsets (SimpleDataSourceStreamReader)
Reads all available data between a specific start offset and end offset.
This method is invoked during failure recovery to re-read a batch deterministically.
Added in Databricks Runtime 15.3
Syntax
readBetweenOffsets(start: dict, end: dict)
Parameters
Parameter | Type | Description |
|---|---|---|
| dict | The start offset to read from. |
| dict | The offset at which reading stops. |
Returns
Iterator[Tuple]
All records between the start offset and end offset.