read (SimpleDataSourceStreamReader)
Reads all available data from the start offset and returns the offset that the next read attempt starts from.
Added in Databricks Runtime 15.3
Syntax
read(start: dict)
Parameters
Parameter | Type | Description |
|---|---|---|
| dict | The start offset to read from. |
Returns
Tuple[Iterator[Tuple], dict]
A tuple of an iterator and a dict. The iterator contains all available records after the start offset. The dict is the end offset of this read attempt and the start offset of the next read attempt.
The iterator must be picklable. Return an iterator over a list, such as iter([...]). Returning a generator fails with TypeError: cannot pickle 'generator' object.