initialOffset (DataSourceStreamReader)

Returns the initial offset of the streaming data source.

A new streaming query starts reading data from the initial offset. If Spark is restarting an existing query, it will restart from the checkpointed offset rather than the initial one.

Added in Databricks Runtime 15.2

Syntax

initialOffset()

Returns

dict

A dict or recursive dict whose key and value are primitive types, which includes Integer, String, and Boolean.

Examples

Python

def initialOffset(self):
    return {"partition-1": {"index": 3, "closed": True}, "partition-2": {"index": 5}}