write
Writes data into the data source.
This method is called once on each executor to write data to the data source. It accepts an iterator of input data and returns a single row representing a commit message, or None if there is no commit message.
The driver collects commit messages, if any, from all executors and passes them to the commit() method if all tasks run successfully. If any task fails, the abort() method will be called with the collected commit messages.
Added in Databricks Runtime 14.3 LTS
Syntax
write(iterator: Iterator[Row])
Parameters
Parameter | Type | Description |
|---|---|---|
| Iterator[Row] | An iterator of input data. |
Returns
WriterCommitMessage
A serializable commit message.