Applies to: Databricks SQL Databricks Runtime 11.3 and later
Returns the file-level state of an Auto Loader or
STRINGliteral. The checkpoint directory for a stream using the Auto Loader source. See What is Auto Loader?.
Returns a table with the following schema:
path STRING NOT NULL PRIMARY KEY
The path of a file.
size BIGINT NOT NULL
The size of a file in bytes.
create_time TIMESTAMP NOT NULL
The time that a file was created.
discovery_time TIMESTAMP NOT NULL
The time that a file was discovered.
The time that a file was committed to the checkpoint after processing.
NULLif the file is not yet processed. A file might be processed, but might be marked as committed arbitrarily later. Marking the file as committed means that Auto Loader does not require the file for processing again.
The time that a file was archived.
NULLif the file has not been archived.
The ID of the Auto Loader source in the streaming query. This value is
'0'for streams that ingest from a single cloud object store location.
You need to have:
OWNERprivileges on the streaming table if using a streaming table identifier.
READ FILESprivileges on the checkpoint location if providing a checkpoint under an external location.