Query history system table reference
Preview
This system table is in Public Preview. To access the table, the schema must be enabled in your system
catalog. For more information, see Enable system table schemas.
This article includes information on the query history system table, including an outline of the table’s schema.
Important
To access the query history system table, you must enable the query
schema. For instructions on enabling system schemas, see Enable system table schemas.
Table path: This system table is located at system.query.history
.
Using the query history table
The query history table includes records for every SQL statement run using SQL warehouses. The table includes account-wide records from all workspaces in the same region from which you access the table.
By default, only admins have access to the system table. If you would like to share the table’s data with a user or group, Databricks recommends creating a dynamic view for each user or group. See Create a dynamic view.
Query history system table schema
The query history table uses the following schema:
Column name |
Data type |
Description |
Example |
---|---|---|---|
|
string |
ID of the account. |
|
|
string |
The ID of the workspace where the query was run. |
|
|
string |
The ID that uniquely identifies the execution of the statement. You can use this ID to find the statement execution in the Query History UI. |
|
|
string |
The Spark session ID. |
|
|
string |
The statement termination state. Possible values are:
|
|
|
struct |
A struct that represents the type of compute resource used to run the statement and the ID of the resource where applicable. The |
|
|
string |
The ID of the user who ran the statement. |
|
|
string |
The email address or username of the user who ran the statement. |
|
|
string |
Text of the SQL statement. If you have configured customer-managed keys, |
|
|
string |
The statement type. For example: |
|
|
string |
Message describing the error condition. If you have configured customer-managed keys, |
|
|
string |
Client application that ran the statement. For example: Databricks SQL, Tableau, and Power BI. |
|
|
string |
The connector used to connect to Databricks to run the statement. For example: Databricks SQL Driver for Go, Databricks ODBC Driver, Databricks JDBC Driver. |
|
|
bigint |
Total execution time of the statement in milliseconds ( excluding result fetch time ). |
|
|
bigint |
Time spent waiting for compute resources to be provisioned in milliseconds. |
|
|
bigint |
Time spent waiting in queue for available compute capacity in milliseconds. |
|
|
bigint |
Time spent executing the statement in milliseconds. |
|
|
bigint |
Time spent loading metadata and optimizing the statement in milliseconds. |
|
|
bigint |
The sum of all task durations in milliseconds. This time represents the combined time it took to run the query across all cores of all nodes. It can be significantly longer than the wall-clock duration if multiple tasks are executed in parallel. It can be shorter than the wall-clock duration if tasks wait for available nodes. |
|
|
bigint |
Time spent, in milliseconds, fetching the statement results after the execution finished. |
|
|
timestamp |
The time when Databricks received the request. Timezone information is recorded at the end of the value with |
|
|
timestamp |
The time the statement execution ended, excluding result fetch time. Timezone information is recorded at the end of the value with |
|
|
timestamp |
The time the statement last received a progress update. Timezone information is recorded at the end of the value with |
|
|
bigint |
The number of partitions read after pruning. |
|
|
bigint |
The number of pruned files. |
|
|
bigint |
The number of files read after pruning. |
|
|
bigint |
Total number of rows read by the statement. |
|
|
bigint |
Total number of rows returned by the statement. |
|
|
bigint |
Total size of data read by the statement in bytes. |
|
|
int |
The percentage of bytes of persistent data read from the IO cache. |
|
|
boolean |
|
|
|
bigint |
Size of data, in bytes, temporarily written to disk while executing the statement. |
|
|
bigint |
The size in bytes of persistent data written to cloud object storage. |
|
|
bigint |
The total amount of data in bytes sent over the network. |
|
|
struct |
A struct that contains key-value pairs representing one or more Databricks entities that were involved in the execution of this statement, such as jobs, notebooks, or dashboards. This field only records Databricks entities. |
|
|
string |
The name of the user or service principal whose privilege was used to run the statement. |
|
|
string |
The ID of the user or service principal whose privilege was used to run the statement. |
|
View the query profile for a record
To navigate to a query’s query profile based on a record in the query history table, do the following:
Identify the record of interest, then copy the record’s
statement_id
.Reference the record’s
workspace_id
to ensure you are logged in to the same workspace as the record.Click Query History in the workspace sidebar.
In the Statement ID field, paste the
statement_id
on the record.Click the name of a query. An overview of query metrics appears.
Click See query profile.