System events

Preview

Lakewatch is in Private Preview. To request access, reach out to your Databricks account team.

A system event is an event record generated when Lakewatch detects a system health or configuration problem, such as a deleted table that still has an associated detection rule. Each system event attaches to exactly one system error, and includes a summary of the issue, the resource type and name involved (for example, a specific detection rule or datasource), and details such as the error message or stack trace.

To view system events, navigate to System errors, open a system error, and view its related events. To query system events directly, use the system_events system table.

What triggers system events?

System events are generated when Lakewatch encounters errors or health issues during automated processes. Common scenarios include:

• Detection rule execution errors: A rule fails while applying filters, running custom SQL or custom functions, generating input views, or writing output.
• Datasource pipeline errors: A datasource pipeline fails during execution, or enters a failed or unexpected state.
• Configuration issues: A mismatch between resources, such as a table that has been deleted while its associated detection rule still exists.
• Job reconciliation failures: One or more resources in a job group fail to reconcile after repeated attempts.
• Workspace connectivity errors: Lakewatch can't reach the API server, or Databricks secret-based authentication fails during housekeeping tasks.
• Metrics reporting failures: The system fails to report detection rule or datasource metrics back to the API server.

Datasource run failures

When a datasource run fails, Lakewatch records a system event (and a corresponding system error) whose resource_type is Datasource and whose resource_name identifies the affected datasource. The summary and details describe the failure (for example, a pipeline error or a source-connectivity problem) without exposing Lakewatch's internal service details.

To investigate and recover:

• Navigate to System errors, open the datasource's error to read the summary and details, or query the system_events system table filtered on resource_type = 'Datasource'.
• From the datasource's overview page, click View job to open the underlying Databricks job run for run history and logs.
• For a source-access failure, restore the run-as identity's access to the source. If the identity is unchanged, the next scheduled run recovers automatically. If you reassign the run-as identity or change another setting, edit the datasource inline and click Save. See Monitor a datasource and recover from failures.

System events system table

Lakewatch provides a system table at system.lakewatch.system_events that contains the full set of system events for your workspace. You can query this table to analyze system events, correlate them with system errors, and build custom reports or dashboards.

note

System events are available in system tables, but system errors are not.

Schema

Column	Type	Description
alert_id	STRING	Unique identifier for the system event.
event_time	TIMESTAMP	When the system event was generated (UTC).
resource_type	STRING	The type of resource involved: Datasource, Rule, Case, or System.
resource_name	STRING	The internal name of the resource.
resource_display_name	STRING	The human-readable display name of the resource.
summary	STRING	A short description of the issue.
details	STRING	Extended information, such as an error message or stack trace.
state	STRING	Current state of the system event: open, dismissed, or exported.
filename	STRING	Source file where the error occurred, if applicable. Empty string if not applicable.
line_no	INT	Line number in the source file, if applicable. -1 if not applicable.
grouping_id	STRING	A SHA-256 hash of the resource type, resource name, summary, and event date. System events with the same grouping_id represent the same issue on the same day.

Column	Type	Description
alert_id	STRING	Unique identifier for the system event.
event_time	TIMESTAMP	When the system event was generated (UTC).
resource_type	STRING	The type of resource involved: Datasource, Rule, Case, or System.
resource_name	STRING	The internal name of the resource.
resource_display_name	STRING	The human-readable display name of the resource.
summary	STRING	A short description of the issue.
details	STRING	Extended information, such as an error message or stack trace.
state	STRING	Current state of the system event: open, dismissed, or exported.
filename	STRING	Source file where the error occurred, if applicable. Empty string if not applicable.
line_no	INT	Line number in the source file, if applicable. -1 if not applicable.
grouping_id	STRING	A SHA-256 hash of the resource type, resource name, summary, and event date. System events with the same grouping_id represent the same issue on the same day.

Additional resources

• System errors