Skip to main content

Logging and Monitoring for Databricks Apps

Effective logging and monitoring help you detect and respond to security events in Databricks Apps. Apps generate both application-level logs and platform audit logs, which you can use for diagnostics, performance tracking, and security analytics.

Application logs​

note

To make logs available in the Databricks Apps UI or through your app’s URL, your app must write output to stdout and stderr.

You can access application logs in the following ways:

  • Apps UI: On the app details page, click the Logs tab to view standard output and error. For details, see View app details.
  • Direct URL: Append /logz to your app URL.
    For example, if your app URL is https://my-app-1234567890.my-instance.databricksapps.com, logs are available at
    https://my-app-1234567890.my-instance.databricksapps.com/logz.

To integrate with external monitoring and real-time alerting systems:

  • Format logs in JSON or other machine-parseable formats.
  • Log security-relevant events with context:
    • Authentication and authorization events, including user identity and outcome
    • Data access details, such as catalog, schema, and table names
    • Security-related errors, such as invalid tokens, permission denials, and suspicious activity
  • Forward logs to external systems. Integrate with Application Performance Monitoring (APM) or log aggregation tools to support real-time alerts, security incident response, usage and performance analytics, and correlation with Databricks system logs.

Security considerations for logging​

Databricks apps are designed with the following built-in controls to prevent data exfiltration:

  • API-only access: Apps can only access Databricks resources through public Databricks APIs. These APIs are auditable through system table logs.
  • Encrypted communication: All API traffic is encrypted using TLS 1.2 or higher to ensure secure data transfer.

Security monitoring with system tables​

Databricks captures audit logs for app-related activities in the system.access.audit table. You can query these logs to track user actions, app configuration changes, and security events.

Use audit logs to support the following common monitoring and security scenarios:

Use the following queries to monitor security-related activities and detect potential issues with your apps.

Monitor app permission changes​

Use this query to detect app permission modifications:

SQL
-- Monitor all app permission modifications in the last 30 days
WITH permission_changes AS (
  SELECT
    event_date,
    workspace_id,
    request_params.request_object_id AS app_name,
    user_identity.email AS modified_by,
    explode(from_json(
      request_params.access_control_list,
      'array<struct<user_name:string,group_name:string,permission_level:string>>'
    )) AS permission
  FROM system.access.audit
  WHERE action_name = 'changeAppsAcl'
    AND event_date >= current_date() - 30
)
SELECT
  event_date,
  app_name,
  modified_by,
  permission.user_name,
  permission.group_name,
  permission.permission_level
FROM permission_changes
ORDER BY event_date DESC

Identify apps with user API scopes​

Use this query to find apps with user API scopes configured:

SQL
-- Find apps created or updated in the last 30 days with user API scopes configured
SELECT
  event_date,
  get_json_object(request_params.app, '$.name') AS app_name,
  user_identity.email AS creator_email,
  get_json_object(request_params.app, '$.user_api_scopes') AS user_api_scopes
FROM system.access.audit
WHERE
  action_name IN ('createApp', 'updateApp')
  AND get_json_object(request_params.app, '$.user_api_scopes') IS NOT NULL
  AND event_date >= current_date() - INTERVAL 30 DAYS

Track user authorization actions​

Use this query to list app actions performed with user authorization:

SQL
-- List app actions performed on behalf of users in the last 30 days
WITH obo_events AS (
  SELECT
    event_date,
    workspace_id,
    audit_level,
    identity_metadata.acting_resource AS app_id,   -- OAuth App ID or name
    user_identity.email AS user_email,             -- Logged-in user
    service_name,
    action_name
  FROM system.access.audit
  WHERE event_date >= current_date() - 30
    AND identity_metadata.acting_resource IS NOT NULL
)
SELECT
  event_date,
  app_id,
  user_email,
  service_name,
  action_name,
  audit_level,
  COUNT(*) AS event_count
FROM obo_events
GROUP BY
  event_date, app_id, user_email, service_name, action_name, audit_level
ORDER BY event_date DESC;

Operational monitoring​

Use system tables to monitor operational aspects of your apps, such as cost and resource usage.

Monitor app costs​

Monitor Databricks Apps costs using the system.billing.usage table. Use the following query to get accurate cost information for apps per day or month:

SQL
-- Get Databricks Apps cost by app per day for the last 30 days
SELECT
  us.usage_date,
  us.usage_metadata.app_id,
  us.usage_metadata.app_name,
  SUM(us.usage_quantity) AS dbus,
  SUM(us.usage_quantity * lp.pricing.effective_list.default) AS dollars
FROM
  system.billing.usage us
LEFT JOIN system.billing.list_prices lp
  ON lp.sku_name = us.sku_name
  AND us.usage_start_time BETWEEN lp.price_start_time AND COALESCE(lp.price_end_time, NOW())
WHERE
  billing_origin_product = 'APPS'
  AND us.usage_unit = 'DBU'
  AND us.usage_date >= DATE_SUB(NOW(), 30)
GROUP BY ALL
Last updated on