Skip to main content
Last updated on

Jira connector limitations

Beta

This feature is in Beta. Workspace admins can control access to this feature from the Previews page. See Manage Databricks previews.

This page lists limitations and considerations for ingesting data from Jira using Databricks Lakeflow Connect.

General SaaS connector limitations​

The limitations in this section apply to all SaaS connectors in Lakeflow Connect.

  • When you run a scheduled pipeline, alerts don't trigger immediately. Instead, they trigger when the next update runs.
  • When a source table is deleted, the destination table is not automatically deleted. You must delete the destination table manually. This behavior is not consistent with Spark Declarative Pipelines on Lakeflow behavior.
  • During source maintenance periods, Databricks might not be able to access your data.
  • If a source table name conflicts with an existing destination table name, the pipeline update fails.
  • Multi-destination pipeline support is API-only.
  • You can optionally rename a table that you ingest. If you rename a table in your pipeline, it becomes an API-only pipeline, and you can no longer edit the pipeline in the UI.
  • If you select a column after a pipeline has already started, the connector does not automatically backfill data for the new column. To ingest historical data, manually run a full refresh on the table.
  • Databricks can't ingest two or more tables with the same name in the same pipeline, even if they come from different source schemas.
  • The source system assumes that the cursor columns are monotonically increasing.
  • The connector ingests raw data without transformations. Use downstream Spark Declarative Pipelines on Lakeflow pipelines for transformations.

Connector-specific limitations​

The limitations in this section are specific to the Jira connector.

Incremental sync​

Delete tracking​

important

By default, the Jira connector does not track deletes for the issues table. Deleted Jira issues remain in the destination table. To propagate deletes, set use_audit_logs to true in jira_options. This requires a paid Jira plan and a connection user with Jira admin permissions.

  • When use_audit_logs is false (the default), the issues table reflects only inserts and updates. When it is true, the connector uses the Jira audit log to also remove deleted issues. When you enable this option on an existing pipeline, the connector reconciles only the deletes still within your Jira plan's audit log retention window. To set the option, see Track deletes for the issues table.
  • Delete tracking through the audit log requires a paid Jira plan, and the connection user must have global admin permissions on the Jira instance.
  • Comments and worklogs deletions are only supported through full refresh.

Filtering​

  • Filtering by Jira project or space is supported using the include_jira_spaces parameter in jira_options. Make sure that you are using exact project keys instead of project names or IDs.

Content ingestion​

  • The connector provides access to 27 tables in total. All data, including data from multiple projects, is organized into these 27 tables.
  • Some tables (for example, issue links) use internal Jira IDs and might require joining with other tables for meaningful output.
On this page