Configure Jira for ingestion
This feature is in Beta.
This page describes how to configure Jira for ingestion into Databricks.
Prerequisites
To ingest from Jira, you must have the following:
- A Jira cloud environment or a Jira on-premises environment with version 8.22.x or later.
- Workspace administrator privileges or
CREATE CONNECTIONprivilege on the Unity Catalog metastore. - OAuth 2.0 credentials for your Jira environment.
Configure Jira for OAuth
Prepare your Jira instance for ingestion. You can either add the required scopes to an existing OAuth app or create a new app. For OAuth machine-to-machine (M2M) authentication on Jira Cloud, set up a service account instead. See Jira Cloud service account (M2M).
Jira Cloud
-
Optionally click Create > OAuth 2.0 integration to create a new OAuth app. If you prefer to use an existing OAuth app, add the required scopes defined in steps 5 and 6.
-
Provide a name for the integration, then click Create.
-
Visit Permissions > Jira API (under API scopes) > Add.
-
Click Classic Scopes > Edit Scopes, then add the following scopes to the OAuth app:
read:jira-work, read:jira-user -
Click Granular Scopes > Edit Scopes, then add the following scopes to the OAuth app.
Not all OAuth scopes are required for all Jira tables. If your organization can't grant certain scopes (such as admin-level scopes), you can still ingest the tables that don't depend on them. See Supported Jira source tables for ingestion to check which scopes are required for each table.
read:project-role:jira, read:project-category:jira, read:user:jira, read:avatar:jira, read:sprint:jira-software, read:workflow:jira, read:board-scope:jira-software, read:issue-details:jira, read:project:jira, read:audit-log:jira, read:issue-security-level:jira, read:issue-security-scheme:jira, read:board-scope.admin:jira-software, read:group:jira, read:application-role:jira -
Go to Authorization > Add and input the redirect URI as follows:
https://<databricks_workspace_URL>/login/oauth/lakehousefederation.htmlFor example:
https://my-databricks-account.cloud.databricks.com/login/oauth/lakehousefederation.html -
Go to Settings and note down the Client ID and Client Secret. You'll use these to create a connection.
Jira Cloud service account (M2M)
Use this setup for OAuth machine-to-machine (M2M) authentication. Databricks authenticates as an Atlassian service account with the OAuth client credentials flow, so no one signs in interactively. M2M supports Jira Cloud only.
- In Atlassian Administration, select your organization, and then go to Directory > Service accounts.
- Create a service account, or select an existing one.
- Click Create credentials > OAuth 2.0, and then click Next.
- Select the same classic and granular scopes that steps 5 and 6 in Jira Cloud list. Databricks doesn't request scopes in the client credentials flow, so the access token gets only the scopes that you assign to the credential.
- Create the credential, and note the Client ID and Client secret. You use these to create a connection. Atlassian doesn't show the client secret again, so store it securely.
- Give the service account access to Jira and the project permissions it needs to read the data that you want to ingest, such as Browse projects. Scopes don't replace Jira permissions.
For details, see Create an OAuth 2.0 credential for service accounts in the Atlassian documentation.
Jira on-premises
Configure OAuth
-
Open Jira Administration.
-
Visit Applications from the settings icon on the top-right.
-
Open Applications > Application Links to see a list of created application links.
-
Optionally click Create Link to create a new application link. If you prefer to use an existing OAuth app, add the required scopes.
-
Select External Application as the application type and Incoming as the direction. Then click Continue.
-
Provide a name for your external application, and input the redirect URL as follows:
https://<databricks_workspace_URL>/login/oauth/lakehousefederation.htmlFor example:
https://my-databricks-account.cloud.databricks.com/login/oauth/lakehousefederation.html -
Set the Application permissions as Admin. Then, click Save.
-
Note down the Client ID and Client Secret. You'll use these to create a connection.
Enable issue deletes in on-premises audit logs
These steps apply only when you set use_audit_logs to true in the pipeline's jira_options to track deletes for the issues table. For more information, see Track deletes for the issues table.
Sometimes issue deletes aren't present in the audit logs on an on-premises instance. To increase audit log coverage in those scenarios:
- Go to Jira Administration > System.
- Open Audit logs under Troubleshooting on the left, and set these three areas to Advanced:
- Global configuration and administration
- Local configuration and administration
- End user activity