Troubleshoot ServiceNow ingestion

Preview

The ServiceNow connector is in Public Preview.

This page describes common issues with the ServiceNow connector in Databricks Lakeflow Connect and how to resolve them.

General pipeline troubleshooting

If a pipeline fails while executing, click on the step that failed and confirm whether the error message provides sufficient information about the nature of the error.

You can also check and download the cluster logs from the pipeline details page by clicking Update details in the right-hand panel, then Logs. Scan the logs for errors or exceptions.

Ingestion for a particular table is slow

As a first step, work with your ServiceNow administrator to enable indexing on the cursor column. The cursor column is selected from the following list, in order of preference: sys_updated_on (first choice), sys_created_on (second choice), sys_archived (third choice). Then, retry your ingestion pipeline. If the issue persists, please file a ticket. For instructions on enabling indexing in ServiceNow, see the ServiceNow documentation.

Authentication error

Error:

com.databricks.pipelines.execution.conduit.common.DataConnectorException: [SERVICENOW_CONNECTION_ERROR.OAUTH_TOKEN_EXCHANGE] Failed to authenticate to ServiceNow. This can be caused by issues with your UC credentials, ServiceNow OAuth setup, and more...

Step 1:

Confirm that you provided the correct username, password, client ID, and client secret when you created the Unity Catalog connection. The username and password for authentication to ServiceNow are not necessarily your personal username and password for SSO. These are the username and password values associated with a user that's intended for authentication and ingestion with external services.

Catalog Explorer

1. In the sidebar of the Databricks workspace, click Catalog.

2. At the top of the Catalog pane, click the gear icon and select Connections from the menu.

   Alternatively, from the Quick access page, click the External data > button and go to the Connections tab.

3. Click the name of the connection.

   Connection details are listed on the Overview tab.

curl

Run the following curl request. Replace the items in brackets with your connection properties.

curl -X POST "https://<instance-id>.service-now.com/oauth_token.do" \
--header "Content-Type: application/x-www-form-urlencoded" \
--data-urlencode "grant_type=password" \
--data-urlencode "client_id=<client-id>" \
--data-urlencode "client_secret=<client-secret>" \
--data-urlencode "username=<username>" \
--data-urlencode "password=<password>"

If the request succeeds, the credentials are correct.

Step 2:

Confirm that the authenticating user has the admin role. Databricks also recommends that the authenticating user has the snc_read_only role to restrict permissions further.

Step 3:

Confirm that the authenticating user is marked as Active:

1. In source-product-name, click Organization > Users.

2. Select the user that you plan to use for authentication.

3. Confirm that the Active checkbox is selected.

   

Step 4:

Confirm that the OAuth 2.0 plugin in ServiceNow is active:

1. In source-product-name, click System Definition > Plugins.

2. Run the following script in the Scripts - Background section:

   var oauthActive = gs.getProperty('com.snc.platform.security.oauth.is.active');
   gs.info('OAuth Active: ' + oauthActive);

   If the script returns true, the plugin installation was successful. If this script returns false, uninstall and then reinstall it.

   

Step 5:

Confirm that the OAuth endpoint is configured correctly:

1. In ServiceNow, visit System OAuth > Application Registry.

2. In the upper-right corner, click NEW.

3. In the OAuth application options, click Create an OAuth API endpoint for external clients.

   

4. Confirm that the Active checkbox is selected.

   

   After you have successfully created the OAuth application, it appears in the list at System OAuth > Application Registry with the type OAuth Client.

Step 6:

If the issue persists after following these troubleshooting steps, file a support ticket.