Connect to Qlik Replicate

Preview

Qlik Replicate helps you pull data from multiple data sources (Oracle, Microsoft SQL Server, SAP, mainframe and more) into Delta Lake. Replicate’s automated change data capture (CDC) helps you avoid the heavy lifting of manually extracting data, transferring using an API script, chopping, staging, and importing. Qlik Compose automates the CDC into Delta Lake.

Note

For information about Qlik Sense, a solution that helps you analyze data in Delta Lake, see Connect to Qlik Sense.

For a general demonstration of Qlik Replicate Replicate, watch the following YouTube video (14 minutes).

For a demonstration of data pipelines with Qlik Replicate Replicate, see the following YouTube video (6 minutes).

Here are the steps for using Qlik Replicate with Databricks.

Step 1: Generate a Databricks personal access token

Qlik Replicate authenticates with Databricks using a Databricks personal access token.

Note

As a security best practice when you authenticate with automated tools, systems, scripts, and apps, Databricks recommends that you use OAuth tokens.

If you use personal access token authentication, Databricks recommends using personal access tokens belonging to service principals instead of workspace users. To create tokens for service principals, see Manage tokens for a service principal.

Step 2: Set up a cluster to support integration needs

Qlik Replicate will write data to an S3 bucket and the Databricks integration cluster will read data from that location. Therefore the integration cluster requires secure access to the S3 bucket.

Secure access to an S3 bucket

To access AWS resources, you can launch the Databricks integration cluster with an instance profile. The instance profile should have access to the staging S3 bucket and the target S3 bucket where you want to write the Delta tables. To create an instance profile and configure the integration cluster to use the role, follow the instructions in Tutorial: Configure S3 access with an instance profile.

As an alternative, you can use IAM credential passthrough, which enables user-specific access to S3 data from a shared cluster.

Specify the cluster configuration

Set Cluster Mode to Standard.
Set Databricks Runtime Version to a Databricks runtime version.
Enable optimized writes and auto compaction by adding the following properties to your Spark configuration:
```
spark.databricks.delta.optimizeWrite.enabled true
spark.databricks.delta.autoCompact.enabled true
```
Configure your cluster depending on your integration and scaling needs.

For cluster configuration details, see Compute configuration reference.

See Get connection details for a Databricks compute resource for the steps to obtain the JDBC URL and HTTP path.

Step 3: Obtain JDBC and ODBC connection details to connect to a cluster

To connect a Databricks cluster to Qlik Replicate you need the following JDBC/ODBC connection properties:

JDBC URL
HTTP Path

Step 4: Configure Qlik Replicate with Databricks

Go to the Qlik login page and follow the instructions.

Additional resources

Support