Skip to main content

Connect to Matillion Data Productivity Cloud

Matillion Data Productivity Cloud is an ETL/ELT tool built specifically for cloud database platforms including Databricks. Matillion Data Productivity Cloud has a modern, browser-based UI, with powerful, push-down ETL/ELT functionality.

You can integrate your Databricks SQL warehouses (formerly Databricks SQL endpoints) and Databricks clusters with Matillion.

Connect to Matillion using Partner Connect​

This section describes how to use Partner Connect to simplify the process of connecting an existing SQL warehouse or cluster in your Databricks workspace to Matillion.

Requirements​

See the requirements for using Partner Connect.

Steps to connect​

To connect to Matillion using Partner Connect, follow the steps in this section.

  1. In the sidebar, click Marketplace icon Marketplace.

  2. In Partner Connect integrations, click View all.

  3. Click the Matillion Data Productivity Cloud tile.

  4. Select a Databricks catalog for Matillion to write to and click Next.

  5. Select an existing Databricks SQL warehouse to use with Matillion. This compute resource is used to execute your pipelines.

  6. Choose the schema Matillion should use to create and manage your data pipelines. Click Add.

  7. Click Next.

  8. Review your connection information and click Next.

  9. Review and accept the terms and conditions for using Partner Connect and click Connect to Matillion Data Productivity Cloud.

  10. The Matillion Data Productivity Cloud page loads. Complete the on-screen instructions to create your 14-day trial account or sign in to your existing Matillion account.

    Matillion might take a few minutes to create the necessary infrastructure and securely connect to your Databricks environment.

  11. After Matillion completes the setup process, the Designer loads.

    note

    If you are not on the Designer page, go back to Databricks and sign in to Matillion again.

Get started with Matillion​

After the setup is complete, you land in the Designer where you can start building data pipelines. Pipelines are the Data Productivity Cloud's way of designing, organizing, and executing workflows.

To ensure your Databricks workspace is connected to Matillion, look for the following:

  • A default project with the Databricks logo in the top left.
  • Your environment is named using Databricks terminology.
  • If you click Schemas in the upper-left, a panel opens and shows your selected schema in Databricks, along with any tables and views.

Explore the contents of the Schema to confirm that Matillion is successfully connected to your Databricks workspace.

After you check that you are connected to Databricks, start creating pipelines on Matillion:

  • Create your first Orchestration Pipeline to move data into Databricks from sources.
  • Create your first Transformation Pipeline to shape, clean, and prepare data that already exists directly within Databricks.
  • Use the visual Designer to build data workflows using a drag-and-drop canvas interface.

Next steps​

Explore one or more of the following resources on the Matillion website:

Last updated on