Skip to main content
Last updated on

MySQL ingestion connector

Preview

The MySQL connector is in Public Preview. Contact your Databricks account team to request access.

This page helps you understand the MySQL ingestion workflow, including the factors that determine your setup approach and the steps involved for different user personas.

What to know before you start​

Topic

Why it matters

Databricks user persona

The workflow depends on your Databricks user persona:

  • Single-user: An admin user configures the source database and creates a Unity Catalog connection, an ingestion gateway, and an ingestion pipeline.
  • Multi-user: An admin user configures the source database and creates a connection for non-admin users to create gateway-pipeline pairs with.

Deployment environment

The source database configuration depends on the MySQL deployment environment (Amazon RDS, Aurora MySQL, Azure Database for MySQL, Google Cloud SQL for MySQL, or MySQL on EC2).

Interface

The steps to create a connection, a gateway, and a pipeline depend on the interface.

Ingestion frequency

The pipeline schedule depends on your latency and cost requirements.

Common patterns

Depending on your ingestion needs, the pipeline might use configurations like history tracking, column selection, and row filtering. Supported configurations vary by connector. See Feature availability.

Start ingesting from MySQL​

The following table provides an overview of the end-to-end MySQL ingestion workflow, based on user type:

User

Steps

Admin

  1. Configure MySQL for ingestion into Databricks.
  2. Either:
    • Use Catalog Explorer to create a connection so that non-admins can create gateway-pipeline pairs. See MySQL.
    • Use the data ingestion UI to create a connection, a gateway, and a pipeline. See Create a MySQL ingestion pipeline.

Non-admin

Use any supported interface to create a gateway and a pipeline. See Create a MySQL ingestion pipeline.