Skip to main content
Last updated on

Microsoft SQL Server ingestion connector

This page helps you understand the SQL Server ingestion workflow, including the factors that determine your setup approach and the steps involved for different user personas.

What to know before you start​

Topic

Why it matters

Databricks user persona

The workflow depends on your Databricks user persona:

  • Single-user: An admin user configures the source database and creates a Unity Catalog connection, an ingestion gateway, and an ingestion pipeline.
  • Multi-user: An admin user configures the source database and creates a connection for non-admin users to create gateway-pipeline pairs with.

Database variation

The source database configuration depends on the SQL Server deployment environment.

Change tracking method

The source database configuration depends on how you choose to track changes in the source.

Authentication method

The steps to create a connection depend on the authentication method you choose.

Interface

The steps to create a connection, a gateway, and a pipeline depend on the interface.

Ingestion frequency

The pipeline schedule depends on your latency and cost requirements.

Overview of steps​

The following table provides an overview of the end-to-end SQL Server ingestion workflow, depending on your user persona:

User

Steps

Admin

  1. Configure Microsoft SQL Server for ingestion into Databricks.
  2. Either:
    • Use Catalog Explorer to create a connection so that non-admins can create gateway-pipeline pairs. See SQL Server.
    • Use the data ingestion UI to create a connection, a gateway, and a pipeline. See Ingest data from SQL Server.

Non-admin

Use any supported interface to create a gateway and a pipeline. See Ingest data from SQL Server.