Oracle integrated CDC connector
This feature is in Beta. Workspace admins can control access to this feature from the Previews page. See Manage Databricks previews.
This page helps you understand the Oracle ingestion workflow, including the factors that determine your setup approach and the steps involved for different user personas.
The Oracle integrated CDC connector uses an integrated CDC pipeline: a single pipeline extracts changes from Oracle using LogMiner and applies them to destination streaming tables, with no separate ingestion gateway. For the component model, see Connector components.
Before you create an Oracle integrated CDC ingestion pipeline, you must configure the source database directly in Oracle: enable archive log mode, enable supplemental logging on the tables you replicate, and create a replication user with the required privileges. Complete these tasks first. See Configure Oracle for ingestion into Databricks.
Feature availability
The same feature availability applies to AWS, Azure, and GCP Databricks.
Feature | Availability |
|---|---|
UI-based pipeline authoring |
|
API-based pipeline authoring |
|
Declarative Automation Bundles |
|
Incremental ingestion |
|
Unity Catalog governance |
|
Orchestration using Lakeflow Jobs |
|
SCD type 2 |
|
API-based column selection and deselection |
|
API-based row filtering |
|
Automated schema evolution: New and deleted columns |
Re-adding a deleted column with the same name is not supported. |
Automated schema evolution: Data type changes |
Type widening only. A non-widening (narrowing) type change requires a full refresh. |
Automated schema evolution: Column renames |
|
Automated schema evolution: Table renames |
|
Automated schema evolution: New tables |
If you ingest the entire schema. See the limitations on the number of tables per pipeline. |
Auto full refresh |
Automatically full-refreshes affected tables to recover from unsupported schema changes. See Auto full refresh policy. |
Maximum number of tables per pipeline | 250 |
Supported
Not supportedAuthentication methods
The connector supports basic authentication with an Oracle username and password only. See Create an Oracle connection.
Ingest from Oracle in 3 steps
Before starting, review the Databricks user persona, database variation, authentication methods, supported interfaces, ingestion frequency, and common patterns.
- Configure Oracle for ingestion (Admins) — Enable archive log mode, enable supplemental logging on the tables you replicate, and create a replication user with the required privileges.
- Create a Unity Catalog connection (Admins) — Create a connection in Catalog Explorer so non-admins can create pipelines.
- Create an ingestion pipeline (Admins or non-admins) — Select any supported interface and create a pipeline from an existing connection.