Oracle integrated CDC connector concepts
This feature is in Beta. Workspace admins can control access to this feature from the Previews page. See Manage Databricks previews.
This page describes how the Oracle integrated CDC connector works, including its core concepts.
How does Databricks connect to Oracle?
Databricks connects to Oracle using a JDBC connection. Credentials are stored securely in Unity Catalog and can only be retrieved if the user running the ingestion flow has appropriate permissions. Databricks recommends creating a separate Oracle user that is solely used for data ingestion. If there are schemas or tables you do not want to expose to this user, you can use built-in Oracle privileges.
How does the connector incrementally pull data?
The connector uses Oracle LogMiner in uncommitted transaction mode to read changes from both the online redo logs and the archive logs. On the first run of a pipeline, or after a full refresh, the connector captures an initial snapshot of each selected table. On subsequent runs, it captures incremental changes (inserts, updates, and deletes) from the logs.
Because the connector reads from the archive logs, the source database must run in archive log mode and retain the archive logs long enough for the pipeline to process them. Databricks recommends retaining archive logs for at least 48 hours. The connector also requires supplemental logging so that LogMiner emits the column values needed to reconstruct each change.
Database variations
The connector supports the following Oracle deployments:
- Single-tenant (non-CDB) databases.
- Multi-tenant (container) databases. For a multi-tenant database, the replication user must be a common user in
CDB$ROOT, and the Unity Catalog connection must use theCDB$ROOTservice name. See Multi-tenant (CDB) databases. - Amazon RDS for Oracle (single-tenant only).
Oracle versions 12c and above are supported (12c, 18c, 19c, 21c, 23ai, and 26ai).
The connector supports Oracle running on cloud infrastructure (including Amazon RDS for Oracle, Oracle on Amazon EC2 or Azure virtual machines, and Oracle Cloud Infrastructure (OCI)) and on-premises Oracle using Azure ExpressRoute, AWS Direct Connect, or VPN if sufficient bandwidth is available.
Authentication methods
The connector supports basic authentication with an Oracle username and password.
Oracle ingestion vs. query federation
In addition to the Oracle integrated CDC connector in Lakeflow Connect, Databricks offers a zero-copy connector in Lakehouse Federation. The zero-copy connector allows you to query data in Oracle without moving it.