Cdf Config

View as Markdown

A Lakebase CDF configuration (CdfConfig): one per Postgres schema per database, replicating that schema's tables into a Unity Catalog schema. Immutable once created.

Get a change data feed configurationGET /api/2.0/postgres/{name=projects/*/branches/*/databases/*/cdf-configs/*}List change data feed configurationsGET /api/2.0/postgres/{parent=projects/*/branches/*/databases/*}/cdf-configsCreate a change data feed configurationPOST /api/2.0/postgres/{parent=projects/*/branches/*/databases/*}/cdf-configsDelete a change data feed configurationDELETE /api/2.0/postgres/{name=projects/*/branches/*/databases/*/cdf-configs/*}

CdfConfig object

A Lakebase CDF configuration (CdfConfig): one per Postgres schema per database, replicating that schema's tables into a Unity Catalog schema. Immutable once created.

namestringBetaIDImmutable

Output only. The full resource name of the CdfConfig. Format: projects/{project}/branches/{branch}/databases/{database}/cdf-configs/{cdf_config}

catalogstringBetaImmutable

The Unity Catalog catalog that replicated tables are written into. Set at creation; the CdfConfig is immutable.

schemastringBetaImmutable

The Unity Catalog schema that replicated tables are written into. Set at creation; the CdfConfig is immutable.

create_timestringBetaOutput only

When the CdfConfig was created.

cdf_config_idstringBetaOutput only

The user-specified id; equals the final segment of name. Defaults to the Postgres schema name for configs without an explicit id.

postgres_schemastringBetaImmutable

The Postgres schema this CdfConfig replicates from. Unique within the parent database. Set at creation; the CdfConfig is immutable.

View as Markdown

The CdfConfig object

{
  "name": "string",
  "catalog": "string",
  "schema": "string",
  "create_time": "string",
  "cdf_config_id": "string",
  "postgres_schema": "string"
}

Get a change data feed configuration Beta

GET /api/2.0/postgres/{name=projects/*/branches/*/databases/*/cdf-configs/*}

Get a single Lakebase CDF configuration, including the source Postgres schema, target Unity Catalog schema, and the identity under which writes are authorized.

API scopes: postgres

Parameters

namestringRequiredpath

The resource name of the CdfConfig to retrieve. Format: projects/{project}/branches/{branch}/databases/{database}/cdf-configs/{cdf_config}

Response

Returns the CdfConfig object.

View as Markdown

Response

{
  "name": "string",
  "catalog": "string",
  "schema": "string",
  "create_time": "string",
  "cdf_config_id": "string",
  "postgres_schema": "string"
}

List change data feed configurations Beta

GET /api/2.0/postgres/{parent=projects/*/branches/*/databases/*}/cdf-configs

List all CDF configurations for a Lakebase database. Each configuration maps a Postgres schema to a Unity Catalog schema where the change data feed is materialized.

API scopes: postgres

Parameters

parentstringRequiredpath

The parent database to list CdfConfigs for. Format: projects/{project}/branches/{branch}/databases/{database}

page_sizeint32query

Maximum number of CdfConfigs to return.

page_tokenstringquery

Pagination token returned by a previous ListCdfConfigs call. Empty on the first page.

Response

Returns a list of CdfConfig objects.

View as Markdown

Response

{
  "cdf_configs": [
    {
      "name": "string",
      "catalog": "string",
      "schema": "string",
      "create_time": "string",
      "cdf_config_id": "string",
      "postgres_schema": "string"
    }
  ],
  "next_page_token": "string"
}

Create a change data feed configuration Beta

POST /api/2.0/postgres/{parent=projects/*/branches/*/databases/*}/cdf-configs

Create a CDF configuration that materializes the change data feed for all tables in a Postgres schema as open-format Delta tables in Unity Catalog. Once created, each table's change history is continuously written to its corresponding Lakehouse table.

API scopes: postgres

Parameters

parentstringRequiredpath

The parent database under which to create the CdfConfig. Format: projects/{project}/branches/{branch}/databases/{database}

cdf_config_idstringquery

The user-specified id for the CdfConfig, forming the final segment of its resource name. Must match the pattern [a-z][a-z0-9_]{0,62}. Defaults to the Postgres schema name when omitted.

Request body

The CdfConfig to create. The catalog, schema, and postgres_schema fields are required; all other fields are output only and ignored on input.

namestringIDImmutable

Output only. The full resource name of the CdfConfig. Format: projects/{project}/branches/{branch}/databases/{database}/cdf-configs/{cdf_config}

catalogstringRequiredImmutable

The Unity Catalog catalog that replicated tables are written into. Set at creation; the CdfConfig is immutable.

schemastringRequiredImmutable

The Unity Catalog schema that replicated tables are written into. Set at creation; the CdfConfig is immutable.

postgres_schemastringRequiredImmutable

The Postgres schema this CdfConfig replicates from. Unique within the parent database. Set at creation; the CdfConfig is immutable.

Response

namestring

The server-assigned name, which is only unique within the same service that originally returns it. If you use the default HTTP mapping, the name should be a resource name ending with operations/{unique_id}.

metadataobject

Service-specific metadata associated with the operation. It typically contains progress information and common metadata such as create time. Some services might not provide such metadata.

doneboolean

If the value is false, it means the operation is still in progress. If true, the operation is completed, and either error or response is available.

errorobject

The error result of the operation in case of failure or cancellation.

Show child attributesHide child attributes

error_codestring

Error codes returned by Databricks APIs to indicate specific failure conditions.

Values:

Show 81 possible values

• UNKNOWN
• INTERNAL_ERROR
• TEMPORARILY_UNAVAILABLE
• IO_ERROR
• BAD_REQUEST
• SERVICE_UNDER_MAINTENANCE
• WORKSPACE_TEMPORARILY_UNAVAILABLE
• DEADLINE_EXCEEDED
• CANCELLED
• RESOURCE_EXHAUSTED
• ABORTED
• NOT_FOUND
• ALREADY_EXISTS
• UNAUTHENTICATED
• UNAVAILABLE
• INVALID_PARAMETER_VALUE
• ENDPOINT_NOT_FOUND
• MALFORMED_REQUEST
• INVALID_STATE
• PERMISSION_DENIED
• FEATURE_DISABLED
• CUSTOMER_UNAUTHORIZED
• REQUEST_LIMIT_EXCEEDED
• RESOURCE_CONFLICT
• UNPARSEABLE_HTTP_ERROR
• NOT_IMPLEMENTED
• DATA_LOSS
• INVALID_STATE_TRANSITION
• COULD_NOT_ACQUIRE_LOCK
• RESOURCE_ALREADY_EXISTS
• RESOURCE_DOES_NOT_EXIST
• QUOTA_EXCEEDED
• MAX_BLOCK_SIZE_EXCEEDED
• MAX_READ_SIZE_EXCEEDED
• PARTIAL_DELETE
• MAX_LIST_SIZE_EXCEEDED
• DRY_RUN_FAILED
• RESOURCE_LIMIT_EXCEEDED
• DIRECTORY_NOT_EMPTY
• DIRECTORY_PROTECTED
• MAX_NOTEBOOK_SIZE_EXCEEDED
• MAX_CHILD_NODE_SIZE_EXCEEDED
• SEARCH_QUERY_TOO_LONG
• SEARCH_QUERY_TOO_SHORT
• MANAGED_RESOURCE_GROUP_DOES_NOT_EXIST
• PERMISSION_NOT_PROPAGATED
• DEPLOYMENT_TIMEOUT
• GIT_CONFLICT
• GIT_UNKNOWN_REF
• GIT_SENSITIVE_TOKEN_DETECTED
• GIT_URL_NOT_ON_ALLOW_LIST
• GIT_REMOTE_ERROR
• PROJECTS_OPERATION_TIMEOUT
• IPYNB_FILE_IN_REPO
• INSECURE_PARTNER_RESPONSE
• MALFORMED_PARTNER_RESPONSE
• METASTORE_DOES_NOT_EXIST
• DAC_DOES_NOT_EXIST
• CATALOG_DOES_NOT_EXIST
• SCHEMA_DOES_NOT_EXIST
• TABLE_DOES_NOT_EXIST
• SHARE_DOES_NOT_EXIST
• RECIPIENT_DOES_NOT_EXIST
• STORAGE_CREDENTIAL_DOES_NOT_EXIST
• EXTERNAL_LOCATION_DOES_NOT_EXIST
• PRINCIPAL_DOES_NOT_EXIST
• PROVIDER_DOES_NOT_EXIST
• METASTORE_ALREADY_EXISTS
• DAC_ALREADY_EXISTS
• CATALOG_ALREADY_EXISTS
• SCHEMA_ALREADY_EXISTS
• TABLE_ALREADY_EXISTS
• SHARE_ALREADY_EXISTS
• RECIPIENT_ALREADY_EXISTS
• STORAGE_CREDENTIAL_ALREADY_EXISTS
• EXTERNAL_LOCATION_ALREADY_EXISTS
• PROVIDER_ALREADY_EXISTS
• CATALOG_NOT_EMPTY
• SCHEMA_NOT_EMPTY
• METASTORE_NOT_EMPTY
• PROVIDER_SHARE_NOT_ACCESSIBLE

messagestring

stack_tracestring

detailsarray of object

responseobject

The normal, successful response of the operation.

View as Markdown

Request

{
  "name": "string",
  "catalog": "string",
  "schema": "string",
  "postgres_schema": "string"
}

Response

{
  "name": "string",
  "metadata": {},
  "done": true,
  "error": {
    "error_code": "UNKNOWN",
    "message": "string",
    "stack_trace": "string",
    "details": [
      {}
    ]
  },
  "response": {}
}

Delete a change data feed configuration Beta

DELETE /api/2.0/postgres/{name=projects/*/branches/*/databases/*/cdf-configs/*}

Delete a CDF configuration and stop materializing the change data feed. When force=true, also drops the Delta tables in Unity Catalog. When force=false (default), the existing tables are preserved at their last state.

API scopes: postgres

Parameters

namestringRequiredpath

The resource name of the CdfConfig to delete. Format: projects/{project}/branches/{branch}/databases/{database}/cdf-configs/{cdf_config}

forcebooleanquery

When true, also drops the replicated Delta tables in Unity Catalog. When false (the default), the replicated tables are preserved at their last synced state.

Response

namestring

The server-assigned name, which is only unique within the same service that originally returns it. If you use the default HTTP mapping, the name should be a resource name ending with operations/{unique_id}.

metadataobject

Service-specific metadata associated with the operation. It typically contains progress information and common metadata such as create time. Some services might not provide such metadata.

doneboolean

If the value is false, it means the operation is still in progress. If true, the operation is completed, and either error or response is available.

errorobject

The error result of the operation in case of failure or cancellation.

Show child attributesHide child attributes

error_codestring

Error codes returned by Databricks APIs to indicate specific failure conditions.

Values:

Show 81 possible values

• UNKNOWN
• INTERNAL_ERROR
• TEMPORARILY_UNAVAILABLE
• IO_ERROR
• BAD_REQUEST
• SERVICE_UNDER_MAINTENANCE
• WORKSPACE_TEMPORARILY_UNAVAILABLE
• DEADLINE_EXCEEDED
• CANCELLED
• RESOURCE_EXHAUSTED
• ABORTED
• NOT_FOUND
• ALREADY_EXISTS
• UNAUTHENTICATED
• UNAVAILABLE
• INVALID_PARAMETER_VALUE
• ENDPOINT_NOT_FOUND
• MALFORMED_REQUEST
• INVALID_STATE
• PERMISSION_DENIED
• FEATURE_DISABLED
• CUSTOMER_UNAUTHORIZED
• REQUEST_LIMIT_EXCEEDED
• RESOURCE_CONFLICT
• UNPARSEABLE_HTTP_ERROR
• NOT_IMPLEMENTED
• DATA_LOSS
• INVALID_STATE_TRANSITION
• COULD_NOT_ACQUIRE_LOCK
• RESOURCE_ALREADY_EXISTS
• RESOURCE_DOES_NOT_EXIST
• QUOTA_EXCEEDED
• MAX_BLOCK_SIZE_EXCEEDED
• MAX_READ_SIZE_EXCEEDED
• PARTIAL_DELETE
• MAX_LIST_SIZE_EXCEEDED
• DRY_RUN_FAILED
• RESOURCE_LIMIT_EXCEEDED
• DIRECTORY_NOT_EMPTY
• DIRECTORY_PROTECTED
• MAX_NOTEBOOK_SIZE_EXCEEDED
• MAX_CHILD_NODE_SIZE_EXCEEDED
• SEARCH_QUERY_TOO_LONG
• SEARCH_QUERY_TOO_SHORT
• MANAGED_RESOURCE_GROUP_DOES_NOT_EXIST
• PERMISSION_NOT_PROPAGATED
• DEPLOYMENT_TIMEOUT
• GIT_CONFLICT
• GIT_UNKNOWN_REF
• GIT_SENSITIVE_TOKEN_DETECTED
• GIT_URL_NOT_ON_ALLOW_LIST
• GIT_REMOTE_ERROR
• PROJECTS_OPERATION_TIMEOUT
• IPYNB_FILE_IN_REPO
• INSECURE_PARTNER_RESPONSE
• MALFORMED_PARTNER_RESPONSE
• METASTORE_DOES_NOT_EXIST
• DAC_DOES_NOT_EXIST
• CATALOG_DOES_NOT_EXIST
• SCHEMA_DOES_NOT_EXIST
• TABLE_DOES_NOT_EXIST
• SHARE_DOES_NOT_EXIST
• RECIPIENT_DOES_NOT_EXIST
• STORAGE_CREDENTIAL_DOES_NOT_EXIST
• EXTERNAL_LOCATION_DOES_NOT_EXIST
• PRINCIPAL_DOES_NOT_EXIST
• PROVIDER_DOES_NOT_EXIST
• METASTORE_ALREADY_EXISTS
• DAC_ALREADY_EXISTS
• CATALOG_ALREADY_EXISTS
• SCHEMA_ALREADY_EXISTS
• TABLE_ALREADY_EXISTS
• SHARE_ALREADY_EXISTS
• RECIPIENT_ALREADY_EXISTS
• STORAGE_CREDENTIAL_ALREADY_EXISTS
• EXTERNAL_LOCATION_ALREADY_EXISTS
• PROVIDER_ALREADY_EXISTS
• CATALOG_NOT_EMPTY
• SCHEMA_NOT_EMPTY
• METASTORE_NOT_EMPTY
• PROVIDER_SHARE_NOT_ACCESSIBLE

messagestring

stack_tracestring

detailsarray of object

responseobject

The normal, successful response of the operation.

View as Markdown

Response

{
  "name": "string",
  "metadata": {},
  "done": true,
  "error": {
    "error_code": "UNKNOWN",
    "message": "string",
    "stack_trace": "string",
    "details": [
      {}
    ]
  },
  "response": {}
}