MANAGED_INGESTION_PIPELINE_SPEC_VALIDATION_ERROR error condition
This error condition has the following derived sub error conditions.
APPEND_ONLY_REQUIRES_CURSOR_COLUMNS
APPEND_ONLY can only be used for incremental ingestion of query based connector. The cursor_columns must be set in such cases.
AUTO_FULL_REFRESH_NOT_SUPPORTED_FOR_SCD_TYPE_2
Auto Full Refresh is not supported for SCD Type 2. Please disable auto full refresh or use a different SCD type.
AUTO_FULL_REFRESH_POLICY_CDC_ONLY
Auto Full Refresh Policy can only be set for CDC ingestion pipelines.
AUTO_FULL_REFRESH_POLICY_EMPTY_ENABLED_FIELD
Auto Full Refresh policy enabled must be set.
AUTO_FULL_REFRESH_POLICY_NOT_ENABLED
Auto Full Refresh Policy is not enabled for the org. Contact Databricks support for enabling the feature.
CANNOT_SET_CONNECTION_NAME
Cannot set connection name because the existing pipeline definition of managed ingestion is missing connection name.
CANNOT_SET_INGESTION_GATEWAY_ID
Cannot set ingestion gateway id because the existing pipeline definition of managed ingestion is missing ingestion gateway id.
CATALOG_MISMATCH
Catalog in the spec does not match the destination_catalog specified in the objects. If you want to use a different destination_catalog, enable DPM via https://docs.databricks.com/aws/en/ingestion/lakeflow-connect/multi-destination-pipeline
CATALOG_REQUIRED_IN_DPM
The catalog must be specified in spec when direct publishing mode is enabled.
CATALOG_SPEC_INCOMPLETE
source_catalog and destination_catalog must be specified for catalog spec.
COLUMN_SELECTION_NOT_ENABLED
Column selection feature is not enabled for database source type <sourceType>: include_columns or exclude_columns can't be specified in table_configurations.
COLUMN_SELECTION_NOT_SUPPORTED_FOR_SOURCE
Column selection (include_columns / exclude_columns) is not supported for <sourceType>.
CONFLICTING_CONNECTOR_OPTIONS
'<sourceOptionName>' must not be specified alongside 'connector_options.<sourceOptionName>'. Use only 'connector_options.<sourceOptionName>', as '<sourceOptionName>' is deprecated.
CONFLUENCE_SPACES_LIMIT_EXCEEDED
include_confluence_spaces in <specType> cannot have more than <maxSpaces> spaces, but <actualSpaces> were specified.
CONNECTION_RETRIEVAL_FAILED
Failed to retrieve connection '<connectionName>' from Unity Catalog. Please check whether the connection exists and contact support if the error persists.
DATA_STAGING_OPTIONS_DIRECT_CDC_ONLY
Data staging options can only be set for Direct CDC ingestion pipelines.
DESTINATION_AND_SOURCE_TABLE_MISMATCH
Destination table should be the same as source table, but got TableSpec: <tableSpec>
DESTINATION_CATALOG_CANNOT_CHANGE
The destination_catalog cannot be changed after pipeline creation. existing destination_catalog: <existingCatalog> new destination_catalog: <newCatalog>.
DESTINATION_ONLY_TABLE_SPEC_INCOMPLETE
destination_catalog, destination_schema and destination_table must be specified for table spec.
DESTINATION_SCHEMA_CANNOT_CHANGE
The destination_schema cannot be changed after pipeline creation. existing destination_schema: <existingSchema> new destination_schema: <newSchema>.
DIRECT_CDC_NOT_ENABLED
Direct Cdc Managed Ingestion Pipelines are not enabled for the org for source : <sourceType>.
Contact Databricks support for enabling the feature.
DUPLICATE_SOURCE_CATALOG
Duplicate source catalog name in sourceConfigurations: <catalog>.
EMPTY_INGESTION_DEFINITION
The ingestion_definition in pipeline spec needs to be configured.
FILE_SOURCE_FORMAT_REQUIRED
File format is required for <sourceType> file source connector but was set to UNSPECIFIED.
FILE_SOURCE_INVALID_ENTITY_TYPE
Invalid entity type '<entityType>' for <sourceType> file source connector. Please specify a valid entity type.
FILE_SOURCE_INVALID_FORMAT_OPTION_KEY
Format option key '<key>' is not valid for file format '<format>'.
FILE_SOURCE_INVALID_SCHEMA_EVOLUTION_MODE
Schema evolution mode must not be UNSPECIFIED for <sourceType> file source connector.
FILE_SOURCE_INVALID_SCHEMA_SPEC
<sourceType> file source connector requires destination_schema and destination_catalog to be set.
source_schema and source_catalog must not be set.
FILE_SOURCE_INVALID_STORAGE_MODE
Structured file source connector for <sourceType> only supports APPEND_ONLY storage mode.
Object '<objectName>' has an unsupported storage mode.
FILE_SOURCE_INVALID_TABLE_SPEC
<sourceType> file source connector requires destination_table, destination_schema, and destination_catalog to be set.
source_table, source_schema, and source_catalog must not be set.
FILE_SOURCE_INVALID_TIMESTAMP_FORMAT
Invalid timestamp format '<timestamp>' in file filter '<filterType>'. Expected format: YYYY-MM-DDTHH:mm:ss (for example, 2020-06-01T13:00:00).
FILE_SOURCE_SCD_TYPE_NOT_SUPPORTED
scd_type is not supported for structured file source connector for <sourceType>.
Object '<objectName>' must not set scd_type. Use storage_mode instead.
FILE_SOURCE_SPEC_NOT_SUPPORTED
File source connectors do not support the provided spec type.
Found: <specType>.
FILE_SOURCE_URL_REQUIRED
URL is required for <sourceType> file source connector.
FOREIGN_CATALOG_SCHEMA_SPEC_INCOMPLETE
source_catalog and source_schema must be specified for foreign_catalog ingestion pipelines in schema spec.
FOREIGN_CATALOG_TABLE_SPEC_INCOMPLETE
source_catalog, source_schema and source_table must be specified for foreign_catalog ingestion pipelines in table spec.
FOREIGN_CATALOG_UNSUPPORTED_SPEC_TYPE
Only table specs and schema specs are supported for foreign_catalog ingestion pipelines. Catalog, report, and identity mapping specs are not allowed.
FULL_REFRESH_WINDOW_CDC_ONLY
Full Refresh Window can only be set for CDC ingestion pipelines.
FULL_REFRESH_WINDOW_NOT_ENABLED
Full Refresh Window is not enabled for the org. Contact Databricks support for enabling the feature.
GOOGLE_DRIVE_CONNECTOR_OPTIONS_NOT_SET_FOR_ALL_OBJECTS
Google Drive connector options must be set on all objects.
Object '<objectName>' is missing Google Drive connector options.
GOOGLE_DRIVE_GROUP_MEMBERSHIP_FORBIDS_URL_AND_SCOPE
Google Drive GROUP_MEMBERSHIP entity type does not accept a URL or scope; group membership is tenant-wide.
GOOGLE_DRIVE_INVALID_SCOPE
Google Drive ingestion scope must not be UNSPECIFIED. Provide a valid scope.
GOOGLE_DRIVE_URL_OR_SCOPE_REQUIRED
Google Drive connector options must specify exactly one of a URL or a scope.
INCLUDE_COLUMNS_EXCLUDE_COLUMNS_BOTH_SPECIFIED
include_columns and exclude_columns can't be specified together in table_configurations.
INGESTION_GATEWAY_ID_EMPTY
Ingestion gateway ID can't be empty.
INGESTION_GATEWAY_INVALID_OR_DOES_NOT_EXIST
The source ingestion gateway is not valid or does not exist.
INGESTION_GATEWAY_NOT_FOUND
Ingestion Gateway pipeline : <pipelineId> does not exist
INVALID_CHANNEL_FOR_CDC
The channel needs to be <allowedChannels> for ingestion pipeline of CDC Connector.
INVALID_FULL_REFRESH_WINDOW_START_HOUR
Full Refresh Window start_hour must be an integer between 0 and 23.
INVALID_FULL_REFRESH_WINDOW_TIMEZONE
Invalid time zone id: <timeZoneId> passed in Full Refresh Window. Please use one of TZ Database time zones (https://en.wikipedia.org/wiki/List_of_tz_database_time_zones).
INVALID_SOURCE_OPTIONS
<optionName> in <specType> is not allowed for source: <sourceType>.
KAFKA_AVRO_SCHEMA_REQUIRED
Avro transformer requires either 'schema' or 'schema_file_path' to be set.
KAFKA_INVALID_MAX_OFFSETS_PER_TRIGGER
Invalid max_offsets_per_trigger '<maxOffsetsPerTrigger>' for Kafka connector.
Value must be a positive integer.
KAFKA_INVALID_STARTING_OFFSET
Invalid starting_offset '<offset>' for Kafka connector.
Valid values are 'latest' and 'earliest'.
KAFKA_INVALID_TABLE_SPEC
Kafka connector requires destination_table, destination_schema, and destination_catalog to be set.
source_table, source_schema, and source_catalog must not be set.
KAFKA_JSON_OPTIONS_EMPTY
JSON transformer options were specified but no parsing configuration was provided.
Specify at least one of 'schema', 'schema_file_path', or 'schema_evolution_mode',
or omit json_options entirely to parse the value as a Variant column.
KAFKA_JSON_SCHEMA_AND_EVOLUTION_EXCLUSIVE
JSON transformer cannot specify both an explicit schema ('schema' or 'schema_file_path') and 'schema_evolution_mode'.
Use an explicit schema for fixed schemas, or 'schema_evolution_mode' for schema inference.
KAFKA_JSON_SCHEMA_EXCLUSIVE
JSON transformer cannot specify both 'schema' and 'schema_file_path'. Use one or the other.
KAFKA_ONLY_TABLE_SPEC_SUPPORTED
Kafka connector only supports table specs.
Schema and catalog specs are not supported. Found: <specType>.
KAFKA_OPTIONS_REQUIRED
Kafka connector options (kafka_options) must be set on all specs.
KAFKA_PROTOBUF_DESC_FILE_PATH_REQUIRED
Protobuf transformer requires 'desc_file_path' to be set.
KAFKA_PROTOBUF_MESSAGE_NAME_REQUIRED
Protobuf transformer requires 'message_name' to be set.
KAFKA_TOPICS_EXCLUSIVE
Exactly one of 'topics' or 'topic_pattern' must be specified in Kafka connector options, not both or neither.
KAFKA_TRANSFORMER_FORMAT_CONFIG_MISMATCH
Transformer format '<format>' does not match the provided config '<config>'.
Use json_options for JSON, avro_options for AVRO, protobuf_options for PROTOBUF.
STRING does not accept any config. JSON parses as Variant when json_options is omitted.
KAFKA_TRANSFORMER_FORMAT_REQUIRED
Transformer format is required when a transformer is specified.
KAFKA_UNSUPPORTED_SCHEMA_EVOLUTION_MODE
Schema evolution mode '<mode>' is not supported for the Kafka connector.
Supported modes can be found here: https://docs.databricks.com/aws/en/ldp/from-json-schema-evolution
LIBRARIES_NOT_ALLOWED
The libraries must be empty for ingestion pipeline.
META_ADS_INVALID_ACTION_ATTRIBUTION_WINDOWS
Invalid action_attribution_windows values: <invalidWindows>. See the Meta documentation for a list of allowed action attribution windows: https://developers.facebook.com/documentation/ads-commerce/graph-api/reference/adgroup/insights/v25.0
META_ADS_INVALID_ACTION_BREAKDOWNS
Invalid action_breakdowns parameters: <unknownActionBreakdowns>. See the Meta documentation for a list of allowed action breakdowns: https://developers.facebook.com/documentation/ads-commerce/graph-api/reference/adgroup/insights/v25.0
META_ADS_INVALID_ACTION_REPORT_TIME
Invalid action_report_time value: <actionReportTime>. Enter one of impression, conversion, mixed or lifetime. See the Meta documentation: https://developers.facebook.com/documentation/ads-commerce/graph-api/reference/adgroup/insights/v25.0
META_ADS_INVALID_BREAKDOWNS
Invalid Breakdown parameters: <unknownBreakdowns>. See the Meta documentation for a list of allowed breakdowns: https://developers.facebook.com/documentation/ads-commerce/graph-api/reference/adgroup/insights/v25.0
META_ADS_INVALID_LEVEL
Invalid Level Parameter: <level>. Enter one of ad, adset, campaign or account. See the Meta documentation: https://developers.facebook.com/documentation/ads-commerce/graph-api/reference/adgroup/insights/v25.0
META_ADS_INVALID_LOOKBACK_WINDOW
Invalid Click Attribution Window Parameter: <window>. Enter one of 1, 7 or 28. See the Meta documentation: https://developers.facebook.com/documentation/ads-commerce/graph-api/reference/adgroup/insights/v25.0
META_ADS_INVALID_START_DATE
Invalid Start Date Parameter: <date>. Enter in correct YYYY-MM-DD format. See the Meta documentation: https://developers.facebook.com/documentation/ads-commerce/graph-api/reference/adgroup/insights/v25.0
META_ADS_INVALID_TIME_INCREMENT
Invalid Time increment parameter: <days>. It has to be either monthly, all_days or a valid positive integer. See the Meta documentation: https://developers.facebook.com/documentation/ads-commerce/graph-api/reference/adgroup/insights/v25.0
MISSING_INGESTION_DEFINITION
Ingestion pipeline settings should contain ingestion_definition.
NETSUITE_JAR_PATH_NOT_SUPPORTED_FOR_SOURCE
netsuite_jar_path is not supported for <sourceType>.
NETSUITE_JAR_PATH_REQUIRED
netsuite_jar_path can't be empty for source: <sourceType>.
NON_EMPTY_MANAGED_DEFINITION
The managed_definition in pipeline spec should not be configured for ingestion pipeline.
NOT_IN_DEFAULT_PUBLISHING_MODE
Only Direct Publishing Mode is supported for the Managed Ingestion Pipeline.
POSTGRES_CONFIG_UNSUPPORTED_FOR_SOURCE
PostgreSQL source configuration should only be provided for POSTGRES connector.
POSTGRES_PUBLICATION_NAME_CONFLICTS_WITH_CONNECTOR_FORMAT
The custom publication name '<publicationName>' is reserved for internal use. Publication names matching the pattern 'dbxpub{id}_<normalizedGatewayId>' are auto-generated by the connector. Please choose a different publication name.
POSTGRES_PUBLICATION_NAME_REQUIRED
PostgreSQL publication name cannot be empty or null.
POSTGRES_SLOT_CONFIG_FOR_ALL_OR_NONE_CATALOGS
PostgreSQL slot configuration must be specified either for all replication catalogs or for none of them.
POSTGRES_SLOT_CONFIG_FOR_REPLICATION_CATALOGS_ONLY
PostgreSQL slot configuration has to be provided for all the replication catalogs and not provided for any non-replication catalog.
POSTGRES_SLOT_CONFIG_NOT_ALLOWED
<reason>
POSTGRES_SLOT_NAME_CONFLICTS_WITH_CONNECTOR_FORMAT
The custom slot name '<slotName>' is reserved for internal use. Slot names matching the pattern 'dbx_{id}_<normalizedGatewayId>' are auto-generated by the connector. Please choose a different slot name.
POSTGRES_SLOT_NAME_REQUIRED
PostgreSQL slot name cannot be empty or null.
QUERY_BASED_CONNECTOR_CONFIG_EXISTS_FOR_CDC_CONNECTOR
Query-based connector configs are not supported for CDC connector with gateway.
RABBITMQ_INVALID_TABLE_SPEC
RabbitMQ connector ingests from queues, not tables.
Found '<offendingField>' set; source_table, source_schema, and source_catalog must not be set.
RABBITMQ_ONLY_TABLE_SPEC_SUPPORTED
RabbitMQ connector only supports table specs.
Schema and catalog specs are not supported. Found: <specType>.
RABBITMQ_OPTIONS_REQUIRED
RabbitMQ connector options (rabbitmq_options) must be set on all specs.
RABBITMQ_QUEUE_REQUIRED
RabbitMQ connector options must specify a non-empty 'queue'.
REPORT_SPEC_INCOMPLETE
destination_catalog and destination_schema must be specified for report spec.
SALESFORCE_FORMULA_FIELDS_NOT_SUPPORTED_FOR_SOURCE
salesforce_include_formula_fields is not supported for <sourceType>.
SCD_TYPE_2_NOT_SUPPORTED_FOR_SALESFORCE
SCD Type 2 ingestion for Salesforce ingestion pipeline is not supported.
SCD_TYPE_2_NOT_SUPPORTED_FOR_SOURCE
SCD Type 2 ingestion is not supported for <sourceType>.
SCD_TYPE_2_NOT_SUPPORTED_FOR_TABLE
SCD Type 2 ingestion is not supported for the following <sourceType> table(s): <sourceTable>.
SCHEMA_MISMATCH
Target in the spec does not match the destination_schema specified in the objects. If you want to use a different destination_schema, enable DPM via https://docs.databricks.com/aws/en/ingestion/lakeflow-connect/multi-destination-pipeline
SCHEMA_NOT_ALLOWED_WITHOUT_DPM
The schema cannot be set when direct publishing mode is not enabled.
SCHEMA_OR_TARGET_REQUIRED_IN_DPM
Either schema or target must be specified in spec when direct publishing mode is enabled.
SCHEMA_REQUIRED_IN_DPM
The schema must be specified in spec when direct publishing mode is enabled.
SCHEMA_SPEC_INCOMPLETE
source_schema, destination_catalog and destination_schema must be specified for schema spec.
SERVERLESS_MODE_REQUIRED
The serverless mode needs to be true for managed ingestion pipeline with <connectorType> connector.
SERVICE_NAME_EMPTY_IN_CONNECTION
Option service_name is empty in the connection: <connectionName>.
Provide a non-empty service_name in the Oracle UC Connection and retry creating the pipeline.
SERVICE_NAME_MISSING_IN_CONNECTION
Missing service_name option in the connection: <connectionName>.
Add the service_name in the Oracle UC Connection and retry creating the pipeline.
SHAREPOINT_CONNECTOR_OPTIONS_NOT_SET_FOR_ALL_OBJECTS
SharePoint connector options must be set on all objects.
Object '<objectName>' is missing SharePoint connector options.
SOURCE_CATALOG_NAME_REQUIRED
Catalog name has to be provided always for source catalog configuration.
SOURCE_TABLE_REQUIRED
Source table should be present in the table spec, but got TableSpec <tableSpec>
SOURCE_TYPE_MISMATCH
Provided Source type '<providedSourceType>' is not consistent with source type '<inferredSourceType>' inferred from connection_name or ingestion_gateway_id. Please fix it or remove the source type from Managed Ingestion Definition.
TABLE_SPEC_INCOMPLETE
source_table, destination_catalog and destination_schema must be specified for table spec.
UNSUPPORTED_CONNECTION_TYPE
Connection type '<connectionType>' is not supported for ingestion pipelines.
UNSUPPORTED_SOURCE_OPTION
Found unsupported source option in ingestion_definition. unsupported source option: <source>. Supported source options are connection_name, ingestion_gateway_id, and ingest_from_uc_foreign_catalog.
UNSUPPORTED_SOURCE_OPTION_FOR_CDC_CONNECTOR
Unsupported source option <source> in ingestion_definition.
Only connection_name or ingestion_gateway_id are supported for cdc connectors.
UNSUPPORTED_SOURCE_OPTION_FOR_QUERY_BASED_CONNECTOR
Unsupported source option <source> in ingestion_definition.
Only connection_name or ingest_from_uc_foreign_catalog are supported for query based connectors.
UNSUPPORTED_SOURCE_TYPE_FOR_CDC_CONNECTOR
<sourceType> is not supported for cdc connector.
UNSUPPORTED_SOURCE_TYPE_FOR_QUERY_BASED_CONNECTOR
<sourceType> is not supported for query based connector.
WORKDAY_DESTINATION_TABLE_REQUIRED
Destination table must be set for Workday Reports.
WORKDAY_PRIMARY_KEYS_REQUIRED
Table configurations must be set and primary keys should be specified for Workday Reports.
WORKDAY_REPORT_URL_DOMAIN_INVALID
Invalid Workday Reports URL domain '<domain>'. The URL must end with one of the supported domains: <allowedDomains>.
WORKDAY_REPORT_URL_FORMAT_INVALID
Workday Report URL must include 'format=json' query parameter. URL: <url>.
WORKDAY_REPORT_URL_SYNTAX_INVALID
Invalid Workday Report URL format: <url>.
WORKDAY_REPORT_URL_VALIDATION_ERROR
Error validating Workday Report URL: <error>.