Skip to main content
Last updated on

Troubleshoot managed ingestion pipelines

This page describes common issues with managed ingestion pipelines in Databricks Lakeflow Connect and how to resolve them. For connector-specific troubleshooting, see the documentation for your connector.

General pipeline troubleshooting​

If a pipeline run fails, click the step that failed and confirm whether the error message provides sufficient information about the nature of the error.

View pipeline event logs in the UI

You can also check and download the cluster logs from the pipeline details page by clicking Update details in the right panel, then clicking Logs. Scan the logs for errors or exceptions.

View pipeline update details in the UI

Column selection using Declarative Automation Bundles​

Issue: You're unable to use the column selection feature when you create a managed ingestion pipeline using Declarative Automation Bundles.

Resolution: Check your Databricks CLI version. If the version is below v0.251.0, reinstall the CLI.

I can't edit my pipeline in the UI​

Pipelines created or updated using Declarative Automation Bundles or the Databricks CLI might include configuration fields that are not supported in the ingestion wizard. When you try to edit a pipeline with any of these fields in the wizard, a warning indicates that visual mode is not available for the pipeline. To edit these pipelines in the UI, use code editor mode instead.

To switch modes, use the buttons at the top of the pipeline editor (Wizard, YAML, JSON).

note

If your pipeline is managed by a Declarative Automation Bundles bundle, edit the pipeline definition file in your bundle directly. Edits made in the UI will be overwritten the next time you run bundle deploy.

CANNOT_WRITE_TO_INACTIVE_TABLES​

CANNOT_WRITE_TO_INACTIVE_TABLES: Table 'XYZ' is marked as inactive and cannot be written to.

This error occurs when a pipeline tries to write to a destination table that has been marked inactive. A table becomes inactive when the corresponding source table is deleted. For more context on how inactive tables work, see Managed connector FAQs.

To resolve this:

  • If you want to resume ingesting from the source table (for example, it was recreated), run a full refresh of the affected table.
  • If the source table was intentionally deleted, drop the inactive destination table and remove it from the pipeline.
On this page