Skip to main content
Last updated on

Troubleshoot UNITY_CATALOG_INITIALIZATION_FAILED

This page describes how to diagnose and resolve the UNITY_CATALOG_INITIALIZATION_FAILED error in Databricks pipelines.

Overview​

UNITY_CATALOG_INITIALIZATION_FAILED is a catch-all error that appears when Unity Catalog cannot initialize storage for a pipeline or workload during cluster startup. Despite its name, the failure is usually not caused by a Unity Catalog misconfiguration. Instead, it almost always indicates an underlying infrastructure problem: a networking issue preventing the cluster from reaching the Databricks control plane, or a permissions issue preventing access to the cloud storage backing the Unity Catalog metastore.

This error surfaces across multiple Databricks products, including Lakeflow Connect pipelines, Lakeflow Spark Declarative Pipelines pipelines, and Vector Search.

General error message​

Encountered an error with Unity Catalog while setting up the pipeline on cluster [CLUSTER_ID].
Ensure that your Unity Catalog configuration is correct, and that required resources
(e.g., catalog, schema) exist and are accessible. Also verify that the cluster has
appropriate permissions to access Unity Catalog.

The error message instructs you to check your Unity Catalog configuration, but the root cause is often a networking or cloud permissions issue as described in the following sections.

Root causes and resolutions​

Unity Catalog resources not configured correctly​

Cause: The catalog, schema, or connection referenced by the pipeline does not exist or is not accessible from the workspace. This is the case described by the error message itself and is less common than the infrastructure issues above.

Resolution:

  1. Verify that the catalog and schema referenced in the pipeline exist and are accessible from the workspace. In Databricks, go to Catalog and confirm that the catalog is visible and that you have at least USE CATALOG and USE SCHEMA privileges.
  2. If you are using Lakeflow Connect, verify that the connection used by the pipeline is valid. Go to Catalog > External Data > Connections and confirm that the connection is listed and accessible.
  3. Verify that the cluster or pipeline has the necessary Unity Catalog privileges. See Manage privileges in Unity Catalog.
On this page