Compute access mode limitations for Unity Catalog
Databricks recommends using Unity Catalog and shared access mode for most workloads. This article outlines limitations and requirements for each access mode with Unity Catalog. For details on access modes, see Access modes.
Databricks recommends using compute policies to simplify configuration options for most users. See Create and manage compute policies.
Note
No-isolation shared is a legacy access mode that does not support Unity Catalog.
Important
Init scripts and libraries have different support across access modes and Databricks Runtime versions. See Where can init scripts be installed? and Cluster-scoped libraries.
Single user access mode limitations on Unity Catalog
Single user access mode on Unity Catalog has the following limitations. These are in addition to the general limitations for all Unity Catalog access mode. See General limitations for Unity Catalog.
Fine-grained access control limitations for Unity Catalog single user access mode
On Databricks Runtime 15.3 and below, fine-grained access control on single user compute is not supported. Specifically:
You cannot access a table that has a row filter or column mask.
You cannot access dynamic views.
To read from any view, you must have
SELECT
on all tables and views that are referenced by the view.
To query dynamic views, views on which you don’t have SELECT
on the underlying tables and views, and tables with row filters or column masks, use one of the following:
A SQL warehouse.
Compute with shared access mode.
Compute with single user access mode on Databricks Runtime 15.4 LTS or above (Public Preview).
Databricks Runtime 15.4 LTS and above support fine-grained access control on single user compute. To take advantage of the data filtering provided in Databricks Runtime 15.4 LTS and above, you must also verify that your workspace is enabled for serverless compute, because the data filtering functionality that supports fine-grained access controls runs on serverless compute. You might therefore be charged for serverless compute resources when you use single user compute to run data filtering operations. See Fine-grained access control on single user compute.
Streaming table and materialized view limitations for Unity Catalog single user access mode
On Databricks Runtime 15.3 and below, you cannot use single user compute to query tables that were created using a Delta Live Tables pipeline, including streaming tables and materialized views, if those tables are owned by other users. The user who creates a table is the owner.
To query tables created by Delta Live Tables and owned by other users, use one of the following:
A SQL warehouse.
Compute with shared access mode on Databricks Runtime 13.3 LTS or above.
Compute with single user access mode on Databricks Runtime 15.4 LTS or above (Public Preview).
Databricks Runtime 15.4 LTS and above support queries on Delta Live Tables-generated tables on single user compute. To take advantage of the data filtering provided in Databricks Runtime 15.4 LTS and above, you must also verify that your workspace is enabled for serverless compute, because the data filtering functionality that supports Delta Live Tables-generated tables runs on serverless compute. You might therefore be charged for serverless compute resources when you use single user compute to run data filtering operations. See Fine-grained access control on single user compute.
Streaming limitations for Unity Catalog single user access mode
Asynchronous checkpointing is not supported in Databricks Runtime 11.3 LTS and below.
StreamingQueryListener
requires Databricks Runtime 15.1 or above to use credentials or interact with objects managed by Unity Catalog on single user compute.
General limitations for Unity Catalog
The following limitations apply to all Unity Catalog-enabled access modes.
UDFs
Graviton instance support for UDFs on Unity Catalog-enabled clusters is available in Databricks Runtime 15.2 and above. Additional limitations exist for shared access mode. See UDF limitations and requirements for Unity Catalog shared access mode.
Streaming limitations for Unity Catalog
Apache Spark continuous processing mode is not supported. See Continuous Processing in the Spark Structured Streaming Programming Guide.
See also Streaming limitations for Unity Catalog single user access mode and Streaming limitations and requirements for Unity Catalog shared access mode.
For more on streaming with Unity Catalog, see Using Unity Catalog with Structured Streaming.