Data access configuration
This article describes the data access configurations performed by Databricks administrators for all SQL warehouses (formerly SQL endpoints) using the UI.
To configure all SQL warehouses using the REST API, see Global SQL Warehouses API.
Important
Changing these settings restarts all running SQL warehouses.
For a general overview of how to enable access to data, see Access control.
Requirements
You must be a Databricks administrator to configure settings for all SQL warehouses.
Configure an instance profile
To configure all warehouses to use an AWS instance profile when accessing AWS storage:
Click
Settings at the bottom of the sidebar and select SQL Admin Console.
Click the SQL Warehouse Settings tab.
In the Instance Profile drop-down, select an instance profile. If there are no profiles:
In a new browser tab, click the sidebar persona switcher to select Data Science & Engineering.
Go to the admin console.
Configure an instance profile.
Go back to the SQL Admin Console browser tab and select the instance profile you just created.
Click Save.
Warning
If a user does not have permission to use the instance profile, all warehouses the user creates will fail to start.
If the instance profile is invalid, all SQL warehouses will become unhealthy.
You can also configure an instance profile the Databricks Terraform provider and databricks_sql_global_config.
Note
You can also edit the Data Access Configuration textbox entries directly.
Important
To set a configuration property to the value of a secret without exposing the secret value to Spark, set the value to {{secrets/<secret-scope>/<secret-name>}}
. Replace <secret-scope>
with the secret scope and <secret-name>
with the secret name. The value must start with {{secrets/ and end with }}. For more information about this syntax, see Syntax for referencing secrets in a Spark configuration property or environment variable.
Configure data access properties for SQL warehouses
To configure all warehouses with data access properties, such as when you use an external metastore instead of the Hive metastore:
Click
Settings at the bottom of the sidebar and select SQL Admin Console.
Click the SQL Warehouse Settings tab.
In the Data Access Configuration textbox, specify key-value pairs containing metastore properties.
Important
To set a Spark configuration property to the value of a secret without exposing the secret value to Spark, set the value to
{{secrets/<secret-scope>/<secret-name>}}
. Replace<secret-scope>
with the secret scope and<secret-name>
with the secret name. The value must start with{{secrets/
and end with}}
. For more information about this syntax, see Syntax for referencing secrets in a Spark configuration property or environment variable.Click Save.
You can also configure data access properties with the Databricks Terraform provider and databricks_sql_global_config.
Supported properties
The following properties are supported for SQL warehouses. For an entry that ends with *
, all properties within that prefix are supported. For example, spark.sql.hive.metastore.*
indicates that both spark.sql.hive.metastore.jars
and spark.sql.hive.metastore.version
are supported, as well as any other properties that start with spark.sql.hive.metastore
.
For properties whose values contain sensitive information, you can store the sensitive information in a secret and set the property’s value to the secret name using the following syntax: secrets/<secret-scope>/<secret-name>
.
spark.databricks.hive.metastore.glueCatalog.enabled
spark.databricks.delta.catalog.update.enabled false
spark.sql.hive.metastore.*
(spark.sql.hive.metastore.jars
andspark.sql.hive.metastore.jars.path
are unsupported for serverless SQL warehouses.)spark.sql.warehouse.dir
spark.hadoop.aws.region
spark.hadoop.datanucleus.*
spark.hadoop.fs.*
spark.hadoop.hive.*
spark.hadoop.javax.jdo.option.*
spark.hive.*
spark.hadoop.aws.glue.*
spark.databricks.cloudfetch.override.enabled
For more information about how to set these properties, see External Hive metastore and AWS Glue data catalog.