Best practices for Unity Catalog secrets

Organize Unity Catalog secrets so that each application can access only its own secrets, most principals use a secret without reading its value, and you control the key that encrypts the values. For how to create, grant, and read secrets, see Secrets in Unity Catalog.

Group secrets by who needs them

Create a schema for each application, environment, or team, and grant CREATE SECRET privileges on the schema instead of on the catalog. A schema holds up to 1,000 secrets, and a metastore up to 10,000.

• Grant on a single secret: A grant on a schema applies to every secret in it, including secrets added later. When a principal needs only one secret, grant READ SECRET on that secret instead of on its schema.
• Give production secrets their own catalog when you need to: Workspace binding and customer-managed keys apply to a whole catalog. To keep production secrets out of development workspaces, or to encrypt them with their own key, put them in a separate catalog. See Workspace-catalog binding and Encrypt Databricks-managed secrets with your own key.

To give the orders application its own schema, and let a reporting job read only one of its secrets, run the following:

SQL

-- The application's service principal reads every secret in its schema
CREATE SCHEMA main.orders_app;
GRANT USE CATALOG ON CATALOG main TO `orders-service-principal`;
GRANT USE SCHEMA, READ SECRET ON SCHEMA main.orders_app TO `orders-service-principal`;

-- The reporting job reads only the warehouse password
GRANT USE CATALOG ON CATALOG main TO `reporting-service-principal`;
GRANT USE SCHEMA ON SCHEMA main.orders_app TO `reporting-service-principal`;
GRANT READ SECRET ON SECRET main.orders_app.warehouse_password TO `reporting-service-principal`;

Manage permissions to discover, rotate, and read secrets separately

Each of these is a separate privilege, so a principal that updates a secret or finds it in Catalog Explorer doesn't need to see its value:

To let a principal	Grant	Returns the value
Find secrets and request access	BROWSE on the catalog. It shows each secret's metadata, such as its name, owner, and expiration date.	No
Rotate a secret by updating its value	WRITE SECRET	No
Read the value, for example with dbutils.secrets in a job	READ SECRET	Yes

To let a principal	Grant	Returns the value
Find secrets and request access	BROWSE on the catalog. It shows each secret's metadata, such as its name, owner, and expiration date.	No
Rotate a secret by updating its value	WRITE SECRET	No
Read the value, for example with dbutils.secrets in a job	READ SECRET	Yes

For the full list of privileges, see Grant access to secrets.

To let everyone find the secrets and request access, and let a rotation job update the warehouse password without reading it, run the following:

SQL

GRANT BROWSE ON CATALOG main TO `account users`;

GRANT USE CATALOG ON CATALOG main TO `rotation-service-principal`;
GRANT USE SCHEMA ON SCHEMA main.orders_app TO `rotation-service-principal`;
GRANT WRITE SECRET ON SECRET main.orders_app.warehouse_password TO `rotation-service-principal`;

BROWSE shows comments to everyone who has it. Use a secret's comment to say who owns it and how to request access. Never put a value, or a hint about one, in a comment.

Encrypt Databricks-managed secrets with your own key

Customer-managed key (CMK) settings on a catalog apply to every Databricks-managed secret in it. To encrypt production secrets with a separate key, put them in their own catalog. For how CMK applies to secrets, see Encrypt secret values with customer-managed keys.

Rotate secrets at the source

For Databricks-managed secrets, update the value in Catalog Explorer or with the REST API. Set an expiration date so that Catalog Explorer shows a warning when the secret expires.

Code that already holds a value keeps it until it reads the secret again. In long-running jobs, read the secret each time you open a connection instead of only at startup.

Let workloads use secrets without reading them

Most users who rely on a secret never need to see its value. A Unity Catalog UDF that declares a secret reads it with its owner's permissions, so callers need only EXECUTE on the UDF, for example to mask or hash personal data. By contrast, anyone who runs a SQL function that calls secret() on a workspace secret scope needs READ on the scope, which exposes every secret in it. See Use secrets in UDFs.

• Grant READ SECRET only to the service principals that run jobs, the owners of UDFs that declare the secret, and the small group that owns the secrets.
• Make a group or service principal the owner of each UDF that declares a secret, so the UDF doesn't fail when someone leaves.
• Use session-scoped UDFs only when every caller can already read the secret. They read secrets with the caller's permissions.

warning

Never return a secret value from a UDF. Redaction doesn't stop UDF code from returning a value in query results.

Control who can grant access, and audit it

• Assign ownership of secrets and their schemas to a group instead of an individual.
• Metastore admins and principals with MANAGE can't read a value, but they can grant themselves READ SECRET. Keep MANAGE to a small group.
• Review secret reads and grants in the system.access.audit system table. See Audit secret access.

Additional resources

• • Secrets in Unity Catalog
  • Create, grant, and read Unity Catalog secrets.
• • Secret management
  • Compare Unity Catalog secrets with workspace secret scopes, and see where you can use each one.