Skip to main content
Last updated on

Workspace secret scopes

A workspace secret scope is a named collection of secrets in one workspace. Secret scope permissions control who can read, write, and manage the secrets in it. Permissions apply to the whole scope, not to individual secrets, so a user who can read one secret in a scope can read all of them. To compare workspace secret scopes with Unity Catalog secrets, see Choose where to store a secret.

To use a workspace secret scope:

  1. Create a secret scope.
  2. Add secrets to the scope.
  3. Grant permissions on the scope.
  4. Reference the secrets in your code, or in a Spark configuration property or environment variable. See Use a secret in a Spark configuration property or environment variable.
warning

Workspace admins, secret creators, and users who have been granted permission can access and read Databricks secrets. Although Databricks attempts to redact secret values in notebook outputs, it is not possible to fully prevent these users from viewing secret contents. Always assign secret access permissions carefully to protect sensitive information.

Manage secret scopes​

Databricks recommends aligning secret scopes to roles or applications rather than individuals.

Secret scopes are stored in an encrypted database owned and managed by Databricks.

After creating a secret scope, you can assign permissions to grant users access to read, write, and manage secret scopes.

Create a secret scope​

Create a secret scope with the Databricks CLI (version 0.205 and above), the Databricks workspace UI, or the Secrets API.

Secret scope names:

  • Must be unique within a workspace.
  • Must consist of alphanumeric characters, dashes, underscores, @, and periods, and cannot exceed 128 characters.
  • Are case insensitive.

Secret scope names are considered non-sensitive and are readable by all users in the workspace.

To create a scope using the Databricks CLI:

Bash
databricks secrets create-scope <scope-name>

By default, scopes are created with MANAGE permission for the user who created the scope. After you have created a Databricks-backed secret scope, you can add secrets to it.

List secret scopes​

To list the existing scopes in a workspace using the CLI:

Bash
databricks secrets list-scopes

You can also list secret scopes using the Secrets API.

Delete a secret scope​

Deleting a secret scope deletes all secrets and ACLs applied to the scope. To delete a scope using the CLI, run the following:

Bash
databricks secrets delete-scope <scope-name>

You can also delete a secret scope using the Secrets API.

Manage secrets​

A secret is a key-value pair that stores sensitive material using a key name that is unique within a secret scope.

Create a secret​

Create a secret with the Databricks CLI (version 0.205 and above), the Databricks SDK for Python in a notebook, or the Secrets API. Secret names are case insensitive.

When you create a secret in a Databricks-backed scope, you can specify the secret value in one of three ways:

  • Specify the value as a string with the --string-value flag.
  • Input the secret when prompted interactively (single-line secrets).
  • Pass the secret using standard input (multi-line secrets).

For example:

Bash
databricks secrets put-secret --json '{
  "scope": "<scope-name>",
  "key": "<key-name>",
  "string_value": "<secret>"
}'

If you are creating a multi-line secret, you can pass the secret using standard input. For example:

Bash
(cat << EOF
this
is
a
multi
line
secret
EOF
) | databricks secrets put-secret <scope-name> <key-name>

Read a secret​

Read a secret with the Databricks CLI (version 0.205 and above), or in a notebook with Secrets utility (dbutils.secrets).

To read the value of a secret using the Databricks CLI, you must decode the base64 encoded value. You can use jq to extract the value and base64 --decode to decode it:

Bash
databricks secrets get-secret <scope-name> <key-name> | jq -r .value | base64 --decode

List secrets​

To list secrets in a given scope:

Bash
databricks secrets list-secrets <scope-name>

The response shows metadata about the secrets, such as their key names. To list this metadata in a notebook or job, use Secrets utility (dbutils.secrets). For example:

Python
dbutils.secrets.list('my-scope')

Delete a secret​

To delete a secret from a scope with the Databricks CLI:

Bash
databricks secrets delete-secret <scope-name> <key-name>

You can also use the Secrets API.

Manage secret scope permissions​

By default, the user who creates a secret scope has the MANAGE permission on it, which lets them read secrets in the scope, write secrets to it, and manage its permissions.

Manage secret scope permissions with the Databricks CLI (version 0.205 and above) or the Secrets API. For secret permission levels, see Secret ACLs.

Grant a user permissions on a secret scope​

note

Granting permissions on a secret scope requires the Premium plan or above.

To grant a user permissions on a secret scope using the Databricks CLI:

Bash
databricks secrets put-acl <scope-name> <principal> <permission>

Making a put request for a principal that already has an applied permission overwrites the existing permission level.

The principal field specifies an existing Databricks principal. A user is specified using their email address, a service principal using its applicationId value, and a group using its group name. For more information, see Principal.

For example, to let a group such as datascience read the secrets in a scope, grant it READ:

Bash
databricks secrets put-acl <scope-name> <group> READ

View secret scope permissions​

To view all secret scope permissions for a given secret scope:

Bash
databricks secrets list-acls <scope-name>

To get the secret scope permissions applied to a principal for a given secret scope:

Bash
databricks secrets get-acl <scope-name> <principal>

If no ACL exists for the given principal and scope, this request fails.

Delete a secret scope permission​

To delete a secret scope permission applied to a principal for a given secret scope:

Bash
databricks secrets delete-acl <scope-name> <principal>

Secret redaction​

Storing credentials as Databricks secrets helps protect them when you run notebooks and jobs. However, you can still accidentally print a secret to standard output buffers or display the value during variable assignment.

To prevent this, Databricks attempts to redact secret values that are read using dbutils.secrets.get() and referenced in a Spark configuration property. When displayed, the secret values are replaced with [REDACTED].

For example, if you set a variable to a secret value using dbutils.secrets.get() and then print that variable, that variable is replaced with [REDACTED].

warning

Redaction applies only to literal secret values. The secret redaction functionality does not prevent deliberate and arbitrary transformations of a secret literal. To control secrets, use access control lists to limit permissions to run commands. This prevents unauthorized access to shared notebook contexts.

Secret redaction in SQL​

Databricks attempts to redact all SQL data query language (DQL) commands that invoke the secret function, including referenced views and user-defined functions. When the secret function is used, the output is replaced with [REDACTED] where possible. Like notebook redaction, this only applies to literal values, not to transformed or indirectly referenced secrets.

For SQL data manipulation language (DML) commands, Databricks allows secret lookups if the secret is considered safe. For example, a secret is safe when wrapped in a cryptographic function such as sha() or aes_encrypt(), which prevent raw values from being stored unencrypted.

Secret validation in SQL​

Databricks also applies validation to block SQL DML commands that could result in unencrypted secrets being saved to tables. The query analyzer attempts to identify and prevent these scenarios, which helps avoid accidental storage of sensitive information in plaintext.

Additional resources​

On this page