Sometimes accessing data requires that you authenticate to external data sources, such as Amazon Redshift, through JDBC. Instead of directly typing your credentials into a notebook, use Databricks secret management to store your credentials and reference them in notebooks and jobs. To manage secrets, you can use the Databricks CLI to access the Secrets API.
Secret management requires Databricks Runtime 4.0 or above and Databricks CLI 0.7.1 or above.
To set up secrets you:
- Create a secret scope.
- Add secrets to the scope.
- If you have the Databricks Operational Security Package, assign access control to the secret scope.
This guide shows you how to perform these setup tasks and manage secrets using the Databricks CLI.
For more information, see:
- An end-to-end example of how to use secret management in your workflows: Secret Management Workflow Example.
- Reference information about the Databricks CLI: Secrets CLI.
- Reference information about the API: Secrets API.
- How to use Databricks Utilities to reference secrets in notebooks and jobs: Secrets utilities.
In this guide:
- Secret Scopes
- Secret Access Control
- Notebook Secret Redaction
- Secret Management Workflow Example