Skip to main content

Best practices for security, compliance, and privacy

The Databricks on Google Cloud Security Best Practices and Threat Model can be downloaded as a PDF document from the Security & Trust Center. The sections in this article list the best practices that can be found in the PDF along the principles of this pillar.

1. Manage identity and access using least privilege

  • Leverage multi-factor authentication
  • Use SCIM to synchronize users and groups
  • Limit the number of admin users
  • Enforce segregation of duties between administrative accounts
  • Restrict workspace admins
  • Manage access according to the principle of least privilege
  • Use OAuth token authentication
  • Enforce token management
  • Restrict cluster creation rights
  • Use compute policies
  • Use service principals to run administrative tasks and production workloads
  • Use compute that supports user isolation
  • Store and use secrets securely
  • Consider post-deployment hardening steps

Details are in the PDF referenced at the beginning of this article.

2. Protect data in transit and at rest

  • Centralise data governance with Unity Catalog
  • Plan your data isolation model
  • Avoid storing production data in DBFS
  • Secure your GCS buckets and prevent public access
  • Use VPC Service Controls
  • Protect your GCS data with soft delete
  • Backup your GCS data with dual-regions
  • Configure customer-managed keys for managed services
  • Configure customer-managed keys for storage
  • Use Delta Sharing
  • Configure a Delta Sharing recipient token lifetime
  • Additionally encrypt sensitive data at rest using Advanced Encryption Standard (AES)
  • Leverage data exfiltration prevention settings within the workspace

Details are in the PDF referenced at the beginning of this article.

3. Secure your network and protect endpoints

  • Use a customer-managed VPC
  • Configure IP access lists
  • Use GCP Private Service Connect
  • Implement network exfiltration protections
  • Isolate sensitive workloads into different networks
  • Configure a firewall for serverless compute access
  • Restrict access to valuable codebases to only trusted networks

Details are in the PDF referenced at the beginning of this article.

4. Meet compliance and data privacy requirements

  • Restart compute on a regular schedule
  • Isolate sensitive workloads into different workspaces
  • Assign Unity Catalog securables to specific workspaces
  • Implement fine-grained access controls
  • Apply tags
  • Use lineage
  • Control and monitor workspace access for Databricks personnel
  • Implement and test a Disaster Recovery strategy

Details are in the PDF referenced at the beginning of this article.

5. Monitor system security

  • Leverage system tables
  • Monitor system activities via GCP Cloud Audit Logs
  • Enable verbose audit logging
  • Manage code versions with Git folders
  • Restrict usage to trusted code repositories
  • Provision infrastructure via infrastructure-as-code
  • Manage code via CI/CD
  • Control library installation
  • Use models and data from only trusted or reputable sources
  • Implement DevSecOps processes
  • Use tagging as part of your cost monitoring and charge-back strategy
  • Use budgets to monitor account spending
  • Use Organization Policies

Details are in the PDF referenced at the beginning of this article.

Additional Resources

- Download and review the Databricks AI Security Framework (DASF) to understand how to mitigate AI security threats based on real-world attack scenarios