Best practices for security, compliance, & privacy

The security best practices can be found in the Databricks Security and Trust Center under Security Features.

For details, see this downloadable guide: Databricks on AWS Security Best Practices and Threat Model.

For generative AI, Databricks provides an actionable framework for managing AI security, the Databricks AI Security Framework (DASF).

The following sections list the best practices that can be found in the PDF along the principles of this pillar.

1. Manage identity and access using least privilege

  • Configure single sign-on and unified login.

  • Use multi-factor authentication.

  • Separate admin accounts from normal user accounts.

  • Use token management.

  • SCIM synchronization of users and groups.

  • Limit cluster creation rights.

  • Store and use secrets securely.

  • Cross-account IAM role configuration.

  • Customer-approved workspace login.

  • Use clusters that support user isolation.

  • Use service principals to run production jobs.

Details are in the PDF referenced near the beginning of this article.

2. Protect data in transit and at rest

  • Avoid storing production data in DBFS.

  • Secure access to cloud storage.

  • Use data exfiltration settings within the admin console.

  • Use bucket versioning.

  • Encrypt storage and restrict access.

  • Add a customer-managed key for managed services.

  • Add a customer-managed key for workspace storage.

Details are in the PDF referenced near the beginning of this article.

3. Secure your network, and identify and protect endpoints

  • Deploy with a customer-managed VPC or VNet.

  • Use IP access lists.

  • Implement network exfiltration protections.

  • Apply VPC service controls.

  • Use VPC endpoint policies.

  • Configure PrivateLink.

Details are in the PDF referenced near the beginning of this article.

4. Review the shared responsibility model

  • Review the shared responsibility model.

Details are in the PDF referenced near the beginning of this article.

5. Meet compliance and data privacy requirements

  • Review the Databricks compliance standards.

Details are in the PDF referenced near the beginning of this article.

6. Monitor system security

  • Use Databricks audit log delivery.

  • Configure tagging to monitor usage and enable charge-back.

  • Monitor workspace using Overwatch.

  • Monitor provisioning activities.

  • Use Enhanced Security Monitoring or Compliance Security Profile.

Details are in the PDF referenced near the beginning of this article.

Generic controls

  • Service quotas.

  • Controlling libraries.

  • Isolate sensitive workloads into different workspaces.

  • Use CI/CD processes to scan code for hard-coded secrets.

  • Use AWS Nitro instances.

Details are in the PDF referenced near the beginning of this article.