Skip to main content

Set up your Databricks infrastructure

Set up the foundation that every team in your organization builds on: workspaces, single sign-on (SSO), users, groups, metastore administration, and cost monitoring. Complete these tasks before anyone loads data into Databricks, because workspace layout, identity, and metastore administration are difficult to change after teams start to depend on them.

After infrastructure setup, plan how to organize your data in Unity Catalog. See Plan your data governance strategy.

Infrastructure setup tasks​

Each task builds on the one before it. Complete the tasks in the following order:

Task

Outcome

Create development, staging, and production workspaces

Separate development, staging, and production workspaces, deployed in the UI or with Terraform

Configure single sign-on for your Databricks account

SSO, so that users authenticate to Databricks through your identity provider

Add users to your Databricks account

Users in your Databricks account, added manually or synced from your identity provider

Create groups for access

Account groups with members, assigned to workspaces or synced from your identity provider

Assign a group as the metastore admin

A group, rather than an individual user, as the metastore admin

Monitor and control your Databricks costs

Usage dashboards, tags on compute and jobs, and budgets, so that you can see and attribute spend from the first day

Task

Outcome

Create development, staging, and production workspaces

Separate development, staging, and production workspaces, deployed in the UI or with Terraform

Configure single sign-on for your Databricks account

SSO, so that users authenticate to Databricks through your identity provider

Add users to your Databricks account

Users in your Databricks account, added manually or synced from your identity provider

Create groups for access

Account groups with members, assigned to workspaces or synced from your identity provider

Assign a group as the metastore admin

A group, rather than an individual user, as the metastore admin

Monitor and control your Databricks costs

Usage dashboards, tags on compute and jobs, and budgets, so that you can see and attribute spend from the first day

Next steps​