Skip to main content
Last updated on

Manage Databricks Git folders using Terraform

Manage Databricks Git folders in a fully automated environment using Terraform and the Databricks Terraform provider.

Authentication with personal access tokens​

This approach uses Git personal access tokens for repository authentication with a Databricks service principal.

Terraform evaluates provider configurations before creating any resources, so you can't reference a resource value (such as an on-behalf-of (OBO) token) in a provider block within the same Terraform configuration. To work around this limitation, split the setup into two separate configurations:

  • Part 1: Create the service principal: Creates the Databricks service principal and generates an OBO token.
  • Part 2: Configure Git credentials: Uses the OBO token to authenticate as the Databricks service principal, then configures Git credentials and creates the Git folder.

Part 1: Create the Databricks service principal​

Create a setup/ directory with a main.tf file.

Step 1: Declare the provider and variables​

terraform {
  required_providers {
    databricks = {
      source = "databricks/databricks"
    }
  }
}

variable "databricks_host" {}

variable "databricks_admin_token" {
  sensitive = true
}

variable "service_principal_name" {}

provider "databricks" {
  host  = var.databricks_host
  token = var.databricks_admin_token
}

Step 2: Create the Databricks service principal​

Choose a display name for the Databricks service principal. Terraform creates it using this name.

resource "databricks_service_principal" "sp" {
  display_name = var.service_principal_name
}

Step 3: Create and export the authorization token​

Generate an OBO token for the Databricks service principal and export it for use in the next configuration.

resource "databricks_obo_token" "this" {
  application_id   = databricks_service_principal.sp.application_id
  comment          = "PAT on behalf of ${databricks_service_principal.sp.display_name}"
  lifetime_seconds = 3600
}

output "obo_token_value" {
  value     = databricks_obo_token.this.token_value
  sensitive = true
}

Step 4: Apply the configuration​

From the setup/ directory, initialize, apply, and retrieve the token:

Bash
terraform init
terraform apply
terraform output -raw obo_token_value

Step 5: Prepare variables for Part 2​

Create a terraform.tfvars file in the git-credentials/ directory with the OBO token and other required values:

databricks_host           = "https://<your-workspace>.cloud.databricks.com"
obo_token_value           = "<token from previous step>"
git_username              = "<your-git-username>"
git_provider              = "<gitHub|gitLab|azureDevOpsServices|...>"
git_personal_access_token = "<your-git-PAT>"
repo_url                  = "https://github.com/<your-org>/<your-repo>.git"
note

Add terraform.tfvars to your .gitignore file to avoid committing sensitive values to version control. Terraform automatically reads this file when you run terraform apply.

Part 2: Configure Git credentials​

Create a separate git-credentials/ directory with its own main.tf file.

Step 1: Declare the provider and variables​

Pass the OBO token from the setup/ configuration output as obo_token_value to authenticate as the Databricks service principal.

terraform {
  required_providers {
    databricks = {
      source = "databricks/databricks"
    }
  }
}

variable "databricks_host" {}

variable "obo_token_value" {
  sensitive = true
}

variable "git_username" {}
variable "git_provider" {}

variable "git_personal_access_token" {
  sensitive = true
}

variable "repo_url" {}

provider "databricks" {
  alias = "sp"
  host  = var.databricks_host
  token = var.obo_token_value
}

Step 2: Configure Git credentials​

Set the Git credentials that the Databricks service principal will use to access your Git repository.

resource "databricks_git_credential" "sp" {
  provider              = databricks.sp
  git_username          = var.git_username
  git_provider          = var.git_provider
  personal_access_token = var.git_personal_access_token
}

Step 3: Create the Git folder​

resource "databricks_repo" "this" {
  provider   = databricks.sp
  url        = var.repo_url
  depends_on = [databricks_git_credential.sp]
}

Step 4: Apply the Git credentials configuration​

From the git-credentials/ directory, initialize and apply:

Bash
terraform init
terraform apply
On this page