GitHub Version Control

This topic describes how to set up version control for notebooks using GitHub through the UI. Although this document describes how to set up GitHub integration through the UI, you can also integrate through the Databricks CLI or Workspace API.

Configure version control

Configuring version control involves creating access credentials in your version control provider and adding those credentials to Databricks.

Get an access token

Go to GitHub and create a personal access token that allows access to your repositories:

  1. From GitHub, access the menu on the upper right, next to your Gravitar, and select Settings.

  2. Go to the Personal access tokens tab.

  3. Click the Generate New Token button to create a new token.

  4. Select the repo and public_repo permissions, and click the Generate Token button.

    github-newtoken-settings

  5. Copy the token to your clipboard. You enter this token in Databricks in the next step.

See the GitHub documentation to learn more about how to create personal access tokens.

Save your access token to Databricks

  1. Click the User icon Account Icon at the top right of your screen and select User Settings.

    AccountSettings
  2. Click the Git Integration tab.

  3. If you have previously entered credentials, click the Change token or app password button.

  4. In the Git provider drop-down, select GitHub.

    github-provider-credentials

  5. Paste your token into the Token or app password field and click Save.

Work with notebook revisions

You work with notebook revisions in the History panel. Open the history panel by clicking Revision history at the top right of the notebook.

../../_images/revision-history-open.png

Note

You cannot modify a notebook while the History panel is open.

Save a notebook to GitHub

While the changes that you make to your notebook are saved automatically to the Databricks revision history, changes do not automatically persist to GitHub.

  1. Open the History panel.

    SaveNow
  2. Click Save Now to save your notebook to GitHub. The Save Notebook Revision dialog displays.

  3. Optionally, enter a message to describe your change.

  4. Make sure that Also commit to Git is selected.

    ../../_images/save-revision.png
  5. Click Save.

Revert or update a notebook to a version from GitHub

Once you link a notebook, Databricks syncs your history with Git every time you re-open the History panel. Versions that sync to Git have commit hashes as part of the entry.

  1. Open the History panel.

    github-sync
  2. Choose an entry in the History panel. Databricks displays that version.

  3. Click Restore this version.

  4. Click Confirm to confirm that you want to restore that version.

Branch support

You can work on any branch of your repository and create new branches inside Databricks.

Create a branch

  1. Open History panel.

  2. Click the Git status bar to open the GitHub panel.

  3. Click the Branch dropdown.

  4. Enter a branch name.

    ../../_images/github-branch-create.png
  5. Select the Create Branch option at the bottom of the dropdown. The parent branch is indicated. You always branch from your current selected branch.

Create a pull request

  1. Open History panel.

  2. Click the Git status bar to open the GitHub panel.

    ../../_images/git-create-pr.png
  3. Click Create PR. GitHub opens to a pull request page for the branch.

Rebase a branch

You can also rebase your branch inside Databricks. The Rebase link displays if new commits are available in the parent branch. Only rebasing on top of the default branch of the parent repository is supported.

../../_images/github-rebase.png

For example, assume that you are working on databricks/reference-apps. You fork it into your own account (for example, brkyvz) and start working on a branch called my-branch. If a new update is pushed to databricks:master, then the Rebase button displays, and you will be able to pull the changes into your branch brkyvz:my-branch.

Rebasing works a little differently in Databricks. Assume the following branch structure:

../../_images/rebase-before.png

After a rebase, the branch structure will look like:

../../_images/rebase-after.png

What’s different here is that Commits C5 and C6 will not apply on top of C4. They will appear as local changes in your notebook. Any merge conflict will show up as follows:

../../_images/merge-conflict.png

You can then commit to GitHub once again using the Save Now button.

What happens if someone branched off from my branch that I just rebased?
If your branch (for example, branch-a) was the base for another branch (branch-b), and you rebase, you need not worry! Once a user also rebases branch-b, everything will work out. The best practice in this situation is to use separate branches for separate notebooks.

Best practices for code reviews

Databricks supports Git branching.

  • You can link a notebook to your own fork and choose a branch.
  • We recommend using separate branches for each notebook.
  • Once you are happy with your changes, you can use the Create PR link in the Git Preferences dialog to take you to GitHub’s pull request page.
  • The Create PR link displays only if you’re not working on the default branch of the parent repository.

GitHub Enterprise

Important

Integration with GitHub Enterprise is not supported. However, you can use the Workspace API to programmatically create notebooks and manage the code base in GitHub Enterprise.

Troubleshooting

If you receive errors related to syncing GitHub history, verify the following:

  1. You have initialized the repository on GitHub, and it isn’t empty. Try the URL that you entered and verify that it forwards to your GitHub repository.
  2. Your personal access token is active.
  3. If the repository is private, you must have at least read level permissions (through GitHub) on the repository.