Select a workspace directory for the Databricks extension for Visual Studio Code

This article describes how to select a Databricks workspace directory for the Databricks extension for Visual Studio Code to use. See What is the Databricks extension for Visual Studio Code?

Note

This article applies only to the Databricks extension for Visual Studio Code, version 1.

The Databricks extension for Visual Studio Code, version 2, currently in Private Preview, automatically determines the Databricks workspace directory to use based on settings in the project’s associated Databricks Asset Bundle. If you are using the Databricks extension for Visual Studio Code, version 2, skip ahead to Development tasks for the Databricks extension for Visual Studio Code.

The following information assumes that you have that you have already set up the Databricks extension for Visual Studio Code for authentication with your Databricks workspace. See Authentication setup for the Databricks extension for VS Code.

Note

If you have already completed the VSCode extension for Databricks tutorial: Run Python on a cluster and as a job, and you want to keep using the workspace directory from that tutorial, skip ahead to Development tasks for the Databricks extension for Visual Studio Code.

Note

The Databricks extension for Visual Studio Code works only with workspace directories that it creates. You cannot use an existing workspace directory in your workspace unless it was created by the extension.

To use workspace directories with the Databricks extension for Visual Studio Code, you must use version 0.3.5 or higher of the extension, and your Databricks cluster must have Databricks Runtime 11.3 LTS or higher installed.

Workspace directories is the default setting, beginning with version 0.3.15 of the extension. However, if you set the extension to use a Databricks Git folder instead of a workspace directory, you can set the extension back to using workspace directories as follows:

With the extension and your code project opened, and a Databricks configuration profile already set, in the Command Palette (View > Command Palette), type Preferences: Open User Settings, and then click Preferences: Open User Settings.
On the User tab, expand Extensions, and click Databricks.
For Sync: Destination Type, select workspace.
Quit and restart Visual Studio Code.

Note that after you change the setting from using a Databricks Git folder to using workspace directories, you might need to manually resync your code to your workspace. To do this, after you create a new workspace directory in the next procedure, click the arrowed circle (Start synchronization) icon next to Sync Destination.

With the extension and your code project opened, create a new Databricks workspace directory that you want to use, or select an existing workspace directory that you created earlier with the extension.

Create a new workspace directory

To create a new workspace directory, do the following:

In the Configuration pane, next to Sync Destination, click the gear (Configure sync destination) icon.
In the Command Palette, click Create New Sync Destination.
Type a name for the new workspace directory, and then press Enter.

The extension creates a directory with the specified directory name within /Users/<your-username>/.ide in the workspace and then adds the workspace directory’s path to the code project’s .databricks/project.json file, for example "workspacePath": "/Users/<your-username>/.ide/<your-directory-name>".

Note

If the remote workspace directory’s name does not match your local code project’s name, a warning icon appears with this message: The remote sync destination name does not match the current Visual Studio Code workspace name. You can ignore this warning if you do not require the names to match.
After you set the workspace directory, begin synchronizing with the workspace directory by clicking the arrowed circle (Start synchronization) icon next to Sync Destination.

Important

The Databricks extension for Visual Studio Code only performs one-way, automatic synchronization of file changes from your local Visual Studio Code project to the related workspace directory in your remote Databricks workspace. The files in this remote workspace directory are intended to be transient. Do not initiate changes to these files from within your remote workspace, as these changes will not be synchronized back to your local project.

Continue with Development tasks for the Databricks extension for Visual Studio Code.

Reuse an existing workspace directory

If you have an existing workspace directory that you created earlier with the Databricks extension for Visual Studio Code and want to reuse in your current Visual Studio Code project, then do the following:

In the Configuration pane, next to Sync Destination, click the gear (Configure sync destination) icon.
In the Command Palette, select the workspace directory’s name from the list.

The extension adds the workspace directory’s path to the code project’s .databricks/project.json file, for example "workspacePath": "/Users/<your-username>/.ide/<your-directory-name>".

Note

If the remote workspace directory’s name does not match your local code project’s name, a warning icon appears with this message: The remote sync destination name does not match the current Visual Studio Code workspace name. You can ignore this warning if you do not require the names to match.

After you set the workspace directory, begin synchronizing with the workspace directory by clicking the arrowed circle (Start synchronization) icon next to Sync Destination.

Important

The Databricks extension for Visual Studio Code only performs one-way, automatic synchronization of file changes from your local Visual Studio Code project to the related workspace directory in your remote Databricks workspace. The files in this remote workspace directory are intended to be transient. Do not initiate changes to these files from within your remote workspace, as these changes will not be synchronized back to your local project.

Continue with Development tasks for the Databricks extension for Visual Studio Code.