Select a cluster for the Databricks extension for Visual Studio Code

This article describes how to select a Databricks cluster for the Databricks extension for Visual Studio Code to use. See What is the Databricks extension for Visual Studio Code?

The following information assumes that you have already set up the Databricks extension for Visual Studio Code for authentication with your Databricks workspace. See Authentication setup for the Databricks extension for Visual Studio Code.

With the extension and your code project opened, select an existing Databricks cluster that you want to use, or create a new Databricks cluster and use it.

Use an existing cluster

If you have an existing Databricks cluster that you want to use, do one of the following:

• In the Clusters pane, do the following:

  Note

  The Clusters pane appears only if the views.cluster experimental feature is enabled. See Settings for the Databricks extension for Visual Studio Code.

  1. Next to the cluster that you want to use, click the plug (Attach cluster) icon.

     

     Tip

     If the cluster is not visible in the Clusters pane, click the filter (Filter clusters) icon to see All clusters, clusters that are Created by me, or Running clusters. Or, click the arrowed circle (Refresh) icon next to the filter icon.

     

  The extension adds the cluster’s ID to your code project’s .databricks/project.json file, for example "clusterId": "1234-567890-abcd12e3".

  This procedure is complete. Continue with Select a workspace directory for the Databricks extension for Visual Studio Code.

• In the Configuration pane, do the following:

  1. Next to Cluster, click the gear (Configure cluster) icon.

     

  2. In the Command Palette, click the cluster that you want to use.

  The extension adds the cluster’s ID to your code project’s .databricks/project.json file, for example "clusterId": "1234-567890-abcd12e3".

  Continue with Select a workspace directory for the Databricks extension for Visual Studio Code.

Create a new cluster

If you do not have an existing Databricks cluster, or you want to create a new one and use it, do the following:

1. In the Configuration pane, next to Cluster, click the gear (Configure cluster) icon.

   

2. In the Command Palette, click Create New Cluster.

3. When prompted to open the external website (your Databricks workspace), click Open.

4. If prompted, sign in to your Databricks workspace.

5. Follow the instructions to create a cluster.

   Note

   Databricks recommends that you create a Personal Compute cluster. This enables you to start running workloads immediately, minimizing compute management overhead.

6. After the cluster is created and is running, go back to Visual Studio Code.

7. Do one of the following:

   • In the Clusters pane, next to the cluster that you want to use, click the plug (Attach cluster) icon.

     Note

     The Clusters pane appears only if the views.cluster experimental feature is enabled. See Settings for the Databricks extension for Visual Studio Code.

     

     Tip

     If the cluster is not visible, click the filter (Filter clusters) icon to see All clusters, clusters that are Created by me, or Running clusters. Or, click the arrowed circle (Refresh) icon.

     

     The extension adds the cluster’s ID to the code project’s .databricks/project.json file, for example "clusterId": "1234-567890-abcd12e3".

     This procedure is complete. Continue with Select a workspace directory for the Databricks extension for Visual Studio Code.

   • In the Configuration pane, next to Cluster, click the gear (Configure cluster) icon.

     

     In the Command Palette, click the cluster that you want to use.

     The extension adds the cluster’s ID to the code project’s .databricks/project.json file, for example "clusterId": "1234-567890-abcd12e3".

     Continue with Select a workspace directory for the Databricks extension for Visual Studio Code.