Cluster configuration for Databricks Connect

Note

This article covers Databricks Connect for Databricks Runtime 13.0 and above.

This article lists configuration requirements and settings for Databricks clusters before Databricks Connect can connect to them. This information applies to the Python and Scala version of Databricks Connect unless stated otherwise.

Databricks Connect enables you to connect popular IDEs such as PyCharm, notebook servers, and other custom applications to Databricks clusters. See What is Databricks Connect?.

Requirements

  • You have a Databricks workspace and its corresponding account that are enabled for Unity Catalog. See Set up and manage Unity Catalog and Enable a workspace for Unity Catalog.

  • For Python, a Databricks cluster with Databricks Runtime 13.0 or higher installed.

  • For Scala, a Databricks cluster with Databricks Runtime 13.3 LTS or above installed.

    Note

    Scala is not supported on Databricks Connect for Databricks Runtime 13.2 and below.

  • The cluster must use a cluster access mode of Assigned or Shared. See Access modes.

Disabling Databricks Connect

Databricks Connect (and the underlying Spark Connect) services can be disabled on any given cluster.

To disable the Databricks Connect service, set the following Spark configuration on the cluster.

spark.databricks.service.server.enabled false