Databricks Connect release notes

This page lists releases and maintenance updates issued for Databricks Connect.

The Databricks Connect major and minor package version must always match your Databricks Runtime version. Databricks recommends that you always use the most recent patch version of Databricks Connect that matches your Databricks Runtime version. For example, when you use a Databricks Runtime 7.3 cluster, use the latest databricks-connect==7.3.* package.

Databricks Connect for Databricks Runtime 7.3 LTS

Databricks Connect 7.3.7

January 20, 2021

  • Databricks Connect client update to support Databricks Runtime 7.3 maintenance release of January 20, 2021

Databricks Connect 7.3.6

January 12, 2021

  • [ES-59054] Fix Delta Optimize Table command through Databricks Connect
  • [ES-53497] Fix Databricks Connect on Databricks Container Services (DCS)

Databricks Connect 7.3.5

December 9, 2020

Databricks Connect 7.3 is now GA.

  • [ES-49865] Prevent 500 Internal Server Errors due to automatic synchronization of dependent libraries when using Databricks Connect with SBT.

Databricks Connect 7.3.4 Beta

November 24, 2020

  • [ES-44382] Set spark.databricks.service.allowTerminatedClusterStart=false to prevent Databricks Connect from starting a terminated Databricks Runtime cluster.
  • Fix an error where client could not synchronize state with server correctly if INFO logging level was disabled.

Databricks Connect 7.3.3 Beta

November 3, 2020

Initial Databricks Connect release for Databricks Runtime 7.3. This release includes:

  • Support for Delta Lake time travel.
  • Easy transition between Databricks Connect clients and Databricks Runtime jobs or notebooks when using DBUtils. See Access DBUtils.

Known issues:

Databricks Runtime version validation is disabled in this beta release. Make sure to connect this client to a 7.3 Databricks Runtime cluster.

Databricks Connect for Databricks Runtime 7.1

Databricks Connect 7.1.13

January 20, 2021

  • Databricks Connect client update to support Databricks Runtime 7.1 maintenance release of January 20, 2021

Databricks Connect 7.1.12

January 12, 2021

  • [ES-59054] Fix Delta Optimize Table command through Databricks Connect
  • [ES-53497] Fix Databricks Connect on Databricks Container Services (DCS)

Databricks Connect 7.1.11

December 9, 2020

  • [ES-49865] Prevent 500 Internal Server Errors due to automatic synchronization of dependent libraries when using Databricks Connect with SBT.

Databricks Connect 7.1.10

November 24, 2020

  • Prevent status polls from restarting the cluster.
  • [ES-43656] Fix serialization error related to Azure Data Lake Storage Gen2 support.
  • [ES-30594] Allow larger HTTP headers and enlarge buffers in Databricks Connect.
  • [ES-34186] Avoid out of memory exception related to logging.
  • [ES-39493] Prevent recursive load error in databricks-connect test.
  • [ES-44382] Set spark.databricks.service.allowTerminatedClusterStart=false to prevent Databricks Connect from starting a Databricks Runtime cluster that is terminated.
  • Fix an error in which client could not synchronize state with server correctly if INFO logging level was disabled.
  • Minor fixes.

Databricks Connect 7.1.1

September 23, 2020

  • [ES-32536] Fix an issue with serialization of nested timezone-aware expressions.
  • [ES-33705] Support Azure Data Lake Storage Gen2 filesystem in DButils.

Databricks Connect 7.1.0

August 13, 2020

Initial Databricks Connect release for Databricks Runtime 7.1.

  • Support new Spark 3.0 DataSource V2 APIs.
  • Support using multiple SparkSessions in a single Databricks Connect application.
  • Optimize Databricks Connect state synchronization between client and server. Avoid excessive polling for status updates when the server is idle.

Databricks Connect for Databricks Runtime 6.4

Databricks Connect 6.4.22

January 12, 2021

  • [ES-59054] Fix Delta Optimize Table command through Databricks Connect
  • [ES-53497] Fix Databricks Connect on Databricks Container Services (DCS)

Databricks Connect 6.4.21

December 9, 2020

  • [ES-25890] Fix an error where in some cases DataFrame cache() would have no effect and the query would be executed again.
  • [ES-26196] Support using NATURAL and USING syntax in joins.
  • [ES-33475][ES-34186] Avoid out of memory exception related to logging.
  • [ES-39493] Prevent recursive load error in databricks-connect test.
  • [ES-43656] Fix serialization error related to Azure Data Lake Storage Gen2 support.
  • [ES-30594] Allow larger HTTP headers and enlarge buffers in Databricks Connect.
  • Minor fixes

Databricks Connect 6.4.2

November 5, 2020

  • [ES-47216] Fix version check compatibility issue with Databricks Runtime 6.4 November 3, 2020 maintenance update.

Databricks Connect 6.4.1

April 8, 2020

Minor fixes and updates.

Databricks Connect 6.4.0

March 7, 2020

Initial Databricks Connect release for Databricks Runtime 6.4.