Databricks Runtime 16.4 LTS

The following release notes provide information about Databricks Runtime 16.4 LTS, powered by Apache Spark 3.5.2.

Databricks released this LTS version in May 2025. There are 2 variants for this release, one supporting Scala 2.12 and one supporting Scala 2.13.

Starting with DBR 17 (Spark 4), only Scala 2.13 will be supported. To help you transition, two images are available in 16.4 LTS: one with Scala 2.12 and one with Scala 2.13. Use the 2.13 image to test and update your code base to Scala 2.13 before migrating to DBR 17.

Scala 2.13 migration guidance

All other features are the same in both Databricks Runtime 16.4 LTS releases.
Dependency versions may differ between the Databricks Runtime 16.4 Scala 2.12 image and the 16.4 Scala 2.13 image. The versions of the libraries supported in each image are listed below on this page.
Ensure your code is written in Scala 2.13 and your executable is compiled with Scala 2.13.
Build your project against Spark 3.5.

If your code uses the sparklyr R library, you must use the image that supports Scala 2.12 instead.

Breaking changes between Scala 2.12 and 2.13

Major breaking changes

Databricks Runtime considers a breaking change major when it requires you to make significant code changes to support it.

Collection incompatibility: Read this official Scala docs page for details on migrating collections to Scala 2.13. If your code uses an earlier version of Scala code, collections will be primary source of incompatibilities when using Databricks, particularly with API parameters and return types.
Hash algorithm: When reviewing your code built with Scala 2.12, do not rely on the implicit order of data structures that do not guarantee ordering, such as HashMap and Set. Collections that do implicit ordering may order their elements differently when run under 2.13 (vs. 2.12) when you iterate through them.

Minor breaking changes

Databricks Runtime considers a breaking change minor when there are no specific error messages raised due to version changes, but your code would generally fail to compile under the version. In this case, the error messages may provide sufficient information for updating your code.

Stricter type inference: When compiling code developed for Scala 2.12 with 2.13, the compiler may report that it cannot infer the type or hint at type ambiguity. Explicitly annotating the variables with their data type often solves this problem.
Removal of specific syntax: Some syntax has been deprecated in Scala 2.13, such as the single-quote string literal, the + operator for string concatenation when used with a non-String type on the left, and the postfix operator (use dot notation instead). For more details, see: Scala dropped features in the official Scala docs.
Output messages: Text results for messages returned from REPL can be different between Scala 2.12 and 2.13. Any discrepancy doesn't mean the syntax or computation is wrong; rather, the message describes the issue or context differently or presents it in a different representational format. For example, f: Foo = Foo(1) may become f: Foo = Foo(i = 1) in some messages.

The list of library versions supported by each Scala version of Databricks Runtime 16.4 LTS:

DBR 16.4 LTS Scala 2.12 libraries
DBR 16.4 LTS Scala 2.13 libraries

DBR 16.4 LTS new and updated features

Liquid clustering auto-compaction improvement
Auto Loader can now clean processed files in the source directory
Type widening support added for streaming from Delta tables
IDENTIFIER support now available in DBSQL for catalog operations
Collated expressions now provide autogenerated transient aliases
Add filter pushdown API support to Python data sources
Python UDF traceback improvement
UNION/EXCEPT/INTERSECT inside a view and EXECUTE IMMEDIATE now return correct results
Data source cached plan conf and migration guide
Add flag to hide recursive Common Table Expressions (CTEs)
New listagg and string_agg functions

Liquid clustering auto-compaction improvement

Unity Catalog-managed liquid clustering tables now trigger auto-compaction to automatically reduce small file problems between OPTIMIZE runs.

For more details, see Auto compaction for Delta Lake on Databricks.

Auto Loader can now clean processed files in the source directory

Customers can now instruct Auto Loader to automatically move or delete files that have been processed. Opt in to this feature by using the cloudFiles.cleanSource Auto Loader option.

For more details, see Auto Loader options under cloudFiles.cleanSource.

Type widening support added for streaming from Delta tables

This release adds support for streaming from a Delta table that has type-widened column data, and for sharing a Delta table with type widening enabled using Databricks-to-Databricks Delta Sharing. The type widening is feature is currently in Public Preview.

For more details, see Type widening.

IDENTIFIER support now available in DBSQL for catalog operations

Databricks customers can now use the IDENTIFIER clause when performing the following catalog operations:

CREATE CATALOG
DROP CATALOG
COMMENT ON CATALOG
ALTER CATALOG

This new syntax allows customers to dynamically specify catalog names using parameters defined for these operations, enabling more flexible and reusable SQL workflows. As an example of the syntax, consider CREATE CATALOG IDENTIFIER(:param) where param is a parameter provided to specify a catalog name.

For more details, see IDENTIFIER clause.

Collated expressions now provide autogenerated transient aliases

Autogenerated aliases for collated expressions will now always deterministically incorporate COLLATE information. Autogenerated aliases are transient (unstable) and should not be relied on. Instead, as a best practice, use expression AS alias consistently and explicitly.

Add filter pushdown API support to Python data sources

In Databricks Runtime 16.4 LTS, Databricks added support for filter pushdown to Python data source batch read as an a API similar to SupportsPushDownFilters interface. You can now implement DataSourceReader.pushFilters to receive filters that may be pushed down. You can also implement this API to provide logic to select filters to push down, to track them, and to return the remaining filters for application by Apache Spark.

Filter pushdown allows the data source to handle a subset of filters. This can improve performance by reducing the amount of data that needs to be processed by Spark.

Filter pushdown is only supported for batch reads, not for streaming reads. The new API must be added to DataSourceReader and not to DataSource or DataSourceStreamReader. The list of filters must be interpreted as the logical AND of the elements in your implementation. This method is called once during query planning. In the default implementation, it returns all filters which indicates that no filters can be pushed down. Use subclasses to override this method and implement your logic for filter pushdown.

Initially and to keep the API simple, Databricks only supports V1 filters that have a column, a boolean operator, and a literal value. The filter serialization is a placeholder and will be implemented in a future PR. For example:

Python
class DataSourceReader(ABC):

  ...
  def pushFilters(self, filters: List["Filter"]) -> Iterable["Filter"]:

Databricks recommends you implement this method only for data sources that natively support filtering, such as databases and GraphQL APIs.

Python UDF traceback improvement

The Python UDF traceback now includes frames from both the driver and executor along with client frames, resulting in better error messages that show greater and more relevant details (such as the line content of frames inside a UDF).

UNION/EXCEPT/INTERSECT inside a view and EXECUTE IMMEDIATE now return correct results

Queries for temporary and persistent view definitions with top-level UNION/EXCEPT/INTERSECT and un-aliased columns previously returned incorrect results because UNION/EXCEPT/INTERSECT keywords were considered aliases. Now those queries will correctly perform the whole set operation.

EXECUTE IMMEDIATE ... INTO with a top-level UNION/EXCEPT/INTERSECT and un-aliased columns also wrote an incorrect result of a set operation into the specified variable due to the parser interpreting these keywords as aliases. Similarly, SQL queries with invalid tail text were also allowed. Set operations in these cases now write a correct result into the specified variable, or fail in case of invalid SQL text.

Data source cached plan conf and migration guide

Since DBR 16.4.0, reading from a file source table will correctly respect query options, e.g. delimiters. Previously, the first query plan was cached and subsequent option changes ignored. To restore the previous behavior, set spark.sql.legacy.readFileSourceTableCacheIgnoreOptions to true.

Add flag to hide recursive Common Table Expressions (CTEs)

Recursive CTE execution is now hidden behind spark.sql.cte.recursion.enabled.

New `listagg` and `string_agg` functions

Starting with this release you can use the listagg or string_agg functions to aggregate STRING and BINARY values within a group. See string_agg for more details.

DBR 16.4 LTS behavioral changes

Delta tables: "DESCRIBE DETAIL table" will now show the clusterByAuto status of the table
Fix to respect options for data source cached plans
Moved redaction rule from analyzer to optimizer
variant_get and get_json_object now consider leading spaces in paths in Apache Spark
Enable flag to disallow disabling source materialization for MERGE operations
Move partition metadata log enablement anchor to table
Update snowflake-jdbc 3.16.1 -> 3.22.0
Downgrade Json4s from 4.0.7 to 3.7.0-M11 in DBR 16.2 and 16.3

Delta tables: "DESCRIBE DETAIL table" will now show the clusterByAuto status of the table

DESCRIBE DETAIL {table} will now show the clusterByAuto status of the table (true or false) next to the current clustering columns. For more details on clusterByAuto, see:Automatic liquid clustering.

Fix to respect options for data source cached plans

This update ensures table reads respect options set for all data source plans when cached, not just the first cached table read.

Previously, data source table reads cached the first plan but failed to account for different options in subsequent queries.

For example, the following query:

spark.sql("CREATE TABLE t(a string, b string) USING CSV".stripMargin)
spark.sql("INSERT INTO TABLE t VALUES ('a;b', 'c')")

spark.sql("SELECT * FROM t").show()
spark.sql("SELECT * FROM t WITH ('delimiter' = ';')")

would produce this output:

+----+----+
|col1|col2|
+----+----+
| a;b| c  |
+----+----+

+----+----+
|col1|col2|
+----+----+
| a;b| c  |
+----+----+

With this fix, it now returns the expected output:

+----+----+
|col1|col2|
+----+----+
| a;b| c  |
+----+----+

+----+----+
|col1|col2|
+----+----+
| a  | b,c|
+----+----+

If your workloads have dependencies on the previous incorrect behavior, you may see different results after this change.

Moved redaction rule from analyzer to optimizer

Previously, DataFrames could create tables that contained redacted values when valid SECRET SQL functions were used. This change removes redaction when saving DataFrames with valid secret access to a table, and the redaction rule has moved from the analyzer to the optimizer.

variant_get and get_json_object now consider leading spaces in paths in Apache Spark

Prior to this change, leading whitespaces and tabs in paths in the variant_get and get_json_object expressions were being ignored with Photon disabled. For example, select get_json_object('{" key": "value"}', '$[' key']') would not be effective in extracting the value of " key". However, users will be able to extract such keys.

Enable flag to disallow disabling source materialization for MERGE operations

Previously, users could disable source materialization in MERGE by setting merge.materializeSource to none. With the new flag enabled, this will be forbidden and cause an error. Databricks plans to enable the flag only for customers who haven't used this configuration flag before, so no customer should notice any change in behavior.

Move partition metadata log enablement anchor to table

The partition metadata log feature has been changed so that once a table is created with spark.databricks.nonDelta.partitionLog.enabled = true, you can anchor it to a table so a cluster doesn't set spark.databricks.nonDelta.partitionLog.enabled = true for all tables processed by the cluster.

Update snowflake-jdbc 3.16.1 -> 3.22.0

Updated the dependency snowflake-jdbc from 3.16.1 to 3.22.0. This may impact users if they directly use the l3.16.1 version of the library.

Downgrade Json4s from 4.0.7 to 3.7.0-M11 in DBR 16.2 and 16.3

Customers cannot use databricks-connect 16.1+ and Apache Spark™ 3.5.x together in the same application because of significant discrepancies in API behavior between Json4s version 3.7.0-M11 and version 4.0.7. To address this, Databricks has downgraded Json4s to 3.7.0-M11.

Library upgrades (applies to the Scala 2.12 image only)

Upgraded R libraries:
- sparklyr from 1.8.4 to 1.8.6 (only applies to the Scala 2.12 image for the Databricks Runtime 16.4 LTS release)
Upgraded Java libraries:
- io.delta.delta-sharing-client_2.12 from 1.2.3 to 1.2.4
- org.json4s.json4s-ast_2.12 from 4.0.7 to 3.7.0-M11
- org.json4s.json4s-core_2.12 from 4.0.7 to 3.7.0-M11
- org.json4s.json4s-jackson_2.12 from 4.0.7 to 3.7.0-M11
- org.json4s.json4s-scalap_2.12 from 4.0.7 to 3.7.0-M11

The Scala 2.13 Databricks Runtime release is considered a "new" version and may have different library versions from 2.12. Refer to the table below for specific library versions in that release image. It does not include sparklyr in this release.

Apache Spark

Databricks Runtime 16.4 LTS includes Apache Spark 3.5.2. This release includes all Spark fixes and improvements included in Databricks Runtime 16.3, as well as the following additional bug fixes and improvements made to Spark:

[SPARK-51206] [PYTHON][connect] Move Arrow conversion helpers out of Spark Connect
[SPARK-51566] [PYTHON] Python UDF traceback improvement
[SPARK-51675] [SS] Fix col family creation after opening local DB to avoid snapshot creation, if not necessary
[SPARK-50892] [SQL]Add UnionLoopExec, physical operator for recursion, to perform execution of recursive queries
[SPARK-51670] [SQL] Refactor Intersect and Except to follow Union example to reuse in single-pass Analyzer
[SPARK-47895] [BEHAVE-266][sql] group by alias should be idempotent
[SPARK-51614] [SQL] Introduce ResolveUnresolvedHaving rule in the Analyzer
[SPARK-51662] [SQL] Make OrcFileFormat comparable
[SPARK-51374] Revert "[CORE] Switch to Using java.util.Map in Logging APIs"
[SPARK-51652] [SQL] Refactor SetOperation computation out to reuse it in the single-pass Analyzer
[SPARK-51622] [UI] Titling sections on ExecutionPage
[SPARK-51586] [SS] initialize input partitions independent of columnar support in continuous mode
[SPARK-51070] [SQL] Use scala.collection.Set instead of Set in ValidateExternalType
[SPARK-51559] [SQL] Make max broadcast table size configurable
[SPARK-51496] [SQL] CaseInsensitiveStringMap comparison should ignore case
[SPARK-50416] [CORE] A more portable terminal / pipe test needed for bin/load-spark-env.sh
[SPARK-51509] [CORE][ui] Make Spark Master Environment page support filters
[SPARK-51374] [CORE] Switch to Using java.util.Map in Logging APIs
[SPARK-50820] [SQL] DSv2: Conditional nullification of metadata columns in DML
[SPARK-48922] [SQL] Avoid redundant array transform of identical expression for map type
[SPARK-51446] [SQL] Improve the codecNameMap for the compression codec
[SPARK-51341] [CORE] Cancel time task with suitable way.
[SPARK-51452] [UI] Improve Thread dump table search
[SPARK-51444] [CORE] Remove the unreachable if branch from TaskSchedulerImpl#statusUpdate
[SPARK-51401] [SQL] Change ExplainUtils.generateFieldString to directly call QueryPlan.generateFieldString
[SPARK-49507] [SQL] Fix the case issue after enabling metastorePartitionPruningFastFallback
[SPARK-51506] [PYTHON][ss] Do not enforce users to implement close() in TransformWithStateInPandas
[SPARK-51624] [SQL] Propagate GetStructField metadata in CreateNamedStruct.dataType
[SPARK-51201] [SQL] Make Partitioning Hints support byte and short values
[SPARK-51186] [PYTHON] Add StreamingPythonRunnerInitializationException to PySpark base exception
[SPARK-50286] [SQL] Correctly propagate SQL options to WriteBuilder
[SPARK-51616] [SQL] Run CollationTypeCasts before ResolveAliases and ResolveAggregateFunctions
[SPARK-51023] [CORE] log remote address on RPC exception
[SPARK-51087] [PYTHON][connect] Raise a warning when memory-profiler is not installed for memory profiling
[SPARK-51073] [SQL] Remove Unstable from SparkSessionExtensionsProvider trait
[SPARK-51471] [SS] RatePerMicroBatchStream - classify the ASSERT error when offset/timestamp in startOffset is larger than the endOffset
[SPARK-51062] [PYTHON] Fix assertSchemaEqual to compare decimal precision and scale
[SPARK-51612] [SQL] Display Spark confs set at view creation in Desc As Json
[SPARK-51573] [SS] Fix Streaming State Checkpoint v2 checkpointInfo race condition
[SPARK-51252] [SS] Add instance metrics for last uploaded snapshot version in HDFS State Stores
[SPARK-51552] [SQL] Disallow temporary variables in persisted views when under identifier
[SPARK-51625] [SQL] Command in CTE relations should trigger inline
[SPARK-51584] [SQL] Add rule that pushes Project through Offset and Suite that tests it
[SPARK-51593] [SQL] Refactor QueryExecutionMetering instantiation
[SPARK-49082] [SQL] Support widening Date to TimestampNTZ in Avro reader
[SPARK-50220] [PYTHON] Support listagg in PySpark
[SPARK-51581] [CORE][sql] Use nonEmpty/isEmpty for empty check for explicit Iterable
[SPARK-42746] [SQL] Fix in error class classification for new exceptions with listagg
[SPARK-51580] [SQL] Throw proper user facing error message when lambda function is out of place in HigherOrderFunction
[SPARK-51589] [SQL] Fix small bug failing to check for aggregate functions in |> SELECT
[SPARK-50821] [PYTHON] Upgrade Py4J from 0.10.9.8 to 0.10.9.9
[SPARK-42746] [SQL] Implement LISTAGG function
[SPARK-48399] [SQL] Teradata: ByteType should map to BYTEINT instead of BYTE(binary)
[SPARK-43221] [CORE] Host local block fetching should use a block status of a block stored on disk
[SPARK-51467] [UI] Make tables of the environment page filterable
[SPARK-51569] [SQL] Don't reinstantiate InSubquery in InTypeCoercion if there are no type changes
[SPARK-51280] [CONNECT] Improve RESPONSE_ALREADY_RECEIVED error class
[SPARK-51525] [SQL] Collation field for Desc As JSON StringType
[SPARK-51565] [SQL] Support SQL parameters in window frame clause
[SPARK-51544] [SQL] Add only unique and necessary metadata columns
[SPARK-49349] [SQL] Improve error message for LCA with Generate
[SPARK-51418] [SQL] Fix DataSource PARTITON TABLE w/ Hive type incompatible partition columns
[SPARK-51269] [SQL] Simplify AvroCompressionCodec by removing defaultCompressionLevel
[SPARK-50618] [SS][sql] Make DataFrameReader and DataStreamReader leverage the analyzer more
[SPARK-51097] [SS] Re-introduce RocksDB state store's last uploaded snapshot version instance metrics
[SPARK-51321] [SQL] Add rpad and lpad support for PostgresDialect and MsSQLServerDialect expression pushdown
[SPARK-51428] [BEHAVE-264][sql] Reassign Aliases for collated expression trees deterministically
[SPARK-51528] [SQL] Don't compare hash codes directly in ProtobufCatalystDataConversionSuite
[SPARK-51443] [SS] Fix singleVariantColumn in DSv2 and readStream.
[SPARK-51397] [SS] Fix maintenance pool shutdown handling issue causing long test times
[SPARK-51522] [SQL] Disable lazy union children test for single-pass Analyzer
[SPARK-51079] [Fix-forward] Support large variable types in pandas UDF, createDataFrame and toPandas with Arrow
[SPARK-51112] [ES-1348599][connect] Avoid using pyarrow's to_pandas on an empty table
[SPARK-51438] [SQL] Make CatalystDataToProtobuf and ProtobufDataToCatalyst properly comparable and hashable
[SPARK-50855] [DBR16.x][ss][CONNECT] Spark Connect Support for TransformWithState In Scala
[SPARK-51208] [SQL] ColumnDefinition.toV1Column should preserve EXISTS_DEFAULT resolution
[SPARK-51468] [SQL] Revert "From json/xml should not change collations in the given schema"
[SPARK-51453] [SQL] AssertTrue uses toPrettySQL instead of simpleString
[SPARK-51409] [SS] Add error classification in the changelog writer creation path
[SPARK-51440] [SS] classify the NPE when null topic field value is in kafka message data and there is no topic option
[SPARK-51425] [Connect] Add client API to set custom operation_id
[SPARK-50652] [SS] Add checks to RocksDB V2 backward compatibility
[SPARK-49164] [SQL] Fix not NullSafeEqual in predicate of SQL query in JDBC Relation
[SPARK-50880] [SQL] Add a new visitBinaryComparison method to V2ExpressionSQLBuilder
[SPARK-50838] [SQL]Performs additional checks inside recursive CTEs to throw an error if forbidden case is encountered
[SPARK-51307] [SQL] locationUri in CatalogStorageFormat shall be decoded for display
[SPARK-51079] [PYTHON] Support large variable types in pandas UDF, createDataFrame and toPandas with Arrow
[SPARK-50792] [SQL] Format binary data as a binary literal in JDBC.

Databricks ODBC/JDBC driver support

Databricks supports ODBC/JDBC drivers released in the past 2 years. Download the recently released drivers here:

System environment

Operating System: Ubuntu 24.04.2 LTS.
- Note: This is the Ubuntu version used by the Databricks Runtime containers. The Databricks Runtime containers run on the cloud provider's virtual machines, which might use a different Ubuntu version or Linux distribution.
Java: Zulu17.54+21-CA
Scala: 2.12.15 or 2.13.10
Python: 3.12.3
R: 4.4.0
Delta Lake: 3.3.1

Installed Python libraries

Library	Version	Library	Version	Library	Version
annotated-types	0.7.0	asttokens	2.0.5	astunparse	1.6.3
autocommand	2.2.2	azure-core	1.31.0	azure-storage-blob	12.23.0
azure-storage-file-datalake	12.17.0	backports.tarfile	1.2.0	black	24.4.2
blinker	1.7.0	boto3	1.34.69	botocore	1.34.69
cachetools	5.3.3	certifi	2024.6.2	cffi	1.16.0
chardet	4.0.0	charset-normalizer	2.0.4	click	8.1.7
cloudpickle	2.2.1	comm	0.2.1	contourpy	1.2.0
cryptography	42.0.5	cycler	0.11.0	Cython	3.0.11
databricks-sdk	0.30.0	dbus-python	1.3.2	debugpy	1.6.7
decorator	5.1.1	Deprecated	1.2.14	distlib	0.3.8
docstring-to-markdown	0.11	executing	0.8.3	facets-overview	1.1.1
filelock	3.15.4	fonttools	4.51.0	gitdb	4.0.11
GitPython	3.1.37	google-api-core	2.20.0	google-auth	2.35.0
google-cloud-core	2.4.1	google-cloud-storage	2.18.2	google-crc32c	1.6.0
google-resumable-media	2.7.2	googleapis-common-protos	1.65.0	grpcio	1.60.0
grpcio-status	1.60.0	httplib2	0.20.4	idna	3.7
importlib-metadata	6.0.0	importlib_resources	6.4.0	inflect	7.3.1
ipyflow-core	0.0.201	ipykernel	6.28.0	ipython	8.25.0
ipython-genutils	0.2.0	ipywidgets	7.7.2	isodate	0.6.1
jaraco.context	5.3.0	jaraco.functools	4.0.1	jaraco.text	3.12.1
jedi	0.19.1	jmespath	1.0.1	joblib	1.4.2
jupyter_client	8.6.0	jupyter_core	5.7.2	kiwisolver	1.4.4
launchpadlib	1.11.0	lazr.restfulclient	0.14.6	lazr.uri	1.0.6
matplotlib	3.8.4	matplotlib-inline	0.1.6	mccabe	0.7.0
mlflow-skinny	2.19.0	more-itertools	10.3.0	mypy	1.10.0
mypy-extensions	1.0.0	nest-asyncio	1.6.0	nodeenv	1.9.1
numpy	1.26.4	oauthlib	3.2.2	opentelemetry-api	1.27.0
opentelemetry-sdk	1.27.0	opentelemetry-semantic-conventions	0.48b0	packaging	24.1
pandas	1.5.3	parso	0.8.3	pathspec	0.10.3
patsy	0.5.6	pexpect	4.8.0	pillow	10.3.0
pip	24.2	platformdirs	3.10.0	plotly	5.22.0
pluggy	1.0.0	prompt-toolkit	3.0.43	proto-plus	1.24.0
protobuf	4.24.1	psutil	5.9.0	psycopg2	2.9.3
ptyprocess	0.7.0	pure-eval	0.2.2	pyarrow	15.0.2
pyasn1	0.4.8	pyasn1-modules	0.2.8	pyccolo	0.0.65
pycparser	2.21	pydantic	2.8.2	pydantic_core	2.20.1
pyflakes	3.2.0	Pygments	2.15.1	PyGObject	3.48.2
PyJWT	2.7.0	pyodbc	5.0.1	pyparsing	3.0.9
pyright	1.1.294	python-dateutil	2.9.0.post0	python-lsp-jsonrpc	1.1.2
python-lsp-server	1.10.0	pytoolconfig	1.2.6	pytz	2024.1
PyYAML	6.0.1	pyzmq	25.1.2	requests	2.32.2
rope	1.12.0	rsa	4.9	s3transfer	0.10.2
scikit-learn	1.4.2	scipy	1.13.1	seaborn	0.13.2
setuptools	74.0.0	six	1.16.0	smmap	5.0.0
sqlparse	0.5.1	ssh-import-id	5.11	stack-data	0.2.0
statsmodels	0.14.2	tenacity	8.2.2	threadpoolctl	2.2.0
tokenize-rt	4.2.1	tomli	2.0.1	tornado	6.4.1
traitlets	5.14.3	typeguard	4.3.0	types-protobuf	3.20.3
types-psutil	5.9.0	types-pytz	2023.3.1.1	types-PyYAML	6.0.0
types-requests	2.31.0.0	types-setuptools	68.0.0.0	types-six	1.16.0
types-urllib3	1.26.25.14	typing_extensions	4.11.0	ujson	5.10.0
unattended-upgrades	0.1	urllib3	1.26.16	virtualenv	20.26.2
wadllib	1.3.6	wcwidth	0.2.5	whatthepatch	1.0.2
wheel	0.43.0	wrapt	1.14.1	yapf	0.33.0
zipp	3.17.0

Installed R libraries

R libraries are installed from the Posit Package Manager CRAN snapshot on 2025-03-20:

Library	Version	Library	Version	Library	Version
arrow	16.1.0	askpass	1.2.0	assertthat	0.2.1
backports	1.5.0	base	4.4.0	base64enc	0.1-3
bigD	0.2.0	bit	4.0.5	bit64	4.0.5
bitops	1.0-8	blob	1.2.4	boot	1.3-30
brew	1.0-10	brio	1.1.5	broom	1.0.6
bslib	0.8.0	cachem	1.1.0	callr	3.7.6
caret	6.0-94	cellranger	1.1.0	chron	2.3-61
class	7.3-22	cli	3.6.3	clipr	0.8.0
clock	0.7.1	cluster	2.1.6	codetools	0.2-20
colorspace	2.1-1	commonmark	1.9.1	compiler	4.4.0
config	0.3.2	conflicted	1.2.0	cpp11	0.4.7
crayon	1.5.3	credentials	2.0.1	curl	5.2.1
data.table	1.15.4	datasets	4.4.0	DBI	1.2.3
dbplyr	2.5.0	desc	1.4.3	devtools	2.4.5
diagram	1.6.5	diffobj	0.3.5	digest	0.6.36
downlit	0.4.4	dplyr	1.1.4	dtplyr	1.3.1
e1071	1.7-14	ellipsis	0.3.2	evaluate	0.24.0
fansi	1.0.6	farver	2.1.2	fastmap	1.2.0
fontawesome	0.5.2	forcats	1.0.0	foreach	1.5.2
foreign	0.8-86	forge	0.2.0	fs	1.6.4
future	1.34.0	future.apply	1.11.2	gargle	1.5.2
generics	0.1.3	gert	2.1.0	ggplot2	3.5.1
gh	1.4.1	git2r	0.33.0	gitcreds	0.1.2
glmnet	4.1-8	globals	0.16.3	glue	1.7.0
googledrive	2.1.1	googlesheets4	1.1.1	gower	1.0.1
graphics	4.4.0	grDevices	4.4.0	grid	4.4.0
gridExtra	2.3	gsubfn	0.7	gt	0.11.0
gtable	0.3.5	hardhat	1.4.0	haven	2.5.4
highr	0.11	hms	1.1.3	htmltools	0.5.8.1
htmlwidgets	1.6.4	httpuv	1.6.15	httr	1.4.7
httr2	1.0.2	ids	1.0.1	ini	0.3.1
ipred	0.9-15	isoband	0.2.7	iterators	1.0.14
jquerylib	0.1.4	jsonlite	1.8.8	juicyjuice	0.1.0
KernSmooth	2.23-22	knitr	1.48	labeling	0.4.3
later	1.3.2	lattice	0.22-5	lava	1.8.0
lifecycle	1.0.4	listenv	0.9.1	lubridate	1.9.3
magrittr	2.0.3	markdown	1.13	MASS	7.3-60.0.1
Matrix	1.6-5	memoise	2.0.1	methods	4.4.0
mgcv	1.9-1	mime	0.12	miniUI	0.1.1.1
mlflow	2.14.1	ModelMetrics	1.2.2.2	modelr	0.1.11
munsell	0.5.1	nlme	3.1-165	nnet	7.3-19
numDeriv	2016.8-1.1	openssl	2.2.0	parallel	4.4.0
parallelly	1.38.0	pillar	1.9.0	pkgbuild	1.4.4
pkgconfig	2.0.3	pkgdown	2.1.0	pkgload	1.4.0
plogr	0.2.0	plyr	1.8.9	praise	1.0.0
prettyunits	1.2.0	pROC	1.18.5	processx	3.8.4
prodlim	2024.06.25	profvis	0.3.8	progress	1.2.3
progressr	0.14.0	promises	1.3.0	proto	1.0.0
proxy	0.4-27	ps	1.7.7	purrr	1.0.2
R6	2.5.1	ragg	1.3.2	randomForest	4.7-1.1
rappdirs	0.3.3	rcmdcheck	1.4.0	RColorBrewer	1.1-3
Rcpp	1.0.13	RcppEigen	0.3.4.0.0	reactable	0.4.4
reactR	0.6.0	readr	2.1.5	readxl	1.4.3
recipes	1.1.0	rematch	2.0.0	rematch2	2.1.2
remotes	2.5.0	reprex	2.1.1	reshape2	1.4.4
rlang	1.1.4	rmarkdown	2.27	RODBC	1.3-23
roxygen2	7.3.2	rpart	4.1.23	rprojroot	2.0.4
Rserve	1.8-13	RSQLite	2.3.7	rstudioapi	0.16.0
rversions	2.1.2	rvest	1.0.4	sass	0.4.9
scales	1.3.0	selectr	0.4-2	sessioninfo	1.2.2
shape	1.4.6.1	shiny	1.9.1	sourcetools	0.1.7-1
sparklyr (see note below)	1.8.6	SparkR	3.5.2	spatial	7.3-17
splines	4.4.0	sqldf	0.4-11	SQUAREM	2021.1
stats	4.4.0	stats4	4.4.0	stringi	1.8.4
stringr	1.5.1	survival	3.6-4	swagger	5.17.14.1
sys	3.4.2	systemfonts	1.1.0	tcltk	4.4.0
testthat	3.2.1.1	textshaping	0.4.0	tibble	3.2.1
tidyr	1.3.1	tidyselect	1.2.1	tidyverse	2.0.0
timechange	0.3.0	timeDate	4032.109	tinytex	0.52
tools	4.4.0	tzdb	0.4.0	urlchecker	1.0.1
usethis	3.0.0	utf8	1.2.4	utils	4.4.0
uuid	1.2-1	V8	4.4.2	vctrs	0.6.5
viridisLite	0.4.2	vroom	1.6.5	waldo	0.5.2
whisker	0.4.1	withr	3.0.1	xfun	0.46
xml2	1.3.6	xopen	1.0.1	xtable	1.8-4
yaml	2.3.10	zeallot	0.1.0	zip	2.3.1

nota

sparklyr is only supported in the Databricks Runtime 16.4 LTS release image with support for Scala 2.12. It is not supported in the DBR 16.4 release image with Scala 2.13 support.

Installed Java and Scala libraries (Scala 2.13 cluster version)

Group ID	Artifact ID	Version
antlr	antlr	2.7.7
com.amazonaws	amazon-kinesis-client	1.12.0
com.amazonaws	aws-java-sdk-autoscaling	1.12.638
com.amazonaws	aws-java-sdk-cloudformation	1.12.638
com.amazonaws	aws-java-sdk-cloudfront	1.12.638
com.amazonaws	aws-java-sdk-cloudhsm	1.12.638
com.amazonaws	aws-java-sdk-cloudsearch	1.12.638
com.amazonaws	aws-java-sdk-cloudtrail	1.12.638
com.amazonaws	aws-java-sdk-cloudwatch	1.12.638
com.amazonaws	aws-java-sdk-cloudwatchmetrics	1.12.638
com.amazonaws	aws-java-sdk-codedeploy	1.12.638
com.amazonaws	aws-java-sdk-cognitoidentity	1.12.638
com.amazonaws	aws-java-sdk-cognitosync	1.12.638
com.amazonaws	aws-java-sdk-config	1.12.638
com.amazonaws	aws-java-sdk-core	1.12.638
com.amazonaws	aws-java-sdk-datapipeline	1.12.638
com.amazonaws	aws-java-sdk-directconnect	1.12.638
com.amazonaws	aws-java-sdk-directory	1.12.638
com.amazonaws	aws-java-sdk-dynamodb	1.12.638
com.amazonaws	aws-java-sdk-ec2	1.12.638
com.amazonaws	aws-java-sdk-ecs	1.12.638
com.amazonaws	aws-java-sdk-efs	1.12.638
com.amazonaws	aws-java-sdk-elasticache	1.12.638
com.amazonaws	aws-java-sdk-elasticbeanstalk	1.12.638
com.amazonaws	aws-java-sdk-elasticloadbalancing	1.12.638
com.amazonaws	aws-java-sdk-elastictranscoder	1.12.638
com.amazonaws	aws-java-sdk-emr	1.12.638
com.amazonaws	aws-java-sdk-glacier	1.12.638
com.amazonaws	aws-java-sdk-glue	1.12.638
com.amazonaws	aws-java-sdk-iam	1.12.638
com.amazonaws	aws-java-sdk-importexport	1.12.638
com.amazonaws	aws-java-sdk-kinesis	1.12.638
com.amazonaws	aws-java-sdk-kms	1.12.638
com.amazonaws	aws-java-sdk-lambda	1.12.638
com.amazonaws	aws-java-sdk-logs	1.12.638
com.amazonaws	aws-java-sdk-machinelearning	1.12.638
com.amazonaws	aws-java-sdk-opsworks	1.12.638
com.amazonaws	aws-java-sdk-rds	1.12.638
com.amazonaws	aws-java-sdk-redshift	1.12.638
com.amazonaws	aws-java-sdk-route53	1.12.638
com.amazonaws	aws-java-sdk-s3	1.12.638
com.amazonaws	aws-java-sdk-ses	1.12.638
com.amazonaws	aws-java-sdk-simpledb	1.12.638
com.amazonaws	aws-java-sdk-simpleworkflow	1.12.638
com.amazonaws	aws-java-sdk-sns	1.12.638
com.amazonaws	aws-java-sdk-sqs	1.12.638
com.amazonaws	aws-java-sdk-ssm	1.12.638
com.amazonaws	aws-java-sdk-storagegateway	1.12.638
com.amazonaws	aws-java-sdk-sts	1.12.638
com.amazonaws	aws-java-sdk-support	1.12.638
com.amazonaws	aws-java-sdk-swf-libraries	1.11.22
com.amazonaws	aws-java-sdk-workspaces	1.12.638
com.amazonaws	jmespath-java	1.12.638
com.clearspring.analytics	stream	2.9.6
com.databricks	Rserve	1.8-3
com.databricks	databricks-sdk-java	0.27.0
com.databricks	jets3t	0.7.1-0
com.databricks.scalapb	scalapb-runtime_2.13	0.4.15-11
com.esotericsoftware	kryo-shaded	4.0.2
com.esotericsoftware	minlog	1.3.0
com.fasterxml	classmate	1.3.4
com.fasterxml.jackson.core	jackson-annotations	2.15.2
com.fasterxml.jackson.core	jackson-core	2.15.2
com.fasterxml.jackson.core	jackson-databind	2.15.2
com.fasterxml.jackson.dataformat	jackson-dataformat-cbor	2.15.2
com.fasterxml.jackson.dataformat	jackson-dataformat-yaml	2.15.2
com.fasterxml.jackson.datatype	jackson-datatype-joda	2.15.2
com.fasterxml.jackson.datatype	jackson-datatype-jsr310	2.16.0
com.fasterxml.jackson.module	jackson-module-paranamer	2.15.2
com.fasterxml.jackson.module	jackson-module-scala_2.13	2.15.2
com.github.ben-manes.caffeine	caffeine	2.9.3
com.github.fommil	jniloader	1.1
com.github.fommil.netlib	native_ref-java	1.1
com.github.fommil.netlib	native_ref-java	1.1-natives
com.github.fommil.netlib	native_system-java	1.1
com.github.fommil.netlib	native_system-java	1.1-natives
com.github.fommil.netlib	netlib-native_ref-linux-x86_64	1.1-natives
com.github.fommil.netlib	netlib-native_system-linux-x86_64	1.1-natives
com.github.luben	zstd-jni	1.5.5-4
com.github.wendykierp	JTransforms	3.1
com.google.code.findbugs	jsr305	3.0.0
com.google.code.gson	gson	2.10.1
com.google.crypto.tink	tink	1.9.0
com.google.errorprone	error_prone_annotations	2.10.0
com.google.flatbuffers	flatbuffers-java	23.5.26
com.google.guava	guava	15.0
com.google.protobuf	protobuf-java	3.25.1
com.helger	profiler	1.1.1
com.ibm.icu	icu4j	75.1
com.jcraft	jsch	0.1.55
com.jolbox	bonecp	0.8.0.RELEASE
com.lihaoyi	sourcecode_2.13	0.1.9
com.microsoft.azure	azure-data-lake-store-sdk	2.3.9
com.microsoft.sqlserver	mssql-jdbc	11.2.2.jre8
com.microsoft.sqlserver	mssql-jdbc	11.2.3.jre8
com.ning	compress-lzf	1.1.2
com.sun.mail	javax.mail	1.5.2
com.sun.xml.bind	jaxb-core	2.2.11
com.sun.xml.bind	jaxb-impl	2.2.11
com.tdunning	json	1.8
com.thoughtworks.paranamer	paranamer	2.8
com.trueaccord.lenses	lenses_2.13	0.4.13
com.twitter	chill-java	0.10.0
com.twitter	chill_2.13	0.10.0
com.twitter	util-app_2.13	19.8.1
com.twitter	util-core_2.13	19.8.1
com.twitter	util-function_2.13	19.8.1
com.twitter	util-jvm_2.13	19.8.1
com.twitter	util-lint_2.13	19.8.1
com.twitter	util-registry_2.13	19.8.1
com.twitter	util-stats_2.13	19.8.1
com.typesafe	config	1.4.3
com.typesafe.scala-logging	scala-logging_2.13	3.9.2
com.uber	h3	3.7.3
com.univocity	univocity-parsers	2.9.1
com.zaxxer	HikariCP	4.0.3
commons-cli	commons-cli	1.5.0
commons-codec	commons-codec	1.16.0
commons-collections	commons-collections	3.2.2
commons-dbcp	commons-dbcp	1.4
commons-fileupload	commons-fileupload	1.5
commons-httpclient	commons-httpclient	3.1
commons-io	commons-io	2.13.0
commons-lang	commons-lang	2.6
commons-logging	commons-logging	1.1.3
commons-pool	commons-pool	1.5.4
dev.ludovic.netlib	arpack	3.0.3
dev.ludovic.netlib	blas	3.0.3
dev.ludovic.netlib	lapack	3.0.3
info.ganglia.gmetric4j	gmetric4j	1.0.10
io.airlift	aircompressor	0.27
io.delta	delta-sharing-client_2.13	1.2.3
io.dropwizard.metrics	metrics-annotation	4.2.19
io.dropwizard.metrics	metrics-core	4.2.19
io.dropwizard.metrics	metrics-graphite	4.2.19
io.dropwizard.metrics	metrics-healthchecks	4.2.19
io.dropwizard.metrics	metrics-jetty9	4.2.19
io.dropwizard.metrics	metrics-jmx	4.2.19
io.dropwizard.metrics	metrics-json	4.2.19
io.dropwizard.metrics	metrics-jvm	4.2.19
io.dropwizard.metrics	metrics-servlets	4.2.19
io.github.java-diff-utils	java-diff-utils	4.12
io.netty	netty-all	4.1.108.Final
io.netty	netty-buffer	4.1.108.Final
io.netty	netty-codec	4.1.108.Final
io.netty	netty-codec-http	4.1.108.Final
io.netty	netty-codec-http2	4.1.108.Final
io.netty	netty-codec-socks	4.1.108.Final
io.netty	netty-common	4.1.108.Final
io.netty	netty-handler	4.1.108.Final
io.netty	netty-handler-proxy	4.1.108.Final
io.netty	netty-resolver	4.1.108.Final
io.netty	netty-tcnative-boringssl-static	2.0.61.Final-db-r16
io.netty	netty-tcnative-boringssl-static	2.0.61.Final-db-r16-linux-aarch_64
io.netty	netty-tcnative-boringssl-static	2.0.61.Final-db-r16-linux-x86_64
io.netty	netty-tcnative-boringssl-static	2.0.61.Final-db-r16-osx-aarch_64
io.netty	netty-tcnative-boringssl-static	2.0.61.Final-db-r16-osx-x86_64
io.netty	netty-tcnative-boringssl-static	2.0.61.Final-db-r16-windows-x86_64
io.netty	netty-tcnative-classes	2.0.61.Final
io.netty	netty-transport	4.1.108.Final
io.netty	netty-transport-classes-epoll	4.1.108.Final
io.netty	netty-transport-classes-kqueue	4.1.108.Final
io.netty	netty-transport-native-epoll	4.1.108.Final
io.netty	netty-transport-native-epoll	4.1.108.Final-linux-aarch_64
io.netty	netty-transport-native-epoll	4.1.108.Final-linux-riscv64
io.netty	netty-transport-native-epoll	4.1.108.Final-linux-x86_64
io.netty	netty-transport-native-kqueue	4.1.108.Final-osx-aarch_64
io.netty	netty-transport-native-kqueue	4.1.108.Final-osx-x86_64
io.netty	netty-transport-native-unix-common	4.1.108.Final
io.prometheus	simpleclient	0.7.0
io.prometheus	simpleclient_common	0.7.0
io.prometheus	simpleclient_dropwizard	0.7.0
io.prometheus	simpleclient_pushgateway	0.7.0
io.prometheus	simpleclient_servlet	0.7.0
io.prometheus.jmx	collector	0.12.0
jakarta.annotation	jakarta.annotation-api	1.3.5
jakarta.servlet	jakarta.servlet-api	4.0.3
jakarta.validation	jakarta.validation-api	2.0.2
jakarta.ws.rs	jakarta.ws.rs-api	2.1.6
javax.activation	activation	1.1.1
javax.el	javax.el-api	2.2.4
javax.jdo	jdo-api	3.0.1
javax.transaction	jta	1.1
javax.transaction	transaction-api	1.1
javax.xml.bind	jaxb-api	2.2.11
javolution	javolution	5.5.1
jline	jline	2.14.6
joda-time	joda-time	2.12.1
net.java.dev.jna	jna	5.8.0
net.razorvine	pickle	1.3
net.sf.jpam	jpam	1.1
net.sf.opencsv	opencsv	2.3
net.sf.supercsv	super-csv	2.2.0
net.snowflake	snowflake-ingest-sdk	0.9.6
net.sourceforge.f2j	arpack_combined_all	0.1
org.acplt.remotetea	remotetea-oncrpc	1.1.2
org.antlr	ST4	4.0.4
org.antlr	antlr-runtime	3.5.2
org.antlr	antlr4-runtime	4.9.3
org.antlr	stringtemplate	3.2.1
org.apache.ant	ant	1.10.11
org.apache.ant	ant-jsch	1.10.11
org.apache.ant	ant-launcher	1.10.11
org.apache.arrow	arrow-format	15.0.0
org.apache.arrow	arrow-memory-core	15.0.0
org.apache.arrow	arrow-memory-netty	15.0.0
org.apache.arrow	arrow-vector	15.0.0
org.apache.avro	avro	1.11.3
org.apache.avro	avro-ipc	1.11.3
org.apache.avro	avro-mapred	1.11.3
org.apache.commons	commons-collections4	4.4
org.apache.commons	commons-compress	1.23.0
org.apache.commons	commons-crypto	1.1.0
org.apache.commons	commons-lang3	3.12.0
org.apache.commons	commons-math3	3.6.1
org.apache.commons	commons-text	1.10.0
org.apache.curator	curator-client	2.13.0
org.apache.curator	curator-framework	2.13.0
org.apache.curator	curator-recipes	2.13.0
org.apache.datasketches	datasketches-java	3.1.0
org.apache.datasketches	datasketches-memory	2.0.0
org.apache.derby	derby	10.14.2.0
org.apache.hadoop	hadoop-client-runtime	3.3.6
org.apache.hive	hive-beeline	2.3.9
org.apache.hive	hive-cli	2.3.9
org.apache.hive	hive-jdbc	2.3.9
org.apache.hive	hive-llap-client	2.3.9
org.apache.hive	hive-llap-common	2.3.9
org.apache.hive	hive-serde	2.3.9
org.apache.hive	hive-shims	2.3.9
org.apache.hive	hive-storage-api	2.8.1
org.apache.hive.shims	hive-shims-0.23	2.3.9
org.apache.hive.shims	hive-shims-common	2.3.9
org.apache.hive.shims	hive-shims-scheduler	2.3.9
org.apache.httpcomponents	httpclient	4.5.14
org.apache.httpcomponents	httpcore	4.4.16
org.apache.ivy	ivy	2.5.2
org.apache.logging.log4j	log4j-1.2-api	2.22.1
org.apache.logging.log4j	log4j-api	2.22.1
org.apache.logging.log4j	log4j-core	2.22.1
org.apache.logging.log4j	log4j-layout-template-json	2.22.1
org.apache.logging.log4j	log4j-slf4j2-impl	2.22.1
org.apache.orc	orc-core	1.9.2-shaded-protobuf
org.apache.orc	orc-mapreduce	1.9.2-shaded-protobuf
org.apache.orc	orc-shims	1.9.2
org.apache.thrift	libfb303	0.9.3
org.apache.thrift	libthrift	0.12.0
org.apache.ws.xmlschema	xmlschema-core	2.3.0
org.apache.xbean	xbean-asm9-shaded	4.23
org.apache.yetus	audience-annotations	0.13.0
org.apache.zookeeper	zookeeper	3.9.2
org.apache.zookeeper	zookeeper-jute	3.9.2
org.checkerframework	checker-qual	3.31.0
org.codehaus.jackson	jackson-core-asl	1.9.13
org.codehaus.jackson	jackson-mapper-asl	1.9.13
org.codehaus.janino	commons-compiler	3.0.16
org.codehaus.janino	janino	3.0.16
org.datanucleus	datanucleus-api-jdo	4.2.4
org.datanucleus	datanucleus-core	4.1.17
org.datanucleus	datanucleus-rdbms	4.1.19
org.datanucleus	javax.jdo	3.2.0-m3
org.eclipse.collections	eclipse-collections	11.1.0
org.eclipse.collections	eclipse-collections-api	11.1.0
org.eclipse.jetty	jetty-client	9.4.52.v20230823
org.eclipse.jetty	jetty-continuation	9.4.52.v20230823
org.eclipse.jetty	jetty-http	9.4.52.v20230823
org.eclipse.jetty	jetty-io	9.4.52.v20230823
org.eclipse.jetty	jetty-jndi	9.4.52.v20230823
org.eclipse.jetty	jetty-plus	9.4.52.v20230823
org.eclipse.jetty	jetty-proxy	9.4.52.v20230823
org.eclipse.jetty	jetty-security	9.4.52.v20230823
org.eclipse.jetty	jetty-server	9.4.52.v20230823
org.eclipse.jetty	jetty-servlet	9.4.52.v20230823
org.eclipse.jetty	jetty-servlets	9.4.52.v20230823
org.eclipse.jetty	jetty-util	9.4.52.v20230823
org.eclipse.jetty	jetty-util-ajax	9.4.52.v20230823
org.eclipse.jetty	jetty-webapp	9.4.52.v20230823
org.eclipse.jetty	jetty-xml	9.4.52.v20230823
org.eclipse.jetty.websocket	websocket-api	9.4.52.v20230823
org.eclipse.jetty.websocket	websocket-client	9.4.52.v20230823
org.eclipse.jetty.websocket	websocket-common	9.4.52.v20230823
org.eclipse.jetty.websocket	websocket-server	9.4.52.v20230823
org.eclipse.jetty.websocket	websocket-servlet	9.4.52.v20230823
org.fusesource.leveldbjni	leveldbjni-all	1.8
org.glassfish.hk2	hk2-api	2.6.1
org.glassfish.hk2	hk2-locator	2.6.1
org.glassfish.hk2	hk2-utils	2.6.1
org.glassfish.hk2	osgi-resource-locator	1.0.3
org.glassfish.hk2.external	aopalliance-repackaged	2.6.1
org.glassfish.hk2.external	jakarta.inject	2.6.1
org.glassfish.jersey.containers	jersey-container-servlet	2.40
org.glassfish.jersey.containers	jersey-container-servlet-core	2.40
org.glassfish.jersey.core	jersey-client	2.40
org.glassfish.jersey.core	jersey-common	2.40
org.glassfish.jersey.core	jersey-server	2.40
org.glassfish.jersey.inject	jersey-hk2	2.40
org.hibernate.validator	hibernate-validator	6.1.7.Final
org.ini4j	ini4j	0.5.4
org.javassist	javassist	3.29.2-GA
org.jboss.logging	jboss-logging	3.3.2.Final
org.jdbi	jdbi	2.63.1
org.jetbrains	annotations	17.0.0
org.jline	jline	3.21.0
org.joda	joda-convert	1.7
org.jodd	jodd-core	3.5.2
org.json4s	json4s-ast_2.13	3.7.0-M11
org.json4s	json4s-core_2.13	3.7.0-M11
org.json4s	json4s-jackson_2.13	3.7.0-M11
org.json4s	json4s-scalap_2.13	3.7.0-M11
org.lz4	lz4-java	1.8.0-databricks-1
org.mlflow	mlflow-spark_2.13	2.9.1
org.objenesis	objenesis	2.5.1
org.postgresql	postgresql	42.6.1
org.roaringbitmap	RoaringBitmap	0.9.45-databricks
org.roaringbitmap	shims	0.9.45-databricks
org.rocksdb	rocksdbjni	9.2.1
org.rosuda.REngine	REngine	2.1.0
org.scala-lang	scala-compiler_2.13	2.13.10
org.scala-lang	scala-library_2.13	2.13.10
org.scala-lang	scala-reflect_2.13	2.13.10
org.scala-lang.modules	scala-collection-compat_2.13	2.11.0
org.scala-lang.modules	scala-java8-compat_2.13	0.9.1
org.scala-lang.modules	scala-parallel-collections_2.13	1.0.4
org.scala-lang.modules	scala-parser-combinators_2.13	1.1.2
org.scala-lang.modules	scala-xml_2.13	1.2.0
org.scala-sbt	test-interface	1.0
org.scalacheck	scalacheck_2.13	1.14.2
org.scalactic	scalactic_2.13	3.2.16
org.scalanlp	breeze-macros_2.13	2.1.0
org.scalanlp	breeze_2.13	2.1.0
org.scalatest	scalatest-compatible	3.2.16
org.scalatest	scalatest-core_2.13	3.2.16
org.scalatest	scalatest-diagrams_2.13	3.2.16
org.scalatest	scalatest-featurespec_2.13	3.2.16
org.scalatest	scalatest-flatspec_2.13	3.2.16
org.scalatest	scalatest-freespec_2.13	3.2.16
org.scalatest	scalatest-funspec_2.13	3.2.16
org.scalatest	scalatest-funsuite_2.13	3.2.16
org.scalatest	scalatest-matchers-core_2.13	3.2.16
org.scalatest	scalatest-mustmatchers_2.13	3.2.16
org.scalatest	scalatest-propspec_2.13	3.2.16
org.scalatest	scalatest-refspec_2.13	3.2.16
org.scalatest	scalatest-shouldmatchers_2.13	3.2.16
org.scalatest	scalatest-wordspec_2.13	3.2.16
org.scalatest	scalatest_2.13	3.2.16
org.slf4j	jcl-over-slf4j	2.0.7
org.slf4j	jul-to-slf4j	2.0.7
org.slf4j	slf4j-api	2.0.7
org.slf4j	slf4j-simple	1.7.25
org.threeten	threeten-extra	1.7.1
org.tukaani	xz	1.9
org.typelevel	algebra_2.13	2.8.0
org.typelevel	cats-kernel_2.13	2.8.0
org.typelevel	spire-macros_2.13	0.18.0
org.typelevel	spire-platform_2.13	0.18.0
org.typelevel	spire-util_2.13	0.18.0
org.typelevel	spire_2.13	0.18.0
org.wildfly.openssl	wildfly-openssl	1.1.3.Final
org.xerial	sqlite-jdbc	3.42.0.0
org.xerial.snappy	snappy-java	1.1.10.3
org.yaml	snakeyaml	2.0
oro	oro	2.0.8
pl.edu.icm	JLargeArrays	1.5
software.amazon.cryptools	AmazonCorrettoCryptoProvider	2.4.1-linux-x86_64
stax	stax-api	1.0.1

Installed Java and Scala libraries (Scala 2.12 cluster version)

Group ID	Artifact ID	Version
antlr	antlr	2.7.7
com.amazonaws	amazon-kinesis-client	1.12.0
com.amazonaws	aws-java-sdk-autoscaling	1.12.638
com.amazonaws	aws-java-sdk-cloudformation	1.12.638
com.amazonaws	aws-java-sdk-cloudfront	1.12.638
com.amazonaws	aws-java-sdk-cloudhsm	1.12.638
com.amazonaws	aws-java-sdk-cloudsearch	1.12.638
com.amazonaws	aws-java-sdk-cloudtrail	1.12.638
com.amazonaws	aws-java-sdk-cloudwatch	1.12.638
com.amazonaws	aws-java-sdk-cloudwatchmetrics	1.12.638
com.amazonaws	aws-java-sdk-codedeploy	1.12.638
com.amazonaws	aws-java-sdk-cognitoidentity	1.12.638
com.amazonaws	aws-java-sdk-cognitosync	1.12.638
com.amazonaws	aws-java-sdk-config	1.12.638
com.amazonaws	aws-java-sdk-core	1.12.638
com.amazonaws	aws-java-sdk-datapipeline	1.12.638
com.amazonaws	aws-java-sdk-directconnect	1.12.638
com.amazonaws	aws-java-sdk-directory	1.12.638
com.amazonaws	aws-java-sdk-dynamodb	1.12.638
com.amazonaws	aws-java-sdk-ec2	1.12.638
com.amazonaws	aws-java-sdk-ecs	1.12.638
com.amazonaws	aws-java-sdk-efs	1.12.638
com.amazonaws	aws-java-sdk-elasticache	1.12.638
com.amazonaws	aws-java-sdk-elasticbeanstalk	1.12.638
com.amazonaws	aws-java-sdk-elasticloadbalancing	1.12.638
com.amazonaws	aws-java-sdk-elastictranscoder	1.12.638
com.amazonaws	aws-java-sdk-emr	1.12.638
com.amazonaws	aws-java-sdk-glacier	1.12.638
com.amazonaws	aws-java-sdk-glue	1.12.638
com.amazonaws	aws-java-sdk-iam	1.12.638
com.amazonaws	aws-java-sdk-importexport	1.12.638
com.amazonaws	aws-java-sdk-kinesis	1.12.638
com.amazonaws	aws-java-sdk-kms	1.12.638
com.amazonaws	aws-java-sdk-lambda	1.12.638
com.amazonaws	aws-java-sdk-logs	1.12.638
com.amazonaws	aws-java-sdk-machinelearning	1.12.638
com.amazonaws	aws-java-sdk-opsworks	1.12.638
com.amazonaws	aws-java-sdk-rds	1.12.638
com.amazonaws	aws-java-sdk-redshift	1.12.638
com.amazonaws	aws-java-sdk-route53	1.12.638
com.amazonaws	aws-java-sdk-s3	1.12.638
com.amazonaws	aws-java-sdk-ses	1.12.638
com.amazonaws	aws-java-sdk-simpledb	1.12.638
com.amazonaws	aws-java-sdk-simpleworkflow	1.12.638
com.amazonaws	aws-java-sdk-sns	1.12.638
com.amazonaws	aws-java-sdk-sqs	1.12.638
com.amazonaws	aws-java-sdk-ssm	1.12.638
com.amazonaws	aws-java-sdk-storagegateway	1.12.638
com.amazonaws	aws-java-sdk-sts	1.12.638
com.amazonaws	aws-java-sdk-support	1.12.638
com.amazonaws	aws-java-sdk-swf-libraries	1.11.22
com.amazonaws	aws-java-sdk-workspaces	1.12.638
com.amazonaws	jmespath-java	1.12.638
com.azure	azure-core	1.30.0
com.azure	azure-core-http-netty	1.12.3
com.azure	azure-identity	1.5.3
com.azure	azure-security-keyvault-keys	4.4.4
com.clearspring.analytics	stream	2.9.6
com.databricks	Rserve	1.8-3
com.databricks	databricks-sdk-java	0.27.0
com.databricks	jets3t	0.7.1-0
com.databricks.scalapb	scalapb-runtime_2.12	0.4.15-10
com.esotericsoftware	kryo-shaded	4.0.2
com.esotericsoftware	minlog	1.3.0
com.fasterxml	classmate	1.3.4
com.fasterxml.jackson.core	jackson-annotations	2.15.2
com.fasterxml.jackson.core	jackson-core	2.15.2
com.fasterxml.jackson.core	jackson-databind	2.15.2
com.fasterxml.jackson.dataformat	jackson-dataformat-cbor	2.15.2
com.fasterxml.jackson.dataformat	jackson-dataformat-xml	2.13.3
com.fasterxml.jackson.dataformat	jackson-dataformat-yaml	2.15.2
com.fasterxml.jackson.datatype	jackson-datatype-joda	2.15.2
com.fasterxml.jackson.datatype	jackson-datatype-jsr310	2.16.0
com.fasterxml.jackson.module	jackson-module-paranamer	2.15.2
com.fasterxml.jackson.module	jackson-module-scala_2.12	2.15.2
com.fasterxml.woodstox	woodstox-core	6.4.0
com.github.ben-manes.caffeine	caffeine	2.9.3
com.github.fommil	jniloader	1.1
com.github.fommil.netlib	native_ref-java	1.1
com.github.fommil.netlib	native_ref-java	1.1-natives
com.github.fommil.netlib	native_system-java	1.1
com.github.fommil.netlib	native_system-java	1.1-natives
com.github.fommil.netlib	netlib-native_ref-linux-x86_64	1.1-natives
com.github.fommil.netlib	netlib-native_system-linux-x86_64	1.1-natives
com.github.luben	zstd-jni	1.5.5-4
com.github.stephenc.jcip	jcip-annotations	1.0-1
com.github.wendykierp	JTransforms	3.1
com.google.code.findbugs	jsr305	3.0.0
com.google.code.gson	gson	2.10.1
com.google.crypto.tink	tink	1.9.0
com.google.errorprone	error_prone_annotations	2.10.0
com.google.flatbuffers	flatbuffers-java	23.5.26
com.google.guava	guava	15.0
com.google.protobuf	protobuf-java	3.25.1
com.helger	profiler	1.1.1
com.ibm.icu	icu4j	75.1
com.jcraft	jsch	0.1.55
com.jolbox	bonecp	0.8.0.RELEASE
com.lihaoyi	sourcecode_2.12	0.1.9
com.microsoft.azure	azure-data-lake-store-sdk	2.3.9
com.microsoft.azure	msal4j	1.13.0
com.microsoft.azure	msal4j-persistence-extension	1.1.0
com.microsoft.sqlserver	mssql-jdbc	11.2.2.jre8
com.microsoft.sqlserver	mssql-jdbc	11.2.3.jre8
com.nimbusds	content-type	2.2
com.nimbusds	lang-tag	1.6
com.nimbusds	nimbus-jose-jwt	9.31
com.nimbusds	oauth2-oidc-sdk	9.35
com.ning	compress-lzf	1.1.2
com.sun.mail	javax.mail	1.5.2
com.sun.xml.bind	jaxb-core	2.2.11
com.sun.xml.bind	jaxb-impl	2.2.11
com.tdunning	json	1.8
com.thoughtworks.paranamer	paranamer	2.8
com.trueaccord.lenses	lenses_2.12	0.4.12
com.twitter	chill-java	0.10.0
com.twitter	chill_2.12	0.10.0
com.twitter	util-app_2.12	7.1.0
com.twitter	util-core_2.12	7.1.0
com.twitter	util-function_2.12	7.1.0
com.twitter	util-jvm_2.12	7.1.0
com.twitter	util-lint_2.12	7.1.0
com.twitter	util-registry_2.12	7.1.0
com.twitter	util-stats_2.12	7.1.0
com.typesafe	config	1.4.3
com.typesafe.scala-logging	scala-logging_2.12	3.7.2
com.uber	h3	3.7.3
com.univocity	univocity-parsers	2.9.1
com.zaxxer	HikariCP	4.0.3
commons-cli	commons-cli	1.5.0
commons-codec	commons-codec	1.16.0
commons-collections	commons-collections	3.2.2
commons-dbcp	commons-dbcp	1.4
commons-fileupload	commons-fileupload	1.5
commons-httpclient	commons-httpclient	3.1
commons-io	commons-io	2.13.0
commons-lang	commons-lang	2.6
commons-logging	commons-logging	1.1.3
commons-pool	commons-pool	1.5.4
dev.ludovic.netlib	arpack	3.0.3
dev.ludovic.netlib	blas	3.0.3
dev.ludovic.netlib	lapack	3.0.3
info.ganglia.gmetric4j	gmetric4j	1.0.10
io.airlift	aircompressor	0.27
io.delta	delta-sharing-client_2.12	1.2.4
io.dropwizard.metrics	metrics-annotation	4.2.19
io.dropwizard.metrics	metrics-core	4.2.19
io.dropwizard.metrics	metrics-graphite	4.2.19
io.dropwizard.metrics	metrics-healthchecks	4.2.19
io.dropwizard.metrics	metrics-jetty9	4.2.19
io.dropwizard.metrics	metrics-jmx	4.2.19
io.dropwizard.metrics	metrics-json	4.2.19
io.dropwizard.metrics	metrics-jvm	4.2.19
io.dropwizard.metrics	metrics-servlets	4.2.19
io.netty	netty-all	4.1.108.Final
io.netty	netty-buffer	4.1.108.Final
io.netty	netty-codec	4.1.108.Final
io.netty	netty-codec-dns	4.1.77.Final
io.netty	netty-codec-http	4.1.108.Final
io.netty	netty-codec-http2	4.1.108.Final
io.netty	netty-codec-socks	4.1.108.Final
io.netty	netty-common	4.1.108.Final
io.netty	netty-handler	4.1.108.Final
io.netty	netty-handler-proxy	4.1.108.Final
io.netty	netty-resolver	4.1.108.Final
io.netty	netty-resolver-dns	4.1.77.Final
io.netty	netty-resolver-dns-classes-macos	4.1.77.Final
io.netty	netty-resolver-dns-native-macos	4.1.77.Final-osx-x86_64
io.netty	netty-tcnative-boringssl-static	2.0.61.Final-db-r16
io.netty	netty-tcnative-boringssl-static	2.0.61.Final-db-r16-linux-aarch_64
io.netty	netty-tcnative-boringssl-static	2.0.61.Final-db-r16-linux-x86_64
io.netty	netty-tcnative-boringssl-static	2.0.61.Final-db-r16-osx-aarch_64
io.netty	netty-tcnative-boringssl-static	2.0.61.Final-db-r16-osx-x86_64
io.netty	netty-tcnative-boringssl-static	2.0.61.Final-db-r16-windows-x86_64
io.netty	netty-tcnative-classes	2.0.61.Final
io.netty	netty-transport	4.1.108.Final
io.netty	netty-transport-classes-epoll	4.1.108.Final
io.netty	netty-transport-classes-kqueue	4.1.108.Final
io.netty	netty-transport-native-epoll	4.1.108.Final
io.netty	netty-transport-native-epoll	4.1.108.Final-linux-aarch_64
io.netty	netty-transport-native-epoll	4.1.108.Final-linux-riscv64
io.netty	netty-transport-native-epoll	4.1.108.Final-linux-x86_64
io.netty	netty-transport-native-kqueue	4.1.108.Final-osx-aarch_64
io.netty	netty-transport-native-kqueue	4.1.108.Final-osx-x86_64
io.netty	netty-transport-native-unix-common	4.1.108.Final
io.projectreactor	reactor-core	3.4.19
io.projectreactor.netty	reactor-netty-core	1.0.20
io.projectreactor.netty	reactor-netty-http	1.0.20
io.prometheus	simpleclient	0.7.0
io.prometheus	simpleclient_common	0.7.0
io.prometheus	simpleclient_dropwizard	0.7.0
io.prometheus	simpleclient_pushgateway	0.7.0
io.prometheus	simpleclient_servlet	0.7.0
io.prometheus.jmx	collector	0.12.0
jakarta.annotation	jakarta.annotation-api	1.3.5
jakarta.servlet	jakarta.servlet-api	4.0.3
jakarta.validation	jakarta.validation-api	2.0.2
jakarta.ws.rs	jakarta.ws.rs-api	2.1.6
javax.activation	activation	1.1.1
javax.el	javax.el-api	2.2.4
javax.jdo	jdo-api	3.0.1
javax.transaction	jta	1.1
javax.transaction	transaction-api	1.1
javax.xml.bind	jaxb-api	2.2.11
javolution	javolution	5.5.1
jline	jline	2.14.6
joda-time	joda-time	2.12.1
net.java.dev.jna	jna	5.8.0
net.java.dev.jna	jna-platform	5.6.0
net.minidev	accessors-smart	2.4.11
net.minidev	json-smart	2.4.11
net.razorvine	pickle	1.3
net.sf.jpam	jpam	1.1
net.sf.opencsv	opencsv	2.3
net.sf.supercsv	super-csv	2.2.0
net.snowflake	snowflake-ingest-sdk	0.9.6
net.sourceforge.f2j	arpack_combined_all	0.1
org.acplt.remotetea	remotetea-oncrpc	1.1.2
org.antlr	ST4	4.0.4
org.antlr	antlr-runtime	3.5.2
org.antlr	antlr4-runtime	4.9.3
org.antlr	stringtemplate	3.2.1
org.apache.ant	ant	1.10.11
org.apache.ant	ant-jsch	1.10.11
org.apache.ant	ant-launcher	1.10.11
org.apache.arrow	arrow-format	15.0.0
org.apache.arrow	arrow-memory-core	15.0.0
org.apache.arrow	arrow-memory-netty	15.0.0
org.apache.arrow	arrow-vector	15.0.0
org.apache.avro	avro	1.11.3
org.apache.avro	avro-ipc	1.11.3
org.apache.avro	avro-mapred	1.11.3
org.apache.commons	commons-collections4	4.4
org.apache.commons	commons-compress	1.23.0
org.apache.commons	commons-crypto	1.1.0
org.apache.commons	commons-lang3	3.12.0
org.apache.commons	commons-math3	3.6.1
org.apache.commons	commons-text	1.10.0
org.apache.curator	curator-client	2.13.0
org.apache.curator	curator-framework	2.13.0
org.apache.curator	curator-recipes	2.13.0
org.apache.datasketches	datasketches-java	3.1.0
org.apache.datasketches	datasketches-memory	2.0.0
org.apache.derby	derby	10.14.2.0
org.apache.hadoop	hadoop-client-runtime	3.3.6
org.apache.hive	hive-beeline	2.3.9
org.apache.hive	hive-cli	2.3.9
org.apache.hive	hive-jdbc	2.3.9
org.apache.hive	hive-llap-client	2.3.9
org.apache.hive	hive-llap-common	2.3.9
org.apache.hive	hive-serde	2.3.9
org.apache.hive	hive-shims	2.3.9
org.apache.hive	hive-storage-api	2.8.1
org.apache.hive.shims	hive-shims-0.23	2.3.9
org.apache.hive.shims	hive-shims-common	2.3.9
org.apache.hive.shims	hive-shims-scheduler	2.3.9
org.apache.httpcomponents	httpclient	4.5.14
org.apache.httpcomponents	httpcore	4.4.16
org.apache.ivy	ivy	2.5.2
org.apache.logging.log4j	log4j-1.2-api	2.22.1
org.apache.logging.log4j	log4j-api	2.22.1
org.apache.logging.log4j	log4j-core	2.22.1
org.apache.logging.log4j	log4j-layout-template-json	2.22.1
org.apache.logging.log4j	log4j-slf4j2-impl	2.22.1
org.apache.orc	orc-core	1.9.2-shaded-protobuf
org.apache.orc	orc-mapreduce	1.9.2-shaded-protobuf
org.apache.orc	orc-shims	1.9.2
org.apache.thrift	libfb303	0.9.3
org.apache.thrift	libthrift	0.12.0
org.apache.ws.xmlschema	xmlschema-core	2.3.0
org.apache.xbean	xbean-asm9-shaded	4.23
org.apache.yetus	audience-annotations	0.13.0
org.apache.zookeeper	zookeeper	3.9.2
org.apache.zookeeper	zookeeper-jute	3.9.2
org.bouncycastle	bcprov-jdk15on	1.70
org.checkerframework	checker-qual	3.31.0
org.codehaus.jackson	jackson-core-asl	1.9.13
org.codehaus.jackson	jackson-mapper-asl	1.9.13
org.codehaus.janino	commons-compiler	3.0.16
org.codehaus.janino	janino	3.0.16
org.codehaus.woodstox	stax2-api	4.2.1
org.datanucleus	datanucleus-api-jdo	4.2.4
org.datanucleus	datanucleus-core	4.1.17
org.datanucleus	datanucleus-rdbms	4.1.19
org.datanucleus	javax.jdo	3.2.0-m3
org.eclipse.collections	eclipse-collections	11.1.0
org.eclipse.collections	eclipse-collections-api	11.1.0
org.eclipse.jetty	jetty-client	9.4.52.v20230823
org.eclipse.jetty	jetty-continuation	9.4.52.v20230823
org.eclipse.jetty	jetty-http	9.4.52.v20230823
org.eclipse.jetty	jetty-io	9.4.52.v20230823
org.eclipse.jetty	jetty-jndi	9.4.52.v20230823
org.eclipse.jetty	jetty-plus	9.4.52.v20230823
org.eclipse.jetty	jetty-proxy	9.4.52.v20230823
org.eclipse.jetty	jetty-security	9.4.52.v20230823
org.eclipse.jetty	jetty-server	9.4.52.v20230823
org.eclipse.jetty	jetty-servlet	9.4.52.v20230823
org.eclipse.jetty	jetty-servlets	9.4.52.v20230823
org.eclipse.jetty	jetty-util	9.4.52.v20230823
org.eclipse.jetty	jetty-util-ajax	9.4.52.v20230823
org.eclipse.jetty	jetty-webapp	9.4.52.v20230823
org.eclipse.jetty	jetty-xml	9.4.52.v20230823
org.eclipse.jetty.websocket	websocket-api	9.4.52.v20230823
org.eclipse.jetty.websocket	websocket-client	9.4.52.v20230823
org.eclipse.jetty.websocket	websocket-common	9.4.52.v20230823
org.eclipse.jetty.websocket	websocket-server	9.4.52.v20230823
org.eclipse.jetty.websocket	websocket-servlet	9.4.52.v20230823
org.fusesource.leveldbjni	leveldbjni-all	1.8
org.glassfish.hk2	hk2-api	2.6.1
org.glassfish.hk2	hk2-locator	2.6.1
org.glassfish.hk2	hk2-utils	2.6.1
org.glassfish.hk2	osgi-resource-locator	1.0.3
org.glassfish.hk2.external	aopalliance-repackaged	2.6.1
org.glassfish.hk2.external	jakarta.inject	2.6.1
org.glassfish.jersey.containers	jersey-container-servlet	2.40
org.glassfish.jersey.containers	jersey-container-servlet-core	2.40
org.glassfish.jersey.core	jersey-client	2.40
org.glassfish.jersey.core	jersey-common	2.40
org.glassfish.jersey.core	jersey-server	2.40
org.glassfish.jersey.inject	jersey-hk2	2.40
org.hibernate.validator	hibernate-validator	6.1.7.Final
org.ini4j	ini4j	0.5.4
org.javassist	javassist	3.29.2-GA
org.jboss.logging	jboss-logging	3.3.2.Final
org.jdbi	jdbi	2.63.1
org.jetbrains	annotations	17.0.0
org.joda	joda-convert	1.7
org.jodd	jodd-core	3.5.2
org.json4s	json4s-ast_2.12	3.7.0-M11
org.json4s	json4s-core_2.12	3.7.0-M11
org.json4s	json4s-jackson_2.12	3.7.0-M11
org.json4s	json4s-scalap_2.12	3.7.0-M11
org.lz4	lz4-java	1.8.0-databricks-1
org.mlflow	mlflow-spark_2.12	2.9.1
org.objenesis	objenesis	2.5.1
org.ow2.asm	asm	9.3
org.postgresql	postgresql	42.6.1
org.reactivestreams	reactive-streams	1.0.4
org.roaringbitmap	RoaringBitmap	0.9.45-databricks
org.roaringbitmap	shims	0.9.45-databricks
org.rocksdb	rocksdbjni	9.2.1
org.rosuda.REngine	REngine	2.1.0
org.scala-lang	scala-compiler_2.12	2.12.15
org.scala-lang	scala-library_2.12	2.12.15
org.scala-lang	scala-reflect_2.12	2.12.15
org.scala-lang.modules	scala-collection-compat_2.12	2.11.0
org.scala-lang.modules	scala-java8-compat_2.12	0.9.1
org.scala-lang.modules	scala-parser-combinators_2.12	1.1.2
org.scala-lang.modules	scala-xml_2.12	1.2.0
org.scala-sbt	test-interface	1.0
org.scalacheck	scalacheck_2.12	1.14.2
org.scalactic	scalactic_2.12	3.2.16
org.scalanlp	breeze-macros_2.12	2.1.0
org.scalanlp	breeze_2.12	2.1.0
org.scalatest	scalatest-compatible	3.2.16
org.scalatest	scalatest-core_2.12	3.2.16
org.scalatest	scalatest-diagrams_2.12	3.2.16
org.scalatest	scalatest-featurespec_2.12	3.2.16
org.scalatest	scalatest-flatspec_2.12	3.2.16
org.scalatest	scalatest-freespec_2.12	3.2.16
org.scalatest	scalatest-funspec_2.12	3.2.16
org.scalatest	scalatest-funsuite_2.12	3.2.16
org.scalatest	scalatest-matchers-core_2.12	3.2.16
org.scalatest	scalatest-mustmatchers_2.12	3.2.16
org.scalatest	scalatest-propspec_2.12	3.2.16
org.scalatest	scalatest-refspec_2.12	3.2.16
org.scalatest	scalatest-shouldmatchers_2.12	3.2.16
org.scalatest	scalatest-wordspec_2.12	3.2.16
org.scalatest	scalatest_2.12	3.2.16
org.slf4j	jcl-over-slf4j	2.0.7
org.slf4j	jul-to-slf4j	2.0.7
org.slf4j	slf4j-api	2.0.7
org.slf4j	slf4j-simple	1.7.25
org.threeten	threeten-extra	1.7.1
org.tukaani	xz	1.9
org.typelevel	algebra_2.12	2.0.1
org.typelevel	cats-kernel_2.12	2.1.1
org.typelevel	spire-macros_2.12	0.17.0
org.typelevel	spire-platform_2.12	0.17.0
org.typelevel	spire-util_2.12	0.17.0
org.typelevel	spire_2.12	0.17.0
org.wildfly.openssl	wildfly-openssl	1.1.3.Final
org.xerial	sqlite-jdbc	3.42.0.0
org.xerial.snappy	snappy-java	1.1.10.3
org.yaml	snakeyaml	2.0
oro	oro	2.0.8
pl.edu.icm	JLargeArrays	1.5
software.amazon.cryptools	AmazonCorrettoCryptoProvider	2.4.1-linux-x86_64
stax	stax-api	1.0.1

dica

To see release notes for Databricks Runtime versions that have reached end-of-support (EoS), see End-of-support Databricks Runtime release notes. The EoS Databricks Runtime versions have been retired and might not be updated.

Scala 2.13 migration guidance​

Breaking changes between Scala 2.12 and 2.13​

Major breaking changes​

Minor breaking changes​

DBR 16.4 LTS new and updated features​

Liquid clustering auto-compaction improvement​

Auto Loader can now clean processed files in the source directory​

Type widening support added for streaming from Delta tables​

IDENTIFIER support now available in DBSQL for catalog operations​

Collated expressions now provide autogenerated transient aliases​

Add filter pushdown API support to Python data sources​

Python UDF traceback improvement​

UNION/EXCEPT/INTERSECT inside a view and EXECUTE IMMEDIATE now return correct results​

Data source cached plan conf and migration guide​

Add flag to hide recursive Common Table Expressions (CTEs)​

New listagg and string_agg functions​

DBR 16.4 LTS behavioral changes​

Delta tables: "DESCRIBE DETAIL table" will now show the clusterByAuto status of the table​

Fix to respect options for data source cached plans​

Moved redaction rule from analyzer to optimizer​

variant_get and get_json_object now consider leading spaces in paths in Apache Spark​

Enable flag to disallow disabling source materialization for MERGE operations​

Move partition metadata log enablement anchor to table​

Update snowflake-jdbc 3.16.1 -> 3.22.0​

Downgrade Json4s from 4.0.7 to 3.7.0-M11 in DBR 16.2 and 16.3​

Library upgrades (applies to the Scala 2.12 image only)​

Apache Spark​

Databricks ODBC/JDBC driver support​

System environment​

Installed Python libraries​

Installed R libraries​

Installed Java and Scala libraries (Scala 2.13 cluster version)​

Installed Java and Scala libraries (Scala 2.12 cluster version)​