Databricks Runtime 17.3 LTS (Beta)

The following release notes provide information about Databricks Runtime 17.3 LTS (Beta), powered by Apache Spark 4.0.0.

Databricks released this LTS version in October 2025.

Beta

Databricks Runtime 17.3 LTS is in Beta. The contents of the supported environments might change during the Beta. Changes can include the list of packages or versions of installed packages.

New features and improvements

• Allow spark.sql.files.maxPartitionBytes in serverless compute
• Appending to files in Unity Catalog volumes returns correct error

Allow spark.sql.files.maxPartitionBytes in serverless compute

You can now configure spark.sql.files.maxPartitionBytes in serverless compute. This setting was already available in serverless SQL warehouses, and is now allowlisted for serverless compute as well. See Performance tuning.

Appending to files in Unity Catalog volumes returns correct error

Any code that attempts to append content to a Unity Catalog volume file now fails with an “Illegal seek” error message. Previously, it failed with an incorrect “operation not supported” error.

Behavioral changes

• Support MV/ST refresh information in DESCRIBE EXTENDED AS JSON
• Fix temporal value conversion in struct literals when using Connect mode
• Support for complex types in observations in the Spark Connect Scala client
• Preserve null values in array, map, and struct literals
• Default mode change for FSCK REPAIR TABLE command
• Preserve nullability for typed literals
• Fix case class conversion in array and map literals
• Improved handling of null structs in Parquet
• Upgrade aws-msk-iam-auth library for Kafka

Support MV/ST refresh information in DESCRIBE EXTENDED AS JSON

Databricks now generates a section for materialized view and streaming table refresh information in the DESCRIBE EXTENDED AS JSON output, including last refreshed time, refresh type, status, and schedule.

Fix temporal value conversion in struct literals when using Connect mode

In Spark Connect mode, struct literals in TypedLit now correctly handle temporal values instead of throwing exceptions.

Support for complex types in observations in the Spark Connect Scala client

The Spark Connect Scala client now supports array, map, and struct types in observations, bringing behavior in line with dedicated clusters.

Preserve null values in array, map, and struct literals

In Spark Connect mode, null values inside array, map, and struct literals are now preserved instead of being replaced with protobuf default values.

Default mode change for FSCK REPAIR TABLE command

The FSCK REPAIR TABLE command now includes an initial metadata repair step that validates checkpoints and partition values before removing references to missing data files.

Preserve nullability for typed literals

The Spark Connect Scala client now correctly preserves the nullability of array and map types for typed literals. Previously, array elements and map values were always treated as nullable.

Fix case class conversion in array and map literals

In Spark Connect mode, array and map literals in TypedLit now correctly handle case class values instead of throwing exceptions.

Improved handling of null structs in Parquet

Databricks now correctly detects null structs when all requested struct fields are missing from a Parquet file, which makes behavior consistent between Photon and non-Photon readers.

Upgrade aws-msk-iam-auth library for Kafka

The aws-msk-iam-auth library has been upgraded to version 2.0.1+ to support setting a regional Security Token Service (STS) endpoint for Managed Streaming for Apache Kafka (MSK) IAM authentication.

Library upgrades

Upgraded Python libraries:

No Python libraries were upgraded in this version.

Upgraded R libraries:

No R libraries were upgraded in this version.

Upgraded Java libraries:

No Java libraries were upgraded in this version.

Apache Spark

Databricks Runtime 17.3 (Beta) includes Apache Spark 4.0.0. This release includes all Spark fixes and improvements included in version 17.2, as well as the following additional bug fixes and improvements made to Spark:

• SPARK-53507 Add breaking change info to errors
• SPARK-53574 Fix AnalysisContext being wiped during nested plan resolution
• SPARK-53444 Rework execute immediate
• SPARK-53560 Crash looping when retrying uncommitted batch in Kafka source and AvailableNow trigger
• SPARK-53127 Enable LIMIT ALL to override recursion row limit
• SPARK-53574 Revert "[SC-206548] Fix AnalysisContext being wipe…
• SPARK-53674 Handle single-pass analyzer LCAs when assigning aliases
• SPARK-53492 Reject second ExecutePlan with an operation id that has already completed
• SPARK-53677 Improve debuggability for JDBC data source when query contains syntax error
• SPARK-53490 Fix Protobuf conversion in observed metrics
• SPARK-53671 Exclude 0-args from @udf eval type inference
• SPARK-53592 Rework "Make @udf support vectorized UDF"
• SPARK-53654 Rework "Support seed in function uuid"
• SPARK-53429 Support Direct Passthrough Partitioning in the PySpark Dataframe API
• SPARK-53372 Shared Testing Runtime Package for LDP
• SPARK-53574 Fix AnalysisContext being wiped during nested plan resolution
• SPARK-53654 Revert "[SC-207022][sql][PYTHON] Support seed in function uuid"
• SPARK-53574 Revert "[SC-206548] Fix AnalysisContext being wiped during nested plan resolution"
• SPARK-53559 Fix HLL sketch updates to use raw collation key bytes
• SPARK-53654 Support seed in function uuid
• SPARK-52449 Make datatypes for Expression.Literal.Map/Array optional
• SPARK-53574 Fix AnalysisContext being wiped during nested plan resolution
• SPARK-53625 Propagate metadata columns through projections to address ApplyCharTypePadding incompatibility
• SPARK-53558 Revert "[SC-206544][sql] Show fully qualified table name including the catalog name in the exception message when the table is not found"
• SPARK-53558 Show fully qualified table name including the catalog name in the exception message when the table is not found
• SPARK-53357 Update pandas to 2.3.2
• SPARK-52659Misleading modulo error message in ansi mode
• SPARK-53592 Revert "[SC-206971][python] Make @udf support vectorized UDF"
• SPARK-53592 Make @udf support vectorized UDF
• SPARK-52601 Support primitive types in TransformingEncoder
• SPARK-53355 fix numpy 1.x repr in type tests
• SPARK-53387 Add support for Arrow UDTFs with PARTITION BY
• SPARK-52991 Implement MERGE INTO with SCHEMA EVOLUTION for V2 Data Source
• SPARK-53568 Fix several small bugs in Spark Connect Python client error handling logic
• SPARK-43579 optim: Cache the converter between Arrow and pandas for reuse
• SPARK-53524 Fix temporal value conversion in LiteralValueProtoConverter
• SPARK-53600 Revise SessionHolder last access time log message
• SPARK-53529 Fix pyspark connect client to support IPv6
• SPARK-53537 Adding Support for Parsing CONTINUE HANDLER
• SPARK-53623 improve reading large table properties performance
• SPARK-53523 Named parameters respect spark.sql.caseSensitive
• SPARK-53449 Simply options for builtin Datasource Scan related classes
• SPARK-53620 SparkSubmit should print stacktrace when exitFn is called
• SPARK-53518 No truncation for catalogString of User Defined Type
• SPARK-53568 Revert "[SC-206538][connect][PYTHON] Fix several small bugs in Spark Connect Python client error handling logic"
• SPARK-53602 Profile dump improvement and profiler doc fix
• SPARK-53402 Support Direct Passthrough Partitioning Dataset API in Spark Connect in Scala
• SPARK-53491 Fix exponential formatting of inputRowsPerSecond and processedRowsPerSecond in progress metrics JSON
• SPARK-53413 Shuffle cleanup for commands
• SPARK-53518 Revert "[SC-205989][sql] No truncation for catalogString of User Defined Type"
• SPARK-53568 Fix several small bugs in Spark Connect Python client error handling logic
• SPARK-53552 Optimize substr SQL function
• SPARK-53527 Improve fallback of analyzeExistenceDefaultValue
• SPARK-53584 Improve process_column_param validation and column parameter docstring
• SPARK-53498 Correctly Reference pyspark/pipelines/cli.py from spark-pipelines Binary
• SPARK-53518 No truncation for catalogString of User Defined Type
• SPARK-53233 Make the code related to streaming uses the correct package name
• SPARK-53598 Check the existence of numParts before reading large table property
• SPARK-53372 SDP End to End Testing Suite
• SPARK-53563 Optimize: sql_processor by avoiding inefficient string concatenation
• SPARK-53323 Enable Spark Connect tests for df.asTable() in Arrow UDTF
• SPARK-53157 Decouple driver and executor polling intervals
• SPARK-53555 Fix: SparkML-connect can't load SparkML (legacy mode) saved model
• SPARK-53361 Optimizing JVM–Python Communication in TWS by Grouping Multiple Keys into One Arrow Batch
• SPARK-53233 Revert "[SC-203716][sql][SS][mllib][CONNECT] Make the code related to streaming uses the correct package name"
• SPARK-53233 Make the code related to streaming uses the correct package name
• SPARK-53525 Spark Connect ArrowBatch Result Chunking
• SPARK-53444 Revert "[SC-206536][sql] Rework execute immediate"
• SPARK-53594 Make arrow UDF respect user-specified eval type
• SPARK-53444 Rework execute immediate
• SPARK-53582 Extend isExtractable so it can be applied on UnresolvedExtractValue
• SPARK-53572 Avoid throwing from ExtractValue.isExtractable
• SPARK-52346 Fix counter initialization and increment logic for flow retries
• SPARK-53561 Catch Interruption Exception in TransformWithStateInPySparkStateServer during outputStream.flush to avoid the worker crash
• SPARK-53512 Better unification of DSv2 PushDownUtils
• SPARK-53459 Use ReadOnlySparkConf in more places
• SPARK-53549 Always close the arrow allocator when list state request process is completed
• SPARK-53332 Enable StateDataSource with state checkpoint v2 (only snapshotStartBatchId option)
• SPARK-53550 Union output partitioning should compare canonicalized attributes
• SPARK-53506 Disallow % between Decimal and float under ANSI
• SPARK-52238 Rename Pipeline Spec Field "definitions" to 'libraries'
• SPARK-53538 ExpandExec should initialize the unsafe projections
• SPARK-53521 Refactor Star expression
• SPARK-53358 Improve arrow Python UDTF output type mismatch error message
• SPARK-53531 Better error message for HadoopRDD.getInputFormat
• SPARK-53434 ColumnarRow's get should also check isNullAt
• SPARK-53008 Add OSS SQL UDF data access routine validation
• SPARK-44856 Improve Python UDTF arrow serializer performance
• SPARK-53488 Update CollationTypeCoercion to be compatible with single-pass Analyzer logic
• SPARK-53029 Support return type coercion for Arrow Python UDTFs
• SPARK-53479 Align == behavior with pandas when comparing against scalar under ANSI
• SPARK-53497 Disallow /|//|* between Decimal and float under ANSI
• SPARK-53441 Bools |/&/^ None should fail under ANSI
• SPARK-53474 Add conf DATA_SOURCE_V2_EXPR_FOLDING
• SPARK-53333 Enable StateDataSource with state checkpoint v2 (only readChangeFeed)
• SPARK-53502 Improve the naming of methods in LiteralValueProtoConverter
• SPARK-52694 Add o.a.s.sql.Encoders#udtAPI
• SPARK-53499 from_arrow_type should respect containsNull in ArrayType
• SPARK-53515 Remove unused private lazy val from SchemaOfCsv/Xml
• SPARK-53480 Refactor arrow python runner code
• SPARK-53481 Make Hybrid queue class
• SPARK-53401 Enable Direct Passthrough Partitioning in the DataFrame API
• SPARK-52930 Use DataType.Array/Map for Array/Map Literals
• SPARK-52346 Propagate partition columns from destination for BatchTableWrite
• SPARK-53342 Fix Arrow converter to handle multiple record batches in single IPC stream
• SPARK-53421 Propagate Logical Plan ID in SDP Analysis
• SPARK-53408 Remove unused functions from QueryCompilationErrors
• SPARK-53044 Change Declarative Pipelines import alias convention from "sdp" to "dp"
• SPARK-53450 Nulls are filled unexpectedly after converting hive table scan to logical relation
• SPARK-53290 Fix Metadata backward-compatibility breaking
• SPARK-52982 Disallow lateral join with Arrow Python UDTFs
• SPARK-52851 Remove public APIs for append once flows
• SPARK-52511 Support dry-run mode in spark-pipelines command
• SPARK-53290 Revert "[SC-204277][sql][CONNECT] Fix Metadata backward-compatibility breaking"
• SPARK-53290 Fix Metadata backward-compatibility breaking
• SPARK-52577 Add tests for Declarative Pipelines DatasetManager with Hive catalog
• SPARK-53012 Support Arrow Python UDTF in Spark Connect
• SPARK-53251 Enable DataFrame API testing with asTable() for Arrow Python UDTFs
• SPARK-52432 Scope DataflowGraphRegistry to Session
• SPARK-52853 Prevent imperative PySpark methods in declarative pipelines
• SPARK-52745 Ensure one of the schema and columns in the Table interface is implemented and columns is preferable
• SPARK-52991 Revert "[SC-204944][sql] Implement MERGE INTO with SCHEMA EVOLUTION for V2 Data Source"
• SPARK-52981 Add table argument support for Arrow Python UDTFs
• SPARK-53487 Improve comments in NormalizePlan.scala
• SPARK-52991 Implement MERGE INTO with SCHEMA EVOLUTION for V2 Data Source
• SPARK-52281 Change ALTER TABLE ALTER COLUMN TYPE STRING not to apply default collation if original data type was instance of StringType
• SPARK-52759 Throw exception if pipeline has no tables or persisted views
• SPARK-51575 Combine Python Data Source pushdown & plan read workers
• SPARK-53359 Fix Arrow UDTF to handle the results as iterator
• SPARK-52757 Rename "plan" field in DefineFlow to "relation"
• SPARK-52431 Finishing touches on Declarative Pipelines runner
• SPARK-52591 Validate streaming-ness of DFs returned by SDP table and standalone flow definitions
• SPARK-53030 Support Arrow writer for streaming Python data sources
• SPARK-52980 Support Arrow Python UDTFs
• SPARK-42841Assign a name to the error class _LEGACY_ERROR_TEMP_2005
• SPARK-52423 Add missing golang package name to pipelines.proto
• SPARK-52409 Only use PipelineRunEventBuffer in tests
• SPARK-52533 Support enabling only driver profiler
• SPARK-52716 Remove comment from Flow trait and references
• SPARK-52348 Add support for Spark Connect handlers for pipeline commands
• SPARK-52281 Revert "[SC-198058][sql] Change ALTER TABLE ALTER COLUMN TYPE STRING not to apply default collation if original data type was instance of StringType"
• SPARK-49386 Add memory based thresholds for shuffle spill
• SPARK-52281 Change ALTER TABLE ALTER COLUMN TYPE STRING not to apply default collation if original data type was instance of StringType
• SPARK-53329 Improve exception handling when adding artifacts
• SPARK-52772 Revert "[SC-202707][engrm-327][SQL] Inconsistent table attribute handling during updates"
• SPARK-52772 Inconsistent table attribute handling during updates
• SPARK-51739 Validate Arrow schema from mapInArrow & mapInPandas & DataSource
• SPARK-53355 test python udf type behavior
• SPARK-53443 Update python to SPDX-license
• SPARK-49968 The split function produces incorrect results with an empty regex and a limit
• SPARK-52582 Improve the memory usage of XML parser
• SPARK-53452 from_arrow_type should respect valueContainsNull
• SPARK-53103 Revert "[SC-204946][ss] Throw an error if state directory is not empty when query starts"
• SPARK-53103 Throw an error if state directory is not empty when query starts
• SPARK-51575 Revert "[SC-192406][python] Combine Python Data Source pushdown & plan read workers"
• SPARK-51575 Combine Python Data Source pushdown & plan read workers
• SPARK-53095 Support HMS v4.1
• SPARK-53126 Refactor SparkErrorUtils#stackTraceToString to handle null input and use it to replace ExceptionUtils#getStackTrace
• SPARK-49489 HMS client respects hive.thrift.client.maxmessage.size
• SPARK-53108 Implement the time_diff function in Scala
• SPARK-45265 Support Hive 4.0 metastore
• SPARK-53255 Ban org.apache.parquet.Preconditions
• SPARK-52142 Display table constraints in SHOW CREATE TABLE COMMAND
• SPARK-52875 Simplify V2 expression translation if the input is context-independent-foldable
• SPARK-53240 Ban com.google.common.collect.(ArrayList)?Multimap
• SPARK-52875 Revert "[SC-201880][sql] Simplify V2 expression translation if the input is context-independent-foldable"
• SPARK-52141 Display constraints in DESC commands
• SPARK-53386 Support query parameter ending with semicolon in JDBC connectors
• SPARK-52875 Simplify V2 expression translation if the input is context-independent-foldable
• SPARK-53437 InterpretedUnsafeProjection shall setNull4Bytes for YearMonthIntervalType field
• SPARK-53341 Expand golden test coverage on multivariable DECLARE
• SPARK-53156 Track Driver Memory Metrics when the Application ends
• SPARK-52689 Send DML Metrics to V2Write
• SPARK-52641 Primary key columns should not be nullable
• SPARK-48547 Add opt-in flag to have SparkSubmit automatically call System.exit after user code main method exits
• SPARK-52575 Introduce contextIndependentFoldable attribute for Expressions
• SPARK-52860 Support V2 write schema evolution in InMemoryTable
• SPARK-53435 Fix race condition in CachedRDDBuilder
• SPARK-51987 DSv2 expressions in column defaults on write
• SPARK-52235 Add implicit cast to DefaultValue V2 Expressions passed to DSV2
• SPARK-52236 Standardize analyze exceptions for default value
• SPARK-53294 Enable StateDataSource with state checkpoint v2 (only batchId option)
• SPARK-52095 Alter table alter column to pass V2Expression to DSV2
• SPARK-53415 Simply options for builtin FileFormats
• SPARK-53318 Support the time type by make_timestamp_ltz()
• SPARK-53143 Fix self join in DataFrame API - Join is not the only expected output from analyzer
• SPARK-53423 Move all the single-pass resolver related tags to ResolverTag
• SPARK-52116 Improve exception for non-deterministic default values
• SPARK-51906 Dsv2 expressions in alter table add columns
• SPARK-53403 Improve add/sub tests under ANSI
• SPARK-53418 Support TimeType in ColumnAccessor
• SPARK-52777 Enable shuffle cleanup mode configuration in Spark SQL
• SPARK-52485 Declarative Pipelines code cleanup
• SPARK-53236 Use Java ArrayList constructors instead of Lists.newArrayList in Java code
• SPARK-53392 Move SpecializedArray handling to connect-common
• SPARK-53176 Spark launcher should respect --load-spark-defaults
• SPARK-53237 Use Java Base64 instead of org.apache.commons.codec.binary.Base64 instance
• SPARK-53390 Raise error when bools with None astype to ints under ANSI
• SPARK-53109 Support TIME in the make_timestamp_ntz and try_make_timestamp_ntz functions in Scala
• SPARK-53393 Disable memory profiler for Arrow Scalar Iterator UDFs
• SPARK-53367 add int to decimal coercion for Arrow UDFs
• SPARK-53365 Unify code for persisting of configs in views and UDFs
• SPARK-53228 Use Java Map constructors instead of Maps.new*HashMap()
• SPARK-53197 Use java.util.Objects#requireNonNull instead of com.google.common.base.Preconditions#checkNotNull
• SPARK-53216 Move is*(Blank|Empty) from object SparkStringUtils to trait SparkStringUtils
• SPARK-53385 Refactor Identifier evaluation out
• SPARK-53384 Refactor variable resolution out
• SPARK-53195 Use Java InputStream.readNBytes instead of ByteStreams.read
• SPARK-53205 Support createParentDirs in SparkFileUtils
• SPARK-53196 Use Java OutputStream.nullOutputStream instead of ByteStreams.nullOutputStream
• SPARK-53381 Avoid creating temporary collections in toCatalystStruct
• SPARK-53275 Handle stateful expressions when ordering in interpreted mode
• SPARK-51585 Oracle dialect supports pushdown datetime functions
• SPARK-53200 Use Java Files.newInputStream instead of Files.asByteSource().openStream()
• SPARK-53190 Use Java InputStream.transferTo instead of ByteStreams.copy
• SPARK-53188 Support readFully in SparkStreamUtils and JavaUtils
• SPARK-53354 Simplify LiteralValueProtoConverter.toCatalystStruct
• SPARK-52873 Further restrict when SHJ semi/anti join can ignore duplicate keys on the build side
• SPARK-53180 Use Java InputStream.skipNBytes instead of ByteStreams.skipFully
• SPARK-53369 Fix error message for UDFs with CHAR/VARCHAR return type
• SPARK-53330 Fix Arrow UDF with DayTimeIntervalType (bounds != start/end)
• SPARK-53366 Apply formatting rules to sql/connect/shims
• SPARK-53348 Always persist ANSI value when creating a view or assume it when querying if not stored
• SPARK-53119 Support touch in SparkFileUtils
• SPARK-52592 Support creating a ps.Series from a ps.Series
• SPARK-53360 Once strategy with ConstantFolding's idempotence should not be broken
• SPARK-53135 Support copyURLToFile in SparkFileUtils and JavaUtils
• SPARK-53150 Improve list(File|Path)s to handle non-existent, non-directory, symlink input
• SPARK-53135 Revert "[SC-203164][core][SQL] Support copyURLToFile in SparkFileUtils and JavaUtils"
• SPARK-53137 Support forceDeleteOnExit in SparkFileUtils and JavaUtils
• SPARK-53352 Refine the error message for unsupported return type
• SPARK-53353 Fail Scalar Iterator Arrow UDF with 0-arg
• SPARK-53135 Support copyURLToFile in SparkFileUtils and JavaUtils
• SPARK-53101 Support (left|right)Pad in SparkStringUtils
• SPARK-53117 Support moveDirectory in SparkFileUtils and JavaUtils
• SPARK-53121 Use deleteRecursively instead of FileUtils.forceDelete
• SPARK-53100 Use Java String.substring instead of StringUtils.substring
• SPARK-53092 Ban org.apache.commons.lang3.SystemUtils
• SPARK-53328 Improve debuggability for SparkML-connect
• SPARK-52065 Produce another plan tree with output columns (name, data type, nullability) in plan change logging
• SPARK-51182 DataFrameWriter should throw dataPathNotSpecifiedError when path is not specified
• SPARK-52410 Deprecate PipelineConf and use SqlConf directly
• SPARK-52852 Remove unused spark_conf in create_streaming_table
• SPARK-52714 Remove unused comment arg in append_flow decorator
• SPARK-52663 Introduce name field to pipeline spec
• SPARK-53091 Ban org.apache.commons.lang3.Strings
• SPARK-52393 Pipeline SQL Graph Element Registration
• SPARK-52912 Improve SparkStringUtils to support is(Not)?(Blank|Empty)
• SPARK-53307 Remove RetriesExceeded error from Spark Connect Python and Scala clients
• SPARK-52346 Declarative Pipeline DataflowGraph execution and event logging
• SPARK-52877 Improve Python UDF Arrow Serializer Performance
• SPARK-51920 Fix composite/nested type in value state for python TWS
• SPARK-52741 RemoveFiles ShuffleCleanup mode doesnt work with non-adaptive execution
• SPARK-52238 Python client for Declarative Pipelines
• SPARK-51926 Add INVALID_CONF_VALUE.subXXX error class to conf errors
• SPARK-53155 Global lower agggregation should not be replaced with a project
• SPARK-52911 Remove StringUtils.(split|chop) usage
• SPARK-51926 Revert "[SC-195096][core][SQL] Add INVALID_CONF_VALUE.subXXX error class to conf errors"
• SPARK-53346 Avoid creating temporary collections in toCatalystArray/toCatalystMap
• SPARK-51926 Add INVALID_CONF_VALUE.subXXX error class to conf errors
• SPARK-53136 tryWithResource & tryInitializeResource shall close resource quietly
• SPARK-53132 Support list(File|Path)s in SparkFileUtils and JavaUtils
• SPARK-51896 Add Java Enum Support for TypedConfigBuilder
• SPARK-53311 Make PullOutNonDeterministic use canonicalized expressions
• SPARK-51527 Make codegen log level configurable via SQLConf
• SPARK-52223 Add SDP Spark Connect Protos
• SPARK-52283 Declarative Pipelines DataflowGraph creation and resolution
• SPARK-52880 Improve toString by JEP-280 instead of ToStringBuilder
• SPARK-53284 Adjust imports of Spark config in tests
• SPARK-53050 Enable MultiIndex.to_series() to return struct for each entry
• SPARK-52988 Fix race conditions at CREATE TABLE and FUNCTION when IF NOT EXISTS is used
• SPARK-52874 Support o.a.s.util.Pair Java Record
• SPARK-52710 DESCRIBE SCHEMA should print collation
• SPARK-49984 Fix supplementJava(Module|IPv6)Options to update extraJavaOptions only

Databricks ODBC/JDBC driver support

Databricks supports ODBC/JDBC drivers released in the past 2 years. Please download the recently released drivers and upgrade (download ODBC, download JDBC).

Maintenance updates

See Databricks Runtime maintenance updates.

System environment

• Operating System: Ubuntu 24.04.3 LTS
• Java: Zulu17.58+21-CA
• Scala: 2.13.16
• Python: 3.12.3
• R: 4.4.2
• Delta Lake: 4.0.0

Installed Python libraries

Library	Version	Library	Version	Library	Version
annotated-types	0.7.0	anyio	4.6.2	argon2-cffi	21.3.0
argon2-cffi-bindings	21.2.0	arrow	1.3.0	asttokens	2.0.5
astunparse	1.6.3	async-lru	2.0.4	attrs	24.3.0
autocommand	2.2.2	azure-common	1.1.28	azure-core	1.34.0
azure-identity	1.20.0	azure-mgmt-core	1.5.0	azure-mgmt-web	8.0.0
azure-storage-blob	12.23.0	azure-storage-file-datalake	12.17.0	babel	2.16.0
backports.tarfile	1.2.0	beautifulsoup4	4.12.3	black	24.10.0
bleach	6.2.0	blinker	1.7.0	boto3	1.36.2
botocore	1.36.3	cachetools	5.5.1	certifi	2025.1.31
cffi	1.17.1	chardet	4.0.0	charset-normalizer	3.3.2
click	8.1.7	cloudpickle	3.0.0	comm	0.2.1
contourpy	1.3.1	cryptography	43.0.3	cycler	0.11.0
Cython	3.0.12	databricks-sdk	0.49.0	dbus-python	1.3.2
debugpy	1.8.11	decorator	5.1.1	defusedxml	0.7.1
Deprecated	1.2.13	distlib	0.3.9	docstring-to-markdown	0.11
executing	0.8.3	facets-overview	1.1.1	fastapi	0.115.12
fastjsonschema	2.21.1	filelock	3.18.0	fonttools	4.55.3
fqdn	1.5.1	fsspec	2023.5.0	gitdb	4.0.11
GitPython	3.1.43	google-api-core	2.20.0	google-auth	2.40.0
google-cloud-core	2.4.3	google-cloud-storage	3.1.0	google-crc32c	1.7.1
google-resumable-media	2.7.2	googleapis-common-protos	1.65.0	grpcio	1.67.0
grpcio-status	1.67.0	h11	0.14.0	httpcore	1.0.2
httplib2	0.20.4	httpx	0.27.0	idna	3.7
importlib-metadata	6.6.0	importlib_resources	6.4.0	inflect	7.3.1
iniconfig	1.1.1	ipyflow-core	0.0.209	ipykernel	6.29.5
ipython	8.30.0	ipython-genutils	0.2.0	ipywidgets	7.8.1
isodate	0.6.1	isoduration	20.11.0	jaraco.context	5.3.0
jaraco.functools	4.0.1	jaraco.text	3.12.1	jedi	0.19.2
Jinja2	3.1.5	jmespath	1.0.1	joblib	1.4.2
json5	0.9.25	jsonpointer	3.0.0	jsonschema	4.23.0
jsonschema-specifications	2023.7.1	jupyter-events	0.10.0	jupyter-lsp	2.2.0
jupyter_client	8.6.3	jupyter_core	5.7.2	jupyter_server	2.14.1
jupyter_server_terminals	0.4.4	jupyterlab	4.3.4	jupyterlab-pygments	0.1.2
jupyterlab-widgets	1.0.0	jupyterlab_server	2.27.3	kiwisolver	1.4.8
launchpadlib	1.11.0	lazr.restfulclient	0.14.6	lazr.uri	1.0.6
markdown-it-py	2.2.0	MarkupSafe	3.0.2	matplotlib	3.10.0
matplotlib-inline	0.1.7	mccabe	0.7.0	mdurl	0.1.0
mistune	2.0.4	mlflow-skinny	3.0.1	mmh3	5.1.0
more-itertools	10.3.0	msal	1.32.3	msal-extensions	1.3.1
mypy-extensions	1.0.0	nbclient	0.8.0	nbconvert	7.16.4
nbformat	5.10.4	nest-asyncio	1.6.0	nodeenv	1.9.1
notebook	7.3.2	notebook_shim	0.2.3	numpy	2.1.3
oauthlib	3.2.2	opentelemetry-api	1.32.1	opentelemetry-sdk	1.32.1
opentelemetry-semantic-conventions	0.53b1	overrides	7.4.0	packaging	24.2
pandas	2.2.3	pandocfilters	1.5.0	parso	0.8.4
pathspec	0.10.3	patsy	1.0.1	pexpect	4.8.0
pillow	11.1.0	pip	25.0.1	platformdirs	3.10.0
plotly	5.24.1	pluggy	1.5.0	prometheus_client	0.21.0
prompt-toolkit	3.0.43	proto-plus	1.26.1	protobuf	5.29.4
psutil	5.9.0	psycopg2	2.9.3	ptyprocess	0.7.0
pure-eval	0.2.2	pyarrow	19.0.1	pyasn1	0.4.8
pyasn1-modules	0.2.8	pyccolo	0.0.71	pycparser	2.21
pydantic	2.10.6	pydantic_core	2.27.2	pyflakes	3.2.0
Pygments	2.15.1	PyGObject	3.48.2	pyiceberg	0.9.0
PyJWT	2.10.1	pyodbc	5.2.0	pyparsing	3.2.0
pyright	1.1.394	pytest	8.3.5	python-dateutil	2.9.0.post0
python-json-logger	3.2.1	python-lsp-jsonrpc	1.1.2	python-lsp-server	1.12.0
pytoolconfig	1.2.6	pytz	2024.1	PyYAML	6.0.2
pyzmq	26.2.0	referencing	0.30.2	requests	2.32.3
rfc3339-validator	0.1.4	rfc3986-validator	0.1.1	rich	13.9.4
rope	1.12.0	rpds-py	0.22.3	rsa	4.9.1
s3transfer	0.11.3	scikit-learn	1.6.1	scipy	1.15.1
seaborn	0.13.2	Send2Trash	1.8.2	setuptools	74.0.0
six	1.16.0	smmap	5.0.0	sniffio	1.3.0
sortedcontainers	2.4.0	soupsieve	2.5	sqlparse	0.5.3
ssh-import-id	5.11	stack-data	0.2.0	starlette	0.46.2
statsmodels	0.14.4	strictyaml	1.7.3	tenacity	9.0.0
terminado	0.17.1	threadpoolctl	3.5.0	tinycss2	1.4.0
tokenize_rt	6.1.0	tomli	2.0.1	tornado	6.4.2
traitlets	5.14.3	typeguard	4.3.0	types-python-dateutil	2.9.0.20241206
typing_extensions	4.12.2	tzdata	2024.1	ujson	5.10.0
unattended-upgrades	0.1	uri-template	1.3.0	urllib3	2.3.0
uvicorn	0.34.2	virtualenv	20.29.3	wadllib	1.3.6
wcwidth	0.2.5	webcolors	24.11.1	webencodings	0.5.1
websocket-client	1.8.0	whatthepatch	1.0.2	wheel	0.45.1
widgetsnbextension	3.6.6	wrapt	1.17.0	yapf	0.40.2
zipp	3.21.0

Installed R libraries

R libraries are installed from the Posit Package Manager CRAN snapshot on 2025-03-20.

Library	Version	Library	Version	Library	Version
arrow	19.0.1	askpass	1.2.1	assertthat	0.2.1
backports	1.5.0	base	4.4.2	base64enc	0.1-3
bigD	0.3.0	bit	4.6.0	bit64	4.6.0-1
bitops	1.0-9	blob	1.2.4	boot	1.3-30
brew	1.0-10	brio	1.1.5	broom	1.0.7
bslib	0.9.0	cachem	1.1.0	callr	3.7.6
caret	7.0-1	cellranger	1.1.0	chron	2.3-62
class	7.3-22	cli	3.6.5	clipr	0.8.0
clock	0.7.2	cluster	2.1.6	codetools	0.2-20
colorspace	2.1-1	commonmark	1.9.5	compiler	4.4.2
config	0.3.2	conflicted	1.2.0	cpp11	0.5.2
crayon	1.5.3	credentials	2.0.2	curl	6.4.0
data.table	1.17.0	datasets	4.4.2	DBI	1.2.3
dbplyr	2.5.0	desc	1.4.3	devtools	2.4.5
diagram	1.6.5	diffobj	0.3.5	digest	0.6.37
downlit	0.4.4	dplyr	1.1.4	dtplyr	1.3.1
e1071	1.7-16	ellipsis	0.3.2	evaluate	1.0.3
fansi	1.0.6	farver	2.1.2	fastmap	1.2.0
fontawesome	0.5.3	forcats	1.0.0	foreach	1.5.2
foreign	0.8-86	forge	0.2.0	fs	1.6.5
future	1.34.0	future.apply	1.11.3	gargle	1.5.2
generics	0.1.4	gert	2.1.4	ggplot2	3.5.1
gh	1.4.1	git2r	0.35.0	gitcreds	0.1.2
glmnet	4.1-8	globals	0.18.0	glue	1.8.0
googledrive	2.1.1	googlesheets4	1.1.1	gower	1.0.2
graphics	4.4.2	grDevices	4.4.2	grid	4.4.2
gridExtra	2.3	gsubfn	0.7	gt	0.11.1
gtable	0.3.6	hardhat	1.4.1	haven	2.5.4
highr	0.11	hms	1.1.3	htmltools	0.5.8.1
htmlwidgets	1.6.4	httpuv	1.6.15	httr	1.4.7
httr2	1.1.1	ids	1.0.1	ini	0.3.1
ipred	0.9-15	isoband	0.2.7	iterators	1.0.14
jquerylib	0.1.4	jsonlite	1.9.1	juicyjuice	0.1.0
KernSmooth	2.23-22	knitr	1.50	labeling	0.4.3
later	1.4.1	lattice	0.22-5	lava	1.8.1
lifecycle	1.0.4	listenv	0.9.1	lubridate	1.9.4
magrittr	2.0.3	markdown	1.13	MASS	7.3-60.0.1
Matrix	1.6-5	memoise	2.0.1	methods	4.4.2
mgcv	1.9-1	mime	0.13	miniUI	0.1.1.1
mlflow	2.20.4	ModelMetrics	1.2.2.2	modelr	0.1.11
munsell	0.5.1	nlme	3.1-164	nnet	7.3-19
numDeriv	2016.8-1.1	openssl	2.3.3	parallel	4.4.2
parallelly	1.42.0	pillar	1.11.0	pkgbuild	1.4.6
pkgconfig	2.0.3	pkgdown	2.1.1	pkgload	1.4.0
plogr	0.2.0	plyr	1.8.9	praise	1.0.0
prettyunits	1.2.0	pROC	1.18.5	processx	3.8.6
prodlim	2024.06.25	profvis	0.4.0	progress	1.2.3
progressr	0.15.1	promises	1.3.2	proto	1.0.0
proxy	0.4-27	ps	1.9.0	purrr	1.0.4
R6	2.6.1	ragg	1.3.3	randomForest	4.7-1.2
rappdirs	0.3.3	rcmdcheck	1.4.0	RColorBrewer	1.1-3
Rcpp	1.0.14	RcppEigen	0.3.4.0.2	reactable	0.4.4
reactR	0.6.1	readr	2.1.5	readxl	1.4.5
recipes	1.2.0	rematch	2.0.0	rematch2	2.1.2
remotes	2.5.0	reprex	2.1.1	reshape2	1.4.4
rlang	1.1.6	rmarkdown	2.29	RODBC	1.3-26
roxygen2	7.3.2	rpart	4.1.23	rprojroot	2.0.4
Rserve	1.8-15	RSQLite	2.3.9	rstudioapi	0.17.1
rversions	2.1.2	rvest	1.0.4	sass	0.4.9
scales	1.3.0	selectr	0.4-2	sessioninfo	1.2.3
shape	1.4.6.1	shiny	1.10.0	sourcetools	0.1.7-1
sparklyr	1.9.1	SparkR	4.0.0	sparsevctrs	0.3.1
spatial	7.3-17	splines	4.4.2	sqldf	0.4-11
SQUAREM	2021.1	stats	4.4.2	stats4	4.4.2
stringi	1.8.7	stringr	1.5.1	survival	3.5-8
swagger	5.17.14.1	sys	3.4.3	systemfonts	1.2.1
tcltk	4.4.2	testthat	3.2.3	textshaping	1.0.0
tibble	3.3.0	tidyr	1.3.1	tidyselect	1.2.1
tidyverse	2.0.0	timechange	0.3.0	timeDate	4041.110
tinytex	0.56	tools	4.4.2	tzdb	0.5.0
urlchecker	1.0.1	usethis	3.1.0	utf8	1.2.6
utils	4.4.2	uuid	1.2-1	V8	6.0.2
vctrs	0.6.5	viridisLite	0.4.2	vroom	1.6.5
waldo	0.6.1	whisker	0.4.1	withr	3.0.2
xfun	0.51	xml2	1.3.8	xopen	1.0.1
xtable	1.8-4	yaml	2.3.10	zeallot	0.1.0
zip	2.3.2

Installed Java and Scala libraries (Scala 2.13 cluster version)

Group ID	Artifact ID	Version
antlr	antlr	2.7.7
com.amazonaws	amazon-kinesis-client	1.12.0
com.amazonaws	aws-java-sdk-autoscaling	1.12.638
com.amazonaws	aws-java-sdk-cloudformation	1.12.638
com.amazonaws	aws-java-sdk-cloudfront	1.12.638
com.amazonaws	aws-java-sdk-cloudhsm	1.12.638
com.amazonaws	aws-java-sdk-cloudsearch	1.12.638
com.amazonaws	aws-java-sdk-cloudtrail	1.12.638
com.amazonaws	aws-java-sdk-cloudwatch	1.12.638
com.amazonaws	aws-java-sdk-cloudwatchmetrics	1.12.638
com.amazonaws	aws-java-sdk-codedeploy	1.12.638
com.amazonaws	aws-java-sdk-cognitoidentity	1.12.638
com.amazonaws	aws-java-sdk-cognitosync	1.12.638
com.amazonaws	aws-java-sdk-config	1.12.638
com.amazonaws	aws-java-sdk-core	1.12.638
com.amazonaws	aws-java-sdk-datapipeline	1.12.638
com.amazonaws	aws-java-sdk-directconnect	1.12.638
com.amazonaws	aws-java-sdk-directory	1.12.638
com.amazonaws	aws-java-sdk-dynamodb	1.12.638
com.amazonaws	aws-java-sdk-ec2	1.12.638
com.amazonaws	aws-java-sdk-ecs	1.12.638
com.amazonaws	aws-java-sdk-efs	1.12.638
com.amazonaws	aws-java-sdk-elasticache	1.12.638
com.amazonaws	aws-java-sdk-elasticbeanstalk	1.12.638
com.amazonaws	aws-java-sdk-elasticloadbalancing	1.12.638
com.amazonaws	aws-java-sdk-elastictranscoder	1.12.638
com.amazonaws	aws-java-sdk-emr	1.12.638
com.amazonaws	aws-java-sdk-glacier	1.12.638
com.amazonaws	aws-java-sdk-glue	1.12.638
com.amazonaws	aws-java-sdk-iam	1.12.638
com.amazonaws	aws-java-sdk-importexport	1.12.638
com.amazonaws	aws-java-sdk-kinesis	1.12.638
com.amazonaws	aws-java-sdk-kms	1.12.638
com.amazonaws	aws-java-sdk-lambda	1.12.638
com.amazonaws	aws-java-sdk-logs	1.12.638
com.amazonaws	aws-java-sdk-machinelearning	1.12.638
com.amazonaws	aws-java-sdk-opsworks	1.12.638
com.amazonaws	aws-java-sdk-rds	1.12.638
com.amazonaws	aws-java-sdk-redshift	1.12.638
com.amazonaws	aws-java-sdk-route53	1.12.638
com.amazonaws	aws-java-sdk-s3	1.12.638
com.amazonaws	aws-java-sdk-ses	1.12.638
com.amazonaws	aws-java-sdk-simpledb	1.12.638
com.amazonaws	aws-java-sdk-simpleworkflow	1.12.638
com.amazonaws	aws-java-sdk-sns	1.12.638
com.amazonaws	aws-java-sdk-sqs	1.12.638
com.amazonaws	aws-java-sdk-ssm	1.12.638
com.amazonaws	aws-java-sdk-storagegateway	1.12.638
com.amazonaws	aws-java-sdk-sts	1.12.638
com.amazonaws	aws-java-sdk-support	1.12.638
com.amazonaws	aws-java-sdk-swf-libraries	1.11.22
com.amazonaws	aws-java-sdk-workspaces	1.12.638
com.amazonaws	jmespath-java	1.12.638
com.clearspring.analytics	stream	2.9.8
com.databricks	Rserve	1.8-3
com.databricks	databricks-sdk-java	0.27.0
com.databricks	jets3t	0.7.1-0
com.databricks.scalapb	scalapb-runtime_2.13	0.4.15-11
com.esotericsoftware	kryo-shaded	4.0.3
com.esotericsoftware	minlog	1.3.0
com.fasterxml	classmate	1.5.1
com.fasterxml.jackson.core	jackson-annotations	2.18.2
com.fasterxml.jackson.core	jackson-core	2.18.2
com.fasterxml.jackson.core	jackson-databind	2.18.2
com.fasterxml.jackson.dataformat	jackson-dataformat-cbor	2.18.2
com.fasterxml.jackson.dataformat	jackson-dataformat-yaml	2.15.2
com.fasterxml.jackson.datatype	jackson-datatype-joda	2.18.2
com.fasterxml.jackson.datatype	jackson-datatype-jsr310	2.18.2
com.fasterxml.jackson.module	jackson-module-paranamer	2.18.2
com.fasterxml.jackson.module	jackson-module-scala_2.13	2.18.2
com.github.ben-manes.caffeine	caffeine	2.9.3
com.github.blemale	scaffeine_2.13	4.1.0
com.github.fommil	jniloader	1.1
com.github.fommil.netlib	native_ref-java	1.1
com.github.fommil.netlib	native_ref-java	1.1-natives
com.github.fommil.netlib	native_system-java	1.1
com.github.fommil.netlib	native_system-java	1.1-natives
com.github.fommil.netlib	netlib-native_ref-linux-x86_64	1.1-natives
com.github.fommil.netlib	netlib-native_system-linux-x86_64	1.1-natives
com.github.luben	zstd-jni	1.5.6-10
com.github.virtuald	curvesapi	1.08
com.github.wendykierp	JTransforms	3.1
com.google.api.grpc	proto-google-common-protos	2.5.1
com.google.code.findbugs	jsr305	3.0.0
com.google.code.gson	gson	2.11.0
com.google.crypto.tink	tink	1.16.0
com.google.errorprone	error_prone_annotations	2.36.0
com.google.flatbuffers	flatbuffers-java	24.3.25
com.google.guava	failureaccess	1.0.2
com.google.guava	guava	33.4.0-jre
com.google.guava	listenablefuture	9999.0-empty-to-avoid-conflict-with-guava
com.google.j2objc	j2objc-annotations	3.0.0
com.google.protobuf	protobuf-java	3.25.5
com.google.protobuf	protobuf-java-util	3.25.5
com.helger	profiler	1.1.1
com.ibm.icu	icu4j	75.1
com.jcraft	jsch	0.1.55
com.lihaoyi	sourcecode_2.13	0.1.9
com.microsoft.azure	azure-data-lake-store-sdk	2.3.10
com.microsoft.sqlserver	mssql-jdbc	11.2.2.jre8
com.microsoft.sqlserver	mssql-jdbc	11.2.3.jre8
com.ning	compress-lzf	1.1.2
com.sun.mail	javax.mail	1.5.2
com.sun.xml.bind	jaxb-core	2.2.11
com.sun.xml.bind	jaxb-impl	2.2.11
com.tdunning	json	1.8
com.thoughtworks.paranamer	paranamer	2.8
com.trueaccord.lenses	lenses_2.13	0.4.13
com.twitter	chill-java	0.10.0
com.twitter	chill_2.13	0.10.0
com.twitter	util-app_2.13	19.8.1
com.twitter	util-core_2.13	19.8.1
com.twitter	util-function_2.13	19.8.1
com.twitter	util-jvm_2.13	19.8.1
com.twitter	util-lint_2.13	19.8.1
com.twitter	util-registry_2.13	19.8.1
com.twitter	util-stats_2.13	19.8.1
com.typesafe	config	1.4.3
com.typesafe.scala-logging	scala-logging_2.13	3.9.2
com.uber	h3	3.7.3
com.univocity	univocity-parsers	2.9.1
com.zaxxer	HikariCP	4.0.3
com.zaxxer	SparseBitSet	1.3
commons-cli	commons-cli	1.9.0
commons-codec	commons-codec	1.17.2
commons-collections	commons-collections	3.2.2
commons-dbcp	commons-dbcp	1.4
commons-fileupload	commons-fileupload	1.5
commons-httpclient	commons-httpclient	3.1
commons-io	commons-io	2.18.0
commons-lang	commons-lang	2.6
commons-logging	commons-logging	1.1.3
commons-pool	commons-pool	1.5.4
dev.ludovic.netlib	arpack	3.0.3
dev.ludovic.netlib	blas	3.0.3
dev.ludovic.netlib	lapack	3.0.3
info.ganglia.gmetric4j	gmetric4j	1.0.10
io.airlift	aircompressor	2.0.2
io.delta	delta-sharing-client_2.13	1.3.5
io.dropwizard.metrics	metrics-annotation	4.2.30
io.dropwizard.metrics	metrics-core	4.2.30
io.dropwizard.metrics	metrics-graphite	4.2.30
io.dropwizard.metrics	metrics-healthchecks	4.2.30
io.dropwizard.metrics	metrics-jetty9	4.2.30
io.dropwizard.metrics	metrics-jmx	4.2.30
io.dropwizard.metrics	metrics-json	4.2.30
io.dropwizard.metrics	metrics-jvm	4.2.30
io.dropwizard.metrics	metrics-servlets	4.2.30
io.github.java-diff-utils	java-diff-utils	4.15
io.netty	netty-all	4.1.118.Final
io.netty	netty-buffer	4.1.118.Final
io.netty	netty-codec	4.1.118.Final
io.netty	netty-codec-http	4.1.118.Final
io.netty	netty-codec-http2	4.1.118.Final
io.netty	netty-codec-socks	4.1.118.Final
io.netty	netty-common	4.1.118.Final
io.netty	netty-handler	4.1.118.Final
io.netty	netty-handler-proxy	4.1.118.Final
io.netty	netty-resolver	4.1.118.Final
io.netty	netty-tcnative-boringssl-static	2.0.70.Final-db-r0-linux-aarch_64
io.netty	netty-tcnative-boringssl-static	2.0.70.Final-db-r0-linux-x86_64
io.netty	netty-tcnative-boringssl-static	2.0.70.Final-db-r0-osx-aarch_64
io.netty	netty-tcnative-boringssl-static	2.0.70.Final-db-r0-osx-x86_64
io.netty	netty-tcnative-boringssl-static	2.0.70.Final-db-r0-windows-x86_64
io.netty	netty-tcnative-classes	2.0.70.Final
io.netty	netty-transport	4.1.118.Final
io.netty	netty-transport-classes-epoll	4.1.118.Final
io.netty	netty-transport-classes-kqueue	4.1.118.Final
io.netty	netty-transport-native-epoll	4.1.118.Final
io.netty	netty-transport-native-epoll	4.1.118.Final-linux-aarch_64
io.netty	netty-transport-native-epoll	4.1.118.Final-linux-riscv64
io.netty	netty-transport-native-epoll	4.1.118.Final-linux-x86_64
io.netty	netty-transport-native-kqueue	4.1.118.Final-osx-aarch_64
io.netty	netty-transport-native-kqueue	4.1.118.Final-osx-x86_64
io.netty	netty-transport-native-unix-common	4.1.118.Final
io.prometheus	simpleclient	0.16.1-databricks
io.prometheus	simpleclient_common	0.16.1-databricks
io.prometheus	simpleclient_dropwizard	0.16.1-databricks
io.prometheus	simpleclient_pushgateway	0.16.1-databricks
io.prometheus	simpleclient_servlet	0.16.1-databricks
io.prometheus	simpleclient_servlet_common	0.16.1-databricks
io.prometheus	simpleclient_tracer_common	0.16.1-databricks
io.prometheus	simpleclient_tracer_otel	0.16.1-databricks
io.prometheus	simpleclient_tracer_otel_agent	0.16.1-databricks
io.prometheus.jmx	collector	0.18.0
jakarta.annotation	jakarta.annotation-api	1.3.5
jakarta.servlet	jakarta.servlet-api	4.0.3
jakarta.validation	jakarta.validation-api	2.0.2
jakarta.ws.rs	jakarta.ws.rs-api	2.1.6
javax.activation	activation	1.1.1
javax.annotation	javax.annotation-api	1.3.2
javax.el	javax.el-api	2.2.4
javax.jdo	jdo-api	3.0.1
javax.media	jai_core	jai_core_dummy
javax.transaction	jta	1.1
javax.transaction	transaction-api	1.1
javax.xml.bind	jaxb-api	2.2.11
javolution	javolution	5.5.1
jline	jline	2.14.6
joda-time	joda-time	2.13.0
net.java.dev.jna	jna	5.8.0
net.razorvine	pickle	1.5
net.sf.jpam	jpam	1.1
net.sf.opencsv	opencsv	2.3
net.sf.supercsv	super-csv	2.2.0
net.snowflake	snowflake-ingest-sdk	0.9.6
net.sourceforge.f2j	arpack_combined_all	0.1
org.acplt.remotetea	remotetea-oncrpc	1.1.2
org.antlr	ST4	4.0.4
org.antlr	antlr-runtime	3.5.2
org.antlr	antlr4-runtime	4.13.1
org.antlr	stringtemplate	3.2.1
org.apache.ant	ant	1.10.11
org.apache.ant	ant-jsch	1.10.11
org.apache.ant	ant-launcher	1.10.11
org.apache.arrow	arrow-format	18.2.0
org.apache.arrow	arrow-memory-core	18.2.0
org.apache.arrow	arrow-memory-netty	18.2.0
org.apache.arrow	arrow-memory-netty-buffer-patch	18.2.0
org.apache.arrow	arrow-vector	18.2.0
org.apache.avro	avro	1.12.0
org.apache.avro	avro-ipc	1.12.0
org.apache.avro	avro-mapred	1.12.0
org.apache.commons	commons-collections4	4.4
org.apache.commons	commons-compress	1.27.1
org.apache.commons	commons-crypto	1.1.0
org.apache.commons	commons-lang3	3.17.0
org.apache.commons	commons-math3	3.6.1
org.apache.commons	commons-text	1.13.0
org.apache.curator	curator-client	5.7.1
org.apache.curator	curator-framework	5.7.1
org.apache.curator	curator-recipes	5.7.1
org.apache.datasketches	datasketches-java	6.1.1
org.apache.datasketches	datasketches-memory	3.0.2
org.apache.derby	derby	10.14.2.0
org.apache.hadoop	hadoop-client-runtime	3.4.1
org.apache.hive	hive-beeline	2.3.10
org.apache.hive	hive-cli	2.3.10
org.apache.hive	hive-jdbc	2.3.10
org.apache.hive	hive-llap-client	2.3.10
org.apache.hive	hive-llap-common	2.3.10
org.apache.hive	hive-serde	2.3.10
org.apache.hive	hive-shims	2.3.10
org.apache.hive	hive-storage-api	2.8.1
org.apache.hive.shims	hive-shims-0.23	2.3.10
org.apache.hive.shims	hive-shims-common	2.3.10
org.apache.hive.shims	hive-shims-scheduler	2.3.10
org.apache.httpcomponents	httpclient	4.5.14
org.apache.httpcomponents	httpcore	4.4.16
org.apache.ivy	ivy	2.5.3
org.apache.logging.log4j	log4j-1.2-api	2.24.3
org.apache.logging.log4j	log4j-api	2.24.3
org.apache.logging.log4j	log4j-core	2.24.3
org.apache.logging.log4j	log4j-layout-template-json	2.24.3
org.apache.logging.log4j	log4j-slf4j2-impl	2.24.3
org.apache.orc	orc-core	2.1.1-shaded-protobuf
org.apache.orc	orc-format	1.1.0-shaded-protobuf
org.apache.orc	orc-mapreduce	2.1.1-shaded-protobuf
org.apache.orc	orc-shims	2.1.1
org.apache.poi	poi	5.4.1
org.apache.poi	poi-ooxml	5.4.1
org.apache.poi	poi-ooxml-full	5.4.1
org.apache.poi	poi-ooxml-lite	5.4.1
org.apache.thrift	libfb303	0.9.3
org.apache.thrift	libthrift	0.16.0
org.apache.ws.xmlschema	xmlschema-core	2.3.1
org.apache.xbean	xbean-asm9-shaded	4.26
org.apache.xmlbeans	xmlbeans	5.3.0
org.apache.yetus	audience-annotations	0.13.0
org.apache.zookeeper	zookeeper	3.9.3
org.apache.zookeeper	zookeeper-jute	3.9.3
org.checkerframework	checker-qual	3.43.0
org.codehaus.janino	commons-compiler	3.0.16
org.codehaus.janino	janino	3.0.16
org.datanucleus	datanucleus-api-jdo	4.2.4
org.datanucleus	datanucleus-core	4.1.17
org.datanucleus	datanucleus-rdbms	4.1.19
org.datanucleus	javax.jdo	3.2.0-m3
org.eclipse.jetty	jetty-client	9.4.53.v20231009
org.eclipse.jetty	jetty-continuation	9.4.53.v20231009
org.eclipse.jetty	jetty-http	9.4.53.v20231009
org.eclipse.jetty	jetty-io	9.4.53.v20231009
org.eclipse.jetty	jetty-jndi	9.4.53.v20231009
org.eclipse.jetty	jetty-plus	9.4.53.v20231009
org.eclipse.jetty	jetty-proxy	9.4.53.v20231009
org.eclipse.jetty	jetty-security	9.4.53.v20231009
org.eclipse.jetty	jetty-server	9.4.53.v20231009
org.eclipse.jetty	jetty-servlet	9.4.53.v20231009
org.eclipse.jetty	jetty-servlets	9.4.53.v20231009
org.eclipse.jetty	jetty-util	9.4.53.v20231009
org.eclipse.jetty	jetty-util-ajax	9.4.53.v20231009
org.eclipse.jetty	jetty-webapp	9.4.53.v20231009
org.eclipse.jetty	jetty-xml	9.4.53.v20231009
org.eclipse.jetty.websocket	websocket-api	9.4.53.v20231009
org.eclipse.jetty.websocket	websocket-client	9.4.53.v20231009
org.eclipse.jetty.websocket	websocket-common	9.4.53.v20231009
org.eclipse.jetty.websocket	websocket-server	9.4.53.v20231009
org.eclipse.jetty.websocket	websocket-servlet	9.4.53.v20231009
org.fusesource.leveldbjni	leveldbjni-all	1.8
org.glassfish.hk2	hk2-api	2.6.1
org.glassfish.hk2	hk2-locator	2.6.1
org.glassfish.hk2	hk2-utils	2.6.1
org.glassfish.hk2	osgi-resource-locator	1.0.3
org.glassfish.hk2.external	aopalliance-repackaged	2.6.1
org.glassfish.hk2.external	jakarta.inject	2.6.1
org.glassfish.jersey.containers	jersey-container-servlet	2.41
org.glassfish.jersey.containers	jersey-container-servlet-core	2.41
org.glassfish.jersey.core	jersey-client	2.41
org.glassfish.jersey.core	jersey-common	2.41
org.glassfish.jersey.core	jersey-server	2.41
org.glassfish.jersey.inject	jersey-hk2	2.41
org.hibernate.validator	hibernate-validator	6.2.5.Final
org.ini4j	ini4j	0.5.4
org.javassist	javassist	3.29.2-GA
org.jboss.logging	jboss-logging	3.4.1.Final
org.jdbi	jdbi	2.63.1
org.jetbrains	annotations	17.0.0
org.jline	jline	3.27.1-jdk8
org.joda	joda-convert	1.7
org.jodd	jodd-core	3.5.2
org.json4s	json4s-ast_2.13	4.0.7
org.json4s	json4s-core_2.13	4.0.7
org.json4s	json4s-jackson-core_2.13	4.0.7
org.json4s	json4s-jackson_2.13	4.0.7
org.json4s	json4s-scalap_2.13	4.0.7
org.lz4	lz4-java	1.8.0-databricks-1
org.mlflow	mlflow-spark_2.13	2.9.1
org.objenesis	objenesis	3.3
org.postgresql	postgresql	42.6.1
org.roaringbitmap	RoaringBitmap	1.2.1
org.rocksdb	rocksdbjni	9.8.4
org.rosuda.REngine	REngine	2.1.0
org.scala-lang	scala-compiler_2.13	2.13.16
org.scala-lang	scala-library_2.13	2.13.16
org.scala-lang	scala-reflect_2.13	2.13.16
org.scala-lang.modules	scala-collection-compat_2.13	2.11.0
org.scala-lang.modules	scala-java8-compat_2.13	0.9.1
org.scala-lang.modules	scala-parallel-collections_2.13	1.2.0
org.scala-lang.modules	scala-parser-combinators_2.13	2.4.0
org.scala-lang.modules	scala-xml_2.13	2.3.0
org.scala-sbt	test-interface	1.0
org.scalacheck	scalacheck_2.13	1.18.0
org.scalactic	scalactic_2.13	3.2.19
org.scalanlp	breeze-macros_2.13	2.1.0
org.scalanlp	breeze_2.13	2.1.0
org.scalatest	scalatest-compatible	3.2.19
org.scalatest	scalatest-core_2.13	3.2.19
org.scalatest	scalatest-diagrams_2.13	3.2.19
org.scalatest	scalatest-featurespec_2.13	3.2.19
org.scalatest	scalatest-flatspec_2.13	3.2.19
org.scalatest	scalatest-freespec_2.13	3.2.19
org.scalatest	scalatest-funspec_2.13	3.2.19
org.scalatest	scalatest-funsuite_2.13	3.2.19
org.scalatest	scalatest-matchers-core_2.13	3.2.19
org.scalatest	scalatest-mustmatchers_2.13	3.2.19
org.scalatest	scalatest-propspec_2.13	3.2.19
org.scalatest	scalatest-refspec_2.13	3.2.19
org.scalatest	scalatest-shouldmatchers_2.13	3.2.19
org.scalatest	scalatest-wordspec_2.13	3.2.19
org.scalatest	scalatest_2.13	3.2.19
org.slf4j	jcl-over-slf4j	2.0.16
org.slf4j	jul-to-slf4j	2.0.16
org.slf4j	slf4j-api	2.0.16
org.slf4j	slf4j-simple	1.7.25
org.threeten	threeten-extra	1.8.0
org.tukaani	xz	1.10
org.typelevel	algebra_2.13	2.8.0
org.typelevel	cats-kernel_2.13	2.8.0
org.typelevel	spire-macros_2.13	0.18.0
org.typelevel	spire-platform_2.13	0.18.0
org.typelevel	spire-util_2.13	0.18.0
org.typelevel	spire_2.13	0.18.0
org.wildfly.openssl	wildfly-openssl	1.1.3.Final
org.xerial	sqlite-jdbc	3.42.0.0
org.xerial.snappy	snappy-java	1.1.10.3
org.yaml	snakeyaml	2.0
oro	oro	2.0.8
pl.edu.icm	JLargeArrays	1.5
software.amazon.cryptools	AmazonCorrettoCryptoProvider	2.5.0-linux-x86_64
stax	stax-api	1.0.1

ヒント

To see release notes for Databricks Runtime versions that have reached end-of-support (EoS), see End-of-support Databricks Runtime release notes. The EoS Databricks Runtime versions have been retired and might not be updated.