Pular para o conteúdo principal
Última atualização em

Databricks Runtime 18.1 (Beta)

The following release notes provide information about Databricks Runtime 18.1 (Beta).

This version incorporates all features, improvements, and bug fixes from all previous Databricks Runtime releases. Databricks released this version in February 2026.

Beta

Databricks Runtime 18.1 is in Beta. The contents of the supported environments might change during the Beta. Changes can include the list of packages or versions of installed packages.

New features and improvements​

Auto Loader uses file events by default when available​

Auto Loader uses file events by default when loading from an external location with file events enabled, which reduces list operations and cost compared to directory listing. See Auto Loader with file events overview. File events aren't used if your stream code sets useIncrementalListing or useNotifications. To use directory listing instead, set useManagedFileEvents to false.

Optimized writes for partitioned Unity Catalog tables created with CRTAS​

Optimized writes apply to partitioned Unity Catalog tables created by CREATE OR REPLACE TABLE ... AS SELECT (CRTAS), producing fewer, larger files. This behavior is enabled by default. To disable, set spark.databricks.delta.optimizeWrite.UCTableCRTAS.enabled to false.

DATETIMEOFFSET data type support for Microsoft Azure Synapse​

The DATETIMEOFFSET data type is supported for Microsoft Azure Synapse connections.

Google BigQuery table comments​

Google BigQuery table descriptions are resolved and exposed as table comments.

Schema evolution with INSERT statements​

Use the WITH SCHEMA EVOLUTION clause with SQL INSERT statements to automatically evolve the target table's schema during insert operations. The clause is supported for INSERT INTO, INSERT OVERWRITE, and INSERT INTO ... REPLACE forms. For example:

SQL
INSERT WITH SCHEMA EVOLUTION INTO students TABLE visiting_students_with_additional_id;

The target Delta Lake table's schema is updated to accommodate additional columns or widened types from the source. For details, see schema evolution and INSERT statement syntax.

Preserved NULL struct values in INSERT operations​

INSERT operations with schema evolution or implicit casting preserve NULL struct values when the source and target tables have differing struct field orders.

Delta Sharing multi-statement transaction support​

Delta Sharing tables that use pre-signed URL or cloud token sharing modes support multi-statement transactions. On first access within a transaction, the table version is pinned and reused for all subsequent reads in that transaction.

Limitations:

  • Time travel, change data feed, and streaming aren't supported.
  • Multi-statement transactions aren't supported on shared tables without history.
  • Shared views and foreign entities aren't allowed on untrusted compute.

DataFrame checkpoints in volume paths​

DataFrame checkpoints support Unity Catalog volume paths. Configure the checkpoint path using SparkContext.setCheckpointDir on dedicated compute or the spark.checkpoint.dir configuration on standard compute.

SQL commands no longer re-run when calling cache()​

SQL commands are no longer re-run when calling .cache() on the result DataFrame. This includes commands such as SHOW TABLES and SHOW NAMESPACES.

parse_timestamp SQL function​

The parse_timestamp SQL function parses timestamp strings against multiple patterns. It's photonized for faster evaluation with timestamps in various formats. See Datetime patterns for information about datetime pattern formatting.

SQL cursor support in compound statements​

SQL scripting compound statements now support cursor processing. Use DECLARE CURSOR to define a cursor, then OPEN statement, FETCH statement, and CLOSE statement to run the query and consume rows one at a time. Cursors can use parameter markers and condition handlers such as NOT FOUND for row-by-row processing.

Approximate top-k sketch functions​

New functions enable building and combining approximate top-K sketches for distributed top-K aggregation:

For more information, see approx_top_k aggregate function and Built-in functions.

Tuple sketch functions​

New aggregate and scalar functions for tuple sketch support distinct counting and aggregation over key-summary pairs.

Aggregate functions:

Scalar functions:

  • tuple_sketch_estimate
  • tuple_sketch_summary
  • tuple_sketch_theta
  • tuple_union
  • tuple_intersection
  • tuple_difference

See Built-in functions.

Behavioral changes​

FILTER clause for MEASURE aggregate functions​

MEASURE aggregate functions now support FILTER clauses. Previously, filters were silently ignored.

Optimized writes for Unity Catalog CRTAS operations​

CREATE OR REPLACE TABLE AS SELECT (CRTAS) operations on partitioned Unity Catalog tables now apply optimized writes by default. To disable, set spark.databricks.delta.optimizeWrite.UCTableCRTAS.enabled to false.

DataFrame cache refresh for fine-grained access control tables​

Writing to fine-grained access control tables on dedicated compute now refreshes cached DataFrames that depend on the table.

Timestamp partition values use session timezone​

Timestamp partition values use the Spark session timezone instead of the JVM timezone. If you have timestamp partitions written before Databricks Runtime 18.0, run SHOW PARTITIONS to verify your partition metadata before writing new data.

DESCRIBE FLOW reserved keyword​

The DESCRIBE FLOW command is now available. If you have a table named flow, use DESCRIBE schema.flow, DESCRIBE TABLE flow, or DESCRIBE `flow` with backticks.

Geospatial operations performance improvement​

Geospatial boolean set operations use a new implementation with improved performance. Minor precision differences might occur after the 15th decimal place.

Exception types for SQLSTATE​

Exception types are updated to support SQLSTATE. If your code parses exceptions by string matching or catches specific exception types, update your error handling logic.

Automatic streaming type widening​

Streaming reads on Delta tables automatically handle column type widening. To require manual acknowledgement, set spark.databricks.delta.typeWidening.enableStreamingSchemaTracking to true.

Library upgrades​

No libraries were upgraded in this version.

Apache Spark​

Databricks Runtime 18.1 includes Apache Spark 4.1.0. This release includes all Spark fixes and improvements included in Databricks Runtime 18.0, as well as the following additional bug fixes and improvements made to Spark:

  • SPARK-55350 Fix row count loss when creating DataFrame from pandas with 0 columns
  • SPARK-55364 Make SupportsIAdd and SupportsOrdering protocol more reasonable
  • SPARK-53656 Refactor MemoryStream to use SparkSession instead of SQLContext
  • SPARK-55472 Raise AttributeError from methods removed in pandas 3
  • SPARK-55224 Use Spark DataType as ground truth in Pandas-Arrow serialization
  • SPARK-55402 Move streamingSourceIdentifyingName from CatalogTable to DataSource
  • SPARK-55459 Fix 3x performance regression in applyInPandas for large groups
  • SPARK-55317 Add SequentialUnion logical plan node and planning rule
  • SPARK-55424 Explicitly pass the series name in convert_numpy
  • SPARK-55175 Extract to_pandas transformer from serializers
  • SPARK-55304 Introduce support of Admission Control and Trigger.AvailableNow in Python data source - streaming reader
  • SPARK-55382 Make Executor to log Running Spark version
  • SPARK-55408 Handle unexpected keyword argument errors related to datetime with pandas 3
  • SPARK-55345 Do not pass unit and closed to Timedelta for pandas 3
  • SPARK-54759 Properly guard logic in Scripting Variable Manager after Cursor introduction
  • SPARK-55409 Handle an unexpected keyword argument error from read_excel with pandas 3
  • SPARK-55403 Fix no attribute 'draw' error in the plot tests with pandas 3
  • SPARK-55256 Revert "[SC-218596][SQL] Support IGNORE NULLS / RESPECT NULLS for array_agg and collect_list"
  • SPARK-55256 Support IGNORE NULLS / RESPECT NULLS for array_agg and collect_list
  • SPARK-55395 Disable RDD cache in DataFrame.zipWithIndex
  • SPARK-55131 Introduce a new merge operator delimiter for RocksDB to empty string to concat without delimiter
  • SPARK-55376 Make numeric_only argument in groupby functions accept only boolean with pandas 3
  • SPARK-55334 Enable TimestampType and TimestampNTZType in convert_numpy
  • SPARK-55281 Add ipykernel and IPython to mypy optional package list
  • SPARK-55336 Let createDF use create_batch logic for decoupling
  • SPARK-55366 Remove errorOnDuplicatedFieldNames from Python UDFs
  • SPARK-54759 SQL scripting Cursor support
  • SPARK-55302 Fix custom metrics in case of KeyGroupedPartitioning
  • SPARK-55228 Implement Dataset.zipWithIndex in Scala API
  • SPARK-55373 Improve noHandlerFoundForExtension error message
  • SPARK-55356 Support alias for PIVOT clause
  • SPARK-55359 Promote TaskResourceRequest to Stable
  • SPARK-55365 Generalize the utils for arrow array conversion
  • SPARK-55106 Add Repartition Integration test for TransformWithState Operators
  • SPARK-55086 Add DataSourceReader.pushFilters to Python Data Source API docs
  • SPARK-46165 Add support for DataFrame.all axis=None
  • SPARK-55289 Revert "[SC-218749][SQL] Fix flaky test in-set-operations.sql by disabling broadcast join"
  • SPARK-55297 Restore timedelta dtype based on the original dtype
  • SPARK-55291 Pre-process metadata headers at client interceptor construction time
  • SPARK-55155 Reapply "[SC-218401][SQL] Support foldable expressions in SET CATALOG statement
  • SPARK-55318 Performance Optimizations for vector_avg/vector_sum
  • SPARK-55295 Extend the ST_GeomFromWKB function to take an optional SRID value
  • SPARK-55280 Add GetStatus proto to support execution status monitoring
  • SPARK-54969 Implement new arrow->pandas conversion
  • SPARK-55176 Extract arrow_to_pandas converter into ArrowArrayToPandasConversion
  • SPARK-55111 Revert "[SC-217817][SC-210791][SS] Unfinished repartitioning detection on query restart"
  • SPARK-55252 Improve HttpSecurityFilter to add Content-Security-Policy header
  • SPARK-55111 Unfinished repartitioning detection on query restart
  • SPARK-55105 Add Integration Test for Join Operator
  • SPARK-55260 Implement Parquet write support for Geo types
  • SPARK-54523 Fix default resolution during variant pushdown
  • SPARK-55328 Reuse PythonArrowInput.codec in GroupedPythonArrowInput
  • SPARK-55246 Add Test for Pyspark TWS and TWSInPandas and Fix StatePartitionAllColumnFamiliesWriter Bug
  • SPARK-55289 Fix flaky test in-set-operations.sql by disabling broadcast join
  • SPARK-55040 Refactor TaskContext and related worker protocol with engine pyspark fix.
  • SPARK-47996 support cross merge in pandas API
  • SPARK-55031 Add vector avg/sum aggregation function expressions
  • SPARK-54410 Fix read support for the variant logical type annotation
  • SPARK-54776 Revert "[SC-216482][SQL] Improved the logs message regarding lambda function with SQL UDF"
  • SPARK-55123 Add SequentialUnionOffset for tracking sequential source processing
  • SPARK-54972 Improve NOT IN subqueries with non-nullable columns
  • SPARK-54776 Improved the logs message regarding lambda function with SQL UDF
  • SPARK-53807 Fix race condition issues between unlock and releaseAllLocksForTask in BlockInfoManager
  • SPARK-51831 Revert "[SC-207389][SQL] Column pruning with existsJoin for Datasource V2"
  • SPARK-54881 Improve BooleanSimplification to handle negation of conjunction and disjunction in one pass
  • SPARK-54696 Clean-up ArrowBuffers in Connect
  • SPARK-55009 Remove unnecessary memory copy from the constructor of LevelDBTypeInfo/RocksDBTypeInfo.Index
  • SPARK-54877 Make display stacktrace on UI error page configurable
  • SPARK-51831 Column pruning with existsJoin for Datasource V2
  • SPARK-55285 Fix the initialization of PythonArrowInput
  • SPARK-53960 Let approx_top_k_accumulate/combine/estimate handle NULLs
  • SPARK-49110 Simplify SubqueryAlias.metadataOutput to always propagate metadata columns
  • SPARK-55155 Revert "[SC-218401][SQL] Support foldable expressions in SET CATALOG statement"
  • SPARK-55155 Support foldable expressions in SET CATALOG statement
  • SPARK-49110 Revert "[SC-218594][SQL] Simplify SubqueryAlias.metadataOutput to always propagate metadata columns"
  • SPARK-54399 Implement the st_setsrid function in Scala and PySpark
  • SPARK-49110 Simplify SubqueryAlias.metadataOutput to always propagate metadata columns
  • SPARK-55133 Fix race condition in IsolatedSessionState lifecycle management
  • SPARK-55262 Block Geo types in all file based data sources except Parquet
  • SPARK-54202 Allow casting from GeometryType(srid) to GeometryType(ANY)
  • SPARK-54142 Implement the st_srid function in Scala and PySpark
  • SPARK-54887 Always set a sql state in spark connect client
  • SPARK-55237 Suppress annoying messages when looking up nonexistent DBs
  • SPARK-55040 Revert "[SC-217628][PYTHON] Refactor TaskContext and related worker protocol"
  • SPARK-55243 Allow setting binary headers via the -bin suffix in the Scala Connect client
  • SPARK-55259 Implement Parquet schema conversion for Geo types
  • SPARK-55282 Avoid using worker_util in the Driver-side
  • SPARK-54151 Introduce the framework for adding ST functions in PySpark
  • SPARK-55194 Remove GroupArrowUDFSerializer by moving flatten logic to mapper
  • SPARK-55020 Disable gc when executing gRPC command
  • SPARK-55053 Refactor data source/udtf analysis workers so they have a unified entry
  • SPARK-55040 Refactor TaskContext and related worker protocol
  • SPARK-55244 Use np.nan as default value for pandas string types
  • SPARK-55225 Restore to the original dtype for Datetime
  • SPARK-55154 Remove fastpath to pd.Series for pandas 3
  • SPARK-55030 Add vector norm/normalize function expressions support
  • SPARK-55202 Fix UNEXPECTED_USE_OF_PARAMETER_MARKER when using param…
  • SPARK-54201 Allow casting from GeographyType(srid) to GeographyType(ANY)
  • SPARK-54244 Introduce type coercion support for GEOMETRY data types
  • SPARK-54160 Implement the ST_SetSrid expression in SQL
  • SPARK-55046 PySpark add udf processing time metric
  • SPARK-54101 Introduce the framework for adding ST functions in Scala
  • SPARK-54683 Unify geo and time types blocking
  • SPARK-55249 Make DataFrame.toJSON able to return dataframe
  • SPARK-54521 Add to and from WKB for Geometry type
  • SPARK-54162 Allow casting from GeographyType to GeometryType
  • SPARK-55146 Recheckin State Repartition API for PySpark
  • SPARK-55140 Do not map builtin functions to numpy version for pandas 3
  • SPARK-54243 Introduce type coercion support for GEOGRAPHY data types
  • SPARK-55058 Throw error on inconsistent checkpoint metadata
  • SPARK-55108 Use the latest pandas-stubs for type check
  • SPARK-54091 Implement the ST_Srid expression in SQL
  • SPARK-55104 Add Spark Connect support for DataStreamReader.name()
  • SPARK-54996 Report ArrivalTime for records from LowLatencyMemoryStream
  • SPARK-55118 Replace ASM Opcodes wildcard imports
  • SPARK-54365 Add Repartition Integration Test for Aggregate, Dedup, FMGWS and SessionWindow Operators
  • SPARK-55119 Fix Continue Handler: prevent INTERNAL_ERROR and incorrect conditional statements interruption
  • SPARK-54104 Disallow casting geospatial types to/from other data types
  • SPARK-55240 Refactor LazyTry stacktrace handling to use wrapper instead of suppressed exception
  • SPARK-55238 Move Geo SRS mapping logic from main/scala to main/java
  • SPARK-55179 Skip eager column name validation in df.col_name
  • SPARK-55055 Support SparkSession.Builder.create for PySpark Classic
  • SPARK-55186 Make ArrowArrayToPandasConversion.convert_legacy able to return pd.DataFrame
  • SPARK-54079 Introduce the framework for adding ST expressions in Catalyst
  • SPARK-54096 Support Spatial Reference System mapping in PySpark
  • SPARK-54801 Mark a few new 4.1 configs as internal
  • SPARK-55146 Revert "[SC-217936][SC-210779][SS] State Repartition API for PySpark"
  • SPARK-54559 Use runnerConf to pass profiler option
  • SPARK-55205 Fix tests where it assumes string type will convert to object
  • SPARK-55171 Fix memory profiler on iter UDF
  • SPARK-55226 Recognize datetime and timedelta with units other than [ns]
  • SPARK-55027 Move writeConf to PythonWorkerUtils
  • SPARK-55197 Extract _write_stream_start helper to deduplicate START_ARROW_STREAM signal logic
  • SPARK-54179 Add Native Support for Apache Tuple Sketches (#190848) (#191111)
  • SPARK-55151 Fix RocksDBSuite testWithStateStoreCheckpointIds
  • SPARK-55146 State Repartition API for PySpark
  • SPARK-54232 Enable Arrow serialization for Geography and Geometry types
  • SPARK-53957 Support GEOGRAPHY and GEOMETRY in the SpatialReferenceSystemMapper
  • SPARK-55169 Use ArrowBatchTransformer.flatten_struct in ArrowStreamArrowUDTFSerializer
  • SPARK-55134 Fix BasicExecutorFeatureStep to throw IllegalArgumentException for executor cpu misconfigs
  • SPARK-54166 Introduce type encoders for geospatial types in PySpark
  • SPARK-55138 Fix convertToMapData throwing a NullPointerException
  • SPARK-55168 Use ArrowBatchTransformer.flatten_struct in GroupArrowUDFSerializer
  • SPARK-55032 Refactor profilers in workers.py
  • SPARK-54990 Fix how classproperty is implemented in session.py
  • SPARK-55076 Fix the type hint issue in ml/mllib and add scipy requirement
  • SPARK-55162 Extract transformers from ArrowStreamUDFSerializer
  • SPARK-55121 Add DataStreamReader.name() to Classic PySpark
  • SPARK-54887 Add optional sqlstate to sparkthrowables
  • SPARK-54169 Introduce Geography and Geometry types in Arrow writer
  • SPARK-51658 Introduce Geometry and Geography in-memory wrapper formats
  • SPARK-54110 Introduce type encoders for Geography and Geometry types
  • SPARK-54956 Unify the indeterminate shuffle retry solution
  • SPARK-55137 Refactor GroupingAnalyticsTransformer and Analyzer code
  • SPARK-54103 Introduce client-side Geography and Geometry classes
  • SPARK-55160 Directly pass input schema to serializers
  • SPARK-55170 Extract grouped stream reading pattern from serializers
  • SPARK-55125 Remove redundant __init__ methods in Arrow serializers
  • SPARK-55126 Remove unused timezone and assign_cols_by_name from ArrowStreamArrowUDFSerializer
  • SPARK-54980 Convert the JDBC isolation level to a string
  • SPARK-55051 Byte string accepts KiB, MiB, GiB, TiB, PiB
  • SPARK-55025 Improve performance in pandas by using list comprehension
  • SPARK-46165 Add support for pandas.DataFrame.all axis=1
  • SPARK-55037 Re-implement Observation Without Using the QueryExecutionListener
  • SPARK-54965 Factor out legacy pa.Array -> pd.Series converter
  • SPARK-55016 Make SQLConf a direct property of SparkSession to prevent stack overflow
  • SPARK-55091 Reduce Hive RPC calls for DROP TABLE command
  • SPARK-55097 Fix re-adding cached artifacts drops blocks silently issue
  • SPARK-55026 Optimize BestEffortLazyVal.
  • SPARK-55091 Revert "[SC-217410][SQL] Reduce Hive RPC calls for DROP TABLE command"
  • SPARK-54590 Support Checkpoint V2 for State Rewriter and Repartitioning
  • SPARK-55016 Make SQLConf a direct property of SparkSession to prevent stack overflow
  • SPARK-55091 Reduce Hive RPC calls for DROP TABLE command
  • SPARK-55016 Revert "[SC-217401][SQL] Make SQLConf a direct property of SparkSession to prevent stack overflow"
  • SPARK-55016 Make SQLConf a direct property of SparkSession to prevent stack overflow
  • SPARK-55098 Vectorized UDFs with output batch size control fail with memory leak
  • SPARK-54824 Add support for multiGet and deleteRange for Rocksdb State Store
  • SPARK-55054 Add IDENTIFIED BY support for streaming table-valued functions
  • SPARK-55029 Propagate streaming source identifying name through resolution pipeline
  • SPARK-55071 Make spark.addArtifact work with Windows paths
  • SPARK-54103 Revert "[SC-210400][Geo][SQL] Introduce client-side Geography and Geometry classes"
  • SPARK-54103 Introduce client-side Geography and Geometry classes
  • SPARK-54033 Introduce Catalyst server-side geospatial execution classes
  • SPARK-54176 Introduce Geography and Geometry data types to PySpark Connect
  • SPARK-55089 Correct the output schema of toJSON
  • SPARK-55035 Perform shuffle cleanup in child executions
  • SPARK-55036 Add ArrowTimestampConversion for arrow timezone handling
  • SPARK-55090 Implement DataFrame.toJSON in Python Client
  • SPARK-54873 Simplify V2TableReference resolution as only temp view may contain it
  • SPARK-52828 Make hashing for collated strings collation agnostic
  • SPARK-54175 Add Geography and Geometry types to Spark Connect proto
  • SPARK-54961 Introduce GroupingAnalyticsTransformer
  • SPARK-55088 Keep the metadata in to/from_arrow_type/schema
  • SPARK-55070 Allow hidden column in dataframe column resolution
  • SPARK-55044 Keep the metadata in toArrowSchema/fromArrowSchema
  • SPARK-55043 Fix time travel with subquery containing table references
  • SPARK-54987 Change the default value of prefer_timestamp_ntz to True in from_arrow_type/from_arrow_schema
  • SPARK-54866 Refactor Drop/RefreshFunction to avoid catalog lookup
  • SPARK-55024 Use REQUIRES_SINGLE_PART_NAMESPACE error for session catalog namespace validation
  • SPARK-54992 Replace cast with a runtime check for make_timestamp
  • SPARK-55024 Revert "[SC-216987][SQL] Use REQUIRES_SINGLE_PART_NAMESPACE error for session catalog namespace validation"
  • SPARK-54866 Revert "[SC-216753][SQL] Refactor Drop/RefreshFunction to avoid catalog lookup"
  • SPARK-55024 Use REQUIRES_SINGLE_PART_NAMESPACE error for session catalog namespace validation
  • SPARK-54866 Refactor Drop/RefreshFunction to avoid catalog lookup
  • SPARK-54991 Correct type hint for streaming/listener.py
  • SPARK-54925 Add the capability to dump threads for pyspark
  • SPARK-54803 Support BY NAME with INSERT ... REPLACE WHERE
  • SPARK-54785 Add support for binary sketch aggregations in KLL (#188370) (#188860)
  • SPARK-54949 Move pyproject.toml to repo root
  • SPARK-54954 Fix remote related type hints in util.py
  • SPARK-54922 Unify how args are passed to python workers
  • SPARK-54870 Collation support for char/varchar and CTAS/RTAS
  • SPARK-54762 Fix _create_converter and covert overload signature
  • SPARK-55019 Allow DROP TABLE to drop VIEW
  • SPARK-53103 Throw an error if state directory is not empty when query starts
  • SPARK-54995 Make a fast path for foreachPartition
  • SPARK-54634 Add clear error message for empty IN predicate
  • SPARK-54984 State Repartition execution and integrate with State Rewriter
  • SPARK-54443 Integrate PartitionKeyExtractor in Re-partition reader
  • SPARK-54907 Introduce NameStreamingSources analyzer rule for streaming source evolution
  • SPARK-54609 Update TIME type config to match the OSS one
  • SPARK-54988 Simplify the implementation of ObservationManager.tryComplete
  • SPARK-54959 Disable shuffle checksum mismatch fully retry when push based shuffle enabled
  • SPARK-54940 Add tests for pa.scalar type inference
  • SPARK-54634 Revert "[SC-216478][SQL] Add clear error message for empty IN predicate"
  • SPARK-54337 Add support for PyCapsule to Pyspark
  • SPARK-54634 Add clear error message for empty IN predicate
  • SPARK-53785 Memory Source for RTM
  • SPARK-54883 Clean up error messages for CLI and add new error mode DEBUG
  • SPARK-54713 Add vector similarity/distance function expressions support
  • SPARK-54962 Fix nullable integers handling in Pandas UDF
  • SPARK-54864 Add rCTE nodes to NormalizePlan
  • SPARK-53847 Add ContinuousMemorySink for Real-time Mode testing
  • SPARK-54865 Add foreachWithSubqueriesAndPruning method to QueryPlan
  • SPARK-54930 Remove redundant _accumulatorRegistry.clear() call in worker.py
  • SPARK-54929 Fix taskContext._resources reset in loop causes only last resource to be kept
  • SPARK-54963 Make createDataFrame respect prefer_timestamp_ntz when infer_pandas_dict_as_map
  • SPARK-54920 Move grouping analytics extraction logic to a common GroupingAnalyticsExtractor
  • SPARK-54924 State Rewriter to read state, transform it and write new state
  • SPARK-54872 Unify column default value handling between v1 and v2 commands
  • SPARK-54905 Simplify foreachWithSubqueries implementation in QueryPlan
  • SPARK-54682 Support showing parameters in DESCRIBE PROCEDURE
  • SPARK-54933 Avoid repeatedly fetching config binary_as_bytes in toLocalIterator
  • SPARK-54872 Revert "[SC-216260][SQL] Unify column default value handling between v1 and v2 commands"
  • SPARK-51936 ReplaceTableAsSelect should overwrite the new table instead of append
  • SPARK-54771 Remove ResolveUserSpecifiedColumns rule from RuleIdCollection
  • SPARK-54872 Unify column default value handling between v1 and v2 commands
  • SPARK-54313 Add --extra-properties-file option for configuration layering
  • SPARK-54468 Add missing error classes
  • SPARK-46741 Cache Table with CTE should work when CTE in plan expression subquery
  • SPARK-46741 Cache Table with CTE won't work
  • SPARK-54615 Always pass runner_conf to python worker
  • SPARK-53737 Add Real-time Mode trigger
  • SPARK-54541 Rename _LEGACY_ERROR_TEMP_1007 and add sqlState
  • SPARK-54718 Preserve attributes names during CTE newInstance()
  • SPARK-54621 Merge Into Update Set * preserve nested fields if …coerceNestedTypes is enabled
  • SPARK-54595 Keep existing behavior of MERGE INTO without SCHEMA EVOLUTION clause
  • SPARK-54903 Make to_arrow_schema/to_arrow_type able to set timezone
  • SPARK-52326 Add partitions related ExternalCatalogEvent and post them in corresponding operations
  • SPARK-54541 Revert "[SC-215212][SQL] Rename _LEGACY_ERROR_TEMP_1007 and add sqlState"
  • SPARK-54578 Perform Code Cleanup on AssignmentUtils
  • SPARK-54830 Enable checksum based indeterminate shuffle retry by default
  • SPARK-54525 Disable nested struct coercion in MERGE INTO under a config
  • SPARK-53784 Additional Source APIs needed to support RTM execution
  • SPARK-54496 Fix Merge Into Schema Evolution for Dataframe API
  • SPARK-54835 Avoid unnecessary temp QueryExecution for nested command execution
  • SPARK-54541 Rename _LEGACY_ERROR_TEMP_1007 and add sqlState
  • SPARK-54289 Allow MERGE INTO to preserve existing struct fields for UPDATE SET * when source struct has less nested fields than target struct
  • SPARK-54720 Add SparkSession.emptyDataFrame with a schema
  • SPARK-54800 Changed default implementation for isObjectNotFoundException
  • SPARK-54686 Relax DSv2 table checks in temp views to allow new top-level columns
  • SPARK-54619 Add a sanity check for configuration numbers
  • SPARK-54726 Improve a bit of performance for InsertAdaptiveSparkPlan
  • SPARK-51966 Replace select.select() with select.poll() when running on POSIX os
  • SPARK-54749 Fix incorrect numOutputRows metric in OneRowRelationExec
  • SPARK-54411 Introduce Repartition Writer that Support Multi-CF
  • SPARK-54835 Revert "[SC-215823][SQL] Avoid unnecessary temp QueryExecution for nested command execution"
  • SPARK-54867 Introduce NamedStreamingRelation wrapper for source identification during analysis
  • SPARK-54835 Avoid unnecessary temp QueryExecution for nested command execution
  • SPARK-54491 Fix insert into temp view on DSv2 table failure
  • SPARK-54871 Trim aliases from grouping and aggregate expressions before handling grouping analytics
  • SPARK-51920 Fix type handling of namedTuple for transfromWithState
  • SPARK-54526 Rename _LEGACY_ERROR_TEMP_1133 and add sqlState
  • SPARK-54424 Failures during recaching must not fail operations
  • SPARK-54894 Fix argument passing in to_arrow_type
  • SPARK-53448 Conversion of a pyspark DataFrame with a Variant column to pandas fails with an error
  • SPARK-54882 Remove legacy PYARROW_IGNORE_TIMEZONE
  • SPARK-54504 Fix version refresh for DSv2 tables with subqueries
  • SPARK-54444 Relax DSv2 table checks to restore previous behavior
  • SPARK-54859 Arrow by default PySpark UDF API reference doc
  • SPARK-54387 Revert "[ES-1688666] Revert "[SC-212394][SQL] Fix recaching of DSv2 tables""
  • SPARK-54753 fix memory leak of ArtifactManager
  • SPARK-54387 Revert "[SC-212394][SQL] Fix recaching of DSv2 tables"
  • SPARK-54436 Fix error formatting for incompatible table metadata checks
  • SPARK-54849 Upgrade the minimum version of pyarrow to 18.0.0
  • SPARK-54022 Make DSv2 table resolution aware of cached tables
  • SPARK-54387 Fix recaching of DSv2 tables
  • SPARK-53924 Reload DSv2 tables in views created using plans on each view access
  • SPARK-54561 Support breakpoint() for run-tests.py
  • SPARK-54157 Fix refresh of DSv2 tables in Dataset
  • SPARK-54830 Revert "[CORE] Enable checksum based indeterminate shuffle retry by default"
  • SPARK-54861 Reset task thread name to IDLE_TASK_THREAD_NAME when task completes
  • SPARK-54834 Add new interfaces SimpleProcedure and SimpleFunction
  • SPARK-54760 DelegatingCatalogExtension as session catalog supports both V1 and V2 functions
  • SPARK-54685 Remove redundant observed-metrics responses
  • SPARK-54853 Always check hive.exec.max.dynamic.partitions on the spark side
  • SPARK-54840 OrcList Pre-allocation
  • SPARK-54830 Enable checksum based indeterminate shuffle retry by default
  • SPARK-54850 Improve extractShuffleIds to find AdaptiveSparkPlanExec anywhere in plan tree
  • SPARK-54843 Try_to_number expression not working for empty string input
  • SPARK-54556 Rollback succeeding shuffle map stages when shuffle checksum mismatch detected
  • SPARK-54760 Revert "[SC-215670][SQL] DelegatingCatalogExtension as session catalog supports both V1 and V2 functions"
  • SPARK-54760 DelegatingCatalogExtension as session catalog supports both V1 and V2 functions
  • SPARK-54818 TaskMemoryManager allocate failed should log errorstack to help check memory usage
  • SPARK-54827 Add helper function TreeNode.containsTag
  • SPARK-54777 Revert "[SC-215740][SQL] Changed dropTable error handling in JDBCTableCatalog.dropTable(...)"
  • SPARK-54777 Changed dropTable error handling in JDBCTableCatalog.dropTable(...)
  • SPARK-54817 Refactor Unpivot resolution logic to UnpivotTransformer
  • SPARK-54820 Make pandas_on_spark_type compatible with numpy 2.4.0
  • SPARK-54799 Refactor UnpivotCoercion
  • SPARK-54754 OrcSerializer should not parse the schema every time it is serialized
  • SPARK-54226 Extend Arrow compression to Pandas UDF
  • SPARK-54787 Use list comprehension instead of for loops in pandas
  • SPARK-54690 Make Frame.__repr__ insensitive to arrow optimization
  • SPARK-46166 Implementation of pandas.DataFrame.any with axis=None
  • SPARK-54696 Clean-up Arrow Buffers - follow-up
  • SPARK-54769 Remove dead code in conversion.py
  • SPARK-54787 Use list comprehension in pandas _bool_column_labels
  • SPARK-54794 Suppress verbose FsHistoryProvider.checkForLogs scanning logs
  • SPARK-54745 Fix PySpark import error caused by missing UnixStreamServer on Windows
  • SPARK-54782 Correct the config versions
  • SPARK-54781 Return model cache information in JSON format
  • SPARK-54419 Offline Repartition State Reader Supports Multi-Column Families
  • SPARK-54722 Register Pandas Grouped Iter Aggregate UDF for SQL usage
  • SPARK-54762 Revert "[SC-215422][PYTHON] Fix _create_converter and covert overload signature"
  • SPARK-54652 Revert "[SC-215452][SQL] Complete conversion of IDENTIFIER()"
  • SPARK-54762 Fix _create_converter and covert overload signature
  • SPARK-52819 Making KryoSerializationCodec serializable to fix java.io.NotSerializableException errors in various use cases
  • SPARK-54711 Add a timeout for daemon created worker connection
  • SPARK-54738 Add Profiler Support for Pandas Grouped Iter Aggregate UDF
  • SPARK-54652 Complete conversion of IDENTIFIER()
  • SPARK-54581 Making fetchsize option case-insensitive for Postgres connector
  • SPARK-54589 Consolidate ArrowStreamAggPandasIterUDFSerializer into ArrowStreamAggPandasUDFSerializer
  • SPARK-41916 Torch distributor: support multiple torchrun processes per task if task.gpu.amount > 1
  • SPARK-54707 Refactor PIVOT resolution main logic to the PivotTransformer
  • SPARK-54706 Make DistributedLDAModel work with local file system
  • SPARK-53616 Introduce iterator API for pandas grouped agg UDF
  • SPARK-54116 Add off-heap mode support for LongHashedRelation
  • SPARK-54656 Refactor SupportsPushDownVariants to be a ScanBuilder mix-in
  • SPARK-54687 Add golden file with generators resolution edge cases
  • SPARK-54708 Optimize ML cache cleanup with lazy directory creation
  • SPARK-54116 Revert "[SC-213108][SQL] Add off-heap mode support for LongHashedRelation"
  • SPARK-54116 Add off-heap mode support for LongHashedRelation
  • SPARK-54443 Partition key extraction for all streaming stateful operators
  • SPARK-54687 Revert "[SC-214791][SQL] Add golden file with generators resolution edge cases"
  • SPARK-54116 Revert "[SC-213108][SQL] Add off-heap mode support for LongHashedRelation"
  • SPARK-54687 Add golden file with generators resolution edge cases
  • SPARK-54116 Add off-heap mode support for LongHashedRelation
  • SPARK-54420 Introduce Offline Repartitioning StatePartitionWriter for Single Column Family
  • SPARK-54689 Make org.apache.spark.sql.pipelines internal package and make EstimatorUtils private
  • SPARK-54673 Refactor SQL pipe syntax analysis code for sharing and reuse
  • SPARK-54668 Add tests for CTEs in multi-children operators
  • SPARK-54669 Remove redundant casting in rCTEs
  • SPARK-54587 Consolidate all runner_conf related code
  • SPARK-54628 Remove all unnecessary explicit super() arguments
  • SPARK-54675 Add configurable force shutdown timeout for StateStore maintenance thread pool
  • SPARK-54639 Avoid unnecessary Table creation in Arrow serializers
  • SPARK-49635 Remove ANSI config suggestion in CAST error messages
  • SPARK-54664 Clean up the code related to listenerCache from connect.StreamingQueryManager
  • SPARK-54640 Replace select.select with select.poll on UNIX
  • SPARK-54662 Add viztracer and debugpy in dev/requirements.txt
  • SPARK-54632 Add the option to use ruff for lint
  • SPARK-54585 Fix State Store rollback when thread is in interrupted state
  • SPARK-54172 Merge Into Schema Evolution should only add referenced columns
  • SPARK-54438 Consolidate ArrowStreamAggArrowIterUDFSerializer into ArrowStreamAggArrowUDFSerializer
  • SPARK-54627 Remove redundant initializations in serializers
  • SPARK-54631 Add profiler support for Arrow Grouped Iter Aggregate UDF
  • SPARK-54316 Reapply [SC-213971][CORE][PYTHON][SQL] Consolidate GroupPandasIterUDFSerializer with GroupPandasUDFSerializer
  • SPARK-54392 Optimize JVM-Python communication for TWS initial state
  • SPARK-54617 Enable Arrow Grouped Iter Aggregate UDF registration for SQL
  • SPARK-54544 Enable flake8 F811 check
  • SPARK-54650 Move int to decimal conversion into _create_converter_from_pandas
  • SPARK-54316 Revert "[SC-213971][CORE][PYTHON][SQL] Consolidate GroupPandasIterUDFSerializer with GroupPandasUDFSerializer"
  • SPARK-53687 Introduce WATERMARK clause in SQL statement
  • SPARK-54316 Consolidate GroupPandasIterUDFSerializer with GroupPandasUDFSerializer
  • SPARK-54598 Extract logic to read UDFs
  • SPARK-54622 Promote RequiresDistributionAndOrdering and its required interfaces to Evolving
  • SPARK-54624 Ensure user name in historypage get escaped
  • SPARK-54580 Consider Hive 4.1 in HiveVersionSuite and HiveClientImpl
  • SPARK-54068 Fix to_feather to support PyArrow 22.0.0
  • SPARK-54618 Promote LocalScan to Stable
  • SPARK-54616 Mark SupportsPushDownVariants as Experimental
  • SPARK-54607 Remove unused method toStringHelper from AbstractFetchShuffleBlocks.java
  • SPARK-53615 Introduce iterator API for Arrow grouped aggregation UDF
  • SPARK-54608 Avoid double caching of type converter in UDTF
  • SPARK-54600 Don't use pickle to save/load models in pyspark.ml.connect
  • SPARK-54592 Make estimatedSize private
  • SPARK-54388 Introduce StatePartitionReader that scan raw bytes for Single ColFamily
  • SPARK-54570 Propagate the error class correctly in Spark Connect
  • SPARK-54577 Optimize Py4J calls in schema inference
  • SPARK-54568 Avoid unnecessary pandas conversion in create dataframe from ndarray
  • SPARK-54576 Add documentation for new Datasketches-based aggregate functions
  • SPARK-54574 Reenable FPGrowth on connect
  • SPARK-54557 Make CSV/JSON/XmlOptions and CSV/JSON/XmlInferSchema comparable
  • SPARK-52798 Add function approx_top_k_combine
  • SPARK-54446 FPGrowth supports local filesystem with Arrow file format
  • SPARK-54547 Rename hostPort variable to host in TaskSchedulerImpl.(executorLost|logExecutorLoss) methods
  • SPARK-54558 Fix Internal Exception when Exception Handlers with no BEGIN/END are used
  • SPARK-52923 Allow ShuffleManager to control push merge during shuffle registration
  • SPARK-54474 Discard the XML report on tests that are supposed to fail
  • SPARK-54473 Add Avro read and write support for TIME type
  • SPARK-54472 Add ORC read and write support for TIME type
  • SPARK-54463 Add CSV serialization and deserialization support for TIME type
  • SPARK-52588 Approx_top_k: accumulate and estimate
  • SPARK-54461 Add XML serialization and deserialization support for TIME type
  • SPARK-54451 Add JSON serialization and deserialization support for TIME type
  • SPARK-54537 Fix SparkConnectDatabaseMetaData getSchemas/getTables on catalogs missing namespace capbalitity
  • SPARK-54442 Add numeric conversion functions for TIME type
  • SPARK-54451 Revert "[SC-212861][SQL] Add JSON serialization and deserialization support for TIME type"
  • SPARK-54492 Rename _LEGACY_ERROR_TEMP_1201 and add sqlState
  • SPARK-54531 Introduce ArrowStreamAggPandasUDFSerializer
  • SPARK-54223 Add task context and data metrics to Python runner logs
  • SPARK-54272 Add aggTime for SortAggregateExec
  • SPARK-53469 Ability to cleanup shuffle in Thrift server
  • SPARK-54219 Support spark.cleaner.referenceTracking.blocking.timeout config
  • SPARK-54475 Add master-server, branch-4.0-client, Python 3.11
  • SPARK-54451 Add JSON serialization and deserialization support for TIME type
  • SPARK-54285 Cache timezone info to avoid expensive timestamp conversion
  • SPARK-49133 Make member MemoryConsumer#used atomic to avoid user code causing deadlock
  • SPARK-46166 Implementation of pandas.DataFrame.any with axis=1
  • SPARK-54532 Add support for sqlstate for PySparkException
  • SPARK-54435 spark-pipelines init should avoid overwriting existing directory
  • SPARK-54247 Explictly close socket for util._load_from_socket

Databricks ODBC/JDBC driver support​

Databricks supports ODBC/JDBC drivers released in the past 2 years. Please download the recently released drivers and upgrade (download ODBC, download JDBC).

System environment​

  • Operating System: Ubuntu 24.04.3 LTS
  • Java: Zulu21.42+19-CA
  • Scala: 2.13.16
  • Python: 3.12.3
  • R: 4.5.1
  • Delta Lake: 4.0.1

Installed Python libraries​

Library

Version

Library

Version

Library

Version

aiohappyeyeballs

2.4.4

aiohttp

3.11.10

aiosignal

1.2.0

annotated-doc

0.0.4

annotated-types

0.7.0

anyio

4.7.0

argon2-cffi

21.3.0

argon2-cffi-bindings

21.2.0

arro3-core

0.6.5

arrow

1.3.0

asttokens

3.0.0

astunparse

1.6.3

async-lru

2.0.4

attrs

24.3.0

autocommand

2.2.2

azure-common

1.1.28

azure-core

1.37.0

azure-identity

1.20.0

azure-mgmt-core

1.6.0

azure-mgmt-web

8.0.0

azure-storage-blob

12.28.0

azure-storage-file-datalake

12.22.0

babel

2.16.0

backports.tarfile

1.2.0

beautifulsoup4

4.12.3

black

24.10.0

bleach

6.2.0

blinker

1.7.0

boto3

1.40.45

botocore

1.40.45

cachetools

5.5.1

certifi

2025.4.26

cffi

1.17.1

chardet

4.0.0

charset-normalizer

3.3.2

click

8.1.8

cloudpickle

3.0.0

comm

0.2.1

contourpy

1.3.1

cryptography

44.0.1

cycler

0.11.0

Cython

3.1.5

databricks-agents

1.9.1

databricks-sdk

0.67.0

dataclasses-json

0.6.7

dbus-python

1.3.2

debugpy

1.8.11

decorator

5.1.1

defusedxml

0.7.1

deltalake

1.1.4

Deprecated

1.2.18

distlib

0.3.9

docstring-to-markdown

0.11

executing

1.2.0

facets-overview

1.1.1

fastapi

0.128.0

fastjsonschema

2.21.1

filelock

3.17.0

fonttools

4.55.3

fqdn

1.5.1

frozenlist

1.5.0

fsspec

2023.5.0

gitdb

4.0.11

GitPython

3.1.43

google-api-core

2.28.1

google-auth

2.47.0

google-cloud-core

2.5.0

google-cloud-storage

3.7.0

google-crc32c

1.8.0

google-resumable-media

2.8.0

googleapis-common-protos

1.65.0

grpcio

1.67.0

grpcio-status

1.67.0

h11

0.16.0

hf-xet

1.2.0

httpcore

1.0.9

httplib2

0.20.4

httpx

0.28.1

huggingface_hub

1.2.4

idna

3.7

importlib_metadata

8.5.0

inflect

7.3.1

iniconfig

1.1.1

ipyflow-core

0.0.209

ipykernel

6.29.5

ipython

8.30.0

ipython-genutils

0.2.0

ipywidgets

7.8.1

isodate

0.7.2

isoduration

20.11.0

jaraco.collections

5.1.0

jaraco.context

5.3.0

jaraco.functools

4.0.1

jaraco.text

3.12.1

jedi

0.19.2

Jinja2

3.1.6

jiter

0.12.0

jmespath

1.0.1

joblib

1.4.2

json5

0.9.25

jsonpatch

1.33

jsonpointer

3.0.0

jsonschema

4.23.0

jsonschema-specifications

2023.7.1

jupyter-events

0.12.0

jupyter-lsp

2.2.5

jupyter_client

8.6.3

jupyter_core

5.7.2

jupyter_server

2.15.0

jupyter_server_terminals

0.5.3

jupyterlab

4.3.4

jupyterlab_pygments

0.3.0

jupyterlab_server

2.27.3

jupyterlab_widgets

1.1.11

kiwisolver

1.4.8

langchain-core

1.2.6

langchain-openai

1.1.6

langsmith

0.6.1

launchpadlib

1.11.0

lazr.restfulclient

0.14.6

lazr.uri

1.0.6

litellm

1.75.9

markdown-it-py

2.2.0

MarkupSafe

3.0.2

marshmallow

3.26.2

matplotlib

3.10.0

matplotlib-inline

0.1.7

mccabe

0.7.0

mdurl

0.1.0

mistune

3.1.2

mlflow-skinny

3.8.1

mmh3

5.2.0

more-itertools

10.3.0

msal

1.34.0

msal-extensions

1.3.1

multidict

6.1.0

mypy-extensions

1.0.0

nbclient

0.10.2

nbconvert

7.16.6

nbformat

5.10.4

nest-asyncio

1.6.0

nodeenv

1.10.0

notebook

7.3.2

notebook_shim

0.2.4

numpy

2.1.3

oauthlib

3.2.2

openai

2.14.0

opentelemetry-api

1.39.1

opentelemetry-proto

1.39.1

opentelemetry-sdk

1.39.1

opentelemetry-semantic-conventions

0.60b1

orjson

3.11.5

overrides

7.4.0

packaging

24.2

pandas

2.2.3

pandocfilters

1.5.0

parso

0.8.4

pathspec

0.10.3

patsy

1.0.1

pexpect

4.8.0

pillow

11.1.0

pip

25.0.1

platformdirs

4.3.7

plotly

5.24.1

pluggy

1.5.0

prometheus_client

0.21.1

prompt-toolkit

3.0.43

propcache

0.3.1

proto-plus

1.27.0

protobuf

5.29.4

psutil

5.9.0

psycopg2

2.9.11

ptyprocess

0.7.0

pure-eval

0.2.2

pyarrow

21.0.0

pyasn1

0.4.8

pyasn1-modules

0.2.8

pyccolo

0.0.71

pycparser

2.21

pydantic

2.10.6

pydantic_core

2.27.2

pyflakes

3.2.0

Pygments

2.19.1

PyGObject

3.48.2

pyiceberg

0.10.0

PyJWT

2.10.1

pyodbc

5.2.0

pyparsing

3.2.0

pyright

1.1.394

pyroaring

1.0.3

pytest

8.3.5

python-dateutil

2.9.0.post0

python-dotenv

1.2.1

python-json-logger

3.2.1

python-lsp-jsonrpc

1.1.2

python-lsp-server

1.12.2

pytoolconfig

1.2.6

pytz

2024.1

PyYAML

6.0.2

pyzmq

26.2.0

referencing

0.30.2

regex

2024.11.6

requests

2.32.3

requests-toolbelt

1.0.0

rfc3339-validator

0.1.4

rfc3986-validator

0.1.1

rich

13.9.4

rope

1.13.0

rpds-py

0.22.3

rsa

4.9.1

s3transfer

0.14.0

scikit-learn

1.6.1

scipy

1.15.3

seaborn

0.13.2

Send2Trash

1.8.2

setuptools

78.1.1

shellingham

1.5.4

six

1.17.0

smmap

5.0.0

sniffio

1.3.0

sortedcontainers

2.4.0

soupsieve

2.5

sqlparse

0.5.5

ssh-import-id

5.11

stack-data

0.6.3

starlette

0.50.0

strictyaml

1.7.3

tenacity

9.0.0

terminado

0.17.1

threadpoolctl

3.5.0

tiktoken

0.12.0

tinycss2

1.4.0

tokenize_rt

6.1.0

tokenizers

0.22.2

tomli

2.0.1

tornado

6.5.1

tqdm

4.67.1

traitlets

5.14.3

typeguard

4.3.0

typer-slim

0.21.1

types-python-dateutil

2.9.0.20251115

typing-inspect

0.9.0

typing_extensions

4.12.2

tzdata

2024.1

ujson

5.10.0

unattended-upgrades

0.1

uri-template

1.3.0

urllib3

2.3.0

uuid_utils

0.12.0

uvicorn

0.40.0

virtualenv

20.29.3

wadllib

1.3.6

wcwidth

0.2.5

webcolors

25.10.0

webencodings

0.5.1

websocket-client

1.8.0

whatthepatch

1.0.2

wheel

0.45.1

whenever

0.7.3

widgetsnbextension

3.6.6

wrapt

1.17.0

yapf

0.40.2

yarl

1.18.0

zipp

3.21.0

zstandard

0.23.0

Installed R libraries​

R libraries are installed from the Posit Package Manager CRAN snapshot on 2025-11-20.

Library

Version

Library

Version

Library

Version

arrow

22.0.0

askpass

1.2.1

assertthat

0.2.1

backports

1.5.0

base

4.5.1

base64enc

0.1-3

bigD

0.3.1

bit

4.6.0

bit64

4.6.0-1

bitops

1.0-9

blob

1.2.4

boot

1.3-30

brew

1.0-10

brio

1.1.5

broom

1.0.10

bslib

0.9.0

cachem

1.1.0

callr

3.7.6

caret

7.0-1

cellranger

1.1.0

chron

2.3-62

class

7.3-22

cli

3.6.5

clipr

0.8.0

clock

0.7.3

cluster

2.1.6

codetools

0.2-20

commonmark

2.0.0

compiler

4.5.1

config

0.3.2

conflicted

1.2.0

cpp11

0.5.2

crayon

1.5.3

credentials

2.0.3

curl

7.0.0

data.table

1.17.8

datasets

4.5.1

DBI

1.2.3

dbplyr

2.5.1

desc

1.4.3

devtools

2.4.6

diagram

1.6.5

diffobj

0.3.6

digest

0.6.39

downlit

0.4.5

dplyr

1.1.4

dtplyr

1.3.2

e1071

1.7-16

ellipsis

0.3.2

evaluate

1.0.5

fansi

1.0.7

farver

2.1.2

fastmap

1.2.0

fontawesome

0.5.3

forcats

1.0.1

foreach

1.5.2

foreign

0.8-86

fs

1.6.6

future

1.68.0

future.apply

1.20.0

gargle

1.6.0

generics

0.1.4

gert

2.2.0

ggplot2

4.0.1

gh

1.5.0

git2r

0.36.2

gitcreds

0.1.2

glmnet

4.1-10

globals

0.18.0

glue

1.8.0

googledrive

2.1.2

googlesheets4

1.1.2

gower

1.0.2

graphics

4.5.1

grDevices

4.5.1

grid

4.5.1

gridExtra

2.3

gsubfn

0.7

gt

1.1.0

gtable

0.3.6

hardhat

1.4.2

haven

2.5.5

highr

0.11

hms

1.1.4

htmltools

0.5.8.1

htmlwidgets

1.6.4

httpuv

1.6.16

httr

1.4.7

httr2

1.2.1

ids

1.0.1

ini

0.3.1

ipred

0.9-15

isoband

0.2.7

iterators

1.0.14

jquerylib

0.1.4

jsonlite

2.0.0

juicyjuice

0.1.0

KernSmooth

2.23-22

knitr

1.50

labeling

0.4.3

later

1.4.4

lattice

0.22-5

lava

1.8.2

lifecycle

1.0.4

listenv

0.10.0

litedown

0.8

lubridate

1.9.4

magrittr

2.0.4

markdown

2.0

MASS

7.3-60.0.1

Matrix

1.6-5

memoise

2.0.1

methods

4.5.1

mgcv

1.9-1

mime

0.13

miniUI

0.1.2

mlflow

3.6.0

ModelMetrics

1.2.2.2

modelr

0.1.11

nlme

3.1-164

nnet

7.3-19

numDeriv

2016.8-1.1

openssl

2.3.4

otel

0.2.0

parallel

4.5.1

parallelly

1.45.1

pillar

1.11.1

pkgbuild

1.4.8

pkgconfig

2.0.3

pkgdown

2.2.0

pkgload

1.4.1

plogr

0.2.0

plyr

1.8.9

praise

1.0.0

prettyunits

1.2.0

pROC

1.19.0.1

processx

3.8.6

prodlim

2025.04.28

profvis

0.4.0

progress

1.2.3

progressr

0.18.0

promises

1.5.0

proto

1.0.0

proxy

0.4-27

ps

1.9.1

purrr

1.2.0

R6

2.6.1

ragg

1.5.0

randomForest

4.7-1.2

rappdirs

0.3.3

rcmdcheck

1.4.0

RColorBrewer

1.1-3

Rcpp

1.1.0

RcppEigen

0.3.4.0.2

reactable

0.4.4

reactR

0.6.1

readr

2.1.6

readxl

1.4.5

recipes

1.3.1

rematch

2.0.0

rematch2

2.1.2

remotes

2.5.0

reprex

2.1.1

reshape2

1.4.5

rlang

1.1.6

rmarkdown

2.30

RODBC

1.3-26

roxygen2

7.3.3

rpart

4.1.23

rprojroot

2.1.1

Rserve

1.8-15

RSQLite

2.4.4

rstudioapi

0.17.1

rversions

3.0.0

rvest

1.0.5

S7

0.2.1

sass

0.4.10

scales

1.4.0

selectr

0.4-2

sessioninfo

1.2.3

shape

1.4.6.1

shiny

1.11.1

sourcetools

0.1.7-1

sparklyr

1.9.3

SparkR

4.1.0

sparsevctrs

0.3.4

spatial

7.3-17

splines

4.5.1

sqldf

0.4-11

SQUAREM

2021.1

stats

4.5.1

stats4

4.5.1

stringi

1.8.7

stringr

1.6.0

survival

3.5-8

swagger

5.17.14.1

sys

3.4.3

systemfonts

1.3.1

tcltk

4.5.1

testthat

3.3.0

textshaping

1.0.4

tibble

3.3.0

tidyr

1.3.1

tidyselect

1.2.1

tidyverse

2.0.0

timechange

0.3.0

timeDate

4051.111

tinytex

0.58

tools

4.5.1

tzdb

0.5.0

urlchecker

1.0.1

usethis

3.2.1

utf8

1.2.6

utils

4.5.1

uuid

1.2-1

V8

8.0.1

vctrs

0.6.5

viridisLite

0.4.2

vroom

1.6.6

waldo

0.6.2

whisker

0.4.1

withr

3.0.2

xfun

0.54

xml2

1.5.0

xopen

1.0.1

xtable

1.8-4

yaml

2.3.10

zeallot

0.2.0

zip

2.3.3

Installed Java and Scala libraries (Scala 2.13 cluster version)​

Group ID

Artifact ID

Version

antlr

antlr

2.7.7

com.amazonaws

amazon-kinesis-client

1.15.3

com.amazonaws

aws-java-sdk-autoscaling

1.12.681

com.amazonaws

aws-java-sdk-cloudformation

1.12.681

com.amazonaws

aws-java-sdk-cloudfront

1.12.681

com.amazonaws

aws-java-sdk-cloudhsm

1.12.681

com.amazonaws

aws-java-sdk-cloudsearch

1.12.681

com.amazonaws

aws-java-sdk-cloudtrail

1.12.681

com.amazonaws

aws-java-sdk-cloudwatch

1.12.681

com.amazonaws

aws-java-sdk-cloudwatchmetrics

1.12.681

com.amazonaws

aws-java-sdk-codedeploy

1.12.681

com.amazonaws

aws-java-sdk-cognitoidentity

1.12.681

com.amazonaws

aws-java-sdk-cognitosync

1.12.681

com.amazonaws

aws-java-sdk-config

1.12.681

com.amazonaws

aws-java-sdk-core

1.12.681

com.amazonaws

aws-java-sdk-datapipeline

1.12.681

com.amazonaws

aws-java-sdk-directconnect

1.12.681

com.amazonaws

aws-java-sdk-directory

1.12.681

com.amazonaws

aws-java-sdk-dynamodb

1.12.681

com.amazonaws

aws-java-sdk-ec2

1.12.681

com.amazonaws

aws-java-sdk-ecs

1.12.681

com.amazonaws

aws-java-sdk-efs

1.12.681

com.amazonaws

aws-java-sdk-elasticache

1.12.681

com.amazonaws

aws-java-sdk-elasticbeanstalk

1.12.681

com.amazonaws

aws-java-sdk-elasticloadbalancing

1.12.681

com.amazonaws

aws-java-sdk-elastictranscoder

1.12.681

com.amazonaws

aws-java-sdk-emr

1.12.681

com.amazonaws

aws-java-sdk-glacier

1.12.681

com.amazonaws

aws-java-sdk-glue

1.12.681

com.amazonaws

aws-java-sdk-iam

1.12.681

com.amazonaws

aws-java-sdk-importexport

1.12.681

com.amazonaws

aws-java-sdk-kinesis

1.12.681

com.amazonaws

aws-java-sdk-kms

1.12.681

com.amazonaws

aws-java-sdk-lambda

1.12.681

com.amazonaws

aws-java-sdk-logs

1.12.681

com.amazonaws

aws-java-sdk-machinelearning

1.12.681

com.amazonaws

aws-java-sdk-opsworks

1.12.681

com.amazonaws

aws-java-sdk-rds

1.12.681

com.amazonaws

aws-java-sdk-redshift

1.12.681

com.amazonaws

aws-java-sdk-route53

1.12.681

com.amazonaws

aws-java-sdk-s3

1.12.681

com.amazonaws

aws-java-sdk-ses

1.12.681

com.amazonaws

aws-java-sdk-simpledb

1.12.681

com.amazonaws

aws-java-sdk-simpleworkflow

1.12.681

com.amazonaws

aws-java-sdk-sns

1.12.681

com.amazonaws

aws-java-sdk-sqs

1.12.681

com.amazonaws

aws-java-sdk-ssm

1.12.681

com.amazonaws

aws-java-sdk-storagegateway

1.12.681

com.amazonaws

aws-java-sdk-sts

1.12.681

com.amazonaws

aws-java-sdk-support

1.12.681

com.amazonaws

aws-java-sdk-workspaces

1.12.681

com.amazonaws

jmespath-java

1.12.681

com.clearspring.analytics

stream

2.9.8

com.databricks

Rserve

1.8-3

com.databricks

databricks-sdk-java

0.53.0

com.databricks

jets3t

0.7.1-0

com.databricks.scalapb

scalapb-runtime_2.13

0.4.15-11

com.esotericsoftware

kryo-shaded

4.0.3

com.esotericsoftware

minlog

1.3.0

com.fasterxml

classmate

1.5.1

com.fasterxml.jackson.core

jackson-annotations

2.18.2

com.fasterxml.jackson.core

jackson-core

2.18.2

com.fasterxml.jackson.core

jackson-databind

2.18.2

com.fasterxml.jackson.dataformat

jackson-dataformat-cbor

2.18.2

com.fasterxml.jackson.dataformat

jackson-dataformat-yaml

2.15.2

com.fasterxml.jackson.datatype

jackson-datatype-joda

2.18.2

com.fasterxml.jackson.datatype

jackson-datatype-jsr310

2.18.3

com.fasterxml.jackson.module

jackson-module-paranamer

2.18.2

com.fasterxml.jackson.module

jackson-module-scala_2.13

2.18.2

com.github.ben-manes.caffeine

caffeine

2.9.3

com.github.blemale

scaffeine_2.13

4.1.0

com.github.fommil

jniloader

1.1

com.github.fommil.netlib

native_ref-java

1.1

com.github.fommil.netlib

native_ref-java

1.1-natives

com.github.fommil.netlib

native_system-java

1.1

com.github.fommil.netlib

native_system-java

1.1-natives

com.github.fommil.netlib

netlib-native_ref-linux-x86_64

1.1-natives

com.github.fommil.netlib

netlib-native_system-linux-x86_64

1.1-natives

com.github.luben

zstd-jni

1.5.7-6

com.github.virtuald

curvesapi

1.08

com.github.wendykierp

JTransforms

3.1

com.google.api.grpc

proto-google-common-protos

2.5.1

com.google.auth

google-auth-library-credentials

1.20.0

com.google.auth

google-auth-library-oauth2-http

1.20.0

com.google.auto.value

auto-value-annotations

1.10.4

com.google.code.findbugs

jsr305

3.0.0

com.google.code.gson

gson

2.11.0

com.google.crypto.tink

tink

1.16.0

com.google.errorprone

error_prone_annotations

2.36.0

com.google.flatbuffers

flatbuffers-java

25.2.10

com.google.guava

failureaccess

1.0.3

com.google.guava

guava

33.4.8-jre

com.google.http-client

google-http-client

1.43.3

com.google.http-client

google-http-client-gson

1.43.3

com.google.j2objc

j2objc-annotations

3.0.0

com.google.protobuf

protobuf-java

3.25.5

com.google.protobuf

protobuf-java-util

3.25.5

com.helger

profiler

1.1.1

com.ibm.icu

icu4j

75.1

com.jcraft

jsch

0.1.55

com.lihaoyi

sourcecode_2.13

0.1.9

com.microsoft.azure

azure-data-lake-store-sdk

2.3.10

com.microsoft.sqlserver

mssql-jdbc

12.8.0.jre11

com.microsoft.sqlserver

mssql-jdbc

12.8.0.jre8

com.ning

compress-lzf

1.1.2

com.sun.mail

javax.mail

1.5.2

com.sun.xml.bind

jaxb-core

2.2.11

com.sun.xml.bind

jaxb-impl

2.2.11

com.tdunning

json

1.8

com.thoughtworks.paranamer

paranamer

2.8

com.trueaccord.lenses

lenses_2.13

0.4.13

com.twitter

chill-java

0.10.0

com.twitter

chill_2.13

0.10.0

com.twitter

util-app_2.13

19.8.1

com.twitter

util-core_2.13

19.8.1

com.twitter

util-function_2.13

19.8.1

com.twitter

util-jvm_2.13

19.8.1

com.twitter

util-lint_2.13

19.8.1

com.twitter

util-registry_2.13

19.8.1

com.twitter

util-stats_2.13

19.8.1

com.typesafe

config

1.4.3

com.typesafe.scala-logging

scala-logging_2.13

3.9.2

com.uber

h3

3.7.3

com.univocity

univocity-parsers

2.9.1

com.zaxxer

HikariCP

4.0.3

com.zaxxer

SparseBitSet

1.3

commons-cli

commons-cli

1.10.0

commons-codec

commons-codec

1.19.0

commons-collections

commons-collections

3.2.2

commons-dbcp

commons-dbcp

1.4

commons-fileupload

commons-fileupload

1.6.0

commons-httpclient

commons-httpclient

3.1

commons-io

commons-io

2.21.0

commons-lang

commons-lang

2.6

commons-logging

commons-logging

1.1.3

commons-pool

commons-pool

1.5.4

dev.ludovic.netlib

arpack

3.0.4

dev.ludovic.netlib

blas

3.0.4

dev.ludovic.netlib

lapack

3.0.4

info.ganglia.gmetric4j

gmetric4j

1.0.10

io.airlift

aircompressor

2.0.2

io.delta

delta-sharing-client_2.13

1.3.9

io.dropwizard.metrics

metrics-annotation

4.2.37

io.dropwizard.metrics

metrics-core

4.2.37

io.dropwizard.metrics

metrics-graphite

4.2.37

io.dropwizard.metrics

metrics-healthchecks

4.2.37

io.dropwizard.metrics

metrics-jetty10

4.2.37

io.dropwizard.metrics

metrics-jmx

4.2.37

io.dropwizard.metrics

metrics-json

4.2.37

io.dropwizard.metrics

metrics-jvm

4.2.37

io.dropwizard.metrics

metrics-servlets

4.2.37

io.github.java-diff-utils

java-diff-utils

4.15

io.netty

netty-all

4.2.7.Final

io.netty

netty-buffer

4.2.7.Final

io.netty

netty-codec

4.2.7.Final

io.netty

netty-codec-base

4.2.7.Final

io.netty

netty-codec-classes-quic

4.2.7.Final

io.netty

netty-codec-compression

4.2.7.Final

io.netty

netty-codec-http

4.2.7.Final

io.netty

netty-codec-http2

4.2.7.Final

io.netty

netty-codec-http3

4.2.7.Final

io.netty

netty-codec-marshalling

4.2.7.Final

io.netty

netty-codec-native-quic

4.2.7.Final-linux-aarch_64

io.netty

netty-codec-native-quic

4.2.7.Final-linux-x86_64

io.netty

netty-codec-native-quic

4.2.7.Final-osx-aarch_64

io.netty

netty-codec-native-quic

4.2.7.Final-osx-x86_64

io.netty

netty-codec-native-quic

4.2.7.Final-windows-x86_64

io.netty

netty-codec-protobuf

4.2.7.Final

io.netty

netty-codec-socks

4.2.7.Final

io.netty

netty-common

4.2.7.Final

io.netty

netty-handler

4.2.7.Final

io.netty

netty-handler-proxy

4.2.7.Final

io.netty

netty-resolver

4.2.7.Final

io.netty

netty-tcnative-boringssl-static

2.0.74.Final-db-r0-linux-aarch_64

io.netty

netty-tcnative-boringssl-static

2.0.74.Final-db-r0-linux-x86_64

io.netty

netty-tcnative-boringssl-static

2.0.74.Final-db-r0-osx-aarch_64

io.netty

netty-tcnative-boringssl-static

2.0.74.Final-db-r0-osx-x86_64

io.netty

netty-tcnative-boringssl-static

2.0.74.Final-db-r0-windows-x86_64

io.netty

netty-tcnative-classes

2.0.74.Final

io.netty

netty-transport

4.2.7.Final

io.netty

netty-transport-classes-epoll

4.2.7.Final

io.netty

netty-transport-classes-io_uring

4.2.7.Final

io.netty

netty-transport-classes-kqueue

4.2.7.Final

io.netty

netty-transport-native-epoll

4.2.7.Final

io.netty

netty-transport-native-epoll

4.2.7.Final-linux-aarch_64

io.netty

netty-transport-native-epoll

4.2.7.Final-linux-riscv64

io.netty

netty-transport-native-epoll

4.2.7.Final-linux-x86_64

io.netty

netty-transport-native-io_uring

4.2.7.Final-linux-aarch_64

io.netty

netty-transport-native-io_uring

4.2.7.Final-linux-riscv64

io.netty

netty-transport-native-io_uring

4.2.7.Final-linux-x86_64

io.netty

netty-transport-native-kqueue

4.2.7.Final-osx-aarch_64

io.netty

netty-transport-native-kqueue

4.2.7.Final-osx-x86_64

io.netty

netty-transport-native-unix-common

4.2.7.Final

io.opencensus

opencensus-api

0.31.1

io.opencensus

opencensus-contrib-http-util

0.31.1

io.prometheus

simpleclient

0.16.1-databricks

io.prometheus

simpleclient_common

0.16.1-databricks

io.prometheus

simpleclient_dropwizard

0.16.1-databricks

io.prometheus

simpleclient_pushgateway

0.16.1-databricks

io.prometheus

simpleclient_servlet

0.16.1-databricks

io.prometheus

simpleclient_servlet_common

0.16.1-databricks

io.prometheus

simpleclient_tracer_common

0.16.1-databricks

io.prometheus

simpleclient_tracer_otel

0.16.1-databricks

io.prometheus

simpleclient_tracer_otel_agent

0.16.1-databricks

io.prometheus.jmx

collector

0.18.0

jakarta.annotation

jakarta.annotation-api

1.3.5

jakarta.servlet

jakarta.servlet-api

4.0.3

jakarta.validation

jakarta.validation-api

2.0.2

jakarta.ws.rs

jakarta.ws.rs-api

2.1.6

javax.activation

activation

1.1.1

javax.annotation

javax.annotation-api

1.3.2

javax.el

javax.el-api

2.2.4

javax.jdo

jdo-api

3.0.1

javax.media

jai_core

jai_core_dummy

javax.transaction

jta

1.1

javax.transaction

transaction-api

1.1

javax.xml.bind

jaxb-api

2.2.11

javolution

javolution

5.5.1

jline

jline

2.14.6

joda-time

joda-time

2.14.0

net.java.dev.jna

jna

5.8.0

net.razorvine

pickle

1.5

net.sf.jpam

jpam

1.1

net.sf.opencsv

opencsv

2.3

net.sf.supercsv

super-csv

2.2.0

net.snowflake

snowflake-ingest-sdk

0.9.6

net.sourceforge.f2j

arpack_combined_all

0.1

org.acplt.remotetea

remotetea-oncrpc

1.1.2

org.antlr

ST4

4.0.4

org.antlr

antlr-runtime

3.5.2

org.antlr

antlr4-runtime

4.13.1

org.antlr

stringtemplate

3.2.1

org.apache.ant

ant

1.10.11

org.apache.ant

ant-jsch

1.10.11

org.apache.ant

ant-launcher

1.10.11

org.apache.arrow

arrow-compression

18.3.0

org.apache.arrow

arrow-format

18.3.0

org.apache.arrow

arrow-memory-core

18.3.0

org.apache.arrow

arrow-memory-netty

18.3.0

org.apache.arrow

arrow-memory-netty-buffer-patch

18.3.0

org.apache.arrow

arrow-vector

18.3.0

org.apache.avro

avro

1.12.1

org.apache.avro

avro-ipc

1.12.1

org.apache.avro

avro-mapred

1.12.1

org.apache.commons

commons-collections4

4.5.0

org.apache.commons

commons-compress

1.28.0

org.apache.commons

commons-configuration2

2.11.0

org.apache.commons

commons-crypto

1.1.0

org.apache.commons

commons-lang3

3.19.0

org.apache.commons

commons-math3

3.6.1

org.apache.commons

commons-text

1.14.0

org.apache.curator

curator-client

5.9.0

org.apache.curator

curator-framework

5.9.0

org.apache.curator

curator-recipes

5.9.0

org.apache.datasketches

datasketches-java

6.2.0

org.apache.datasketches

datasketches-memory

3.0.2

org.apache.derby

derby

10.14.2.0

org.apache.hadoop

hadoop-client-runtime

3.4.2

org.apache.hive

hive-beeline

2.3.10

org.apache.hive

hive-cli

2.3.10

org.apache.hive

hive-jdbc

2.3.10

org.apache.hive

hive-llap-client

2.3.10

org.apache.hive

hive-llap-common

2.3.10

org.apache.hive

hive-serde

2.3.10

org.apache.hive

hive-shims

2.3.10

org.apache.hive

hive-storage-api

2.8.1

org.apache.hive.shims

hive-shims-0.23

2.3.10

org.apache.hive.shims

hive-shims-common

2.3.10

org.apache.hive.shims

hive-shims-scheduler

2.3.10

org.apache.httpcomponents

httpclient

4.5.14

org.apache.httpcomponents

httpcore

4.4.16

org.apache.ivy

ivy

2.5.3

org.apache.logging.log4j

log4j-1.2-api

2.24.3

org.apache.logging.log4j

log4j-api

2.24.3

org.apache.logging.log4j

log4j-core

2.24.3

org.apache.logging.log4j

log4j-layout-template-json

2.24.3

org.apache.logging.log4j

log4j-slf4j2-impl

2.24.3

org.apache.orc

orc-core

2.2.0-shaded-protobuf

org.apache.orc

orc-format

1.1.1-shaded-protobuf

org.apache.orc

orc-mapreduce

2.2.0-shaded-protobuf

org.apache.orc

orc-shims

2.2.0

org.apache.poi

poi

5.4.1

org.apache.poi

poi-ooxml

5.4.1

org.apache.poi

poi-ooxml-full

5.4.1

org.apache.poi

poi-ooxml-lite

5.4.1

org.apache.thrift

libfb303

0.9.3

org.apache.thrift

libthrift

0.16.0

org.apache.ws.xmlschema

xmlschema-core

2.3.1

org.apache.xbean

xbean-asm9-shaded

4.28

org.apache.xmlbeans

xmlbeans

5.3.0

org.apache.yetus

audience-annotations

0.13.0

org.apache.zookeeper

zookeeper

3.9.4

org.apache.zookeeper

zookeeper-jute

3.9.4

org.checkerframework

checker-qual

3.43.0

org.codehaus.janino

commons-compiler

3.0.16

org.codehaus.janino

janino

3.0.16

org.datanucleus

datanucleus-api-jdo

4.2.4

org.datanucleus

datanucleus-core

4.1.17

org.datanucleus

datanucleus-rdbms

4.1.19

org.datanucleus

javax.jdo

3.2.0-m3

org.eclipse.jetty

jetty-alpn-client

10.0.26

org.eclipse.jetty

jetty-client

10.0.26

org.eclipse.jetty

jetty-http

10.0.26

org.eclipse.jetty

jetty-io

10.0.26

org.eclipse.jetty

jetty-jndi

10.0.26

org.eclipse.jetty

jetty-plus

10.0.26

org.eclipse.jetty

jetty-proxy

10.0.26

org.eclipse.jetty

jetty-security

10.0.26

org.eclipse.jetty

jetty-server

10.0.26

org.eclipse.jetty

jetty-servlet

10.0.26

org.eclipse.jetty

jetty-servlets

10.0.26

org.eclipse.jetty

jetty-util

10.0.26

org.eclipse.jetty

jetty-webapp

10.0.26

org.eclipse.jetty

jetty-xml

10.0.26

org.fusesource.leveldbjni

leveldbjni-all

1.8

org.glassfish.hk2

hk2-api

2.6.1

org.glassfish.hk2

hk2-locator

2.6.1

org.glassfish.hk2

hk2-utils

2.6.1

org.glassfish.hk2

osgi-resource-locator

1.0.3

org.glassfish.hk2.external

aopalliance-repackaged

2.6.1

org.glassfish.hk2.external

jakarta.inject

2.6.1

org.glassfish.jersey.containers

jersey-container-servlet

2.41

org.glassfish.jersey.containers

jersey-container-servlet-core

2.41

org.glassfish.jersey.core

jersey-client

2.41

org.glassfish.jersey.core

jersey-common

2.41

org.glassfish.jersey.core

jersey-server

2.41

org.glassfish.jersey.inject

jersey-hk2

2.41

org.hibernate.validator

hibernate-validator

6.2.5.Final

org.ini4j

ini4j

0.5.4

org.javassist

javassist

3.29.2-GA

org.jboss.logging

jboss-logging

3.4.1.Final

org.jdbi

jdbi

2.63.1

org.jline

jline

3.27.1-jdk8

org.joda

joda-convert

1.7

org.json4s

json4s-ast_2.13

4.0.7

org.json4s

json4s-core_2.13

4.0.7

org.json4s

json4s-jackson-core_2.13

4.0.7

org.json4s

json4s-jackson_2.13

4.0.7

org.json4s

json4s-scalap_2.13

4.0.7

org.locationtech.jts

jts-core

1.20.0

org.lz4

lz4-java

1.8.0-databricks-1

org.mlflow

mlflow-spark_2.13

2.22.1

org.objenesis

objenesis

3.4

org.postgresql

postgresql

42.6.1

org.roaringbitmap

RoaringBitmap

1.2.1

org.rocksdb

rocksdbjni

9.8.4

org.rosuda.REngine

REngine

2.1.0

org.scala-lang

scala-compiler_2.13

2.13.16

org.scala-lang

scala-library_2.13

2.13.16

org.scala-lang

scala-reflect_2.13

2.13.16

org.scala-lang.modules

scala-collection-compat_2.13

2.11.0

org.scala-lang.modules

scala-java8-compat_2.13

0.9.1

org.scala-lang.modules

scala-parallel-collections_2.13

1.2.0

org.scala-lang.modules

scala-parser-combinators_2.13

2.4.0

org.scala-lang.modules

scala-xml_2.13

2.4.0

org.scala-sbt

test-interface

1.0

org.scalacheck

scalacheck_2.13

1.18.0

org.scalactic

scalactic_2.13

3.2.19

org.scalanlp

breeze-macros_2.13

2.1.0

org.scalanlp

breeze_2.13

2.1.0

org.scalatest

scalatest-compatible

3.2.19

org.scalatest

scalatest-core_2.13

3.2.19

org.scalatest

scalatest-diagrams_2.13

3.2.19

org.scalatest

scalatest-featurespec_2.13

3.2.19

org.scalatest

scalatest-flatspec_2.13

3.2.19

org.scalatest

scalatest-freespec_2.13

3.2.19

org.scalatest

scalatest-funspec_2.13

3.2.19

org.scalatest

scalatest-funsuite_2.13

3.2.19

org.scalatest

scalatest-matchers-core_2.13

3.2.19

org.scalatest

scalatest-mustmatchers_2.13

3.2.19

org.scalatest

scalatest-propspec_2.13

3.2.19

org.scalatest

scalatest-refspec_2.13

3.2.19

org.scalatest

scalatest-shouldmatchers_2.13

3.2.19

org.scalatest

scalatest-wordspec_2.13

3.2.19

org.scalatest

scalatest_2.13

3.2.19

org.slf4j

jcl-over-slf4j

2.0.16

org.slf4j

jul-to-slf4j

2.0.16

org.slf4j

slf4j-api

2.0.16

org.slf4j

slf4j-simple

1.7.25

org.threeten

threeten-extra

1.8.0

org.tukaani

xz

1.10

org.typelevel

algebra_2.13

2.8.0

org.typelevel

cats-kernel_2.13

2.8.0

org.typelevel

spire-macros_2.13

0.18.0

org.typelevel

spire-platform_2.13

0.18.0

org.typelevel

spire-util_2.13

0.18.0

org.typelevel

spire_2.13

0.18.0

org.wildfly.openssl

wildfly-openssl

1.1.3.Final

org.xerial

sqlite-jdbc

3.42.0.0

org.xerial.snappy

snappy-java

1.1.10.3

org.yaml

snakeyaml

2.0

oro

oro

2.0.8

pl.edu.icm

JLargeArrays

1.5

software.amazon.cryptools

AmazonCorrettoCryptoProvider

2.5.0-linux-x86_64

stax

stax-api

1.0.1