Databricks Runtime 10.5 (Unsupported)

The following release notes provide information about Databricks Runtime 10.5, powered by Apache Spark 3.2.1. Databricks released these images in May 2022.

New features and improvements

SQL: New functions to_number and try_to_number

The to_number function and try_to_number function allow you to convert character representations of numbers in various formats to DECIMAL.

SQL: New function try_sum

Unlike sum, the try_sum aggregate function returns NULL instead of an error when an overflow occurs.

SQL: Update to ceil, ceiling, and floor functions

The ceil function, ceiling function, and floor function now support a second optional argument to specify a target scale.

SQL: Update to contains, startswith, and endswith functions

The contains function, startswith function, and endswith function now support binary data types as input in addition to strings.

SQL: New function array_size

The new array_size function calculates the size of an array.

Unity Catalog: SQL LIST output removes is_directory column and adds trailing / for directory paths (Public Preview)

The output of the SQL LIST statement for listing paths governed by Unity Catalog has changed. The is_directory column is no longer displayed. Instead, paths that represent directories now end with a trailing slash (/) for the path and name columns. This change is for consistency with the output of the dbutils.fs.ls command.

Auto Loader: new SQL function CLOUD_FILES_STATE

You can use the new CLOUD_FILES_STATE function to query the internal state of an Auto Loader stream; see Querying files discovered by Auto Loader.

Auto Loader: file notification support is now GA

File notification support for What is Auto Loader? on Google Cloud is now generally available.

Delta Lake: new maxRecordsPerFile option for maximum records written to a single file

When you use the DataFrame APIs to write to a Delta table, you can use the maxRecordsPerFile option to specify the maximum number of records to write to a single file. Specifying a value of zero or a negative value represents no limit. See Configure Delta Lake to control data file size.

Deprecation of Koalas

Koalas is now deprecated. Koalas provides pandas APIs on top of Apache Spark. Koalas was contributed to by the community and has been available as pandas API on Spark beginning in Apache Spark 3.2. Koalas continues to be preinstalled for Databricks Runtime 10.5, but Koalas will no longer be preinstalled beginning with the next Databricks Runtime version.

To learn more about pandas API on Spark, see:

Google BigQuery connector: Arrow compression codec for reads is now configurable

You can now configure the Arrow compression codec that the Google BigQuery connector uses for reads, by setting the arrowCompressionCodec DataFrame option. Available options include:

New _metadata column for all file formats

You can get metadata information for input files with the _metadata column. The _metadata column is hidden. This column is available for all input file formats. To include the _metadata column in a returned DataFrame, you must explicitly reference this column in your query. See File metadata column.

Test your Delta Live Tables pipeline with Databricks Runtime 10.5

Databricks Runtime 10.5 is now the supported preview version for Delta Live Tables. To test your pipeline with Databricks Runtime 10.5, set the channel setting to preview in your pipeline settings. See pipeline settings.

Bug fixes

  • Fixed the display of numUpdatedRows and numCopiedRows metric results for update operations on Delta tables that have change data capture (CDC) enabled.

Library upgrades

  • Upgraded Python libraries:

    • plotly from 5.5.0 to 5.6.0

  • Upgraded R libraries:

    • base from 4.1.2 to 4.1.3

    • caret from 6.0-90 to 6.0-91

    • cluster from 2.1.2 to 2.1.3

    • commonmark from 1.7 to 1.8.0

    • compiler from 4.1.2 to 4.1.3

    • crayon from 1.5.0 to 1.5.1

    • datasets from 4.1.2 to 4.1.3

    • desc from 1.4.0 to 1.4.1

    • fansi from 1.0.2 to 1.0.3

    • glue from 1.6.1 to 1.6.2

    • graphics from 4.1.2 to 4.1.3

    • grDevices from 4.1.2 to 4.1.3

    • grid from 4.1.2 to 4.1.3

    • knitr from 1.37 to 1.38

    • MASS from 7.3-55 to 7.3-56

    • Matrix from 1.4-0 to 1.4-1

    • methods from 4.1.2 to 4.1.3

    • mgcv from 1.8-39 to 1.8-40

    • nlme from 3.1-155 to 3.1-157

    • openssl from 1.4.6 to 2.0.0

    • parallel from 4.1.2 to 4.1.3

    • plyr from 1.8.6 to 1.8.7

    • processx from 3.5.2 to 3.5.3

    • r2d3 from 0.2.5 to 0.2.6

    • Rcpp from 1.0.8 to 1.0.8.3

    • rlang from 1.0.1 to 1.0.2

    • rmarkdown from 2.11 to 2.13

    • RSQLite from 2.2.10 to 2.2.11

    • sass from 0.4.0 to 0.4.1

    • splines from 4.1.2 to 4.1.3

    • stats from 4.1.2 to 4.1.3

    • stats4 from 4.1.2 to 4.1.3

    • tcltk from 4.1.2 to 4.1.3

    • tools from 4.1.2 to 4.1.3

    • utils from 4.1.2 to 4.1.3

    • uuid from 1.0-3 to 1.0-4

    • waldo from 0.3.1 to 0.4.0

    • withr from 2.4.3 to 2.5.0

    • xfun from 0.29 to 0.30

  • Upgraded Java libraries:

    • com.h2database.h2 from 1.4.195 to 2.1.210

    • org.apache.hadoop.hadoop-client-api from 3.3.1-databricks to 3.3.2-databricks

    • org.apache.hadoop.hadoop-client-runtime from 3.3.1 to 3.3.2

    • org.rocksdb.rocksdbjni from 6.24.2 to 6.20.3

Apache Spark

Databricks Runtime 10.5 includes Apache Spark 3.2.1. This release includes all Spark fixes and improvements included in Databricks Runtime 10.4 LTS, as well as the following additional bug fixes and improvements made to Spark:

  • [SPARK-38354] [SC-97886] Revert “[SQL] Add hash probes metric for shuffled hash join”

  • [SPARK-38484] [SAS-103][SAS-111][SAS-113][SC-98102][Cherry-Pick] Instrument PySpark SQL module with general usage logger

  • [SPARK-38333] [SQL] PlanExpression expression should skip addExprTree function in Executor

  • [SPARK-38646] [PYTHON][10.x] Pull a trait out for Python functions

  • [SPARK-38674] [PHOTON] Add a metric to PhotonSubqueryBroadcastExec

  • [SPARK-38787] [SS] Replace found value with non-null element in the remaining list for key and remove remaining null elements from values in keyWithIndexToValue store for stream-stream joins

  • [SPARK-38446] [Core] Fix deadlock between ExecutorClassLoader and FileDownloadCallback caused by Log4j

  • [SPARK-38705] [SQL] Use function identifier in create and drop function command

  • [SPARK-38684] [SS] Fix correctness issue on stream-stream outer join with RocksDB state store provider

  • [SPARK-38655] [SQL] OffsetWindowFunctionFrameBase cannot find the offset row whose input is not-null

  • [SPARK-38204] [SS] Use StatefulOpClusteredDistribution for stateful operators with respecting backward compatibility

  • [SPARK-38616] [SQL] Keep track of SQL query text in Catalyst TreeNode

  • [SPARK-38600] [SQL] Include unit into the sql string of TIMESTAMPADD/DIFF

  • [SPARK-34805] [SQL] Propagate metadata from nested columns in Alias

  • [SPARK-38535] [SQL] Add the datetimeUnit enum and use it in TIMESTAMPADD/DIFF

  • [SPARK-38415] [SQL] Update the histogram_numeric (x, y) result type to make x == the input type

  • [SPARK-38148] [SQL] Do not add dynamic partition pruning if there exists static partition pruning

  • [SPARK-37527] [SQL] Compile COVAR_POP, COVAR_SAMP and CORR in H2Dialet

  • [SPARK-38510] [SQL] Retry ClassSymbol.selfType to work around cyclic references

  • [SPARK-38509] [SQL] Unregister the TIMESTAMPADD/DIFF functions and remove DATE_ADD/DIFF

  • [SPARK-38504] [SQL] Cannot read TimestampNTZ as TimestampLTZ

  • [SPARK-38526] [SQL] Fix misleading function alias name for RuntimeReplaceable

  • [SPARK-38335] [SQL] Implement parser support for DEFAULT column values

  • [SPARK-38593] [SS] Carry over the metric of the number of dropped late events into SessionWindowStateStoreSaveExec

  • [SPARK-38549] [SS] Add numRowsDroppedByWatermark to SessionWindowStateStoreRestoreExec

  • [SPARK-38583] [SQL] Restore the behavior of to_timestamp that allows numeric types

  • [SPARK-38481] [SQL] Substitute Java overflow exception from TIMESTAMPADD by Spark exception

  • [SPARK-37753] [SQL] Fine tune logic to demote Broadcast hash join in DynamicJoinSelection

  • [SPARK-37582] [SPARK-37583][SQL] CONTAINS, STARTSWITH, ENDSWITH should support both string and binary types

  • [SPARK-38345] [SQL] Introduce SQL function ARRAY_SIZE

  • [SPARK-38385] [SQL] Improve error messages of empty statement and <EOF> in ParseException

  • [SPARK-37614] [SQL] Support ANSI Aggregate Function: regr_avgx & regr_avgy

  • [SPARK-38560] [SQL] If Sum, Count, Any accompany with distinct, cannot do partial agg push down.

  • [SPARK-38360] [SQL][SS][PYTHON] Introduce a exists function for TreeNode to eliminate duplicate code patterns

  • [SPARK-38385] [SQL] Improve error messages of ‘mismatched input’ cases from ANTLR

  • [SPARK-38240] [SQL] Improve RuntimeReplaceable and add a guideline for adding new functions

  • [SPARK-38542] [SQL]UnsafeHashedRelation should serialize numKeys out

  • [SPARK-38521] [SQL] Change partitionOverwriteMode from string to variable in Scala

  • [SPARK-38558] [SQL] Remove unnecessary casts between IntegerType and IntDecimal

  • [SPARK-38489] [SQL] Aggregate.groupOnly support foldable expressions

  • [SPARK-38410] [SQL] Support specify initial partition number for rebalance

  • [SPARK-38107] [SQL] Use error classes in the compilation errors of python/pandas UDFs

  • [SPARK-38354] [SQL] Add hash probes metric for shuffled hash join

  • [SPARK-38112] [SQL] Use error classes in the execution errors of date/timestamp handling

  • [SPARK-38361] [SQL] Add factory method getConnection into JDBCDialect.

  • [SPARK-37865] [SQL] Fix union deduplication correctness bug

  • [SPARK-38298] [SQL][Tests] Fix DataExpressionSuite, NullExpressionsSuite, StringExpressionsSuite, complexTypesSuite, CastSuite under ANSI mode

  • [SPARK-38407] [SQL] ANSI Cast: loosen the limitation of casting non-null complex types

  • [SPARK-38352] [SQL] Fix DataFrameAggregateSuite/DataFrameSetOperationsSuite/DataFrameWindowFunctionsSuite under ANSI mode

  • [SPARK-38054] [SQL] Supports list namespaces in JDBC v2 MySQL dialect

  • [SPARK-37960] [SQL] A new framework to represent catalyst expressions in DS v2 APIs

  • [SPARK-38103] [SQL] Migrate parsing errors of transform into the new error framework

  • [SPARK-38519] [SQL] AQE throw exception should respect SparkFatalException

  • [SPARK-38528] [SQL] Eagerly iterate over aggregate sequence when building project list in ExtractGenerator

  • [SPARK-38320] [SS] Fix flatMapGroupsWithState timeout in batch with data for key

  • [SPARK-38442] [SQL] Fix ConstantFoldingSuite/ColumnExpressionSuite/DataFrameSuite/AdaptiveQueryExecSuite under ANSI mode

  • [SPARK-38268] [SQL] Hide the “failOnError” field in the toString method of Abs/CheckOverflow

  • [SPARK-37949] [SQL] Improve Rebalance statistics estimation

  • [SPARK-38450] [SQL] Fix HiveQuerySuite//PushFoldableIntoBranchesSuite/TransposeWindowSuite under ANSI mode

  • [SPARK-38501] [SQL] Fix thriftserver test failures under ANSI mode

  • [SPARK-37895] [SQL] Filter push down column with quoted columns

  • [SPARK-38449] [SQL] Avoid call createTable when ignoreIfExists=true and table exists

  • [SPARK-38406] [SQL] Improve performance of ShufflePartitionsUtil createSkewPartitionSpecs

  • [SPARK-37947] [SQL] Extract generator from GeneratorOuter expression contained by a Generate operator.

  • [SPARK-38309] [CORE] Fix SHS shuffleTotalReads and shuffleTotalBlocks percentile metrics

  • [SPARK-38412] [SS] Fix the swapped sequence of from and to in StateSchemaCompatibilityChecker

  • [SPARK-38267] [CORE][SQL][SS] Replace pattern matches on boolean expressions with conditional statements

  • [SPARK-38104] [SQL] Migrate parsing errors of window into the new error framework

  • [SPARK-38434] [SQL] Correct semantic of CheckAnalysis.getDataTypesAreCompatibleFn method

  • [SPARK-38033] [SS] The SS processing cannot be started because the com…

  • [SPARK-38378] [SQL] Refactoring of the ANTLR grammar definition into separate Parser and Lexer files

  • [SPARK-37475] [SQL] Add scale parameter to floor and ceil functions

  • [SPARK-38411] [CORE] Use UTF-8 when doMergeApplicationListingInternal reads event logs

  • [SPARK-36553] [ML] KMeans avoid compute auxiliary statistics for large K

  • [SPARK-38393] [SQL] Clean up deprecated usage of GenSeq/GenMap

  • [SPARK-38389] [SQL] Add the DATEDIFF() and DATE_DIFF() aliases for TIMESTAMPDIFF()

  • [SPARK-33206] [CORE][3.2] Fix shuffle index cache weight calculation for small index files

  • [SPARK-38353] [PYTHON] Instrument enter and exit magic methods for Pandas API on Spark

  • [SPARK-38323] [SQL][Streaming] Support the hidden file metadata in Streaming

  • [SPARK-38138] [SQL] Materialize QueryPlan subqueries

  • [SPARK-38094] Enable matching schema column names by field ids

  • [SPARK-37923] [SQL] Generate partition transforms for BucketSpec inside parser

  • [SPARK-38363] [SQL] Avoid runtime error in Dataset.summary()/Dataset.describe() when ANSI mode is on

  • [SPARK-38284] [SQL] Add the TIMESTAMPDIFF() function

  • [SPARK-38251] [SQL] Change Cast.toString as “cast” instead of “ansi_cast” under ANSI mode

  • [SPARK-38314] [SQL] Fix of failing to read parquet files after writing the hidden file metadata in

See Databricks Runtime 10.5 maintenance updates.

System environment

  • Operating System: Ubuntu 20.04.4 LTS

  • Java: Zulu 8.56.0.21-CA-linux64

  • Scala: 2.12.14

  • Python: 3.8.10

  • R: 4.1.3

  • Delta Lake: 1.2.1

Installed Python libraries

Library

Version

Library

Version

Library

Version

Antergos Linux

2015.10 (ISO-Rolling)

appdirs

1.4.4

argon2-cffi

20.1.0

async-generator

1.10

attrs

20.3.0

backcall

0.2.0

bidict

0.21.4

bleach

3.3.0

boto3

1.16.7

botocore

1.19.7

certifi

2020.12.5

cffi

1.14.5

chardet

4.0.0

cycler

0.10.0

Cython

0.29.23

dbus-python

1.2.16

decorator

5.0.6

defusedxml

0.7.1

distlib

0.3.4

distro-info

0.23ubuntu1

entrypoints

0.3

facets-overview

1.0.0

filelock

3.6.0

idna

2.10

ipykernel

5.3.4

ipython

7.22.0

ipython-genutils

0.2.0

ipywidgets

7.6.3

jedi

0.17.2

Jinja2

2.11.3

jmespath

0.10.0

joblib

1.0.1

jsonschema

3.2.0

jupyter-client

6.1.12

jupyter-core

4.7.1

jupyterlab-pygments

0.1.2

jupyterlab-widgets

1.0.0

kiwisolver

1.3.1

koalas

1.8.2

MarkupSafe

2.0.1

matplotlib

3.4.2

mistune

0.8.4

nbclient

0.5.3

nbconvert

6.0.7

nbformat

5.1.3

nest-asyncio

1.5.1

notebook

6.3.0

numpy

1.20.1

packaging

20.9

pandas

1.2.4

pandocfilters

1.4.3

parso

0.7.0

patsy

0.5.1

pexpect

4.8.0

pickleshare

0.7.5

Pillow

8.2.0

pip

21.0.1

plotly

5.6.0

prometheus-client

0.10.1

prompt-toolkit

3.0.17

protobuf

3.17.2

psycopg2

2.8.5

ptyprocess

0.7.0

pyarrow

4.0.0

pycparser

2.20

Pygments

2.8.1

PyGObject

3.36.0

pyparsing

2.4.7

pyrsistent

0.17.3

python-apt

2.0.0+ubuntu0.20.4.7

python-dateutil

2.8.1

python-engineio

4.3.0

python-socketio

5.4.1

pytz

2020.5

pyzmq

20.0.0

requests

2.25.1

requests-unixsocket

0.2.0

s3transfer

0.3.7

scikit-learn

0.24.1

scipy

1.6.2

seaborn

0.11.1

Send2Trash

1.5.0

setuptools

52.0.0

six

1.15.0

ssh-import-id

5.10

statsmodels

0.12.2

tenacity

8.0.1

terminado

0.9.4

testpath

0.4.4

threadpoolctl

2.1.0

tornado

6.1

traitlets

5.0.5

unattended-upgrades

0.1

urllib3

1.25.11

virtualenv

20.4.1

wcwidth

0.2.5

webencodings

0.5.1

wheel

0.36.2

widgetsnbextension

3.5.1

Installed R libraries

R libraries are installed from the Microsoft CRAN snapshot on 2022-03-28.

Library

Version

Library

Version

Library

Version

askpass

1.1

assertthat

0.2.1

backports

1.4.1

base

4.1.3

base64enc

0.1-3

bit

4.0.4

bit64

4.0.5

blob

1.2.2

boot

1.3-28

brew

1.0-7

brio

1.1.3

broom

0.7.12

bslib

0.3.1

cachem

1.0.6

callr

3.7.0

caret

6.0-91

cellranger

1.1.0

chron

2.3-56

class

7.3-20

cli

3.2.0

clipr

0.8.0

cluster

2.1.3

codetools

0.2-18

colorspace

2.0-3

commonmark

1.8.0

compiler

4.1.3

config

0.3.1

cpp11

0.4.2

crayon

1.5.1

credentials

1.3.2

curl

4.3.2

data.table

1.14.2

datasets

4.1.3

DBI

1.1.2

dbplyr

2.1.1

desc

1.4.1

devtools

2.4.3

diffobj

0.3.5

digest

0.6.29

dplyr

1.0.8

dtplyr

1.2.1

e1071

1.7-9

ellipsis

0.3.2

evaluate

0.15

fansi

1.0.3

farver

2.1.0

fastmap

1.1.0

fontawesome

0.2.2

forcats

0.5.1

foreach

1.5.2

foreign

0.8-82

forge

0.2.0

fs

1.5.2

future

1.24.0

future.apply

1.8.1

gargle

1.2.0

generics

0.1.2

gert

1.5.0

ggplot2

3.3.5

gh

1.3.0

gitcreds

0.1.1

glmnet

4.1-3

globals

0.14.0

glue

1.6.2

googledrive

2.0.0

googlesheets4

1.0.0

gower

1.0.0

graphics

4.1.3

grDevices

4.1.3

grid

4.1.3

gridExtra

2.3

gsubfn

0.7

gtable

0.3.0

hardhat

0.2.0

haven

2.4.3

highr

0.9

hms

1.1.1

htmltools

0.5.2

htmlwidgets

1.5.4

httpuv

1.6.5

httr

1.4.2

hwriter

1.3.2

hwriterPlus

1.0-3

ids

1.0.1

ini

0.3.1

ipred

0.9-12

isoband

0.2.5

iterators

1.0.14

jquerylib

0.1.4

jsonlite

1.8.0

KernSmooth

2.23-20

knitr

1.38

labeling

0.4.2

later

1.3.0

lattice

0.20-45

lava

1.6.10

lifecycle

1.0.1

listenv

0.8.0

lubridate

1.8.0

magrittr

2.0.2

markdown

1.1

MASS

7.3-56

Matrix

1.4-1

memoise

2.0.1

methods

4.1.3

mgcv

1.8-40

mime

0.12

ModelMetrics

1.2.2.2

modelr

0.1.8

munsell

0.5.0

nlme

3.1-157

nnet

7.3-17

numDeriv

2016.8-1.1

openssl

2.0.0

parallel

4.1.3

parallelly

1.30.0

pillar

1.7.0

pkgbuild

1.3.1

pkgconfig

2.0.3

pkgload

1.2.4

plogr

0.2.0

plyr

1.8.7

praise

1.0.0

prettyunits

1.1.1

pROC

1.18.0

processx

3.5.3

prodlim

2019.11.13

progress

1.2.2

progressr

0.10.0

promises

1.2.0.1

proto

1.0.0

proxy

0.4-26

ps

1.6.0

purrr

0.3.4

r2d3

0.2.6

R6

2.5.1

randomForest

4.7-1

rappdirs

0.3.3

rcmdcheck

1.4.0

RColorBrewer

1.1-2

Rcpp

1.0.8.3

RcppEigen

0.3.3.9.1

readr

2.1.2

readxl

1.3.1

recipes

0.2.0

rematch

1.0.1

rematch2

2.1.2

remotes

2.4.2

reprex

2.0.1

reshape2

1.4.4

rlang

1.0.2

rmarkdown

2.13

RODBC

1.3-19

roxygen2

7.1.2

rpart

4.1.16

rprojroot

2.0.2

Rserve

1.8-10

RSQLite

2.2.11

rstudioapi

0.13

rversions

2.1.1

rvest

1.0.2

sass

0.4.1

scales

1.1.1

selectr

0.4-2

sessioninfo

1.2.2

shape

1.4.6

shiny

1.7.1

sourcetools

0.1.7

sparklyr

1.7.5

SparkR

3.2.0

spatial

7.3-11

splines

4.1.3

sqldf

0.4-11

SQUAREM

2021.1

stats

4.1.3

stats4

4.1.3

stringi

1.7.6

stringr

1.4.0

survival

3.2-13

sys

3.4

tcltk

4.1.3

TeachingDemos

2.10

testthat

3.1.2

tibble

3.1.6

tidyr

1.2.0

tidyselect

1.1.2

tidyverse

1.3.1

timeDate

3043.102

tinytex

0.37

tools

4.1.3

tzdb

0.2.0

usethis

2.1.5

utf8

1.2.2

utils

4.1.3

uuid

1.0-4

vctrs

0.3.8

viridisLite

0.4.0

vroom

1.5.7

waldo

0.4.0

whisker

0.4

withr

2.5.0

xfun

0.30

xml2

1.3.3

xopen

1.0.0

xtable

1.8-4

yaml

2.3.5

zip

2.2.0

Installed Java and Scala libraries (Scala 2.12 cluster version)

Group ID

Artifact ID

Version

antlr

antlr

2.7.7

com.amazonaws

amazon-kinesis-client

1.12.0

com.amazonaws

aws-java-sdk-autoscaling

1.12.189

com.amazonaws

aws-java-sdk-cloudformation

1.12.189

com.amazonaws

aws-java-sdk-cloudfront

1.12.189

com.amazonaws

aws-java-sdk-cloudhsm

1.12.189

com.amazonaws

aws-java-sdk-cloudsearch

1.12.189

com.amazonaws

aws-java-sdk-cloudtrail

1.12.189

com.amazonaws

aws-java-sdk-cloudwatch

1.12.189

com.amazonaws

aws-java-sdk-cloudwatchmetrics

1.12.189

com.amazonaws

aws-java-sdk-codedeploy

1.12.189

com.amazonaws

aws-java-sdk-cognitoidentity

1.12.189

com.amazonaws

aws-java-sdk-cognitosync

1.12.189

com.amazonaws

aws-java-sdk-config

1.12.189

com.amazonaws

aws-java-sdk-core

1.12.189

com.amazonaws

aws-java-sdk-datapipeline

1.12.189

com.amazonaws

aws-java-sdk-directconnect

1.12.189

com.amazonaws

aws-java-sdk-directory

1.12.189

com.amazonaws

aws-java-sdk-dynamodb

1.12.189

com.amazonaws

aws-java-sdk-ec2

1.12.189

com.amazonaws

aws-java-sdk-ecs

1.12.189

com.amazonaws

aws-java-sdk-efs

1.12.189

com.amazonaws

aws-java-sdk-elasticache

1.12.189

com.amazonaws

aws-java-sdk-elasticbeanstalk

1.12.189

com.amazonaws

aws-java-sdk-elasticloadbalancing

1.12.189

com.amazonaws

aws-java-sdk-elastictranscoder

1.12.189

com.amazonaws

aws-java-sdk-emr

1.12.189

com.amazonaws

aws-java-sdk-glacier

1.12.189

com.amazonaws

aws-java-sdk-glue

1.12.189

com.amazonaws

aws-java-sdk-iam

1.12.189

com.amazonaws

aws-java-sdk-importexport

1.12.189

com.amazonaws

aws-java-sdk-kinesis

1.12.189

com.amazonaws

aws-java-sdk-kms

1.12.189

com.amazonaws

aws-java-sdk-lambda

1.12.189

com.amazonaws

aws-java-sdk-logs

1.12.189

com.amazonaws

aws-java-sdk-machinelearning

1.12.189

com.amazonaws

aws-java-sdk-opsworks

1.12.189

com.amazonaws

aws-java-sdk-rds

1.12.189

com.amazonaws

aws-java-sdk-redshift

1.12.189

com.amazonaws

aws-java-sdk-route53

1.12.189

com.amazonaws

aws-java-sdk-s3

1.12.189

com.amazonaws

aws-java-sdk-ses

1.12.189

com.amazonaws

aws-java-sdk-simpledb

1.12.189

com.amazonaws

aws-java-sdk-simpleworkflow

1.12.189

com.amazonaws

aws-java-sdk-sns

1.12.189

com.amazonaws

aws-java-sdk-sqs

1.12.189

com.amazonaws

aws-java-sdk-ssm

1.12.189

com.amazonaws

aws-java-sdk-storagegateway

1.12.189

com.amazonaws

aws-java-sdk-sts

1.12.189

com.amazonaws

aws-java-sdk-support

1.12.189

com.amazonaws

aws-java-sdk-swf-libraries

1.11.22

com.amazonaws

aws-java-sdk-workspaces

1.12.189

com.amazonaws

jmespath-java

1.12.189

com.chuusai

shapeless_2.12

2.3.3

com.clearspring.analytics

stream

2.9.6

com.databricks

Rserve

1.8-3

com.databricks

jets3t

0.7.1-0

com.databricks.scalapb

compilerplugin_2.12

0.4.15-10

com.databricks.scalapb

scalapb-runtime_2.12

0.4.15-10

com.esotericsoftware

kryo-shaded

4.0.2

com.esotericsoftware

minlog

1.3.0

com.fasterxml

classmate

1.3.4

com.fasterxml.jackson.core

jackson-annotations

2.12.3

com.fasterxml.jackson.core

jackson-core

2.12.3

com.fasterxml.jackson.core

jackson-databind

2.12.3

com.fasterxml.jackson.dataformat

jackson-dataformat-cbor

2.12.3

com.fasterxml.jackson.datatype

jackson-datatype-joda

2.12.3

com.fasterxml.jackson.module

jackson-module-paranamer

2.12.3

com.fasterxml.jackson.module

jackson-module-scala_2.12

2.12.3

com.github.ben-manes.caffeine

caffeine

2.3.4

com.github.fommil

jniloader

1.1

com.github.fommil.netlib

core

1.1.2

com.github.fommil.netlib

native_ref-java

1.1

com.github.fommil.netlib

native_ref-java-natives

1.1

com.github.fommil.netlib

native_system-java

1.1

com.github.fommil.netlib

native_system-java-natives

1.1

com.github.fommil.netlib

netlib-native_ref-linux-x86_64-natives

1.1

com.github.fommil.netlib

netlib-native_system-linux-x86_64-natives

1.1

com.github.luben

zstd-jni

1.5.0-4

com.github.wendykierp

JTransforms

3.1

com.google.code.findbugs

jsr305

3.0.0

com.google.code.gson

gson

2.8.6

com.google.crypto.tink

tink

1.6.0

com.google.flatbuffers

flatbuffers-java

1.9.0

com.google.guava

guava

15.0

com.google.protobuf

protobuf-java

2.6.1

com.h2database

h2

2.1.210

com.helger

profiler

1.1.1

com.jcraft

jsch

0.1.50

com.jolbox

bonecp

0.8.0.RELEASE

com.lihaoyi

sourcecode_2.12

0.1.9

com.microsoft.azure

azure-data-lake-store-sdk

2.3.9

com.ning

compress-lzf

1.0.3

com.sun.istack

istack-commons-runtime

3.0.8

com.sun.mail

javax.mail

1.5.2

com.tdunning

json

1.8

com.thoughtworks.paranamer

paranamer

2.8

com.trueaccord.lenses

lenses_2.12

0.4.12

com.twitter

chill-java

0.10.0

com.twitter

chill_2.12

0.10.0

com.twitter

util-app_2.12

7.1.0

com.twitter

util-core_2.12

7.1.0

com.twitter

util-function_2.12

7.1.0

com.twitter

util-jvm_2.12

7.1.0

com.twitter

util-lint_2.12

7.1.0

com.twitter

util-registry_2.12

7.1.0

com.twitter

util-stats_2.12

7.1.0

com.typesafe

config

1.2.1

com.typesafe.scala-logging

scala-logging_2.12

3.7.2

com.univocity

univocity-parsers

2.9.1

com.zaxxer

HikariCP

4.0.3

commons-cli

commons-cli

1.2

commons-codec

commons-codec

1.15

commons-collections

commons-collections

3.2.2

commons-dbcp

commons-dbcp

1.4

commons-fileupload

commons-fileupload

1.3.3

commons-httpclient

commons-httpclient

3.1

commons-io

commons-io

2.8.0

commons-lang

commons-lang

2.6

commons-logging

commons-logging

1.1.3

commons-net

commons-net

3.1

commons-pool

commons-pool

1.5.4

dev.ludovic.netlib

arpack

2.2.1

dev.ludovic.netlib

blas

2.2.1

dev.ludovic.netlib

lapack

2.2.1

hive-2.3__hadoop-3.2

jets3t-0.7

liball_deps_2.12

info.ganglia.gmetric4j

gmetric4j

1.0.10

io.airlift

aircompressor

0.21

io.delta

delta-sharing-spark_2.12

0.4.0

io.dropwizard.metrics

metrics-core

4.1.1

io.dropwizard.metrics

metrics-graphite

4.1.1

io.dropwizard.metrics

metrics-healthchecks

4.1.1

io.dropwizard.metrics

metrics-jetty9

4.1.1

io.dropwizard.metrics

metrics-jmx

4.1.1

io.dropwizard.metrics

metrics-json

4.1.1

io.dropwizard.metrics

metrics-jvm

4.1.1

io.dropwizard.metrics

metrics-servlets

4.1.1

io.netty

netty-all

4.1.68.Final

io.prometheus

simpleclient

0.7.0

io.prometheus

simpleclient_common

0.7.0

io.prometheus

simpleclient_dropwizard

0.7.0

io.prometheus

simpleclient_pushgateway

0.7.0

io.prometheus

simpleclient_servlet

0.7.0

io.prometheus.jmx

collector

0.12.0

jakarta.annotation

jakarta.annotation-api

1.3.5

jakarta.servlet

jakarta.servlet-api

4.0.3

jakarta.validation

jakarta.validation-api

2.0.2

jakarta.ws.rs

jakarta.ws.rs-api

2.1.6

javax.activation

activation

1.1.1

javax.annotation

javax.annotation-api

1.3.2

javax.el

javax.el-api

2.2.4

javax.jdo

jdo-api

3.0.1

javax.transaction

jta

1.1

javax.transaction

transaction-api

1.1

javax.xml.bind

jaxb-api

2.2.2

javax.xml.stream

stax-api

1.0-2

javolution

javolution

5.5.1

jline

jline

2.14.6

joda-time

joda-time

2.10.10

log4j

apache-log4j-extras

1.2.17

log4j

log4j

1.2.17

maven-trees

hive-2.3__hadoop-3.2

liball_deps_2.12

net.java.dev.jna

jna

5.8.0

net.razorvine

pyrolite

4.30

net.sf.jpam

jpam

1.1

net.sf.opencsv

opencsv

2.3

net.sf.supercsv

super-csv

2.2.0

net.snowflake

snowflake-ingest-sdk

0.9.6

net.snowflake

snowflake-jdbc

3.13.3

net.snowflake

spark-snowflake_2.12

2.9.0-spark_3.1

net.sourceforge.f2j

arpack_combined_all

0.1

org.acplt.remotetea

remotetea-oncrpc

1.1.2

org.antlr

ST4

4.0.4

org.antlr

antlr-runtime

3.5.2

org.antlr

antlr4-runtime

4.8

org.antlr

stringtemplate

3.2.1

org.apache.ant

ant

1.9.2

org.apache.ant

ant-jsch

1.9.2

org.apache.ant

ant-launcher

1.9.2

org.apache.arrow

arrow-format

2.0.0

org.apache.arrow

arrow-memory-core

2.0.0

org.apache.arrow

arrow-memory-netty

2.0.0

org.apache.arrow

arrow-vector

2.0.0

org.apache.avro

avro

1.10.2

org.apache.avro

avro-ipc

1.10.2

org.apache.avro

avro-mapred

1.10.2

org.apache.commons

commons-compress

1.21

org.apache.commons

commons-crypto

1.1.0

org.apache.commons

commons-lang3

3.12.0

org.apache.commons

commons-math3

3.4.1

org.apache.commons

commons-text

1.6

org.apache.curator

curator-client

2.13.0

org.apache.curator

curator-framework

2.13.0

org.apache.curator

curator-recipes

2.13.0

org.apache.derby

derby

10.14.2.0

org.apache.hadoop

hadoop-client-api

3.3.2-databricks

org.apache.hadoop

hadoop-client-runtime

3.3.2

org.apache.hive

hive-beeline

2.3.9

org.apache.hive

hive-cli

2.3.9

org.apache.hive

hive-jdbc

2.3.9

org.apache.hive

hive-llap-client

2.3.9

org.apache.hive

hive-llap-common

2.3.9

org.apache.hive

hive-serde

2.3.9

org.apache.hive

hive-shims

2.3.9

org.apache.hive

hive-storage-api

2.7.2

org.apache.hive.shims

hive-shims-0.23

2.3.9

org.apache.hive.shims

hive-shims-common

2.3.9

org.apache.hive.shims

hive-shims-scheduler

2.3.9

org.apache.httpcomponents

httpclient

4.5.13

org.apache.httpcomponents

httpcore

4.4.12

org.apache.ivy

ivy

2.5.0

org.apache.mesos

mesos-shaded-protobuf

1.4.0

org.apache.orc

orc-core

1.6.13

org.apache.orc

orc-mapreduce

1.6.13

org.apache.orc

orc-shims

1.6.13

org.apache.parquet

parquet-column

1.12.0-databricks-0004

org.apache.parquet

parquet-common

1.12.0-databricks-0004

org.apache.parquet

parquet-encoding

1.12.0-databricks-0004

org.apache.parquet

parquet-format-structures

1.12.0-databricks-0004

org.apache.parquet

parquet-hadoop

1.12.0-databricks-0004

org.apache.parquet

parquet-jackson

1.12.0-databricks-0004

org.apache.thrift

libfb303

0.9.3

org.apache.thrift

libthrift

0.12.0

org.apache.xbean

xbean-asm9-shaded

4.20

org.apache.yetus

audience-annotations

0.5.0

org.apache.zookeeper

zookeeper

3.6.2

org.apache.zookeeper

zookeeper-jute

3.6.2

org.checkerframework

checker-qual

3.5.0

org.codehaus.jackson

jackson-core-asl

1.9.13

org.codehaus.jackson

jackson-mapper-asl

1.9.13

org.codehaus.janino

commons-compiler

3.0.16

org.codehaus.janino

janino

3.0.16

org.datanucleus

datanucleus-api-jdo

4.2.4

org.datanucleus

datanucleus-core

4.1.17

org.datanucleus

datanucleus-rdbms

4.1.19

org.datanucleus

javax.jdo

3.2.0-m3

org.eclipse.jetty

jetty-client

9.4.43.v20210629

org.eclipse.jetty

jetty-continuation

9.4.43.v20210629

org.eclipse.jetty

jetty-http

9.4.43.v20210629

org.eclipse.jetty

jetty-io

9.4.43.v20210629

org.eclipse.jetty

jetty-jndi

9.4.43.v20210629

org.eclipse.jetty

jetty-plus

9.4.43.v20210629

org.eclipse.jetty

jetty-proxy

9.4.43.v20210629

org.eclipse.jetty

jetty-security

9.4.43.v20210629

org.eclipse.jetty

jetty-server

9.4.43.v20210629

org.eclipse.jetty

jetty-servlet

9.4.43.v20210629

org.eclipse.jetty

jetty-servlets

9.4.43.v20210629

org.eclipse.jetty

jetty-util

9.4.43.v20210629

org.eclipse.jetty

jetty-util-ajax

9.4.43.v20210629

org.eclipse.jetty

jetty-webapp

9.4.43.v20210629

org.eclipse.jetty

jetty-xml

9.4.43.v20210629

org.eclipse.jetty.websocket

websocket-api

9.4.43.v20210629

org.eclipse.jetty.websocket

websocket-client

9.4.43.v20210629

org.eclipse.jetty.websocket

websocket-common

9.4.43.v20210629

org.eclipse.jetty.websocket

websocket-server

9.4.43.v20210629

org.eclipse.jetty.websocket

websocket-servlet

9.4.43.v20210629

org.fusesource.leveldbjni

leveldbjni-all

1.8

org.glassfish.hk2

hk2-api

2.6.1

org.glassfish.hk2

hk2-locator

2.6.1

org.glassfish.hk2

hk2-utils

2.6.1

org.glassfish.hk2

osgi-resource-locator

1.0.3

org.glassfish.hk2.external

aopalliance-repackaged

2.6.1

org.glassfish.hk2.external

jakarta.inject

2.6.1

org.glassfish.jaxb

jaxb-runtime

2.3.2

org.glassfish.jersey.containers

jersey-container-servlet

2.34

org.glassfish.jersey.containers

jersey-container-servlet-core

2.34

org.glassfish.jersey.core

jersey-client

2.34

org.glassfish.jersey.core

jersey-common

2.34

org.glassfish.jersey.core

jersey-server

2.34

org.glassfish.jersey.inject

jersey-hk2

2.34

org.hibernate.validator

hibernate-validator

6.1.0.Final

org.javassist

javassist

3.25.0-GA

org.jboss.logging

jboss-logging

3.3.2.Final

org.jdbi

jdbi

2.63.1

org.jetbrains

annotations

17.0.0

org.joda

joda-convert

1.7

org.jodd

jodd-core

3.5.2

org.json4s

json4s-ast_2.12

3.7.0-M11

org.json4s

json4s-core_2.12

3.7.0-M11

org.json4s

json4s-jackson_2.12

3.7.0-M11

org.json4s

json4s-scalap_2.12

3.7.0-M11

org.lz4

lz4-java

1.7.1

org.mariadb.jdbc

mariadb-java-client

2.2.5

org.objenesis

objenesis

2.5.1

org.postgresql

postgresql

42.2.19

org.roaringbitmap

RoaringBitmap

0.9.14

org.roaringbitmap

shims

0.9.14

org.rocksdb

rocksdbjni

6.20.3

org.rosuda.REngine

REngine

2.1.0

org.scala-lang

scala-compiler_2.12

2.12.14

org.scala-lang

scala-library_2.12

2.12.14

org.scala-lang

scala-reflect_2.12

2.12.14

org.scala-lang.modules

scala-collection-compat_2.12

2.4.3

org.scala-lang.modules

scala-parser-combinators_2.12

1.1.2

org.scala-lang.modules

scala-xml_2.12

1.2.0

org.scala-sbt

test-interface

1.0

org.scalacheck

scalacheck_2.12

1.14.2

org.scalactic

scalactic_2.12

3.0.8

org.scalanlp

breeze-macros_2.12

1.2

org.scalanlp

breeze_2.12

1.2

org.scalatest

scalatest_2.12

3.0.8

org.slf4j

jcl-over-slf4j

1.7.30

org.slf4j

jul-to-slf4j

1.7.30

org.slf4j

slf4j-api

1.7.30

org.slf4j

slf4j-log4j12

1.7.30

org.spark-project.spark

unused

1.0.0

org.threeten

threeten-extra

1.5.0

org.tukaani

xz

1.8

org.typelevel

algebra_2.12

2.0.1

org.typelevel

cats-kernel_2.12

2.1.1

org.typelevel

macro-compat_2.12

1.1.1

org.typelevel

spire-macros_2.12

0.17.0

org.typelevel

spire-platform_2.12

0.17.0

org.typelevel

spire-util_2.12

0.17.0

org.typelevel

spire_2.12

0.17.0

org.wildfly.openssl

wildfly-openssl

1.0.7.Final

org.xerial

sqlite-jdbc

3.8.11.2

org.xerial.snappy

snappy-java

1.1.8.4

org.yaml

snakeyaml

1.24

oro

oro

2.0.8

pl.edu.icm

JLargeArrays

1.5

software.amazon.ion

ion-java

1.0.2

stax

stax-api

1.0.1