Databricks Runtime 10.0 (EoS)

Note

Support for this Databricks Runtime version has ended. For the end-of-support date, see End-of-support history. For all supported Databricks Runtime versions, see Databricks Runtime release notes versions and compatibility.

The following release notes provide information about Databricks Runtime 10.0 and Databricks Runtime 10.0 Photon, powered by Apache Spark 3.2.0. Databricks released this version in October 2021. Photon is in Public Preview.

New features and improvements

New version of Apache Spark

Databricks Runtime 10.0 and Databricks Runtime 10.0 Photon include Apache Spark 3.2.0. For details, see Apache Spark.

SELECT statement now supports QUALIFY clause to filter window function results

The SELECT statement now supports the QUALIFY clause. QUALIFY can be used to filter the results of window functions. One or more window functions must be present in either the SELECT list or the QUALIFY condition. For example:

SELECT * FROM t QUALIFY SUM(c2) OVER (PARTITION BY c1) > 0;

Cluster support for JDK 11 (Public Preview)

Databricks now provides cluster support for Java Development Kit (JDK) 11.

When you create a cluster, you can specify that the cluster uses JDK 11 (for both the driver and executor). To do this, add the following environment variable to Advanced Options > Spark > Environment Variables:

JNAME=zulu11-ca-amd64

Auto Loader now treats schemas as nullable

Auto Loader now treats all inferred and user-provided schemas as nullable by default. This is to avoid potential data corruption in cases where the data contains null fields for non-nullable columns. A new configuration is introduced to track this behavior, spark.databricks.cloudFiles.schema.forceNullable. By default, this configuration contains the setting of spark.sql.streaming.fileSource.schema.forceNullable, which the FileStreamSource in Apache Spark uses and is set to true by default.

Breaking changes

Breaking changes for all Spark SQL users

  • New spark.databricks.behaviorChange.SC78546CorrelatedPredicate.enabled configuration: when set to true, allows a subset of correlated equality predicates when a subquery is aggregated. The default setting is true.

  • New spark.databricks.behaviorChange.SC82201BlockAutoAlias.enabled configuration: when set to true, blocks autogenerating aliases when views are created. The default setting is true.

  • New spark.databricks.behaviorChange.SC81078CTASWithLocation.enabled configuration: when set to true, disallows CREATE TABLE AS SELECT with a non-empty location. The default setting is true. Note that when spark.sql.legacy.allowNonEmptyLocationInCTAS is also set to true, this configuration has no effect, and CREATE TABLE AS SELECT with a non-empty location is always allowed.

Breaking changes for Spark SQL users who enable ANSI mode

For information about ANSI mode, see ANSI compliance in Databricks Runtime.

  • New spark.databricks.behaviorChange.SC83587NextDayDOW.enabled configuration: when set to true, an invalid dayOfWeek argument to the next_day function throws an IllegalArgumentException in ANSI mode; otherwise it returns null. The default setting is true.

  • New spark.databricks.behaviorChange.SC83796CanCast.enabled configuration: when set to true, enables new explicit cast syntax rules in ANSI mode. The default setting is true.

  • New spark.databricks.behaviorChange.SC79099CastStringToBoolean.enabled configuration: when set to true, a parse error exception is thrown when casting a string to a Boolean; otherwise, returns null. The default setting is true.

  • New spark.databricks.behaviorChange.SC79064AbsOutsideRange.enabled configuration: when set to true, the abs function throws an exception if the input is out of range. The default setting is true.

Breaking changes for all Python users

  • The Python API delta.tables.DeltaTable.convertToDelta now returns the correct DeltaTable Python object which can be used to run Delta Lake commands. Previously it returned an internal object which could not be called directly.

Library upgrades

Apache Hadoop 3 upgrade

  • Databricks Runtime 10.0 upgrades the Hadoop dependency from Hadoop 2.7.4 to Hadoop 3.3.1.

Behavior changes

  • Hadoop 3 uses the hadoop-client-api and hadoop-client-runtime libraries instead of the hadoop-common library, which shades certain third-party dependencies that could potentially be used in Hadoop public APIs or extensions.

  • Hadoop 3 configuration options have changed since Hadoop 2. For Hadoop 3.3.1 options, see core-default.xml.

  • Databricks has updated some of the default configurations for Hadoop 3 to be consistent with Hadoop 2, to make sure storage connectors have the same default authentication settings and levels of performance:

    • fs.azure.authorization.caching.enable = false

    • fs.s3a.attempts.maximum = 10

    • fs.s3a.block.size = 67108864

    • fs.s3a.connection.timeout = 50000

    • fs.s3a.max.total.tasks = 1000

    • fs.s3a.retry.limit = 20

    • fs.s3a.retry.throttle.interval = 500ms

    • fs.s3a.assumed.role.credentials.provider = com.amazonaws.auth.InstanceProfileCredentialsProvider

    • fs.s3a.aws.credentials.provider = BasicAWSCredentialsProvider, DatabricksInstanceProfileCredentialsProvider, EnvironmentVariableCredentialsProvider, AnonymousAWSCredentialsProvider

  • The Google Cloud Storage (GCS) connector has been upgraded from 2.1.6 to 2.2.2.

  • The Amazon Redshift connector now uses the s3a:// scheme. The s3n:// scheme is deprecated.

  • Handles the OSS class names for the Amazon S3 MetadataStore API in addition to shaded classes. This enables processing OSS configurations without requiring the shaded class names.

    • For example, you can specify the org.apache.hadoop.fs.s3a.s3guard.NullMetadataStore class in the Hadoop configuration.

  • Makes the new Configuration() and sparkContext.hadoopConfiguration consistent.

    • Now every time a new Hadoop configuration is created, it will be consistent with the predefined Hadoop configuration in sparkContext.hadoopConfiguration in the Databricks Runtime, including file system schemes and their default configuration.

  • Although Hadoop storage connectors included in the Databricks Runtime are fully compatible with Hadoop 3.3.1, they are not guaranteed to be in sync with OSS Hadoop 3.3.1 connectors and might have different behavior.

    • The Amazon S3 connector still allows (although with a warning) user:secret authentication in S3 URLs, compared to HADOOP-14833 which removes it.

  • globStatus now always returns sorted results(HADOOP-10798)

  • Added fs.s3a.endpoint if unset and fs.s3a.endpoint region is null(SPARK-35878)

    • Databricks provided Amazon S3 region auto resolution might not be triggered in some cases, due to the global endpoint being set. This is not an issue, as the AWS SDK will resolve the region correctly.

  • Add fs.s3a.downgrade.syncable.exceptions if not set(SPARK-35868)

  • LZ4 and Snappy codecs do not rely on the native Hadoop library(HADOOP-17125)

Known issues

  • SPARK-36681 Using SnappyCodec to write sequence file will fail with UnsatisfiedLinkError due to known issue in Hadoop 3.3.1(HADOOP-17891)

Apache Spark

Databricks Runtime 10.0 includes Apache Spark 3.2.0.

Highlights

Core and Spark SQL

ANSI SQL compatibility enhancements

Performance enhancements

  • Query optimization

    • Remove redundant aggregates in the Optimizer(SPARK-33122)

    • Push down limit through Project with Join(SPARK-34622)

    • Cardinality estimation of union, sort and range operator (SPARK-33411)

    • UnwrapCastInBinaryComparison support In/InSet predicate(SPARK-35316)

    • Keep necessary stats after partition pruning(SPARK-34119)

  • Query execution

    • Enable Zstandard buffer pool by default(SPARK-34340, SPARK-34390)

    • Add code-gen for all join types of sort merge join(SPARK-34705)

    • Broadcast nested loop join improvement(SPARK-34706)

    • Support two levels of hash maps for final hash aggregation (SPARK-35141)

    • Allow concurrent writers for writing dynamic partitions and bucket table(SPARK-26164)

    • Improve performance of processing FETCH_PRIOR in Thriftserver(SPARK-33655)

Connector enhancements

  • Parquet

    • Upgrade Parquet to 1.12.1(SPARK-36726)

    • Read parquet unsigned types that are stored as int32 physical type in parquet(SPARK-34817)

    • Read Parquet unsigned int64 logical type that stored as signed int64 physical type to decimal(20, 0)(SPARK-34786)

    • Improve Parquet In filter pushdown(SPARK-32792)

  • ORC

  • Avro

    • Upgrade Avro to version 1.10.2(SPARK-34778)

    • Supporting Avro schema evolution for partitioned Hive tables with “avro.schema.literal”(SPARK-26836)

    • Add new Avro data source options to control datetime rebasing in read(SPARK-34404)

    • Adding support for user provided schema url in Avro(SPARK-34416)

    • Add support for positional Catalyst-to-Avro schema matching(SPARK-34365)

  • JSON

    • Upgrade Jackson to version 2.12.3(SPARK-35550)

    • Allow Json data sources to write non-ascii characters as codepoints(SPARK-35047)

  • JDBC

    • Calculate more precise partition stride in JDBCRelation(SPARK-34843)

  • Hive Metastore support filter by not-in(SPARK-34538)

Feature enhancements

Other notable changes

  • Monitor

    • New metrics to ExternalShuffleService(SPARK-35258)

    • Add new stage-level REST APIs and parameters (SPARK-26399)

    • Support task and executor Metrics Distributions Metrics Distributions in the REST API(SPARK-34488)

    • Add fallback metrics for hash aggregate(SPARK-35529)

  • Add count_distinct as an option to Dataset#summary(SPARK-34165)

  • Implement ScriptTransform in sql/core(SPARK-31936)

  • Make BlockManagerMaster driver heartbeat timeout configurable(SPARK-34278)

  • Make shuffle service name configurable on client side and allow for classpath-based config override on server side(SPARK-34828)

  • ExecutorMetricsPoller should keep stage entry in stageTCMP until a heartbeat occurs(SPARK-34779)

  • Replace if with filter clause in RewriteDistinctAggregates(SPARK-34882)

  • Fix failure when apply CostBasedJoinReorder on self-join(SPARK-34354)

  • CREATE TABLE LIKE should respect the reserved table properties(SPARK-34935)

  • Ship ivySettings file to driver in YARN cluster mode(SPARK-34472)

  • Resolve duplicated common columns from USING/NATURAL JOIN(SPARK-34527)

  • Hide internal view properties for describe table cmd(SPARK-35318)

  • Support resolving missing attrs for distribute/cluster by/repartition hint(SPARK-35331)

  • Ignore the error when checking the path in FileStreamSink.hasMetadata(SPARK-34526)

  • Improve s3a magic committer support by inferring missing configs(SPARK-35383)

  • Allow to omit : in the STRUCT type string (SPARK-35706)

  • Add a new operator to distinguish if AQE can optimize safely (SPARK-35786)

  • Append new nested struct fields rather than sort for unionByName with null filling(SPARK-35290)

  • ArraysZip should retain field names to avoid being re-written by analyzer/optimizer(SPARK-35876)

  • Use Void as the type name of NullType(SPARK-36224)

  • Introduce new API to FileCommitProtocol allow flexible file naming(SPARK-33298)

Changes of behavior

See the migration guides for each component: Spark Core.

Structured Streaming

Major features

  • EventTime based sessionization (session window)(SPARK-10816)

  • Upgrade Kafka client to 2.8.0(SPARK-33913)

  • Trigger.AvailableNow for running streaming queries like Trigger.Once in multiple batches in Scala(SPARK-36533)

Other notable changes

  • Introduce a new Option in Kafka source to specify a minimum number of records to read per trigger(SPARK-35312)

  • Add latest offsets to source progress(SPARK-33955)

PySpark

Project Zen

Other notable changes

  • Enable pinned thread mode by default(SPARK-35303)

  • Add NullType support for Arrow executions(SPARK-33489)

  • Add Arrow self_destruct support to toPandas (SPARK-32953)

  • Add thread target wrapper API for pyspark pin thread mode(SPARK-35498)

Changes of behavior

See the migration guides.

MLlib

Performance improvements

  • BucketedRandomProjectionLSH transform optimization(SPARK-34220)

  • w2v findSynonyms optimization(SPARK-34189)

  • optimize sparse GEMM by skipping bound checking(SPARK-35707)

  • Improve performance of ML ALS recommendForAll by GEMV(SPARK-33518)

Model training improvements

  • Refactor Logistic Aggregator - support virtual centering(SPARK-34797)

  • Binary Logistic Regression with intercept support centering(SPARK-34858, SPARK-34448)

  • Multinomial Logistic Regression with intercept support centering(SPARK-34860)

BLAS improvements

  • Replace fully com.github.fommil.netlib by dev.ludovic.netlib:2.0(SPARK-35295)

  • Add a vectorized BLAS implementation(SPARK-33882)

  • Accelerate fallback BLAS with dev.ludovic.netlib(SPARK-35150)

Other notable changes

  • OVR transform fix potential column conflict(SPARK-34356)

Deprecations and removals

  • Deprecate spark.launcher.childConnectionTimeout(SPARK-33717)

  • deprecate GROUP BY … GROUPING SETS (…) and promote GROUP BY GROUPING SETS (…)(SPARK-34932)

  • Deprecate ps.broadcast API(SPARK-35810)

  • Deprecate the num_files argument(SPARK-35807)

  • Deprecate DataFrame.to_spark_io(SPARK-35811)

System environment

  • Operating System: Ubuntu 20.04.3 LTS

  • Java: Zulu 8.56.0.21-CA-linux64

  • Scala: 2.12.14

  • Python: 3.8.10

  • R: 4.1.1

  • Delta Lake: 1.0.0

Installed Python libraries

Library

Version

Library

Version

Library

Version

Antergos Linux

2015.10 (ISO-Rolling)

appdirs

1.4.4

backcall

0.2.0

boto3

1.16.7

botocore

1.19.7

certifi

2020.12.5

chardet

4.0.0

cycler

0.10.0

Cython

0.29.23

dbus-python

1.2.16

decorator

5.0.6

distlib

0.3.3

distro-info

0.23ubuntu1

facets-overview

1.0.0

filelock

3.0.12

idna

2.10

ipykernel

5.3.4

ipython

7.22.0

ipython-genutils

0.2.0

jedi

0.17.2

jmespath

0.10.0

joblib

1.0.1

jupyter-client

6.1.12

jupyter-core

4.7.1

kiwisolver

1.3.1

koalas

1.8.1

matplotlib

3.4.2

numpy

1.19.2

pandas

1.2.4

parso

0.7.0

patsy

0.5.1

pexpect

4.8.0

pickleshare

0.7.5

Pillow

8.2.0

pip

21.0.1

plotly

5.1.0

prompt-toolkit

3.0.17

protobuf

3.17.2

psycopg2

2.8.5

ptyprocess

0.7.0

pyarrow

4.0.0

Pygments

2.8.1

PyGObject

3.36.0

pyparsing

2.4.7

python-apt

2.0.0+ubuntu0.20.4.6

python-dateutil

2.8.1

pytz

2020.5

pyzmq

20.0.0

requests

2.25.1

requests-unixsocket

0.2.0

s3transfer

0.3.7

scikit-learn

0.24.1

scipy

1.6.2

seaborn

0.11.1

setuptools

52.0.0

six

1.15.0

ssh-import-id

5.10

statsmodels

0.12.2

tenacity

8.0.1

threadpoolctl

2.1.0

tornado

6.1

traitlets

5.0.5

unattended-upgrades

0.1

urllib3

1.25.11

virtualenv

20.4.1

wcwidth

0.2.5

wheel

0.36.2

Installed R libraries

R libraries are installed from the Microsoft CRAN snapshot on 2021-09-21.

Library

Version

Library

Version

Library

Version

askpass

1.1

assertthat

0.2.1

backports

1.2.1

base

4.1.1

base64enc

0.1-3

bit

4.0.4

bit64

4.0.5

blob

1.2.2

boot

1.3-28

brew

1.0-6

brio

1.1.2

broom

0.7.9

bslib

0.3.0

cachem

1.0.6

callr

3.7.0

caret

6.0-88

cellranger

1.1.0

chron

2.3-56

class

7.3-19

cli

3.0.1

clipr

0.7.1

cluster

2.1.2

codetools

0.2-18

colorspace

2.0-2

commonmark

1.7

compiler

4.1.1

config

0.3.1

cpp11

0.3.1

crayon

1.4.1

credentials

1.3.1

curl

4.3.2

data.table

1.14.0

datasets

4.1.1

DBI

1.1.1

dbplyr

2.1.1

desc

1.3.0

devtools

2.4.2

diffobj

0.3.4

digest

0.6.27

dplyr

1.0.7

dtplyr

1.1.0

ellipsis

0.3.2

evaluate

0.14

fansi

0.5.0

farver

2.1.0

fastmap

1.1.0

forcats

0.5.1

foreach

1.5.1

foreign

0.8-81

forge

0.2.0

fs

1.5.0

future

1.22.1

future.apply

1.8.1

gargle

1.2.0

generics

0.1.0

gert

1.4.1

ggplot2

3.3.5

gh

1.3.0

gitcreds

0.1.1

glmnet

4.1-2

globals

0.14.0

glue

1.4.2

googledrive

2.0.0

googlesheets4

1.0.0

gower

0.2.2

graphics

4.1.1

grDevices

4.1.1

grid

4.1.1

gridExtra

2.3

gsubfn

0.7

gtable

0.3.0

haven

2.4.3

highr

0.9

hms

1.1.0

htmltools

0.5.2

htmlwidgets

1.5.4

httpuv

1.6.3

httr

1.4.2

hwriter

1.3.2

hwriterPlus

1.0-3

ids

1.0.1

ini

0.3.1

ipred

0.9-12

isoband

0.2.5

iterators

1.0.13

jquerylib

0.1.4

jsonlite

1.7.2

KernSmooth

2.23-20

knitr

1.34

labeling

0.4.2

later

1.3.0

lattice

0.20-44

lava

1.6.10

lifecycle

1.0.0

listenv

0.8.0

lubridate

1.7.10

magrittr

2.0.1

markdown

1.1

MASS

7.3-54

Matrix

1.3-4

memoise

2.0.0

methods

4.1.1

mgcv

1.8-37

mime

0.11

ModelMetrics

1.2.2.2

modelr

0.1.8

munsell

0.5.0

nlme

3.1-152

nnet

7.3-16

numDeriv

2016.8-1.1

openssl

1.4.5

parallel

4.1.1

parallelly

1.28.1

pillar

1.6.2

pkgbuild

1.2.0

pkgconfig

2.0.3

pkgload

1.2.2

plogr

0.2.0

plyr

1.8.6

praise

1.0.0

prettyunits

1.1.1

pROC

1.18.0

processx

3.5.2

prodlim

2019.11.13

progress

1.2.2

progressr

0.8.0

promises

1.2.0.1

proto

1.0.0

ps

1.6.0

purrr

0.3.4

r2d3

0.2.5

R6

2.5.1

randomForest

4.6-14

rappdirs

0.3.3

rcmdcheck

1.3.3

RColorBrewer

1.1-2

Rcpp

1.0.7

readr

2.0.1

readxl

1.3.1

recipes

0.1.16

rematch

1.0.1

rematch2

2.1.2

remotes

2.4.0

reprex

2.0.1

reshape2

1.4.4

rlang

0.4.11

rmarkdown

2.11

RODBC

1.3-19

roxygen2

7.1.2

rpart

4.1-15

rprojroot

2.0.2

Rserve

1.8-8

RSQLite

2.2.8

rstudioapi

0.13

rversions

2.1.1

rvest

1.0.1

sass

0.4.0

scales

1.1.1

selectr

0.4-2

sessioninfo

1.1.1

shape

1.4.6

shiny

1.6.0

sourcetools

0.1.7

sparklyr

1.7.2

SparkR

3.2.0

spatial

7.3-11

splines

4.1.1

sqldf

0.4-11

SQUAREM

2021.1

stats

4.1.1

stats4

4.1.1

stringi

1.7.4

stringr

1.4.0

survival

3.2-13

sys

3.4

tcltk

4.1.1

TeachingDemos

2.10

testthat

3.0.4

tibble

3.1.4

tidyr

1.1.3

tidyselect

1.1.1

tidyverse

1.3.1

timeDate

3043.102

tinytex

0.33

tools

4.1.1

tzdb

0.1.2

usethis

2.0.1

utf8

1.2.2

utils

4.1.1

uuid

0.1-4

vctrs

0.3.8

viridisLite

0.4.0

vroom

1.5.5

waldo

0.3.1

whisker

0.4

withr

2.4.2

xfun

0.26

xml2

1.3.2

xopen

1.0.0

xtable

1.8-4

yaml

2.2.1

zip

2.2.0

Installed Java and Scala libraries (Scala 2.12 cluster version)

Group ID

Artifact ID

Version

antlr

antlr

2.7.7

com.amazonaws

amazon-kinesis-client

1.12.0

com.amazonaws

aws-java-sdk-autoscaling

1.11.655

com.amazonaws

aws-java-sdk-cloudformation

1.11.655

com.amazonaws

aws-java-sdk-cloudfront

1.11.655

com.amazonaws

aws-java-sdk-cloudhsm

1.11.655

com.amazonaws

aws-java-sdk-cloudsearch

1.11.655

com.amazonaws

aws-java-sdk-cloudtrail

1.11.655

com.amazonaws

aws-java-sdk-cloudwatch

1.11.655

com.amazonaws

aws-java-sdk-cloudwatchmetrics

1.11.655

com.amazonaws

aws-java-sdk-codedeploy

1.11.655

com.amazonaws

aws-java-sdk-cognitoidentity

1.11.655

com.amazonaws

aws-java-sdk-cognitosync

1.11.655

com.amazonaws

aws-java-sdk-config

1.11.655

com.amazonaws

aws-java-sdk-core

1.11.655

com.amazonaws

aws-java-sdk-datapipeline

1.11.655

com.amazonaws

aws-java-sdk-directconnect

1.11.655

com.amazonaws

aws-java-sdk-directory

1.11.655

com.amazonaws

aws-java-sdk-dynamodb

1.11.655

com.amazonaws

aws-java-sdk-ec2

1.11.655

com.amazonaws

aws-java-sdk-ecs

1.11.655

com.amazonaws

aws-java-sdk-efs

1.11.655

com.amazonaws

aws-java-sdk-elasticache

1.11.655

com.amazonaws

aws-java-sdk-elasticbeanstalk

1.11.655

com.amazonaws

aws-java-sdk-elasticloadbalancing

1.11.655

com.amazonaws

aws-java-sdk-elastictranscoder

1.11.655

com.amazonaws

aws-java-sdk-emr

1.11.655

com.amazonaws

aws-java-sdk-glacier

1.11.655

com.amazonaws

aws-java-sdk-glue

1.11.655

com.amazonaws

aws-java-sdk-iam

1.11.655

com.amazonaws

aws-java-sdk-importexport

1.11.655

com.amazonaws

aws-java-sdk-kinesis

1.11.655

com.amazonaws

aws-java-sdk-kms

1.11.655

com.amazonaws

aws-java-sdk-lambda

1.11.655

com.amazonaws

aws-java-sdk-logs

1.11.655

com.amazonaws

aws-java-sdk-machinelearning

1.11.655

com.amazonaws

aws-java-sdk-opsworks

1.11.655

com.amazonaws

aws-java-sdk-rds

1.11.655

com.amazonaws

aws-java-sdk-redshift

1.11.655

com.amazonaws

aws-java-sdk-route53

1.11.655

com.amazonaws

aws-java-sdk-s3

1.11.655

com.amazonaws

aws-java-sdk-ses

1.11.655

com.amazonaws

aws-java-sdk-simpledb

1.11.655

com.amazonaws

aws-java-sdk-simpleworkflow

1.11.655

com.amazonaws

aws-java-sdk-sns

1.11.655

com.amazonaws

aws-java-sdk-sqs

1.11.655

com.amazonaws

aws-java-sdk-ssm

1.11.655

com.amazonaws

aws-java-sdk-storagegateway

1.11.655

com.amazonaws

aws-java-sdk-sts

1.11.655

com.amazonaws

aws-java-sdk-support

1.11.655

com.amazonaws

aws-java-sdk-swf-libraries

1.11.22

com.amazonaws

aws-java-sdk-workspaces

1.11.655

com.amazonaws

jmespath-java

1.11.655

com.chuusai

shapeless_2.12

2.3.3

com.clearspring.analytics

stream

2.9.6

com.databricks

Rserve

1.8-3

com.databricks

jets3t

0.7.1-0

com.databricks.scalapb

compilerplugin_2.12

0.4.15-10

com.databricks.scalapb

scalapb-runtime_2.12

0.4.15-10

com.esotericsoftware

kryo-shaded

4.0.2

com.esotericsoftware

minlog

1.3.0

com.fasterxml

classmate

1.3.4

com.fasterxml.jackson.core

jackson-annotations

2.12.2

com.fasterxml.jackson.core

jackson-core

2.12.2

com.fasterxml.jackson.core

jackson-databind

2.12.2

com.fasterxml.jackson.dataformat

jackson-dataformat-cbor

2.12.2

com.fasterxml.jackson.datatype

jackson-datatype-joda

2.12.2

com.fasterxml.jackson.module

jackson-module-paranamer

2.12.2

com.fasterxml.jackson.module

jackson-module-scala_2.12

2.12.2

com.github.ben-manes.caffeine

caffeine

2.3.4

com.github.fommil

jniloader

1.1

com.github.fommil.netlib

core

1.1.2

com.github.fommil.netlib

native_ref-java

1.1

com.github.fommil.netlib

native_ref-java-natives

1.1

com.github.fommil.netlib

native_system-java

1.1

com.github.fommil.netlib

native_system-java-natives

1.1

com.github.fommil.netlib

netlib-native_ref-linux-x86_64-natives

1.1

com.github.fommil.netlib

netlib-native_system-linux-x86_64-natives

1.1

com.github.luben

zstd-jni

1.5.0-4

com.github.wendykierp

JTransforms

3.1

com.google.code.findbugs

jsr305

3.0.0

com.google.code.gson

gson

2.8.6

com.google.crypto.tink

tink

1.6.0

com.google.flatbuffers

flatbuffers-java

1.9.0

com.google.guava

guava

15.0

com.google.protobuf

protobuf-java

2.6.1

com.h2database

h2

1.4.195

com.helger

profiler

1.1.1

com.jcraft

jsch

0.1.50

com.jolbox

bonecp

0.8.0.RELEASE

com.lihaoyi

sourcecode_2.12

0.1.9

com.microsoft.sqlserver

mssql-jdbc

9.2.1.jre8

com.microsoft.azure

azure-data-lake-store-sdk

2.3.9

com.ning

compress-lzf

1.0.3

com.sun.istack

istack-commons-runtime

3.0.8

com.sun.mail

javax.mail

1.5.2

com.tdunning

json

1.8

com.thoughtworks.paranamer

paranamer

2.8

com.trueaccord.lenses

lenses_2.12

0.4.12

com.twitter

chill-java

0.10.0

com.twitter

chill_2.12

0.10.0

com.twitter

util-app_2.12

7.1.0

com.twitter

util-core_2.12

7.1.0

com.twitter

util-function_2.12

7.1.0

com.twitter

util-jvm_2.12

7.1.0

com.twitter

util-lint_2.12

7.1.0

com.twitter

util-registry_2.12

7.1.0

com.twitter

util-stats_2.12

7.1.0

com.typesafe

config

1.2.1

com.typesafe.scala-logging

scala-logging_2.12

3.7.2

com.univocity

univocity-parsers

2.9.1

com.zaxxer

HikariCP

3.1.0

commons-cli

commons-cli

1.2

commons-codec

commons-codec

1.15

commons-collections

commons-collections

3.2.2

commons-dbcp

commons-dbcp

1.4

commons-fileupload

commons-fileupload

1.3.3

commons-httpclient

commons-httpclient

3.1

commons-io

commons-io

2.8.0

commons-lang

commons-lang

2.6

commons-logging

commons-logging

1.1.3

commons-net

commons-net

3.1

commons-pool

commons-pool

1.5.4

dev.ludovic.netlib

arpack

1.3.2

dev.ludovic.netlib

blas

1.3.2

dev.ludovic.netlib

lapack

1.3.2

hive-2.3__hadoop-3.2

jets3t-0.7

liball_deps_2.12

info.ganglia.gmetric4j

gmetric4j

1.0.10

io.airlift

aircompressor

0.19

io.delta

delta-sharing-spark_2.12

0.1.0

io.dropwizard.metrics

metrics-core

4.1.1

io.dropwizard.metrics

metrics-graphite

4.1.1

io.dropwizard.metrics

metrics-healthchecks

4.1.1

io.dropwizard.metrics

metrics-jetty9

4.1.1

io.dropwizard.metrics

metrics-jmx

4.1.1

io.dropwizard.metrics

metrics-json

4.1.1

io.dropwizard.metrics

metrics-jvm

4.1.1

io.dropwizard.metrics

metrics-servlets

4.1.1

io.netty

netty-all

4.1.63.Final

io.prometheus

simpleclient

0.7.0

io.prometheus

simpleclient_common

0.7.0

io.prometheus

simpleclient_dropwizard

0.7.0

io.prometheus

simpleclient_pushgateway

0.7.0

io.prometheus

simpleclient_servlet

0.7.0

io.prometheus.jmx

collector

0.12.0

jakarta.annotation

jakarta.annotation-api

1.3.5

jakarta.servlet

jakarta.servlet-api

4.0.3

jakarta.validation

jakarta.validation-api

2.0.2

jakarta.ws.rs

jakarta.ws.rs-api

2.1.6

javax.activation

activation

1.1.1

javax.annotation

javax.annotation-api

1.3.2

javax.el

javax.el-api

2.2.4

javax.jdo

jdo-api

3.0.1

javax.transaction

jta

1.1

javax.transaction

transaction-api

1.1

javax.xml.bind

jaxb-api

2.2.2

javax.xml.stream

stax-api

1.0-2

javolution

javolution

5.5.1

jline

jline

2.14.6

joda-time

joda-time

2.10.5

log4j

apache-log4j-extras

1.2.17

log4j

log4j

1.2.17

maven-trees

hive-2.3__hadoop-3.2

liball_deps_2.12

net.java.dev.jna

jna

5.8.0

net.razorvine

pyrolite

4.30

net.sf.jpam

jpam

1.1

net.sf.opencsv

opencsv

2.3

net.sf.supercsv

super-csv

2.2.0

net.snowflake

snowflake-ingest-sdk

0.9.6

net.snowflake

snowflake-jdbc

3.13.3

net.snowflake

spark-snowflake_2.12

2.9.0-spark_3.1

net.sourceforge.f2j

arpack_combined_all

0.1

org.acplt.remotetea

remotetea-oncrpc

1.1.2

org.antlr

ST4

4.0.4

org.antlr

antlr-runtime

3.5.2

org.antlr

antlr4-runtime

4.8

org.antlr

stringtemplate

3.2.1

org.apache.ant

ant

1.9.2

org.apache.ant

ant-jsch

1.9.2

org.apache.ant

ant-launcher

1.9.2

org.apache.arrow

arrow-format

2.0.0

org.apache.arrow

arrow-memory-core

2.0.0

org.apache.arrow

arrow-memory-netty

2.0.0

org.apache.arrow

arrow-vector

2.0.0

org.apache.avro

avro

1.10.2

org.apache.avro

avro-ipc

1.10.2

org.apache.avro

avro-mapred

1.10.2

org.apache.commons

commons-compress

1.20

org.apache.commons

commons-crypto

1.1.0

org.apache.commons

commons-lang3

3.12.0

org.apache.commons

commons-math3

3.4.1

org.apache.commons

commons-text

1.6

org.apache.curator

curator-client

2.13.0

org.apache.curator

curator-framework

2.13.0

org.apache.curator

curator-recipes

2.13.0

org.apache.derby

derby

10.14.2.0

org.apache.hadoop

hadoop-client-runtime

3.3.1

org.apache.hive

hive-beeline

2.3.9

org.apache.hive

hive-cli

2.3.9

org.apache.hive

hive-jdbc

2.3.9

org.apache.hive

hive-llap-client

2.3.9

org.apache.hive

hive-llap-common

2.3.9

org.apache.hive

hive-serde

2.3.9

org.apache.hive

hive-shims

2.3.9

org.apache.hive

hive-storage-api

2.7.2

org.apache.hive.shims

hive-shims-0.23

2.3.9

org.apache.hive.shims

hive-shims-common

2.3.9

org.apache.hive.shims

hive-shims-scheduler

2.3.9

org.apache.htrace

htrace-core4

4.1.0-incubating

org.apache.httpcomponents

httpclient

4.5.13

org.apache.httpcomponents

httpcore

4.4.12

org.apache.ivy

ivy

2.5.0

org.apache.mesos

mesos-shaded-protobuf

1.4.0

org.apache.orc

orc-core

1.6.10

org.apache.orc

orc-mapreduce

1.6.10

org.apache.orc

orc-shims

1.6.10

org.apache.parquet

parquet-column

1.12.0-databricks-0003

org.apache.parquet

parquet-common

1.12.0-databricks-0003

org.apache.parquet

parquet-encoding

1.12.0-databricks-0003

org.apache.parquet

parquet-format-structures

1.12.0-databricks-0003

org.apache.parquet

parquet-hadoop

1.12.0-databricks-0003

org.apache.parquet

parquet-jackson

1.12.0-databricks-0003

org.apache.thrift

libfb303

0.9.3

org.apache.thrift

libthrift

0.12.0

org.apache.xbean

xbean-asm9-shaded

4.20

org.apache.yetus

audience-annotations

0.5.0

org.apache.zookeeper

zookeeper

3.6.2

org.apache.zookeeper

zookeeper-jute

3.6.2

org.checkerframework

checker-qual

3.5.0

org.codehaus.jackson

jackson-core-asl

1.9.13

org.codehaus.jackson

jackson-mapper-asl

1.9.13

org.codehaus.janino

commons-compiler

3.0.16

org.codehaus.janino

janino

3.0.16

org.datanucleus

datanucleus-api-jdo

4.2.4

org.datanucleus

datanucleus-core

4.1.17

org.datanucleus

datanucleus-rdbms

4.1.19

org.datanucleus

javax.jdo

3.2.0-m3

org.eclipse.jetty

jetty-client

9.4.40.v20210413

org.eclipse.jetty

jetty-continuation

9.4.40.v20210413

org.eclipse.jetty

jetty-http

9.4.40.v20210413

org.eclipse.jetty

jetty-io

9.4.40.v20210413

org.eclipse.jetty

jetty-jndi

9.4.40.v20210413

org.eclipse.jetty

jetty-plus

9.4.40.v20210413

org.eclipse.jetty

jetty-proxy

9.4.40.v20210413

org.eclipse.jetty

jetty-security

9.4.40.v20210413

org.eclipse.jetty

jetty-server

9.4.40.v20210413

org.eclipse.jetty

jetty-servlet

9.4.40.v20210413

org.eclipse.jetty

jetty-servlets

9.4.40.v20210413

org.eclipse.jetty

jetty-util

9.4.40.v20210413

org.eclipse.jetty

jetty-util-ajax

9.4.40.v20210413

org.eclipse.jetty

jetty-webapp

9.4.40.v20210413

org.eclipse.jetty

jetty-xml

9.4.40.v20210413

org.eclipse.jetty.websocket

websocket-api

9.4.40.v20210413

org.eclipse.jetty.websocket

websocket-client

9.4.40.v20210413

org.eclipse.jetty.websocket

websocket-common

9.4.40.v20210413

org.eclipse.jetty.websocket

websocket-server

9.4.40.v20210413

org.eclipse.jetty.websocket

websocket-servlet

9.4.40.v20210413

org.fusesource.leveldbjni

leveldbjni-all

1.8

org.glassfish.hk2

hk2-api

2.6.1

org.glassfish.hk2

hk2-locator

2.6.1

org.glassfish.hk2

hk2-utils

2.6.1

org.glassfish.hk2

osgi-resource-locator

1.0.3

org.glassfish.hk2.external

aopalliance-repackaged

2.6.1

org.glassfish.hk2.external

jakarta.inject

2.6.1

org.glassfish.jaxb

jaxb-runtime

2.3.2

org.glassfish.jersey.containers

jersey-container-servlet

2.34

org.glassfish.jersey.containers

jersey-container-servlet-core

2.34

org.glassfish.jersey.core

jersey-client

2.34

org.glassfish.jersey.core

jersey-common

2.34

org.glassfish.jersey.core

jersey-server

2.34

org.glassfish.jersey.inject

jersey-hk2

2.34

org.hibernate.validator

hibernate-validator

6.1.0.Final

org.javassist

javassist

3.25.0-GA

org.jboss.logging

jboss-logging

3.3.2.Final

org.jdbi

jdbi

2.63.1

org.jetbrains

annotations

17.0.0

org.joda

joda-convert

1.7

org.jodd

jodd-core

3.5.2

org.json4s

json4s-ast_2.12

3.7.0-M11

org.json4s

json4s-core_2.12

3.7.0-M11

org.json4s

json4s-jackson_2.12

3.7.0-M11

org.json4s

json4s-scalap_2.12

3.7.0-M11

org.lz4

lz4-java

1.7.1

org.mariadb.jdbc

mariadb-java-client

2.2.5

org.objenesis

objenesis

2.5.1

org.postgresql

postgresql

42.2.19

org.roaringbitmap

RoaringBitmap

0.9.14

org.roaringbitmap

shims

0.9.14

org.rocksdb

rocksdbjni

6.20.3

org.rosuda.REngine

REngine

2.1.0

org.scala-lang

scala-compiler_2.12

2.12.14

org.scala-lang

scala-library_2.12

2.12.14

org.scala-lang

scala-reflect_2.12

2.12.14

org.scala-lang.modules

scala-collection-compat_2.12

2.4.3

org.scala-lang.modules

scala-parser-combinators_2.12

1.1.2

org.scala-lang.modules

scala-xml_2.12

1.2.0

org.scala-sbt

test-interface

1.0

org.scalacheck

scalacheck_2.12

1.14.2

org.scalactic

scalactic_2.12

3.0.8

org.scalanlp

breeze-macros_2.12

1.0

org.scalanlp

breeze_2.12

1.0

org.scalatest

scalatest_2.12

3.0.8

org.slf4j

jcl-over-slf4j

1.7.30

org.slf4j

jul-to-slf4j

1.7.30

org.slf4j

slf4j-api

1.7.30

org.slf4j

slf4j-log4j12

1.7.30

org.spark-project.spark

unused

1.0.0

org.springframework

spring-core

4.1.4.RELEASE

org.springframework

spring-test

4.1.4.RELEASE

org.threeten

threeten-extra

1.5.0

org.tukaani

xz

1.8

org.typelevel

algebra_2.12

2.0.0-M2

org.typelevel

cats-kernel_2.12

2.0.0-M4

org.typelevel

machinist_2.12

0.6.8

org.typelevel

macro-compat_2.12

1.1.1

org.typelevel

spire-macros_2.12

0.17.0-M1

org.typelevel

spire-platform_2.12

0.17.0-M1

org.typelevel

spire-util_2.12

0.17.0-M1

org.typelevel

spire_2.12

0.17.0-M1

org.wildfly.openssl

wildfly-openssl

1.0.7.Final

org.xerial

sqlite-jdbc

3.8.11.2

org.xerial.snappy

snappy-java

1.1.8.4

org.yaml

snakeyaml

1.24

oro

oro

2.0.8

pl.edu.icm

JLargeArrays

1.5

software.amazon.ion

ion-java

1.0.2

stax

stax-api

1.0.1